Commit graph

1371 commits

Author SHA1 Message Date
Joseph Schorr
7d4fed6892 Change error message when trying to pull a deleted or expired tag
Will let the users know they can recover the tag via time machine

Note: This was tested with the Docker protocol, but the new error code is *technically* out of spec; we should make sure its okay.
2017-07-19 17:13:48 -04:00
Joseph Schorr
99d7fde8ee Add UI for viewing and changing the expiration of tags 2017-07-19 17:13:48 -04:00
Joseph Schorr
4663bf4194 Add additional test for tag expiration 2017-07-19 17:13:06 -04:00
Joseph Schorr
4ad3682b9c Make health check failures report their reasons
Note that we add a new block with expanded service info, to avoid breaking compatibility with existing callers of the health endpoint
2017-07-19 16:17:02 +03:00
Joseph Schorr
0dfb6806e3 Add ping method to auth engines to determine if they are reachable 2017-07-19 16:16:41 +03:00
josephschorr
fdb21aa5dc Merge pull request #2777 from coreos-inc/joseph.schorr/QUAY-618/notificationworker-data-interface
Change notificationworker to use data interface
2017-07-13 00:23:15 +03:00
josephschorr
d00c7d299c Merge pull request #2778 from coreos-inc/gc-long-chain-test
Add a long-chain test for GC
2017-07-13 00:23:03 +03:00
Charlton Austin
82488c9102 Merge pull request #2753 from charltonaustin/api_tag_v22
refactor(endpoints/api/tag): refactor code for v22
2017-07-12 15:51:48 -04:00
Charlton Austin
131acde317 refactor(data+endpoints): code review changes
this puts the view logic on the object and adds a parameter for logging

[TESTING->locally with docker compose]

Issue: https://coreosdev.atlassian.net/browse/QUAY-632

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-07-12 14:10:21 -04:00
Joseph Schorr
2b51265b48 Add a long-chain test for GC 2017-07-12 18:03:27 +03:00
Joseph Schorr
b6f1782642 Change notificationworker to use a data interface 2017-07-12 17:40:45 +03:00
Joseph Schorr
bdab367285 Change blobuploadcleanupworker to use a data interface 2017-07-11 16:58:09 +03:00
Charlton Austin
897a091692 style(data+endpoints): ran yapf
### Description of Changes

ran yapf for the branch

[TESTING->locally using docker compose]

Issue: https://coreosdev.atlassian.net/browse/QUAY-632

## Reviewer Checklist

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-07-10 09:46:02 -04:00
Charlton Austin
fc4b3642d3 refactor(endpoints/api/tag): refactor code for v22
this decouples the database models from the api

[TESTING->locally with docker compose]

Issue: https://coreosdev.atlassian.net/browse/QUAY-632

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-07-10 09:40:33 -04:00
EvB
ccca0c9655 refactor(util/tufmetadata/test): move app test to gc suite 2017-07-07 15:14:14 -04:00
josephschorr
89ab94ffbb Merge pull request #2720 from coreos-inc/gc-opt
GC optimizations
2017-06-29 20:55:14 +03:00
Jimmy Zelinskie
ecd5996386 endpoints.v2: new fs layout for data interface
Fixes QUAY-656
2017-06-29 13:19:53 -04:00
josephschorr
0e702c72f2 Merge pull request #2739 from coreos-inc/joseph.schorr/QUAY-664/verbs-interface-refactor
Refactor Verbs model definitions to match new style
2017-06-29 10:07:36 +03:00
Jimmy Zelinskie
80c9b611a4 Merge pull request #2730 from jzelinskie/v2-interface
endpoints.v2: new fs layout for data interface
2017-06-28 13:51:43 -04:00
Joseph Schorr
8b4958dbcc Move verbs model into new file structure 2017-06-28 13:05:02 +03:00
Joseph Schorr
8dcea30d58 Fix build by pre-calling the caches
They were being called in a test-dependent order, which caused any tests which relied on query count to fail
2017-06-27 18:11:46 +03:00
Joseph Schorr
cdd7cb9321 Remove directly referenced images from the candidate set before starting GC iteration
Makes the lookup query underneath the transaction smaller if there are a lot of images referenced directly by tag. We still must do the direct referenced check within the transaction, but this should reduce the scope of the search space a bit.
2017-06-27 17:33:58 +03:00
Joseph Schorr
45c7008078 Change Repo GC to be iterative
This prevents us from creating a massive join when there are a large number of tags in the repository, which can result in locking the entire DB for long periods of time. Instead of the join, we just iteratively lookup any images found to be directly referenced by a tag or found as a parent of another image, both of which should be indexed lookups. Once done, we only remove those images and then iterate until the working set stops changing.
2017-06-27 17:33:58 +03:00
Joseph Schorr
e9a95874ee Move GC tests into pytest 2017-06-27 17:33:58 +03:00
Joseph Schorr
11659f73bf Fix log for reenabling a notification
We forgot to log the event and method names
2017-06-27 15:26:40 +03:00
Jimmy Zelinskie
0e26a03f7e endpoints.v2: new fs layout for data interface
Fixes QUAY-658
2017-06-26 18:21:35 -04:00
Charlton Austin
5e6aa6648b fix(endpoints): added in proper error handling
before we would return a 400 without a message because the errors were not being caught

Issue: https://www.pivotaltracker.com/story/show/145459707

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-06-21 16:59:48 -04:00
josephschorr
b269cb84a3 Merge pull request #2688 from coreos-inc/build-badge-query
Remove expensive call in build badge
2017-06-12 15:06:13 -04:00
Joseph Schorr
a949a44cb2 Remove expensive call in build badge
We don't need to list all the tags to determine if any exist, and showing the repo is ready when it is empty is probably correct behavior anyway
2017-06-12 14:53:54 -04:00
Joseph Schorr
d68b65d90c Optimize GC query join a bit by reducing the surface
We remove the directly referenced images from the join across ancestors, as they will be covered by the first part of the union clause. For some large repositories, this will result in a significantly reduced set of images that have to be joined NxM.
2017-06-12 12:55:32 -04:00
Antoine Legrand
9c68cbd6e8 Fix no search result for apps 2017-06-05 23:03:47 +02:00
Charlton Austin
52750e74ee fix(add notification number): add explicit default
there is a possibility that this will not work with postgres

Issue: https://www.pivotaltracker.com/story/show/144646649

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-05-22 09:55:40 -04:00
Charlton Austin
a71f60a9c1 Merge pull request #2652 from charltonaustin/failing_repository_notifications_to_be_disabled_after_n_failures_in_a_row_144646649
Failing repository notifications to be disabled after n failures in a row 144646649
2017-05-22 09:30:53 -04:00
Charlton Austin
993f2a174c feat(full-stack): disable notifications after 3 failures
This stops notifications from firing over and over again if they are repeatedly failing.

[TESTING -> locally with docker compose, DATABASE MIGRATION -> there is a single migration]

Issue: https://www.pivotaltracker.com/story/show/b144646649n

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-05-19 16:58:46 -04:00
Joseph Schorr
065e327190 Make sure catalog always returns public repositories 2017-05-19 16:24:08 -04:00
Jimmy Zelinskie
702cdf59ff Merge pull request #2637 from jzelinskie/audit-apps
Audit Logs for Apps
2017-05-16 17:06:25 -04:00
Jimmy Zelinskie
4db789b656 add audit logging to app registry endpoints 2017-05-16 15:54:02 -04:00
josephschorr
1dafddd7bf Merge pull request #2623 from coreos-inc/ldap-user-creation
Disable federated login for new users if user creation is disabled
2017-05-15 15:07:52 -04:00
Joseph Schorr
b3d7577473 Disable federated login for new users if user creation is disabled
Fixes https://www.pivotaltracker.com/story/show/144821585
2017-05-15 15:07:08 -04:00
Joseph Schorr
db767b3610 Optimize lookup of org membership on prototype and perms APIs
Fixes a major slowdown when working with permissions under organizations with a lot of members

Fixes https://www.pivotaltracker.com/story/show/144076113
2017-05-08 14:03:59 -04:00
Joseph Schorr
9d97e053b3 Make sure to re-sort the filtered repositories in search
The filtering breaks the ordered we expected, so we need to re-sort
2017-05-03 18:38:46 -04:00
Joseph Schorr
0164b48a24 Switch repository search to order matches in repo names higher
Helps push better results to the top of the results list
2017-05-03 17:02:24 -04:00
josephschorr
19f67bfa1b Merge pull request #2607 from coreos-inc/faster-security-notify
Batch the tag lookups in the security notification worker in an attempt to significant reduce load
2017-05-03 13:49:13 -04:00
Joseph Schorr
68d473bff4 Clarify comment and make pairs code a little nicer 2017-05-03 11:41:39 -04:00
Joseph Schorr
977bbc20a2 Add filtering onto the images query in get_matching_tags_for_images
Should make the query even faster in the security notification case
2017-05-02 18:29:14 -04:00
josephschorr
5a9a231754 Merge pull request #2529 from coreos-inc/search-ui
Implement new search UI
2017-05-02 15:56:59 -04:00
Joseph Schorr
d77463cc8b Add sharing to the images lookup in get_matching_tags_for_images
Should prevent an issue if we ever get a request with thousands and thousands of images where we would exceed the DB's max packet size
2017-05-02 15:50:31 -04:00
Joseph Schorr
74dd0ef8e8 Add a batch get_matching_tags_for_images method
This will be used in the security notification worker to retrieving the tags needed in a set of batch calls, rather than multiple calls per image
2017-05-02 15:38:25 -04:00
Joseph Schorr
b67113e848 Move LDAP controls init into the inner loop
We cannot use it across different DNs, so we need to move it down
2017-05-01 16:04:33 -04:00
Joseph Schorr
30a681343f Make sure to escape LDAP queries
Fixes an issue in team sync around group names that contain *s

Fixes https://www.pivotaltracker.com/story/show/144628235
2017-05-01 14:00:54 -04:00