vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
8984 commits 2 branches 62 tags 205 MiB
Commit graph

2219 commits

Author SHA1 Message Date
Joseph Schorr
23ff49f0c1 Adjust usage of image model in manifest model 2018-08-20 16:47:38 -04:00
Joseph Schorr
7b95082a99 Change image API endpoint to use new registry model 2018-08-20 15:49:51 -04:00
Sam Chow
6afc00bf77 Remove old setup page from main app 
Change page for incomplete setup

Remove old endpoints not in use
 2018-08-20 10:30:21 -04:00
Sam Chow
cc9bedbeb9 refactor approval service key to not need approver 2018-08-15 17:18:41 -04:00
Jimmy Zelinskie
b7573a8c88 endpoints/v2: fail clients not accepting schema v1 2018-08-15 14:49:26 -04:00
Jimmy Zelinskie
568ca1d7ba delete trailing whitespace 2018-08-15 13:21:26 -04:00
Joseph Schorr
701eac5466 Fix registry tests to not terminate prematurely when looking for errors 
Also reorders a check in the V1 protocol to have better error messaging for the users
 2018-08-10 13:36:20 -04:00
Joseph Schorr
590f8f65a3 Allow lookup of "dead" manifests so manifest links can be clicked in the tag history 
Fixes https://jira.coreos.com/browse/QUAY-1021
 2018-08-09 16:05:10 -04:00
Joseph Schorr
56222f95dc Change manifest creation to take in the map of blobs that form the manifest 
We need to lookup the blobs *specific to the images in that manifest*, so we now pass them in from the locations in which we know that information
 2018-08-07 16:28:50 -04:00
Joseph Schorr
e33ccff8cb Fix query count in test 2018-08-07 14:52:33 -04:00
Joseph Schorr
89582438cd Fix the V22 phase 1 migrations to use new tables for mapping rather than editing existing tables 
The ALTER TABLE operations previously used were causing the DB to die when run on the production TagManifest table which has 7 million rows. We instead now use new mapping tables, which is less nice, but these are temporary anyway, so hopefully we only have to deal with their ugliness for a short duration.
 2018-08-06 16:58:27 -04:00
Joseph Schorr
a46660a06f Add new Manifest, ManifestLabel, ManifestLegacyImage and ManifestBlob tables and start writing and GCing to/from them 
This change also starts passing in the manifest interface, rather than the raw data, to the model for writing.

Note that this change does *not* backfill the existing rows in to the new tables; that will occur in a followup PR. The new columns in `tagmanifest` and `tagmanifestlabel` will be used to track the backfill, as it will occur in a worker.
 2018-08-06 13:01:11 -04:00
Joseph Schorr
7325b22c90 Change id column in LogEntry to a BigInt and migrate back to using that table 2018-07-30 12:42:56 -04:00
Joseph Schorr
c34eacb4fa Fix issue accessing a null tagmanifest under a tag in a repository 2018-07-25 16:14:20 -04:00
Joseph Schorr
487edf0ba1 Phase 3 of Appr migration 
Deletes the old models and their code
 2018-07-21 15:43:00 -04:00
Joseph Schorr
223077ef53 Phase 2 of Appr migration 
Backfills the new tables from the old ones, and switches all code to read and write from the new tables
 2018-07-21 15:43:00 -04:00
Joseph Schorr
a3c3c0abc7 Phase 1 of migrating APPR-specific tables to tables with the Appr prefix 
Fixes https://jira.coreos.com/browse/QUAY-950
 2018-07-21 15:43:00 -04:00
Joseph Schorr
acb7d626a4 Phase 2 of Appr migration 
Backfills the new tables from the old ones, and switches all code to read and write from the new tables
 2018-07-21 15:25:04 -04:00
Joseph Schorr
113bb96f29 Phase 1 of migrating APPR-specific tables to tables with the Appr prefix 
Fixes https://jira.coreos.com/browse/QUAY-950
 2018-07-20 13:19:01 -04:00
Joseph Schorr
6622f27c93 Rename oci_model to appr_model 2018-07-20 12:46:19 -04:00
Joseph Schorr
306e79e493 Move trigger tests into pytest 
Note that we split the tests between endpoints/building and buildtrigger
 2018-07-19 11:10:19 -04:00
Joseph Schorr
78317c0e93
Merge pull request #3149 from quay/joseph.schorr/QUAY-1000/remove-chatlio 
Remove chatlio support chat
 2018-07-18 02:54:27 +03:00
Joseph Schorr
6a322c90f4
Merge pull request #3154 from quay/move-manifest-cornercase-test 
Move manifest corner case tests to pytest
 2018-07-17 23:53:49 +03:00
Joseph Schorr
58e553838c Move manifest corner case tests to pytest 2018-07-17 16:40:56 -04:00
Joseph Schorr
fcb9fd3792 Move end to end auth tests for APIs into pytest 2018-07-17 16:28:03 -04:00
Joseph Schorr
c2778c7031 Remove chatlio support chat 
Fixes https://jira.coreos.com/browse/QUAY-1000
 2018-07-17 12:55:17 -04:00
Sam Chow
496d94138c
Merge pull request #3139 from quay/spike/install-certs 
Install certs in the config app, small refactor to LDAP validation
 2018-07-16 12:50:36 -04:00
Joseph Schorr
6c3cf403bd
Merge pull request #3137 from quay/joseph.schorr/QUAY-963/repo-name-check 
Ensure that we limit the length of repository names
 2018-07-15 19:17:40 +03:00
Joseph Schorr
a572fd33c7 Ensure that we limit the length of repository names 
Until now, they'd simply be truncated by the database. Now, we properly check their lengths.

Fixes https://jira.coreos.com/browse/QUAY-963
 2018-07-13 13:22:33 +03:00
Joseph Schorr
d32efc4e17 Move anon checked test to pytest 2018-07-12 15:24:49 +03:00
Joseph Schorr
00d965a301
Merge pull request #3135 from quay/joseph.schorr/QUAY-999/informative-errors 
Make API errors more informative
 2018-07-10 22:41:34 +03:00
Sam Chow
bd54eacbad Add app var for init scripts location to access certs install 2018-07-10 11:43:34 -04:00
Joseph Schorr
924dda296f Fully migrate API security tests into the pytest test suite 
Also adds an additional test that ensures that at least one security test exists for every (api endpoint, http method) pair.
 2018-07-08 18:33:21 +03:00
Joseph Schorr
4f152fd7c7 Make API errors more informative 
Fixes https://jira.coreos.com/browse/QUAY-999
 2018-07-08 11:45:33 +03:00
Brad Ison
73cb7f3228
endpoints/api: Allow null fields in user metadata 
The user metadata fields are nullable in the database, but were not in
the json sechema.  This prevented users from updating some of their
information on the site if they hadn't set the metadata fields.
 2018-06-27 15:34:55 -04:00
Sam Chow
f32bbf1fdc
Merge pull request #3121 from quay/project/upload-tar 
Q.E. User can upload a tarball config to modify
 2018-06-22 14:50:21 -04:00
Joseph Schorr
892cc82b6a Ensure that verbs cannot be performed on disabled namespaces or by disabled users 2018-06-21 14:41:27 -04:00
Sam Chow
b5f630ba29 Fix alembic migrations importing app 
Ensure we connect to loaded config db
 2018-06-20 17:17:35 -04:00
Joseph Schorr
2caaf84f31 Add caching support to catalog 
We will now cache the results of the catalog for 60s and not hit the database at all if cached
 2018-06-20 14:58:01 -04:00
Joseph Schorr
7604e9842b Change repo filtering for users to use a user ID reference, rather than the username 
While this means we need an additional query for initial lookup, it makes the *filtering* query (which is the heavy part) require far fewer joins, thus making it more efficient.

Also adds a new unit test to verify that our filter filters to the correct set of repositories.
 2018-06-19 10:51:30 -04:00
Joseph Schorr
3161b60522 Switch V2 pagination back to using IDs, which should be much faster and easier on the DB 
Also adds a test for the tags endpoint
 2018-06-18 16:11:26 -04:00
Joseph Schorr
0fdefd78e9 Extract app from torrent handling code 
Fixes https://jira.coreos.com/browse/QUAY-969
 2018-06-14 17:29:39 -04:00
Sam Chow
093a58558c
Merge pull request #3104 from quay/project/app-endpoints 
Config App initial entrypoint
 2018-06-12 17:03:59 -04:00
Joseph Schorr
1be22a9a56
Merge pull request #3110 from quay/joseph.schorr/QUAY-966/gitlab-v4 
Reimplement GitLab trigger handler using the V4 API library
 2018-06-12 17:03:31 -04:00
Joseph Schorr
bf966545ba Reimplement GitLab trigger handler using the V4 API library 
GitLab has deprecated and removed the V3 API entirely

Fixes https://jira.coreos.com/browse/QUAY-966
 2018-06-12 15:01:37 -04:00
Sam Chow
e9d24dc5ff Make script paths work in docker and locally for config_app 2018-06-12 14:45:00 -04:00
Sam Chow
d080ca2cc6 Create webpack config for config app 
further improve developer morale

get initial angular loading

Add remote css to config index

Starts work to port endpoints into config app

Add the api blueprint
 2018-06-12 14:44:15 -04:00
Sam Chow
92db413da6 Start moving configapp into separate dir 2018-06-12 14:44:15 -04:00
Sam Chow
81af2d9fcc Use different js bundle with base html 2018-06-12 14:44:11 -04:00
Joseph Schorr
0fa1a1d5fd Implement support for blob mounting via the mount parameter on blob uploads 
Fixes https://jira.coreos.com/browse/QUAY-893
 2018-06-06 11:31:28 -04:00
Sam Chow
708b7ee669 Add config flag that only renders new simple page 2018-06-06 10:28:37 -04:00
Joseph Schorr
ef167ab7e3 Rate limit the catalog endpoint by auth token and IP address 2018-06-05 18:24:31 -04:00
Joseph Schorr
2e0edf8f6e Have catalog endpoint return empty if the namespace is disabled 2018-06-05 17:31:00 -04:00
Sam Chow
7f21d0da58
Merge pull request #3096 from quay/refactor/secscan-simple 
Refactor validators to decouple app
 2018-06-01 15:44:18 -04:00
Sam Chow
301cc6992a Remove jwt validation for jschorr to fix later 
Refactor oauth validate method to take config over entire appconfig
 2018-06-01 15:07:06 -04:00
Sam Chow
7df8ed4a60 Add a security scanner api config object for params 
Change SecScanAPI to use a uri creation func instead of test context

Pass config provider through validator context

Remove app config dependency for validators
 2018-06-01 15:06:50 -04:00
Sam Chow
554d4f47a8 Change validators to use the validator_context 
Change InstanceKeys to take a namedtuple for context
 2018-06-01 14:59:49 -04:00
Joseph Schorr
b2262eaf46 Add feature flag to disable username confirmation 
Fixes https://jira.coreos.com/browse/QUAY-914
 2018-06-01 13:30:50 -04:00
Joseph Schorr
8ee98c1c48 Show starred public repositories in the starred repos list 
We forgot to check if the repository was public in the filter

Fixes https://jira.coreos.com/browse/QUAY-857
 2018-05-31 17:34:17 -04:00
Sam Chow
e967fde3ae Decouple oauth methods from app with a namedtuple 2018-05-31 14:53:27 -04:00
Sam Chow
d45b925155 Move config provider to _init to decouple from app 
remove app references from validators
 2018-05-25 11:15:06 -04:00
Joseph Schorr
f86c087b3b Prevent registry operations against disabled namespaces 
Allows admins to completely wall off a namespace by disabling it

Fixes https://jira.coreos.com/browse/QUAY-869
 2018-05-22 18:36:04 -04:00
Joseph Schorr
6ffafe44d3
Merge pull request #3059 from quay/joseph.schorr/QUAY-906/reg-tests 
Move registry integration tests to py.test
 2018-05-22 17:09:11 -04:00
Joseph Schorr
fc6eb71ab1 Switch from an image view UI to a manifest view UI 
We no longer allow viewing individual images, but instead only manifests. This will help with the transition to Clair V3 (which is manifest based) and, eventually, the the new data model (which will also be manifest based)
 2018-05-22 13:11:04 -04:00
Joseph Schorr
a007332d4c Temporarily change to storing logs in a new LogEntry2 table 
This will prevent us from running out of auto-incrementing ID values until such time as we can upgrade to peewee 3 and change the field type to a BigInt

Fixes https://jira.coreos.com/browse/QUAY-943
 2018-05-18 20:15:16 -04:00
josephschorr
58612955d6
Merge pull request #3079 from quay/joseph.schorr/QUAY-927/robots-limit 
Limit robots displayed in entity search
 2018-05-15 13:59:48 -04:00
Joseph Schorr
5c50161d85 Limit robots displayed in entity search 
Before, we'd load *all* the robots, which can be a huge issue in namespaces with a large number of robots. Now, we only load the top-20 robots (as per recency in login), and we also limit the information returned to the entity search to save some bandwidth.

Fixes https://jira.coreos.com/browse/QUAY-927
 2018-05-15 11:00:57 -04:00
Joseph Schorr
e3248bde47 Small fixes to make loading of logs faster 
Removes filtering of log types where not necessary, removes filtering based on namespace when filtering based on repository (superfluous check that was causing issues in MySQL preventing the use of the correct index) and fix some other small issues around the API

Fixes https://jira.coreos.com/browse/QUAY-931
 2018-05-14 11:41:49 -04:00
Joseph Schorr
2ae69dc651 Further fixes to the Kubernetes config provider, and a new set of proper unit tests 2018-05-10 16:44:18 +03:00
Joseph Schorr
77adf9dd77 Fix bug which allowed for implicit library namespace access via the V1 registry protocol when the feature flag was off 
Now we raise a 400 as expected
 2018-05-01 13:28:24 +03:00
Sam Chow
f89ad30320
Merge pull request #3060 from quay/max-results-help-text 
Configurable options for search, disable next page & add help text when at max results
 2018-04-25 08:17:35 -07:00
Sam Chow
1afedafcbb Configurable options for search, info when at max 
includes the options for  maximum search results per page, and the
maximum number of pages available before help text is shown, and
the next page button is disabled
 2018-04-25 11:12:09 -04:00
Joseph Schorr
3309daa32e Add support for reduced initial build count for new possible abusing users 
If configured, we now check the IP address of the user signing up and, if they are a possible threat, we further reduce their number of allowed maximum builds to the configured value.
 2018-04-20 18:46:32 +03:00
Joseph Schorr
a59c951aa3 Add support for multiple scope parameters on V2 auth requests 
Fixes https://jira.coreos.com/browse/QUAY-892
 2018-04-18 20:16:49 +03:00
Joseph Schorr
86aa93aab5 Add a new test suite for the v2auth generate_registry_jwt endpoint 2018-04-18 20:16:49 +03:00
Joseph Schorr
8146646761 Simplifying queries around images and placements 
Only verbs needs to load placements for multiple images, so we can vastly simplify and optimize most queries by making it two-step, and having the rest of the image loads not worry about placements
 2018-04-03 16:23:49 -04:00
josephschorr
323eb63747
Merge pull request #3032 from coreos-inc/joseph.schorr/QUAY-885/squashed-sym 
Retarget broken hard links in squashed images
 2018-03-26 17:59:52 -04:00
Joseph Schorr
dd470bdc9d Add a reporter for verbs to track number of storage streams are needed 2018-03-26 17:02:39 -04:00
Joseph Schorr
110366f656 Retarget hard links pointing to deleted files by emitting the deleted file contents under the first hard link instance. This fixes a breakage in the squashed TAR where we were pointing hard links to missing data. 
Fixes https://jira.coreos.com/browse/QUAY-885
 2018-03-23 14:00:46 -04:00
josephschorr
7b5c22c699
Merge pull request #1205 from coreos-inc/apiimprov 
Remove 404 when an entity is not a member of a team, but is a robot u…
 2018-03-22 21:41:16 -04:00
Joseph Schorr
6d365418b7 Limit the size of then payload for creating robot accounts 2018-03-22 13:54:56 -04:00
josephschorr
6c43b7ff0d
Merge pull request #3024 from coreos-inc/manageable-robots 
Manageable robots epic
 2018-03-21 18:50:17 -04:00
Joseph Schorr
2ea13e86a0 Add last_accessed information to User and expose for robot accounts 
Fixes https://jira.coreos.com/browse/QUAY-848
 2018-03-21 15:28:34 -04:00
Joseph Schorr
3586955669 Remove license code in Quay 
No longer needed under Red Hat rules \o/

Fixes https://jira.coreos.com/browse/QUAY-883
 2018-03-20 17:03:35 -04:00
josephschorr
ca0772e1b5
Merge pull request #3027 from coreos-inc/joseph.schorr/QUAY-871/oauth-scopes 
Fix OAuth scopes display
 2018-03-16 14:28:40 -04:00
Joseph Schorr
86f898d9bd Fix OAuth scopes display 
Before, we were sending the wrong kind of data (namedtuple instead of dict) in the non-superuser case, which broke prod. Now, we always explicitly send a standard dictionary.

Fixes https://jira.coreos.com/browse/QUAY-871
 2018-03-16 13:03:42 -04:00
Brad Ison
e8429f9194
Add organization collaborators API endpoint 
Adds an API endpoint, `/v1/organization/<orgname>/collaborators`, that
lists an organization's "outside collaborators", i.e. users that have
direct permissions on one or more repositories belonging to the
organization, but who aren't members of any teams in the organization.
 2018-03-14 16:11:53 -04:00
Joseph Schorr
f1da3c452f Remove unused code 2018-03-12 20:32:05 -04:00
Joseph Schorr
254cdfe43a Add support for metadata on robot accounts 
Fixes https://jira.coreos.com/browse/QUAY-847
Fixes https://jira.coreos.com/browse/QUAY-816
 2018-03-12 20:32:05 -04:00
Joseph Schorr
a693771345 Add creation date information to robots API and UI 
Fixes https://jira.coreos.com/browse/QUAY-846
 2018-03-09 13:55:19 -05:00
Joseph Schorr
da45bedcdb Remove 404 when an entity is not a member of a team, but is a robot under the org. 
Fixes #1200
 2018-03-02 16:07:03 -05:00
Joseph Schorr
24b77bbc10 Fix serialization of blobs to/from the cache 
Also makes sure the test checks that serialization
 2018-03-02 14:22:55 -05:00
Joseph Schorr
93d79e777e Automatically disable build triggers with successive failures or internal errors 
We allow users to reenable them manually once disabled
 2018-03-01 16:49:51 -05:00
Joseph Schorr
c35eec0615 Add ability for triggers to be disabled 
Will be used in the followup commit to automatically disable broken triggers
 2018-03-01 16:49:28 -05:00
Joseph Schorr
4be3594ec8 Remove internal_only from superuser APIs for users and orgs 2018-03-01 15:14:39 -05:00
Joseph Schorr
8bc55a5676 Make namespace deletion asynchronous 
Instead of deleting a namespace synchronously as before, we now mark the namespace for deletion, disable it, and rename it. A worker then comes along and deletes the namespace in the background. This results in a *significantly* better user experience, as the namespace deletion operation now "completes" in under a second, where before it could take 10s of minutes at the worse.

Fixes https://jira.coreos.com/browse/QUAY-838
 2018-02-27 13:12:51 -05:00
josephschorr
6220df4f88
Merge pull request #3012 from coreos-inc/access-control-header 
Add X-Requested-With header to allowed CORS headers
 2018-02-21 14:27:36 -05:00
Joseph Schorr
bcd9b680fa Add X-Requested-With header to allowed CORS headers 
Will fix the API explorer
 2018-02-21 14:21:27 -05:00
Joseph Schorr
e446eb5757 Switch build queue limiter query to use total number of alive jobs 
This is slightly more accurate and, not being based on time, will work better under MySQL
 2018-02-21 14:04:40 -05:00
Joseph Schorr
9a452ace11 Add configurable limits for number of builds allowed under a namespace 
We also support that limit being increased automatically once a successful billing charge has gone through
 2018-02-20 16:54:22 -05:00