Jimmy Zelinskie
b68e1b5efc
add "get_" prefix to all db read funcs
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
32a6c22b43
mv data/types image
...
This change also merges formats into the new image module.
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
a516c08deb
v2: refactor auth to use data.types
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
3de6000428
v2: refactor blob.py to use data.types
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
21cbe0bd07
v2: mv _paginate paginate
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
3f722f880e
v2: add pagination decorator
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
5b630ebdb0
v2/manifest: refactor to use types
2016-09-26 14:48:05 -04:00
Joseph Schorr
c8a1b8abab
Add prom stats for repository push, pull and verb actions
2016-09-09 15:13:58 -04:00
Joseph Schorr
357005e33f
Raise a 409 if we try to insert a tag twice at the same time
...
Also fixes handling of labels for existing manifests
Fixes #1775
2016-08-29 16:03:35 -04:00
Joseph Schorr
608ffd9663
Basic labels support
...
Adds basic labels support to the registry code (V2), and the API. Note that this does not yet add any UI related support.
2016-08-26 15:24:26 -04:00
Joseph Schorr
a1a6647ea7
Add additional logs for invalid blob upload
...
Fixes #1668
2016-08-02 12:15:30 -04:00
Joseph Schorr
0fe3e6510a
Prevent invalid tags on builds
...
Fixes #1632
2016-07-25 17:50:35 -07:00
Joseph Schorr
cbf7c2bf44
Add better logging to blob uploads
...
Fixes #1635
2016-07-20 17:53:43 -04:00
Joseph Schorr
a1009af61c
Move aggregator into its own repo and add it to the image
2016-07-05 15:39:51 -04:00
josephschorr
614b9124ae
Merge pull request #1512 from coreos-inc/optimize-queries
...
Optimize various queries
2016-06-16 14:22:59 -04:00
Joseph Schorr
8887f09ba8
Use the instance service key for registry JWT signing
2016-06-07 11:58:10 -04:00
Joseph Schorr
53538f9001
Optimize get_tag_image query
...
No caller uses the image placements or locations, so no need to load them.
2016-06-02 16:36:38 -04:00
josephschorr
cad8746f9d
Merge pull request #1502 from coreos-inc/image-replication
...
Enable storage replication for V2 and add backfill tool
2016-06-02 15:02:53 -04:00
Joseph Schorr
12924784ce
Enable storage replication for V2 and add backfill tool
...
Fixes #1501
2016-06-02 14:36:08 -04:00
Joseph Schorr
48213f9ff9
Reject manifest 2 earlier to make pushes faster
2016-06-02 12:46:20 -04:00
Jake Moshenko
8323c51e6e
Extend registry auth to support notary JWTs.
2016-05-24 13:42:28 -04:00
Jake Moshenko
9221a515de
Use the registry API for security scanning
...
when the storage engine doesn't support direct download url
2016-05-04 18:04:06 -04:00
Joseph Schorr
03489c22ad
Log the pushed tag and add IP address display
...
Fixes #798
2016-04-20 13:00:21 -04:00
Joseph Schorr
57e5141fb5
Fix link-to-parent-with-different-blob issue and add a test
2016-03-14 15:35:18 -04:00
Jimmy Zelinskie
ea2e17cc11
v2: send proper scopes for authorization failures
...
Fixes #1278 .
2016-03-11 13:41:38 -05:00
Jimmy Zelinskie
bb46cc933d
use kwargs for parse_repository_name
2016-03-09 16:20:28 -05:00
Jimmy Zelinskie
01b9606481
Merge pull request #1255 from jzelinskie/wwwauth
...
v2: always send www-authn headers on unauthorized
2016-02-22 12:49:37 -05:00
Jimmy Zelinskie
54a029e2bd
v2: always send www-authn headers on unauthorized
...
Fixes #1254 .
2016-02-22 12:48:19 -05:00
josephschorr
81a36ee3b8
Merge pull request #1217 from coreos-inc/v2pagination
...
Fix V2 catalog and tag pagination
2016-02-16 15:34:49 -05:00
Jake Moshenko
6e05920d6b
Delete bad manifests from the DB
2016-02-16 11:42:19 -05:00
Joseph Schorr
4b24556cb3
Check for the parent's ID in the updated ID map.
...
Fixes #1240
2016-02-15 11:02:52 -05:00
Joseph Schorr
69262282fe
Make sure to encode all V1 metadata strings
...
Fixes #1239
2016-02-15 10:57:20 -05:00
Jake Moshenko
6454b5aeb7
Update the layer rename PR to preserve the original manifest
2016-02-12 16:25:47 -05:00
Joseph Schorr
abd2e3c234
V1 Docker ID <-> V2 layer SHA mismatch fix
...
Fix handling of V1 Docker ID <-> V2 layer SHA mismatch by dynamically rewriting the manifest to use new synthesized IDs for all layers above the mismatch. Also adds a bunch of tests for this and other use cases, fixes a bug around manifest digest uniqueness and fixes the 5.5 migration for MySQL.
2016-02-12 17:39:27 +02:00
Quentin Machu
5c7f2a5c16
Remove abort(500) on checksum mismatch
2016-02-11 18:32:21 -05:00
Joseph Schorr
27f1cc0a13
Add a check that will fail if we try to mislink V1 layers
...
Also logs some useful information
2016-02-11 22:40:00 +02:00
Joseph Schorr
db0eab0461
Fix V2 catalog and tag pagination
2016-02-10 00:25:33 +02:00
Jimmy Zelinskie
463dc98a42
return an error when writing manifest v2 schema2
2016-02-09 14:42:58 -05:00
Joseph Schorr
534ec9cb2b
Add pagination to the repository list API to make it better for public
...
Fixes #1166
2016-02-01 22:42:44 +02:00
Joseph Schorr
a03f0f1970
Fix manifest content type
...
Fixes #1168
2016-01-25 16:39:59 -05:00
josephschorr
b081e234f8
Merge pull request #1162 from coreos-inc/publicnocredauth
...
Fix issue with Docker 1.8.3 and pulling public repos with no creds
2016-01-25 16:07:55 -05:00
Joseph Schorr
8cd38569d6
Fix issue with Docker 1.8.3 and pulling public repos with no creds
...
We now return the valid subset of auth scopes requested.
Adds a test for this case and adds testing of all returned JWTs in the V2 login tests
2016-01-25 15:54:17 -05:00
Jimmy Zelinskie
85ae1a2a0a
Merge pull request #1161 from jzelinskie/torrenthmac
...
misc torrent changes
2016-01-22 23:02:44 -05:00
josephschorr
566a91f003
Merge pull request #1160 from coreos-inc/dockerv2authsucks
...
Fix Docker Auth and our V2 registry paths to support library (i.e. namespace-less) repositories.
2016-01-22 16:00:30 -05:00
Joseph Schorr
e4ffaff869
Fix Docker Auth and our V2 registry paths to support library (i.e. namespace-less) repositories.
...
This support is placed behind a feature flag.
2016-01-22 15:54:06 -05:00
Jimmy Zelinskie
e54b86c6eb
s/TORRENT/BITTORRENT
2016-01-22 15:52:28 -05:00
josephschorr
d00db518df
Merge pull request #1153 from coreos-inc/fixtutorial
...
Fix tutorial by properly publishing user events for V2 API
2016-01-22 12:51:47 -05:00
Joseph Schorr
068301ef1f
Add more debugging statements to V2 auth
...
Also fixes a spurious return
2016-01-20 18:06:46 -05:00
Joseph Schorr
22b8a562be
Fix tutorial by properly publishing user events for V2 API
...
Fixes #1123
2016-01-20 13:42:30 -05:00
Joseph Schorr
e4da61a05d
Fix piece hash calculation
2016-01-12 17:44:19 -05:00
Jake Moshenko
8ab6c8a22d
Fix torrent hash generation to work in mixed stacks
2016-01-11 16:43:46 -05:00
Jake Moshenko
17d3b5e204
Fix a bug with torrent calculation on duplicate v2 uploads
2016-01-08 15:08:12 -05:00
Jake Moshenko
44fcc7e44b
Fix logic when deciding whether to hash for torrent
2016-01-05 12:13:26 -05:00
Jake Moshenko
ce8fcbeaae
Update the pieces to use base64 encoded binary
2016-01-04 16:17:51 -05:00
Jake Moshenko
fe87d3c796
Hash and track layer file chunks for torrenting
2016-01-04 16:17:51 -05:00
Jake Moshenko
766d60493f
Add the ability to blacklist v2 for specific versions
2015-12-15 18:27:10 -05:00
Joseph Schorr
ca7d36bf14
Handle empty scopes and always send the WWW-Authenticate header, as per spec
...
Fixes #1045
2015-12-15 14:59:47 -05:00
Joseph Schorr
ea9ebb98e6
Sample repo pulls in V2
...
Fixes #1058
2015-12-14 17:41:16 -05:00
Joseph Schorr
54095eb5cb
Handle the common case of one chunk when calculating the uncompressed size
...
Reference #992
2015-12-14 15:27:48 -05:00
josephschorr
1323da20e3
Merge pull request #1050 from coreos-inc/v2betterlogging
...
Make our JWT subjects better and log using the info
2015-12-14 15:24:39 -05:00
Silas Sewell
881fd53714
Merge pull request #1056 from coreos-inc/dont-hide-ioerror
...
Handle IOErrors in v2 uploads
2015-12-14 14:46:46 -05:00
Joseph Schorr
4a4eee5e05
Make our JWT subjects better and log using the info
...
Fixes #1039
2015-12-14 14:00:33 -05:00
Silas Sewell
2dcc1f13a6
Handle IOErrors in v2 uploads
2015-12-14 11:58:24 -05:00
Joseph Schorr
d963f7174a
Change manifest delete to mark tag as dead and log
2015-12-10 15:45:53 -05:00
Jake Moshenko
9f13bb8960
Fix the overlap condition on resumed uploads
2015-12-10 13:14:11 -05:00
josephschorr
37dec895ce
Merge pull request #987 from coreos-inc/multimanifest
...
Make manifest generation safe for multiple callers
2015-12-08 14:12:39 -05:00
josephschorr
6c897a7c22
Merge pull request #1035 from coreos-inc/closeforlong
...
Close for long operation before we upload chunks
2015-12-07 14:22:29 -05:00
Joseph Schorr
bcd7f45905
Close for long operation before we upload chunks
2015-12-07 14:11:56 -05:00
Joseph Schorr
b8bd92a64f
Handle 404s on blob uploads
...
Fixes #1033
2015-12-07 12:30:28 -05:00
Joseph Schorr
ee0eb80c8f
Fix blob content types
...
Fixes #990
2015-12-04 16:13:58 -05:00
josephschorr
32fae5533c
Merge pull request #1022 from coreos-inc/manifestexc
...
Only write exceptions for manifest gen when a tag exists
2015-12-04 14:32:06 -05:00
Joseph Schorr
c324ebd7f6
Only write exceptions for manifest gen when a tag exists
...
Fixes #1019
Currently, we just raise an exception to the logs regardless, which can make it appear as if there is an issue (when there isn't).
2015-12-03 16:04:17 -05:00
Silas Sewell
8781cf6e11
Increase nginx proxy timeout and close db before storage operation
2015-12-03 11:19:39 -05:00
Silas Sewell
664a2951cc
Don't send content-length when redirecting v2 blob
...
Fixes #1012
2015-12-02 21:28:11 -05:00
Joseph Schorr
6ed705be15
Make manifest generation safe for multiple callers
...
Fixes #985
2015-11-24 18:38:29 -05:00
josephschorr
0dbd19a236
Merge pull request #976 from coreos-inc/incidentaltests
...
Add login tests and fix scope security issue
2015-11-24 13:42:06 -05:00
Joseph Schorr
75a91f0f92
Add login tests and fix scope security issue
2015-11-24 13:39:16 -05:00
Joseph Schorr
a0e597f957
Send Docker-Content-Digest headers on GET requests
...
Fixes #955
2015-11-23 13:56:34 -05:00
Joseph Schorr
5c8eea2728
Log when pulls occur in V2
...
Fixes #958
2015-11-20 21:30:03 -05:00
Jake Moshenko
0c44949017
Return a 401 when doing a login with bad credentials
2015-11-20 18:37:52 -05:00
Jake Moshenko
b564492ea7
Improve the performance of fetching manifest blobs by checksum.
2015-11-19 11:01:47 -05:00
Jake Moshenko
39d799b1aa
Fix anonymous repository pulls
2015-11-18 20:11:06 -05:00
Jake Moshenko
c27f91f7cf
Fix token pushes for v2 auth, tokens have no user
2015-11-18 19:18:12 -05:00
Jake Moshenko
4cc619f4ca
Clean up v2 branch to no longer warn about readiness
2015-11-16 14:42:43 -05:00
Jake Moshenko
52125bbfed
Fix gc by using the v1/v2 storage location helper everywhere
2015-11-16 14:13:37 -05:00
Jake Moshenko
44d06b0c2e
Fix v1 backward compatibility
2015-11-12 16:22:19 -05:00
Jake Moshenko
c2fcf8bead
Merge remote-tracking branch 'upstream/phase4-11-07-2015' into python-registry-v2
2015-11-06 18:18:29 -05:00
Jake Moshenko
1666ac50fe
Filter down the signing key to only public portion
2015-10-26 16:40:19 -04:00
Jake Moshenko
fc55730db8
Add a feature flag to advertise v2 endpoints
2015-10-26 14:20:51 -04:00
Jake Moshenko
e7a6176594
Merge remote-tracking branch 'upstream/v2-phase4' into python-registry-v2
2015-10-22 16:59:28 -04:00
Matt Jibson
b4554f4d14
Verify signed manifests
...
fixes #394
2015-10-20 02:08:45 -04:00
Joseph Schorr
f393236c9f
Add repo name check to V2
...
Fixes #592
2015-10-05 14:19:52 -04:00
Joseph Schorr
d0dc8fe45d
Add endpoint security tests for the V2 endpoints
...
Fixes #581
2015-10-02 14:01:12 -04:00
Joseph Schorr
35c35d9913
Load images and storage references in bulk during V1 synthesize
...
Currently, we perform multiple queries for each layer, making it much slower (especially cross-region)
Fixes #413
2015-09-29 17:53:39 -04:00
Joseph Schorr
f44ca79391
Add _catalog endpoint as specified by V2 API
...
Fixes #391
2015-09-29 16:02:19 -04:00
Joseph Schorr
decdaa4c79
New tests and small fixes while comparing against the V2 spec
...
Fixes #391
2015-09-29 15:18:48 -04:00
Joseph Schorr
2e694dd3f0
Move Docker V2 key to be loaded from file or generated on server load
...
Fixes #394
2015-09-28 15:43:51 -04:00
Joseph Schorr
09f8ad695b
Fix resumable upload support and add another test
2015-09-28 12:17:17 -04:00
Joseph Schorr
4dc30d6321
Remove yaml and switch to JSON because yaml is so slow
2015-09-24 16:17:42 -04:00
Jake Moshenko
9c3ddf846f
Some fixes and tests for v2 auth
...
Fixes #395
2015-09-10 15:38:57 -04:00
Jake Moshenko
210ed7cf02
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-09-04 16:32:01 -04:00
Jake Moshenko
8269d4ac90
Checkpoint implementing PATCH according to Docker
2015-09-03 16:26:02 -04:00
Joseph Schorr
42dba8655c
Fix auth and add V2 tests!
2015-09-03 12:21:21 -04:00
Jake Moshenko
398202e6fc
Implement some new methods on the storage engines.
2015-08-27 11:29:19 -04:00
josephschorr
4fa37a46d1
Merge pull request #408 from coreos-inc/python-registry-v2-notfun
...
Change to return a proper manifest URL
2015-08-26 16:17:25 -04:00
josephschorr
9d1296da38
Merge pull request #407 from coreos-inc/python-registry-v2-pushevent
...
Spawn the repo_push event in V2
2015-08-26 16:13:02 -04:00
Joseph Schorr
9842857b89
Change to return a proper manifest URL
2015-08-26 14:14:59 -04:00
Joseph Schorr
aa90caa97e
Spawn the repo_push event in V2
...
Fixes #403
2015-08-25 16:02:21 -04:00
Joseph Schorr
84458811d5
Rename wrap_with_hash to a more generic wrap_with_handler
2015-08-25 15:53:13 -04:00
Joseph Schorr
31fdb94436
Enable rate limiting of V2 requests
2015-08-25 14:18:34 -04:00
Joseph Schorr
1450b7e84c
Fix verbs support in V2
2015-08-24 12:05:09 -04:00
Joseph Schorr
cf030e2a98
Save the compressed image size on blob upload completion
2015-08-24 12:05:09 -04:00
Joseph Schorr
d246e68e68
Move shared V1/V2 code into common methods and fix verbs
2015-08-24 12:05:09 -04:00
Joseph Schorr
dead054b43
Fix image API for V2 images
2015-08-19 13:35:47 -04:00
Joseph Schorr
bc4e07343e
Fix PATCH semantics due to recent API changes
2015-08-13 15:35:37 -04:00
Joseph Schorr
f50ea3329a
Fix URL scheme on the authenticate header
2015-08-13 14:44:42 -04:00
Jake Moshenko
e1b3e9e6ae
Another huge batch of registry v2 changes
...
Add patch support and resumeable sha
Implement all actual registry methods
Add a simple database generation option
2015-08-12 16:41:12 -04:00
Matt Jibson
b483209862
Wrap API and registry requests with common metric timings
...
Record response times, codes, and rollup non-2XX responses.
2015-08-12 12:16:00 -04:00
Jake Moshenko
f2508fb48a
Have the auth realm url computed based on the request url.
2015-07-17 11:56:15 -04:00
Jake Moshenko
bc29561f8f
Fix and templatize the logic for external JWT AuthN and registry v2 Auth.
...
Make it explicit that the registry-v2 stuff is not ready for prime time.
2015-07-17 11:56:15 -04:00
Jake Moshenko
3efaa255e8
Accidental refactor, split out legacy.py into separate sumodules and update all call sites.
2015-07-17 11:56:15 -04:00
Jake Moshenko
bea8b9ac53
More changes for registry-v2 in python.
...
Implement the minimal changes to the local filesystem storage driver and feed them through the distributed storage driver.
Create a digest package which contains digest_tools and checksums.
Fix the tests to use the new v1 endpoint locations.
Fix repository.delete_instance to properly filter the generated queries to avoid most subquery deletes, but still generate them when not explicitly filtered.
2015-07-17 11:50:41 -04:00
Jake Moshenko
acbcc2e206
Start of a v2 API.
2015-07-17 11:50:41 -04:00
