vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
3985 commits 2 branches 62 tags 205 MiB
Commit graph

72 commits

Author SHA1 Message Date
Jimmy Zelinskie
9ddad4a1a9 client-side join chunks for GCS 
Boto does not implement GCS's custom multipart API and so we're left to
join them client-side until it does.
 2015-10-02 14:57:39 -04:00
Jimmy Zelinskie
6ed5087a3c add client side chunk join method 2015-10-01 12:28:56 -04:00
Jimmy Zelinskie
abe43a0e07 override upload_chunk_complete for RadosGW 
RadosGW doesn't support server-side copy of keys into multipart, so we
have to always join it on the local side.
 2015-09-30 17:46:59 -04:00
Joseph Schorr
a3ebb9028d Add full unit tests for the file-like objects and fix them 
Fixes #568
 2015-09-30 14:19:25 -04:00
josephschorr
41bfe2ffde Merge pull request #551 from coreos-inc/python-registry-v2-swift 
Add V2 storage methods to Swift storage engine
 2015-09-28 17:09:34 -04:00
Joseph Schorr
6c59161527 Add V2 storage methods to Swift storage engine 
Fixes #508
 2015-09-28 16:46:19 -04:00
Jimmy Zelinskie
c5aa3ca4f0 make registry v2 tests pass for GCS 
Fixes #509.
 2015-09-28 15:42:48 -04:00
Jake Moshenko
26cea9a07c Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-17 16:16:27 -04:00
Joseph Schorr
6f2271d0ae Add support for direct download in Swift storage engine 
Fixes #483
 2015-09-14 18:00:03 -04:00
Joseph Schorr
cccb1651f5 Fixes for direct cloud storage copying 2015-09-08 16:55:47 -04:00
Jake Moshenko
210ed7cf02 Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-04 16:32:01 -04:00
Jake Moshenko
8269d4ac90 Checkpoint implementing PATCH according to Docker 2015-09-03 16:26:02 -04:00
Joseph Schorr
42dba8655c Fix auth and add V2 tests! 2015-09-03 12:21:21 -04:00
Joseph Schorr
587ef85c7f Allow users to choose the version of Swift to use 
Fixes #442
 2015-09-02 17:46:14 -04:00
Matt Jibson
a821ad2b01 Return an error on failed S3 uploads 
The previous change to this file didn't raise the error up to stream_write,
and so the complete_upload function still ran because the loop was only
broken. It errored because the data was already canceled. This is better
than what we had before, which was to silently fail but report success
(even internally to ourselves!) on bad image upload.

This means we discovered a bug where a user could have failed during image
upload, but quay would write that image to the repository, potentially
writing broken images to S3.
 2015-09-01 15:53:32 -04:00
Joseph Schorr
724b1607d7 Add automatic storage replication 
Adds a worker to automatically replicate data between storages and update the database accordingly
 2015-09-01 14:53:32 -04:00
Matt Jibson
ab25542bd7 Measure multipart uploads 
see #304
 2015-08-31 13:48:52 -04:00
Matt Jibson
9aedfc8d2c Cancel failed multipart uploads 2015-08-31 03:07:44 -04:00
Jake Moshenko
398202e6fc Implement some new methods on the storage engines. 2015-08-27 11:29:19 -04:00
Joseph Schorr
c07dec4d39 File reader fixes for verbs 
- Fix local file reader to always read in chunks
- Have gzip stream raise an exception if the full data is requested
 2015-08-25 13:49:21 -04:00
Jake Moshenko
e1b3e9e6ae Another huge batch of registry v2 changes 
Add patch support and resumeable sha
Implement all actual registry methods
Add a simple database generation option
 2015-08-12 16:41:12 -04:00
Joseph Schorr
5ea028c8be Loosen the check for mounted volumes 
Docker apparently does not always use the same mounted volume type

Fixes #352
 2015-08-12 13:16:58 -04:00
Joseph Schorr
9798f871ec Always create a new connection to Swift 
Fixes #335
 2015-08-07 13:41:40 -04:00
Jake Moshenko
18100be481 Refactor the util directory to use subpackages. 2015-08-03 16:04:19 -04:00
Joseph Schorr
26ae629189 Prevent local storage setup on non-mounted paths 
Fixes #269
 2015-07-27 14:32:02 -04:00
Jake Moshenko
3efaa255e8 Accidental refactor, split out legacy.py into separate sumodules and update all call sites. 2015-07-17 11:56:15 -04:00
Jake Moshenko
bea8b9ac53 More changes for registry-v2 in python. 
Implement the minimal changes to the local filesystem storage driver and feed them through the distributed storage driver.
Create a digest package which contains digest_tools and checksums.
Fix the tests to use the new v1 endpoint locations.
Fix repository.delete_instance to properly filter the generated queries to avoid most subquery deletes, but still generate them when not explicitly filtered.
 2015-07-17 11:50:41 -04:00
Joseph Schorr
4333bb9e14 Implement stream_read_file for the Swift storage engine 
Note that Swift doesn't seem to have a file-like interface, so we need to wrap the generator we get back from it.

Fixes #210
 2015-07-02 17:52:43 +03:00
Joseph Schorr
9585e2a765 End-to-end registry tests 2015-06-01 16:35:30 -04:00
Joseph Schorr
375d7670a8 Explain why we re-raise ClientException in the swift storage engine 2015-05-26 16:35:12 -04:00
Joseph Schorr
7001fb05bf Add further comments on the TODO in get_direct_download_url 2015-05-26 16:34:59 -04:00
Joseph Schorr
5845e37e32 Add Swift storage library 2015-05-26 16:05:21 -04:00
Joseph Schorr
53e5fc6265 Have the config setup tool automatically prepare the S3 or GCS storage with CORS config 2015-01-16 16:10:40 -05:00
Joseph Schorr
63504c87fb Get end-to-end configuration setup working, including verification (except for Github, which is in progress) 2015-01-07 16:20:51 -05:00
Jimmy Zelinskie
716d7a737b Strip whitespace from ALL the things. 2014-11-24 16:07:38 -05:00
Joseph Schorr
43555af63d Address review comments 2014-09-17 17:49:46 -04:00
Joseph Schorr
e3c52fa0eb Work in progress. This is currently broken! 2014-09-16 00:18:57 -04:00
Joseph Schorr
e8ad01cb41 Lots of small NPE and other exception fixes 2014-09-15 11:27:33 -04:00
Jake Moshenko
8b3a3178b0 Finish the build logs archiver, add handlers for cloud and local that handle gzip encoded archived content. 2014-09-11 15:33:10 -04:00
Jake Moshenko
548f855f71 Use the pure python io module to avoid some interaction between gunicorn, wsgi, and bufferedreader that prevents gunicorn from properly sending the files. 2014-09-09 22:28:25 -04:00
Jake Moshenko
c9e1648781 Small fixes to bugs in the streaming handler for use with magic and radosgw. 2014-09-09 18:30:14 -04:00
Jake Moshenko
756e8ec848 Send the content type through to the cloud engines. 2014-09-09 16:52:53 -04:00
Jake Moshenko
29d40db5ea Add a new RadosGW storage engine. Allow engines to distinguish not only between those that can support direct uploads and downloads, but those that support doing it through the browser. Rename resumeable->resumable. 2014-09-09 15:54:03 -04:00
Jake Moshenko
29f1b048a3 Add support for Google Cloud Storage. 2014-08-12 02:06:44 -04:00
Joseph Schorr
bcbea37fce Change distributed config format to make it easier for the setup tool 2014-08-07 13:45:15 -04:00
Joseph Schorr
af46d3d455 Remove path from get_supports_resumeable_download 2014-07-07 16:21:27 -04:00
Joseph Schorr
0ac6312c8a Get resumable downloads working in a way that the Docker CLI will actually understand. Also rename the method to conform the the name used in the Docker source code. 2014-07-02 21:18:46 -04:00
Joseph Schorr
0d89fd7f26 Add code for resumable downloads from S3 2014-07-02 00:39:59 -04:00
Jake Moshenko
bf0e01fba5 Fix bugs with pushing, pulling, and deleting. 2014-06-17 16:37:48 -04:00
Jake Moshenko
bf98575feb Add the basics of geographic data distribution and get the tests to work. 2014-06-17 16:03:43 -04:00