Jake Moshenko
668a8edc50
Refactor prometheus integration
...
Move prometheus to SaaS and make it a plugin
Move static callers to use metrics_queue plugin
Change local-docker to support different quay clone dirnames
Change prom_aggregator to use logrus
2016-07-01 14:16:50 -04:00
Matt Jibson
3d9acf2fff
Use prometheus as a metric backend
...
This entails writing a metric aggregation program since each worker has its
own memory, and thus own metrics because of python gunicorn. The python
client is a simple wrapper that makes web requests to it.
2016-07-01 14:16:50 -04:00
Joseph Schorr
343a080833
Make security scan testing much faster
2016-05-05 13:55:24 -04:00
Joseph Schorr
eab6af2b87
Add mocked unit tests for cloud storage engine
2016-03-23 12:13:54 -04:00
Joseph Schorr
b440564df1
Fix client side chunk paths
...
Fixes #1306
2016-03-22 17:34:50 -04:00
Jimmy Zelinskie
2b07b6d8a9
allow HEAD on ACI images
...
Fixes #911 .
2016-02-12 16:28:44 -05:00
Joseph Schorr
68c9d5e432
Add torrent tests
...
Fixes #1128
2016-01-19 17:40:11 -05:00
josephschorr
6fc7d2a230
Merge pull request #1132 from coreos-inc/cleanup
...
Remove unused storage path
2016-01-19 12:08:30 -05:00
Jake Moshenko
909e7d45b7
Add a test for swift path computation
2016-01-15 15:35:04 -05:00
Jake Moshenko
c6d7eba98d
Fix paths in swift driver to be more intelligible
2016-01-15 15:35:04 -05:00
Jake Moshenko
0b1951a4a4
Remove list directory from storage driver
2016-01-15 15:35:04 -05:00
Joseph Schorr
622d2ebbb1
Remove unused storage path
...
Fixes #1131
2016-01-11 13:43:22 -05:00
Joseph Schorr
ab166c4448
Delete the image diff feature
...
Fixes #1077
2015-12-23 13:08:01 -05:00
Silas Sewell
2dcc1f13a6
Handle IOErrors in v2 uploads
2015-12-14 11:58:24 -05:00
Jake Moshenko
e00365b225
Assign the exception to a variable to log it
2015-12-08 18:03:48 -05:00
Silas Sewell
76fd744453
Log stream_write_to_fp ioerrors
2015-12-07 16:26:48 -05:00
Joseph Schorr
ee0eb80c8f
Fix blob content types
...
Fixes #990
2015-12-04 16:13:58 -05:00
Joseph Schorr
f38e7f5b25
Make it explicit that the hostname is a hostname, and not a URL
2015-12-04 15:40:33 -05:00
Joseph Schorr
e9b577104d
Add squash testing code to registry tests
...
Fixes #896
2015-11-20 15:16:11 -05:00
Matt Jibson
26f1d77a69
Merge pull request #889 from mjibson/s3-sigv4-host
...
Allow setting of boto's S3 host for SIGv4
2015-11-18 17:36:15 -05:00
Matt Jibson
b3c2388618
Allow setting of boto's S3 host for SIGv4
...
The problem only happens when a user has configured the new AWS Frankfurt
region for their S3 backend. It is the only region to require the new
v4 signature. All other regions support both v2 and v4. I'm not sure
which version is used by default on US Standard.
We could attempt to figure out where the bucket is hosted based on its
DNS resolution and auto-populate the host field that way. But I think
the amount of effort to have that work correctly outweighs its benefit
for such a simple solution.
fixes #863
fixes #764
2015-11-18 17:19:33 -05:00
Jake Moshenko
52125bbfed
Fix gc by using the v1/v2 storage location helper everywhere
2015-11-16 14:13:37 -05:00
Jake Moshenko
ab340e20ea
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-11-11 16:41:40 -05:00
Silas Sewell
5000b1621c
superuser: add storage replication config
2015-11-09 17:34:22 -05:00
Jake Moshenko
9da64f3aba
Stop writing to deprecated columns for image data.
2015-10-24 14:45:15 -04:00
Jake Moshenko
e7a6176594
Merge remote-tracking branch 'upstream/v2-phase4' into python-registry-v2
2015-10-22 16:59:28 -04:00
Jake Moshenko
ce94931540
Stop writing to deprecated columns for image data.
2015-10-22 12:14:39 -04:00
Jimmy Zelinskie
9ddad4a1a9
client-side join chunks for GCS
...
Boto does not implement GCS's custom multipart API and so we're left to
join them client-side until it does.
2015-10-02 14:57:39 -04:00
Jimmy Zelinskie
6ed5087a3c
add client side chunk join method
2015-10-01 12:28:56 -04:00
Jimmy Zelinskie
abe43a0e07
override upload_chunk_complete for RadosGW
...
RadosGW doesn't support server-side copy of keys into multipart, so we
have to always join it on the local side.
2015-09-30 17:46:59 -04:00
Joseph Schorr
a3ebb9028d
Add full unit tests for the file-like objects and fix them
...
Fixes #568
2015-09-30 14:19:25 -04:00
josephschorr
41bfe2ffde
Merge pull request #551 from coreos-inc/python-registry-v2-swift
...
Add V2 storage methods to Swift storage engine
2015-09-28 17:09:34 -04:00
Joseph Schorr
6c59161527
Add V2 storage methods to Swift storage engine
...
Fixes #508
2015-09-28 16:46:19 -04:00
Jimmy Zelinskie
c5aa3ca4f0
make registry v2 tests pass for GCS
...
Fixes #509 .
2015-09-28 15:42:48 -04:00
Jake Moshenko
26cea9a07c
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-09-17 16:16:27 -04:00
Joseph Schorr
6f2271d0ae
Add support for direct download in Swift storage engine
...
Fixes #483
2015-09-14 18:00:03 -04:00
Joseph Schorr
cccb1651f5
Fixes for direct cloud storage copying
2015-09-08 16:55:47 -04:00
Jake Moshenko
210ed7cf02
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-09-04 16:32:01 -04:00
Jake Moshenko
8269d4ac90
Checkpoint implementing PATCH according to Docker
2015-09-03 16:26:02 -04:00
Joseph Schorr
42dba8655c
Fix auth and add V2 tests!
2015-09-03 12:21:21 -04:00
Joseph Schorr
587ef85c7f
Allow users to choose the version of Swift to use
...
Fixes #442
2015-09-02 17:46:14 -04:00
Matt Jibson
a821ad2b01
Return an error on failed S3 uploads
...
The previous change to this file didn't raise the error up to stream_write,
and so the complete_upload function still ran because the loop was only
broken. It errored because the data was already canceled. This is better
than what we had before, which was to silently fail but report success
(even internally to ourselves!) on bad image upload.
This means we discovered a bug where a user could have failed during image
upload, but quay would write that image to the repository, potentially
writing broken images to S3.
2015-09-01 15:53:32 -04:00
Joseph Schorr
724b1607d7
Add automatic storage replication
...
Adds a worker to automatically replicate data between storages and update the database accordingly
2015-09-01 14:53:32 -04:00
Matt Jibson
ab25542bd7
Measure multipart uploads
...
see #304
2015-08-31 13:48:52 -04:00
Matt Jibson
9aedfc8d2c
Cancel failed multipart uploads
2015-08-31 03:07:44 -04:00
Jake Moshenko
398202e6fc
Implement some new methods on the storage engines.
2015-08-27 11:29:19 -04:00
Joseph Schorr
c07dec4d39
File reader fixes for verbs
...
- Fix local file reader to always read in chunks
- Have gzip stream raise an exception if the full data is requested
2015-08-25 13:49:21 -04:00
Jake Moshenko
e1b3e9e6ae
Another huge batch of registry v2 changes
...
Add patch support and resumeable sha
Implement all actual registry methods
Add a simple database generation option
2015-08-12 16:41:12 -04:00
Joseph Schorr
5ea028c8be
Loosen the check for mounted volumes
...
Docker apparently does not always use the same mounted volume type
Fixes #352
2015-08-12 13:16:58 -04:00
Joseph Schorr
9798f871ec
Always create a new connection to Swift
...
Fixes #335
2015-08-07 13:41:40 -04:00
