josephschorr
432b2d3fe8
Merge pull request #2392 from coreos-inc/search-optimization
...
Optimize repository search by changing our lookup strategy
2017-03-10 15:44:26 -05:00
Joseph Schorr
b5bb76cdea
Optimize repository search by changing our lookup strategy
...
Previous to this change, repositories were looked up unfiltered in six different queries, and then filtered using the permissions model, which issued a query per repository found, making search incredibly slow. Instead, we now lookup a chunk of repositories unfiltered and then filter them via a single query to the database. By layering the filtering on top of the lookup, each as queries, we can minimize the number of queries necessary, without (at the same time) using a super expensive join.
Other changes:
- Remove the 5 page pre-lookup on V1 search and simply return that there is one more page available, until there isn't. While technically not correct, it is much more efficient, and no one should be using pagination with V1 search anyway.
- Remove the lookup for repos without entries in the RAC table. Instead, we now add a new RAC entry when the repository is created for *the day before*, with count 0, so that it is immediately searchable
- Remove lookup of results with a matching namespace; these aren't very relevant anyway, and it overly complicates sorting
2017-03-09 19:47:55 -05:00
Jimmy Zelinskie
850c32ebfb
Merge pull request #2298 from jzelinskie/maintainers
...
MAINTAINERS: init owners to subpkgs
2017-03-09 17:30:38 -05:00
Joseph Schorr
a319c55616
Don't make permissions request in search for public callers
...
They are unnecessary, so we can skip them
2017-02-17 12:22:21 -05:00
Joseph Schorr
92c0b5ac3e
Fix handling of None queries
2017-02-16 15:26:45 -05:00
Joseph Schorr
a0bc0e9488
Implement the full spec for the old Docker V1 registry search API
...
This API is still (apparently) being used by the Docker CLI for `docker search` (why?!) and we therefore have customers expecting this to work the same way as the DockerHub.
2017-02-16 14:45:33 -05:00
Jimmy Zelinskie
64421db0a3
MAINTAINERS: init owners to subpkgs
2017-01-23 17:46:34 -05:00
Brad Ison
df7366eace
Add chunk size metric
2016-12-15 13:20:16 -05:00
Brad Ison
8f59ac1251
Don't record size in chunk upload metrics
2016-12-14 12:16:02 -05:00
Joseph Schorr
4b926ae189
Add new metrics as requested by some customers
...
Note that the `status` field on the pull and push metrics will eventually be set to False for failed pulls and pushes in a followup PR
2016-11-03 15:28:40 -04:00
Joseph Schorr
bab17932ac
Fix namespace lookup in V1 registry search
...
Fixes #2053
2016-10-31 13:24:40 -04:00
Jimmy Zelinskie
a42eb09a3e
util.license: make bp-modification a method
2016-10-17 21:43:45 -04:00
Jimmy Zelinskie
0c5400b7d1
enforce license across registry blueprints
2016-10-17 21:43:45 -04:00
Joseph Schorr
0b7bb6d6c6
Fix issue in V1 registry code with accessing locations under HEAD
...
Fixes #1922
2016-10-03 17:09:12 +03:00
Jimmy Zelinskie
31b77cf232
rename auth.auth to auth.process
...
This fixes some ambiguity around imports.
2016-09-29 15:24:57 -04:00
Jimmy Zelinskie
44eca10c05
update interfaces to use ABC
2016-09-26 14:50:24 -04:00
Joseph Schorr
3c8b87e086
Fix verbs in manifestlist
...
All registry_tests now pass
2016-09-26 14:49:58 -04:00
Jimmy Zelinskie
c06d395f96
create interfaces for v1 and v2 data model
2016-09-26 14:49:23 -04:00
Joseph Schorr
db60df827d
Implement V2 interfaces and remaining V1 interfaces
...
Also adds some tests to registry tests for V1 stuff.
Note: All *registry* tests currently pass, but as verbs are not yet converted, the verb tests in registry_tests.py currently fail.
2016-09-26 14:49:04 -04:00
Jimmy Zelinskie
d67991987b
v1: refactor index
2016-09-26 14:48:42 -04:00
Joseph Schorr
ea18790dfe
Get V1 registry code working with new model methods
2016-09-26 14:47:06 -04:00
Jimmy Zelinskie
8435c254c3
finish v1 registry refactor
2016-09-26 14:47:06 -04:00
Jimmy Zelinskie
c14437e54a
initial v1 refactor to use model methods
2016-09-26 14:47:06 -04:00
Jimmy Zelinskie
4f95a814c0
satisfy the pylint gods
2016-09-26 14:47:06 -04:00
Joseph Schorr
c8a1b8abab
Add prom stats for repository push, pull and verb actions
2016-09-09 15:13:58 -04:00
Joseph Schorr
a1009af61c
Move aggregator into its own repo and add it to the image
2016-07-05 15:39:51 -04:00
Joseph Schorr
12924784ce
Enable storage replication for V2 and add backfill tool
...
Fixes #1501
2016-06-02 14:36:08 -04:00
Jimmy Zelinskie
bb46cc933d
use kwargs for parse_repository_name
2016-03-09 16:20:28 -05:00
Joseph Schorr
accc576a98
Fix V1 push URL to match Docker and fix registry tests
2016-01-29 16:42:15 +02:00
Jimmy Zelinskie
85ae1a2a0a
Merge pull request #1161 from jzelinskie/torrenthmac
...
misc torrent changes
2016-01-22 23:02:44 -05:00
Joseph Schorr
e4ffaff869
Fix Docker Auth and our V2 registry paths to support library (i.e. namespace-less) repositories.
...
This support is placed behind a feature flag.
2016-01-22 15:54:06 -05:00
Jimmy Zelinskie
e54b86c6eb
s/TORRENT/BITTORRENT
2016-01-22 15:52:28 -05:00
Jake Moshenko
476ac8cec9
Add piece hashing to verbs generated image storages
2016-01-06 12:01:15 -05:00
Jake Moshenko
8f80d7064b
Hash v1 uploads for torrent chunks
2016-01-05 14:43:40 -05:00
Joseph Schorr
ab166c4448
Delete the image diff feature
...
Fixes #1077
2015-12-23 13:08:01 -05:00
Joseph Schorr
327258c83e
Add namespace to be returned by docker search
...
Fixes #956
2015-12-17 16:38:06 -05:00
Joseph Schorr
ca7d36bf14
Handle empty scopes and always send the WWW-Authenticate header, as per spec
...
Fixes #1045
2015-12-15 14:59:47 -05:00
josephschorr
1323da20e3
Merge pull request #1050 from coreos-inc/v2betterlogging
...
Make our JWT subjects better and log using the info
2015-12-14 15:24:39 -05:00
Joseph Schorr
4a4eee5e05
Make our JWT subjects better and log using the info
...
Fixes #1039
2015-12-14 14:00:33 -05:00
Joseph Schorr
d963f7174a
Change manifest delete to mark tag as dead and log
2015-12-10 15:45:53 -05:00
Jake Moshenko
0459c3bc54
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-11-16 14:22:54 -05:00
Jake Moshenko
52125bbfed
Fix gc by using the v1/v2 storage location helper everywhere
2015-11-16 14:13:37 -05:00
Joseph Schorr
32a799a067
Remove code that adds images to the image diff queue
2015-11-13 12:42:43 -05:00
Jake Moshenko
7efa6265bf
Merge branch 'newchanges' into python-registry-v2
2015-11-06 18:24:32 -05:00
Jake Moshenko
c2fcf8bead
Merge remote-tracking branch 'upstream/phase4-11-07-2015' into python-registry-v2
2015-11-06 18:18:29 -05:00
Jake Moshenko
4314882fa0
Reverse the order of get_parent_images
2015-11-06 17:47:08 -05:00
Jake Moshenko
ad93425ead
Stop writing to v1 checksum on ImageStorage
2015-11-06 16:40:04 -05:00
Jake Moshenko
75f917f592
Stop reading the v1 checksums from storage
2015-11-06 16:17:12 -05:00
Jake Moshenko
3d0bcbaaeb
Move v1 checksums to image and track v2 separately
2015-11-06 15:17:55 -05:00
Jake Moshenko
9da64f3aba
Stop writing to deprecated columns for image data.
2015-10-24 14:45:15 -04:00
