josephschorr
928b9915ed
Merge pull request #2441 from coreos-inc/repo-score-denormalization
...
Add a RepositorySearchScore table and calculation to the RAC worker
2017-04-10 16:31:09 -04:00
Joseph Schorr
df3f47c79a
Add a RepositorySearchScore table and calculation to the RAC worker
...
This will be used in a followup PR to order search results instead of the RAC join. Currently, the join with the RAC table in search results in a lookup of ~600K rows, which causes searching to take ~6s. This PR denormalizes the data we need, as well as allowing us to score based on a wider band (6 months vs the current 1 week).
2017-04-10 14:29:02 -04:00
Erica
3f79422a52
Merge pull request #2306 from coreos-inc/QUAY-2842-audit-log-strict-config-option
...
feat(config.py): add setting for audit log strictness
2017-04-07 13:43:11 -04:00
EvB
20c4d971c4
refactor(model/log): pull allowed action types into constant
2017-04-07 11:39:54 -04:00
Jake Moshenko
a8ec7865a7
Merge pull request #2511 from jakedt/fixwarnings
...
Fixwarnings
2017-04-06 16:12:19 -04:00
Jake Moshenko
c7241911a5
Fix old-style flask imports to silence deprecation warnings.
2017-04-06 13:15:48 -04:00
Joseph Schorr
f9e6110f73
Add basic user interface for application repos
...
Adds support for creating app repos, viewing app repos and seeing the list of app repos in the Quay UI.
2017-04-05 11:30:09 -04:00
EvB
d0aaaaa1ef
test(data/model/log): add more log_action tests
...
temp
2017-04-05 11:26:10 -04:00
EvB
ea740d15ba
fix(model/log): log exception on swallowed db errors
2017-04-05 11:26:10 -04:00
EvB
625bd66b42
test(data/model): test action logging
2017-04-05 11:26:10 -04:00
EvB
6916d82e0d
feat(endpoints/trackhelper): wrap log op for silent fails
2017-04-05 11:26:10 -04:00
Joseph Schorr
b10608e277
Change teamsync config to be a UTF8 field
2017-04-03 14:16:34 -04:00
Joseph Schorr
bdd07d4f39
Fix flakiness in team sync tests
2017-04-03 11:36:42 -04:00
Joseph Schorr
bd22fb255e
Rename get_federated_user to get_and_link_federated_user_info
...
Better to be explicit wherever possible
2017-04-03 11:36:42 -04:00
Joseph Schorr
1a31d98c44
Clarify variable name in Keystone auth
2017-04-03 11:36:41 -04:00
Joseph Schorr
8c07f733eb
Add pagination tests for LDAP
2017-04-03 11:36:41 -04:00
Joseph Schorr
541aa722c2
Add sleeps to make test non-flaky
...
Sucks, but MySQL only has second-level timing, so we need this to be sure
2017-04-03 11:36:41 -04:00
Joseph Schorr
103186f5e8
Small renames to make team syncing code more clear
2017-04-03 11:36:41 -04:00
Joseph Schorr
7f0aa19292
Code cleanup and style improvements in team sync
2017-04-03 11:36:41 -04:00
Joseph Schorr
84e37b68ee
Change if statement to be more readable
2017-04-03 11:31:30 -04:00
Joseph Schorr
71d52d45ba
Add a test for same user returned twice in team sync
2017-04-03 11:31:30 -04:00
Joseph Schorr
d7825c6720
Add group iteration and syncing support to Keystone auth
2017-04-03 11:31:30 -04:00
Joseph Schorr
47278cc559
Cleanup test fixtures
2017-04-03 11:31:30 -04:00
Joseph Schorr
df603462b8
Add database migration for TeamSync
2017-04-03 11:31:29 -04:00
Joseph Schorr
96b9d6b0cd
Add end-to-end test for team sync
2017-04-03 11:31:29 -04:00
Joseph Schorr
4055158fc4
Fix indentation
2017-04-03 11:31:29 -04:00
Joseph Schorr
938730c076
Move sync team into its own module and add tests
2017-04-03 11:31:29 -04:00
Joseph Schorr
eeadeb9383
Initial interfaces and support for team syncing worker
2017-04-03 11:31:29 -04:00
Joseph Schorr
94b07e6de9
Allow nulls in last_updated field to accurately report the last updated time to users for newly sync teams
2017-04-03 11:31:29 -04:00
Joseph Schorr
8ea3977140
Add ability to enable, disable and view team syncing in UI and API
...
Also extracts out some common testing infrastructure to make testing APIs easier now using pytest
2017-04-03 11:31:29 -04:00
Joseph Schorr
bb20422260
Fix pagination disabling in LDAP with mockldap
...
Since mockldap doesn't support pagination, just disable it globally
2017-04-03 11:31:28 -04:00
Joseph Schorr
ecfac81721
Add check_group_lookup_args and service_metadata to auth providers
2017-04-03 11:31:28 -04:00
Joseph Schorr
1cfc4a8341
Change max size of LDAP pages and add filtering to reduce attributes returned
2017-04-03 11:31:28 -04:00
Joseph Schorr
f5a854c189
Add TeamSync database and API support
...
Teams can now have a TeamSync entry in the database, indicating how they are synced via an external group. If found, then the user membership of the team cannot be changed via the API.
2017-04-03 11:31:28 -04:00
Joseph Schorr
d718829f5d
Initial LDAP group member iteration support
...
Add interface for group member iteration on internal auth providers and implement support in the LDAP interface.
2017-04-03 11:31:28 -04:00
Charlton Austin
9ff189b16e
fix(migration merge issue): missing .save() on migration
2017-03-28 15:17:51 -04:00
Charlton Austin
d559dc7b3e
Fixing the migration path so we don't have incorrect branches.
2017-03-28 14:54:21 -04:00
Charlton Austin
ca99535774
Merge pull request #2449 from charltonaustin/phase_two_config
...
feat(build runner): added in context, dockerfile_location
2017-03-28 14:14:36 -04:00
Charlton Austin
e6d201e0b0
feat(build runner): added in context, dockerfile_location
...
this is a new feature meant to allow people to use any file as
a dockerfile and any folder as a context directory
2017-03-28 13:55:31 -04:00
Antoine Legrand
d2ed37e158
Fix force push causing duplicated entries
2017-03-27 15:39:57 +02:00
Antoine Legrand
22c1a29892
fix strip_sha256
2017-03-24 19:49:52 +01:00
Antoine Legrand
bbd74eabd1
Allow force push for app
2017-03-23 22:50:07 +01:00
Joseph Schorr
ac4a79ae01
Update PR for rebase
2017-03-23 15:57:49 -04:00
Joseph Schorr
651666b60b
Refactor our auth handling code to be cleaner
...
Breaks out the validation code from the auth context modification calls, makes decorators easier to define and adds testing for each individual piece. Will be the basis of better error messaging in the following change.
2017-03-23 15:42:45 -04:00
Joseph Schorr
dd9e4bf3e7
Remove transaction around OCI blobs
...
Fixes https://www.pivotaltracker.com/story/show/142341399
2017-03-23 14:51:37 -04:00
Jimmy Zelinskie
9c0cbbf57c
data.oci_model: sloppily rewrite digest format
...
We expect digests to be in the form 'sha256:digest'
2017-03-23 12:37:32 -04:00
Joseph Schorr
7d66f30d52
Fix filtering of repositories in search
2017-03-23 11:35:17 -04:00
Joseph Schorr
917d5e2550
Fix typos in data model
2017-03-23 11:14:08 -04:00
Joseph Schorr
05ce571e3e
Add missing return statement
2017-03-23 11:11:21 -04:00
Jimmy Zelinskie
d20ff785e6
data.model.repository: add back search fields
2017-03-23 10:46:04 -04:00
Jimmy Zelinskie
2bdd3d4fa1
data.oci_model.tag: add missing import
2017-03-23 00:58:57 -04:00
Jimmy Zelinskie
e872c310d0
data.oci_model: fix imports
2017-03-23 00:21:21 -04:00
Joseph Schorr
bdda74d6df
Make sure GC checks new Blob table as well before deleting CAS storage
2017-03-22 23:53:21 -04:00
Jimmy Zelinskie
ddad957a56
data.model.repository: add app methods
2017-03-22 21:51:55 -04:00
Jimmy Zelinskie
650723430b
data.interfaces.appr: init
2017-03-22 21:51:41 -04:00
Jimmy Zelinskie
9f684fa73f
data.oci_model: init with app methods
2017-03-22 21:51:28 -04:00
Jimmy Zelinskie
3ccf3c5f33
Merge pull request #2447 from jzelinskie/cnr-step2
...
CNR Step 2
2017-03-22 18:45:51 -04:00
Joseph Schorr
df1e7f90e0
Add verb security tests and fix small issues
2017-03-22 18:29:53 -04:00
Jimmy Zelinskie
d5fa2ad0c0
endpoints.verbs: abort 405 for non-container repos
2017-03-22 17:50:58 -04:00
Jimmy Zelinskie
40b638a981
data.migrations: rebase to HEAD of migration tree
2017-03-22 17:26:59 -04:00
Jimmy Zelinskie
48ba59d615
endpoints.v2: only work on docker repositories
2017-03-22 17:26:59 -04:00
Jimmy Zelinskie
72751592a3
data.interfaces.v1: document types
2017-03-22 15:51:19 -04:00
Jimmy Zelinskie
45f14f220d
data.model.repository: optimize by using kind_id
2017-03-22 15:51:19 -04:00
Joseph Schorr
30b532254c
Disallow non-apps-supported APIs for application repositories
2017-03-22 15:51:19 -04:00
Joseph Schorr
c3402fff5a
Add test to ensure we cannot create repos with the same name but different kinds
2017-03-22 14:34:32 -04:00
Jimmy Zelinskie
a2bac7dabd
endpoints.v1: only work on docker repositories
2017-03-22 14:31:22 -04:00
Jimmy Zelinskie
f086eea754
data.interfaces.v1: explicitly use kwargs
...
This fixes the function from accidentally using the wrong arguments
positionally for the new `repo_kind` kwarg.
2017-03-22 13:58:50 -04:00
Jimmy Zelinskie
d2a4c9d05a
data.model.repository: audited for repo_kind usage
2017-03-22 13:58:50 -04:00
Jimmy Zelinskie
074c1bc4a8
data.model._basequery: audited for repo_kind usage
2017-03-22 13:58:49 -04:00
Jimmy Zelinskie
f842bc3a82
data.migrations.migration.sh: wait 25s for mysql
...
Without this, there are frequent race conditions wheres the client fails
to connect to the server when using Docker For Mac.
2017-03-21 15:38:39 -04:00
Jimmy Zelinskie
4492d2f210
data.migrations: add repository kind
2017-03-21 15:38:38 -04:00
Jimmy Zelinskie
5b362da8ac
data.database: add RepositoryKind
2017-03-21 15:38:38 -04:00
Joseph Schorr
ff7f78e990
Have blob uploads be checked against configurable max layer size
2017-03-21 13:16:55 -04:00
Joseph Schorr
76de324ca8
Change blob upload ints into bigints
2017-03-21 13:14:11 -04:00
Charlton Austin
3502d9f61c
Merge pull request #2438 from charltonaustin/phase_one_config
...
refactor(data): add in new config for builder
2017-03-21 10:16:51 -04:00
Jimmy Zelinskie
6a538647e4
data.database: beta classes skip transitive delete
2017-03-20 18:41:39 -04:00
josephschorr
27aa12de7a
Merge pull request #2439 from coreos-inc/remove_redis_log_expiration
...
Switch from expire to delete redis log_entries
2017-03-20 13:41:57 -04:00
Charlton Austin
f701677a8e
refactor(data): add in new config for builder
...
we are doing phase one of the four phase migration on the builder config
2017-03-20 13:03:41 -04:00
Jimmy Zelinskie
0ea600628b
Merge pull request #2436 from jzelinskie/cnr-step1
...
CNR - Step 1
2017-03-17 15:37:29 -04:00
Jimmy Zelinskie
ad029fb331
data.migrations: don't use UTF-8 for unique fields
...
Unique indexes must have less than 767 bytes and UTF-8 encoding with 255
chars is beyond this maximum. Since this is an internal identifier, we
can be confident that we will not require UTF-8 for it in the future.
2017-03-17 15:21:24 -04:00
Jimmy Zelinskie
c915a40531
data.database: rm tag_kind from Tag indexes
...
These shouldn't be necessary.
2017-03-17 11:35:16 -04:00
Jimmy Zelinskie
0e32e77e99
data.database: document all CNR/OCI models
2017-03-17 11:35:16 -04:00
Jimmy Zelinskie
2a117f2d24
data.migrations: change CNR mimetypes to v0
...
Our initial CNR support is of a pre-v1 implementation of the
specification.
2017-03-17 11:33:16 -04:00
Jimmy Zelinskie
1e9ce85af6
data.database/migrations: remove repo_id from db
...
This also manually organizes and removes broken parts of the migration.
2017-03-17 11:33:16 -04:00
Antoine Legrand
8f323154ce
data.migrations: add OCI/CNR models
2017-03-17 11:33:16 -04:00
Antoine Legrand
c61024586d
data.database: add CNR/OCI models
2017-03-17 11:33:16 -04:00
Antoine Legrand
718aeeead8
Fix search group_by clause for PG
2017-03-17 16:30:24 +01:00
Antoine Legrand
ec847ce613
Switch from expire to delete redis log_entries
2017-03-17 15:35:47 +01:00
Joseph Schorr
e25c989fef
Add a cleanup worker for blob uploads
2017-03-16 13:36:59 -04:00
Joseph Schorr
e90cab4d77
Change revert tag into restore tag and add manifest support
2017-03-14 11:34:42 -04:00
Joseph Schorr
af743b156b
Show manifest digests in place of V1 ids in the tag view when possible
2017-03-14 11:34:41 -04:00
Jimmy Zelinskie
123d003d4e
Merge pull request #2424 from jzelinskie/qss-image
...
workers.securityworker: revert to image querying
2017-03-10 17:38:02 -05:00
Jimmy Zelinskie
a780136337
workers.securityworker: revert to image querying
2017-03-10 17:37:40 -05:00
josephschorr
cbac673d58
Merge pull request #2404 from coreos-inc/cas-gc-fix
...
Fix GC handling around CAS paths
2017-03-10 17:34:21 -05:00
josephschorr
432b2d3fe8
Merge pull request #2392 from coreos-inc/search-optimization
...
Optimize repository search by changing our lookup strategy
2017-03-10 15:44:26 -05:00
Jimmy Zelinskie
53eb579459
data.model.tag: find min *alive* tag
2017-03-10 13:15:35 -05:00
Joseph Schorr
b5bb76cdea
Optimize repository search by changing our lookup strategy
...
Previous to this change, repositories were looked up unfiltered in six different queries, and then filtered using the permissions model, which issued a query per repository found, making search incredibly slow. Instead, we now lookup a chunk of repositories unfiltered and then filter them via a single query to the database. By layering the filtering on top of the lookup, each as queries, we can minimize the number of queries necessary, without (at the same time) using a super expensive join.
Other changes:
- Remove the 5 page pre-lookup on V1 search and simply return that there is one more page available, until there isn't. While technically not correct, it is much more efficient, and no one should be using pagination with V1 search anyway.
- Remove the lookup for repos without entries in the RAC table. Instead, we now add a new RAC entry when the repository is created for *the day before*, with count 0, so that it is immediately searchable
- Remove lookup of results with a matching namespace; these aren't very relevant anyway, and it overly complicates sorting
2017-03-09 19:47:55 -05:00
Joseph Schorr
62312e6461
Add warning when CAS paths are skipped and ensure we are under a transaction
2017-03-08 17:01:07 -05:00
Joseph Schorr
69e550d125
Fix GC handling around CAS paths
...
Adds code to ensure we never GC CAS paths that are shared amongst multiple ImageStorage rows, as well as an associated pair of tests to catch the positive and negative cases.
2017-03-07 13:48:07 -05:00
Jimmy Zelinskie
40636d4103
find work based on tag IDs rather than image IDs
2017-03-06 17:09:57 -05:00
