Commit graph

845 commits

Author SHA1 Message Date
Joseph Schorr
fdcedafe91 Add a test for issuing test notifications 2016-10-04 10:57:32 +03:00
Evan Cordell
42ebb0a6c3 Record metrics in a separate etcd record 2016-10-03 16:11:37 -04:00
Joseph Schorr
f72cb1d2ba Fix tags API pagination and add a test 2016-10-03 22:06:31 +03:00
Joseph Schorr
0b7bb6d6c6 Fix issue in V1 registry code with accessing locations under HEAD
Fixes #1922
2016-10-03 17:09:12 +03:00
Joseph Schorr
95b7b47501 Add a registry tests for numeric tags 2016-10-03 16:06:49 +03:00
Evan Cordell
68c5384473 Fixes prometheus start metric 2016-09-30 13:09:03 -04:00
josephschorr
2d9ce6dbe3 Merge pull request #1906 from coreos-inc/gitlab-bug
Gitlab trigger payload bug fixes
2016-09-30 17:52:49 +02:00
Joseph Schorr
f50bb8a1ce Add missing call to set_phase when a build doesn't start
This change fixes the build manager ephemeral executor to tell the overall build server to call set_phase when a build never starts. Before this change, we'd properly adjust the queue item, but not the repo build row or the logs, which is why users just saw "Preparing Build Node", with no indicating the node failed to start.

Fixes #1904
2016-09-30 14:54:49 +02:00
Joseph Schorr
26e8e241da Fix handling of Gitlab payloads with multiple commits
Gitlab sends multiple commits in the order reversed from Github. As this only broke recently, I suspect that they may have changed the ordering. This change makes the code order-agnostic to hopefully remove the problem going forward.

Fixes #1900
2016-09-30 12:14:32 +02:00
Joseph Schorr
c43173576a Fix Gitlab trigger payload bug when commits is empty
Gitlab will occasionally send trigger payloads with an empty commit list (and a null checkout_ha) for branches that have been deleted. Properly handle that case.
2016-09-30 12:03:08 +02:00
josephschorr
684ace3b5a Merge pull request #1761 from coreos-inc/nginx-direct-download
Add feature flag to force all direct download URLs to be proxied
2016-09-29 22:46:57 +02:00
Evan Cordell
832ee89923 Add duration metric collector decorator (#1885)
Track time-to-start for builders
Track time-to-build for builders
Track ec2 builder fallbacks
Track build time
2016-09-29 15:44:06 -04:00
Jimmy Zelinskie
31b77cf232 rename auth.auth to auth.process
This fixes some ambiguity around imports.
2016-09-29 15:24:57 -04:00
Joseph Schorr
6ae3faf7fc Add explicit config parameter to the JWT auth methods 2016-09-29 11:15:20 +02:00
Joseph Schorr
dd2e086a20 Add feature flag to force all direct download URLs to be proxied
Fixes #1667
2016-09-29 11:13:41 +02:00
Jimmy Zelinskie
fc7301be0d *: fix legacy imports
This change reorganizes imports and renames the legacy flask extensions.
2016-09-28 20:17:14 -04:00
Jimmy Zelinskie
ae16d24fd1 license: validate via key instance rather than PEM 2016-09-28 15:44:28 -04:00
Joseph Schorr
fac9d9fc5d Fix broken test after the recent 404 change 2016-09-27 17:14:56 +02:00
josephschorr
e1771abe58 Merge pull request #739 from coreos-inc/license
Add license checking to Quay
2016-09-27 16:52:08 +02:00
Joseph Schorr
476576bb70 Add license checking to Quay
Based off of mjibson's changes

Fixes #499
2016-09-27 10:31:34 +02:00
Joseph Schorr
3c8b87e086 Fix verbs in manifestlist
All registry_tests now pass
2016-09-26 14:49:58 -04:00
Jimmy Zelinskie
2e5a94bc0b create key server data interface 2016-09-26 14:49:23 -04:00
Joseph Schorr
db60df827d Implement V2 interfaces and remaining V1 interfaces
Also adds some tests to registry tests for V1 stuff.
Note: All *registry* tests currently pass, but as verbs are not yet converted, the verb tests in registry_tests.py currently fail.
2016-09-26 14:49:04 -04:00
Joseph Schorr
de990253bc Fix the build manager tests for recent change 2016-09-26 17:28:09 +02:00
Joseph Schorr
a5fef119c9 Add an end-to-end test for the notifications queue 2016-09-21 15:15:35 -04:00
Joseph Schorr
6fffc22b8a Fix build should_perform for empty JSON 2016-09-21 15:00:47 -04:00
Joseph Schorr
af79fde50d Fix build notifications 2016-09-21 14:37:23 -04:00
Joseph Schorr
03d4445a02 Add notification filtering for builds based on ref regex
Fixes #1835
2016-09-14 16:48:17 -04:00
Jake Moshenko
d56f570d3b Improve the imagetree test. 2016-09-07 13:25:19 -04:00
Jake Moshenko
cf83c9a16a Improve the garbage collection tests. 2016-09-07 13:25:19 -04:00
Jake Moshenko
584a5a7ddd Reduce database bandwidth by tracking gc candidate images. 2016-09-07 13:25:19 -04:00
Jake Moshenko
0815f6b6c4 Fix indentation for DB queries. 2016-09-07 10:48:58 -04:00
josephschorr
cd8b45e25b Merge pull request #1754 from coreos-inc/team-add-perms
Better UI and permissions handling for robots and teams
2016-09-06 17:21:19 -04:00
Joseph Schorr
1b7b3ea41d Make sure to filter starred repos to those visible to the user
Fixes #1793
2016-08-31 14:08:31 -04:00
Joseph Schorr
b4939a3cd0 Fix filtering of repos only visible to org admins 2016-08-31 13:51:53 -04:00
Joseph Schorr
1864196d8d TAR creation is not deterministic, so we can't test repush consistently 2016-08-29 16:38:12 -04:00
Joseph Schorr
357005e33f Raise a 409 if we try to insert a tag twice at the same time
Also fixes handling of labels for existing manifests

Fixes #1775
2016-08-29 16:03:35 -04:00
Joseph Schorr
1a2666be07 Fix deletion of labels and add tests 2016-08-26 16:07:49 -04:00
Joseph Schorr
608ffd9663 Basic labels support
Adds basic labels support to the registry code (V2), and the API. Note that this does not yet add any UI related support.
2016-08-26 15:24:26 -04:00
Joseph Schorr
391d70d9ec Add repo permissions dialog for existing teams and robots
Fixes #1686
2016-08-22 14:43:12 -04:00
josephschorr
2caa82d091 Merge pull request #1713 from coreos-inc/enable-iam
Enable IAM support for S3 storage
2016-08-16 16:13:29 -04:00
Joseph Schorr
742e153133 Fix watch of the jobs key in the build manager 2016-08-16 15:43:09 -04:00
Joseph Schorr
06718d1237 Fix S3 tests for IAM 2016-08-15 20:34:17 -04:00
josephschorr
de9be6e993 Merge pull request #1730 from coreos-inc/fix-pagination
Fix pagination of repositories
2016-08-15 17:14:32 -04:00
Joseph Schorr
d78361b041 Cleanup old executions that never start
Fixes #1727
2016-08-15 16:54:02 -04:00
Joseph Schorr
7f5b536ddb Fix pagination of repositories
Fixes #1725
2016-08-15 16:48:04 -04:00
Joseph Schorr
4f5b4e63f2 Really fix the hack (for now) on public repo pagination 2016-08-13 14:40:11 -04:00
josephschorr
d9b65b88e9 Merge pull request #1716 from coreos-inc/unicode-tags
Add a test for unicode tags to ensure they cannot be set
2016-08-11 18:34:44 -04:00
Joseph Schorr
19fb8fcf7c Add a test for unicode tags to ensure they cannot be set
Fixes #1324
2016-08-11 18:21:01 -04:00
Joseph Schorr
bab5cf69c7 Add a test for a deleted ref for the Github trigger
Fixes #1047
2016-08-11 18:01:04 -04:00
Joseph Schorr
4a2acac5dc Fix pagination of public repos, make more efficient and add test 2016-08-10 15:08:06 -04:00
Jimmy Zelinskie
22a25ac2d3 Revert "Merge pull request #1678 from coreos-inc/delete-repo-fix"
This reverts commit df64caf133, reversing
changes made to 0d1e453566.
2016-08-08 12:38:15 -04:00
josephschorr
df64caf133 Merge pull request #1678 from coreos-inc/delete-repo-fix
Have repo deletion not lock all the things
2016-08-04 16:48:03 -04:00
Joseph Schorr
c4daf1cc3d Change permissions model so that non-admins do not get org-wide read
Fixes #1684
2016-08-04 16:47:28 -04:00
Joseph Schorr
0b5cd95693 Have repo deletion not lock all the things 2016-08-04 16:45:59 -04:00
josephschorr
8bc0080aeb Merge pull request #1672 from coreos-inc/off-by-one
Fix off-by-one error in repo tags pagination
2016-08-03 15:00:23 -04:00
josephschorr
de58c1e38b Merge pull request #1661 from coreos-inc/buildman-timeout
Fix TTL on heartbeat in etcd
2016-08-03 11:18:32 -04:00
Joseph Schorr
c29f9ccc7f Fix TTL on heartbeat in etcd
Until now, once the heartbeat has expired, we would issue a TTL that is negative, which causes etcd to either raise an exception or simply ignore the expiration (depending on the version of etcd). This change ensures that once the key is expired, it is removed immediately via a set of a TTL of 0. Also adds tests for this case and the normal expiration case.
2016-08-03 11:15:03 -04:00
Joseph Schorr
b1b0da7afd Fix off-by-one error in repo tags pagination
Fixes #1665
2016-08-02 14:17:33 -04:00
Jake Moshenko
8ac88facab Add a test to make sure the random policy function runs. 2016-08-01 18:42:55 -04:00
Jake Moshenko
05e2773fa7 Get rid of remaining slow query for garbage collection. 2016-08-01 18:22:38 -04:00
josephschorr
b0bffe56ca Merge pull request #1638 from coreos-inc/swift-retry-seek
Add retry support to Swift
2016-08-01 14:04:54 -04:00
josephschorr
46a28617e8 Merge pull request #1651 from coreos-inc/fix-branches
Fix handling of multi-part branches in the build triggers
2016-07-26 16:00:21 -07:00
josephschorr
0162d3da30 Merge pull request #1645 from coreos-inc/gc-query-optimize
Optimize GC query for looking up deletable storages
2016-07-26 16:00:17 -07:00
Joseph Schorr
9e4f8cac03 Optimize GC query for looking up deletable storages 2016-07-26 13:47:15 -07:00
Joseph Schorr
06d52f2c83 Fix handling of multi-part branches in the build triggers
Fixes #1360
2016-07-26 13:41:13 -07:00
Joseph Schorr
a41ccf0356 Add retry support to Swift
Fixes #1636
2016-07-26 09:56:00 -07:00
Joseph Schorr
0fe3e6510a Prevent invalid tags on builds
Fixes #1632
2016-07-25 17:50:35 -07:00
Joseph Schorr
5de1e98d3c Fix LDAP DN building for empty RDN list 2016-07-22 14:40:53 -04:00
Joseph Schorr
392242d20b Another fix for the record keeping in buildman
Adds some more mocked tests as well
2016-07-22 12:01:30 -04:00
josephschorr
cf630838f0 Merge pull request #1624 from coreos-inc/builder-cleanup-tests
Bug fixes, refactoring and "new" tests for the build manager
2016-07-21 13:50:41 -04:00
Jimmy Zelinskie
2ed5723ca9 test_secscan: add a second before reads from queue
Because of the granularity of MySQL's clock, we need to wait a second
before an item becomes available.
2016-07-18 14:19:36 -04:00
Joseph Schorr
2c1880b944 Bug fixes, refactoring and "new" tests for the build manager
- Fixes various bugs introduced in the most recent build system commit
- Refactors state management in the build manager to be cleaner and more contained
- Adds back in the mock-based tests, fixed to not use threads and adjusted for the refactoring
- Adds some more simplified unit tests around non-etch related flows
2016-07-18 13:46:48 -04:00
Joseph Schorr
b0b7b63be9 Fix queue tests for MySQL
MySQL's date time's appear to have a 1 second threshold, so we need to make sure the queue items added for the tests are available as soon as they are added. Before this change, the available_after was set to `datetime.utcnow()`, and, if the `get` was called within 1 second, then its check would fail.
2016-07-15 13:27:50 -04:00
Joseph Schorr
1ed1bc9ed3 Disable prometheus in tests 2016-07-14 15:48:21 -04:00
Joseph Schorr
74b87fa813 Build manager cleanup and more logging 2016-07-14 14:33:14 -04:00
Joseph Schorr
3d558f6090 Disable ACI tests when build under Docker
We need to figure out why they fail on our build fleet
2016-07-13 14:23:30 -04:00
josephschorr
a69266c282 Merge pull request #1605 from coreos-inc/kubernetes-builder
Kubernetes builder
2016-07-12 14:49:10 -04:00
josephschorr
3143da6392 Merge pull request #1608 from coreos-inc/storage-rep
Fix storage replication for CAS and add tests
2016-07-12 13:56:36 -04:00
Joseph Schorr
5cd793331e Fix storage replication for CAS and add tests 2016-07-12 13:46:06 -04:00
Joseph Schorr
c1e4bf79b7 Fix delete team error message for admin teams 2016-07-11 15:47:05 -04:00
Joseph Schorr
811413fe9c Add multiple executor and whitelist support to build manager 2016-07-08 15:50:51 -04:00
Joseph Schorr
6bdbe25cdc Cleanup unused imports 2016-07-08 15:50:51 -04:00
Colin Hom
bc13333f20 Kubernetes build worker 2016-07-08 15:50:51 -04:00
Joseph Schorr
adaeeba5d0 Allow for multiple user RDNs in LDAP
Fixes #1600
2016-07-07 14:46:38 -04:00
Joseph Schorr
e252ee07cb Fix popularity metrics on list repos API 2016-07-06 16:15:54 -04:00
Joseph Schorr
1eec6f53b2 Fix SQL error with pagination around Repositories
Fixes #1591
2016-06-30 17:31:35 -04:00
Joseph Schorr
9558c0e937 Fix handling of Github API paths and add tests 2016-06-30 14:10:22 -04:00
Joseph Schorr
2f771304fa Disable flaky mock-based tests 2016-06-24 16:04:34 -04:00
Joseph Schorr
30ede029d5 Fix GeneratorFile for working with BufferedReader
The user files system uses a BufferedReader along with the magic library to determine the mime type of the user file being served. Currently, BufferedReader fails with an exception on Swift storage, because Swift storage returns a GeneratorFile, which is missing the `readable()` method.
2016-06-23 13:40:57 -04:00
josephschorr
9e6a264f5f Merge pull request #1523 from coreos-inc/verb-tag-cache-fix
Add a uniqueness hash to derived image storage to break caching over …
2016-06-20 16:38:25 -04:00
Joseph Schorr
a43b741f1b Add a uniqueness hash to derived image storage to break caching over tags
This allows converted ACIs and squashed images to be unique based on the specified tag.

Fixes #92
2016-06-20 16:34:52 -04:00
Jake Moshenko
a1cf12e460 Add a sitemap.txt for popular public repos
and reference it from the robots.txt
2016-06-17 14:34:20 -04:00
josephschorr
614b9124ae Merge pull request #1512 from coreos-inc/optimize-queries
Optimize various queries
2016-06-16 14:22:59 -04:00
Joseph Schorr
fea47bdaed Increase test counter to 10 2016-06-13 17:31:42 -04:00
josephschorr
58bef472d9 Merge pull request #1526 from coreos-inc/superuser-grant
Add ability for super users to take ownership of namespaces
2016-06-13 16:23:10 -04:00
Joseph Schorr
20816804e5 Add ability for super users to take ownership of namespaces
Fixes #1395
2016-06-13 16:22:52 -04:00
josephschorr
9263aad2ac Merge pull request #1534 from coreos-inc/flakey-test
Fix flaky tests
2016-06-13 16:12:15 -04:00
Joseph Schorr
4747dea395 Fix flaky tests 2016-06-13 16:00:55 -04:00
josephschorr
92f0db8056 Merge pull request #1514 from coreos-inc/better-logs
Only send heavy log-based stats for repository where required
2016-06-09 14:52:30 -04:00