Commit graph

830 commits

Author SHA1 Message Date
Joseph Schorr
06d52f2c83 Fix handling of multi-part branches in the build triggers
Fixes #1360
2016-07-26 13:41:13 -07:00
Joseph Schorr
a41ccf0356 Add retry support to Swift
Fixes #1636
2016-07-26 09:56:00 -07:00
Joseph Schorr
0fe3e6510a Prevent invalid tags on builds
Fixes #1632
2016-07-25 17:50:35 -07:00
Joseph Schorr
5de1e98d3c Fix LDAP DN building for empty RDN list 2016-07-22 14:40:53 -04:00
Joseph Schorr
392242d20b Another fix for the record keeping in buildman
Adds some more mocked tests as well
2016-07-22 12:01:30 -04:00
josephschorr
cf630838f0 Merge pull request #1624 from coreos-inc/builder-cleanup-tests
Bug fixes, refactoring and "new" tests for the build manager
2016-07-21 13:50:41 -04:00
Jimmy Zelinskie
2ed5723ca9 test_secscan: add a second before reads from queue
Because of the granularity of MySQL's clock, we need to wait a second
before an item becomes available.
2016-07-18 14:19:36 -04:00
Joseph Schorr
2c1880b944 Bug fixes, refactoring and "new" tests for the build manager
- Fixes various bugs introduced in the most recent build system commit
- Refactors state management in the build manager to be cleaner and more contained
- Adds back in the mock-based tests, fixed to not use threads and adjusted for the refactoring
- Adds some more simplified unit tests around non-etch related flows
2016-07-18 13:46:48 -04:00
Joseph Schorr
b0b7b63be9 Fix queue tests for MySQL
MySQL's date time's appear to have a 1 second threshold, so we need to make sure the queue items added for the tests are available as soon as they are added. Before this change, the available_after was set to `datetime.utcnow()`, and, if the `get` was called within 1 second, then its check would fail.
2016-07-15 13:27:50 -04:00
Joseph Schorr
1ed1bc9ed3 Disable prometheus in tests 2016-07-14 15:48:21 -04:00
Joseph Schorr
74b87fa813 Build manager cleanup and more logging 2016-07-14 14:33:14 -04:00
Joseph Schorr
3d558f6090 Disable ACI tests when build under Docker
We need to figure out why they fail on our build fleet
2016-07-13 14:23:30 -04:00
josephschorr
a69266c282 Merge pull request #1605 from coreos-inc/kubernetes-builder
Kubernetes builder
2016-07-12 14:49:10 -04:00
josephschorr
3143da6392 Merge pull request #1608 from coreos-inc/storage-rep
Fix storage replication for CAS and add tests
2016-07-12 13:56:36 -04:00
Joseph Schorr
5cd793331e Fix storage replication for CAS and add tests 2016-07-12 13:46:06 -04:00
Joseph Schorr
c1e4bf79b7 Fix delete team error message for admin teams 2016-07-11 15:47:05 -04:00
Joseph Schorr
811413fe9c Add multiple executor and whitelist support to build manager 2016-07-08 15:50:51 -04:00
Joseph Schorr
6bdbe25cdc Cleanup unused imports 2016-07-08 15:50:51 -04:00
Colin Hom
bc13333f20 Kubernetes build worker 2016-07-08 15:50:51 -04:00
Joseph Schorr
adaeeba5d0 Allow for multiple user RDNs in LDAP
Fixes #1600
2016-07-07 14:46:38 -04:00
Joseph Schorr
e252ee07cb Fix popularity metrics on list repos API 2016-07-06 16:15:54 -04:00
Joseph Schorr
1eec6f53b2 Fix SQL error with pagination around Repositories
Fixes #1591
2016-06-30 17:31:35 -04:00
Joseph Schorr
9558c0e937 Fix handling of Github API paths and add tests 2016-06-30 14:10:22 -04:00
Joseph Schorr
2f771304fa Disable flaky mock-based tests 2016-06-24 16:04:34 -04:00
Joseph Schorr
30ede029d5 Fix GeneratorFile for working with BufferedReader
The user files system uses a BufferedReader along with the magic library to determine the mime type of the user file being served. Currently, BufferedReader fails with an exception on Swift storage, because Swift storage returns a GeneratorFile, which is missing the `readable()` method.
2016-06-23 13:40:57 -04:00
josephschorr
9e6a264f5f Merge pull request #1523 from coreos-inc/verb-tag-cache-fix
Add a uniqueness hash to derived image storage to break caching over …
2016-06-20 16:38:25 -04:00
Joseph Schorr
a43b741f1b Add a uniqueness hash to derived image storage to break caching over tags
This allows converted ACIs and squashed images to be unique based on the specified tag.

Fixes #92
2016-06-20 16:34:52 -04:00
Jake Moshenko
a1cf12e460 Add a sitemap.txt for popular public repos
and reference it from the robots.txt
2016-06-17 14:34:20 -04:00
josephschorr
614b9124ae Merge pull request #1512 from coreos-inc/optimize-queries
Optimize various queries
2016-06-16 14:22:59 -04:00
Joseph Schorr
fea47bdaed Increase test counter to 10 2016-06-13 17:31:42 -04:00
josephschorr
58bef472d9 Merge pull request #1526 from coreos-inc/superuser-grant
Add ability for super users to take ownership of namespaces
2016-06-13 16:23:10 -04:00
Joseph Schorr
20816804e5 Add ability for super users to take ownership of namespaces
Fixes #1395
2016-06-13 16:22:52 -04:00
josephschorr
9263aad2ac Merge pull request #1534 from coreos-inc/flakey-test
Fix flaky tests
2016-06-13 16:12:15 -04:00
Joseph Schorr
4747dea395 Fix flaky tests 2016-06-13 16:00:55 -04:00
josephschorr
92f0db8056 Merge pull request #1514 from coreos-inc/better-logs
Only send heavy log-based stats for repository where required
2016-06-09 14:52:30 -04:00
Joseph Schorr
c3701cea7a Only send heavy log-based stats for repository where required 2016-06-09 14:52:15 -04:00
Joseph Schorr
8887f09ba8 Use the instance service key for registry JWT signing 2016-06-07 11:58:10 -04:00
Joseph Schorr
53538f9001 Optimize get_tag_image query
No caller uses the image placements or locations, so no need to load them.
2016-06-02 16:36:38 -04:00
josephschorr
cad8746f9d Merge pull request #1502 from coreos-inc/image-replication
Enable storage replication for V2 and add backfill tool
2016-06-02 15:02:53 -04:00
Joseph Schorr
12924784ce Enable storage replication for V2 and add backfill tool
Fixes #1501
2016-06-02 14:36:08 -04:00
josephschorr
ec492bb683 Merge pull request #1323 from coreos-inc/secworkerreturn
Move security notification work into its own method to allow for retu…
2016-06-02 13:59:25 -04:00
Joseph Schorr
48213f9ff9 Reject manifest 2 earlier to make pushes faster 2016-06-02 12:46:20 -04:00
Jimmy Zelinskie
2317938bfa Merge pull request #1496 from jzelinskie/ripRMS
dockerfile: add check for GPL pip packages
2016-06-02 12:28:18 -04:00
Jimmy Zelinskie
e5241c6d88 tests: simple test for BuildRequest w/ archive URL 2016-06-02 12:27:49 -04:00
Joseph Schorr
a18c4dd210 Make exponential back off test try multiple times
Slower runtime environments require multiple calls before we hit the 429
2016-06-01 15:00:10 -04:00
josephschorr
a85c3ebff7 Merge pull request #1457 from coreos-inc/xauth
Add support for direct granting of OAuth tokens and add tests
2016-06-01 12:07:12 -04:00
Jimmy Zelinskie
6178371cf5 Merge pull request #1493 from jzelinskie/noorder
queue: explicitly declare ordering requirement
2016-05-31 15:46:39 -04:00
Jimmy Zelinskie
44b56ae2cf queue: explicitly declare ordering requirement
This change defaults the ordering requirement of queue items to be off
and only enables it for the build manager. This should make the queries
for getting queueitems significantly faster for every other use case.
2016-05-27 14:44:30 -04:00
josephschorr
47afbb65dc Merge pull request #1490 from coreos-inc/aci-reproduce
Make ACI generation consistent across calls
2016-05-26 19:37:01 -04:00
Joseph Schorr
4ec3a6c231 Make ACI generation consistent across calls
This will ensure that no matter which signature we write for the generated ACI, it is correct for that image.
2016-05-26 17:09:19 -04:00
Jake Moshenko
8323c51e6e Extend registry auth to support notary JWTs. 2016-05-24 13:42:28 -04:00
josephschorr
fa3b342901 Merge pull request #1483 from coreos-inc/superuser-external-user
Fix setup tool when binding to external auth
2016-05-23 17:17:45 -04:00
Joseph Schorr
7933aecf25 Add support for direct granting of OAuth tokens and add tests
This allows a client (when authorized in a whitelist) to send direct credentials via a Basic auth header and therefore bypass the OAuth approval UI for that user.
2016-05-23 17:17:06 -04:00
Joseph Schorr
60bbca2185 Fix setup tool when binding to external auth
We now query the external auth provider for the external service's identifier before adding the linking row into the database. This fixes the case where the external service resolves a different identifier for the same username.

Fixes #1477
2016-05-23 17:11:36 -04:00
Joseph Schorr
f670c4c7a9 Change Signer to use the config provider and fix tests
Fixes the broken ACI tests
2016-05-23 17:10:03 -04:00
Jimmy Zelinskie
5568cc77b8 remove all default keys (#1485)
This change:
- Generates a new BitTorrent pepper by default
- Generates a new pagination key by default
- Changes the pagination key format to base64
- Removes selfsigned JWT certs
- Moves test keys to test/data
2016-05-23 16:00:48 -04:00
Joseph Schorr
1365492b28 Fix ACI signing tests 2016-05-16 13:31:43 -04:00
Joseph Schorr
64fe11a5f1 Add ACI signing tests 2016-05-13 18:29:57 -04:00
josephschorr
de6b7bc88d Merge pull request #1460 from coreos-inc/queuefilebinarydata
Add a binary data test for queue file
2016-05-13 16:43:18 -04:00
Joseph Schorr
d74198ee66 Add a binary data test for queue file 2016-05-13 15:56:06 -04:00
Joseph Schorr
72fd2b76e2 Add basic ACI conversion tests 2016-05-13 15:50:57 -04:00
Joseph Schorr
a736407611 Fix user:admin scope handling and add test 2016-05-09 11:16:01 +02:00
Joseph Schorr
343a080833 Make security scan testing much faster 2016-05-05 13:55:24 -04:00
Joseph Schorr
232fa42897 Add testing of the new secscan-for-local endpoint and fix a bug 2016-05-04 21:47:03 -04:00
Jake Moshenko
9221a515de Use the registry API for security scanning
when the storage engine doesn't support direct download url
2016-05-04 18:04:06 -04:00
josephschorr
550b9cb2b3 Merge pull request #1428 from coreos-inc/clair-setup-new
Implement setup tool support for Clair
2016-05-04 13:52:54 -04:00
Joseph Schorr
2cbdecb043 Implement setup tool support for Clair
Fixes #1387
2016-05-04 13:40:50 -04:00
Joseph Schorr
6e2df3b339 Fix key server to not list expired keys
Fixes the key server to not list expire keys and by default not return expired or unapproved keys unless explicitly requested.

Fixes #1430
2016-05-03 17:58:47 -04:00
Jimmy Zelinskie
e502f50c88 tests: add test RSA key for torrent test (#1427) 2016-05-03 13:11:02 -04:00
Jimmy Zelinskie
b89d81d748 test: add missing helpers.py file 2016-04-29 14:44:52 -04:00
Joseph Schorr
6091db983b Hide expired keys outside of their staleness window 2016-04-29 14:10:33 -04:00
Joseph Schorr
4f63a50a17 Change account-less logs to use a user and not null
This allows us to skip the migration
2016-04-29 14:09:37 -04:00
Joseph Schorr
28a80ef6a9 Make sure to verify service names on key creation 2016-04-29 14:09:37 -04:00
Joseph Schorr
5d6e5a42e8 Add delete logging and tests for logging 2016-04-29 14:09:09 -04:00
Joseph Schorr
bc08ac2749 Fix timeouts in the JWT endpoint tests 2016-04-29 14:05:16 -04:00
Joseph Schorr
522cf68c5d Lots of smaller fixes:
- Add the rotation_duration to the keys API
- Have the key service UI use the new rotation_duration field
- Fix notification deletion lookup path
- Add proper support for the new notification in the UI
- Only delete expired keys after 7 days (configurable)
- Fix angular digest loop
- Fix unit tests
- Regenerate initdb
2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
2805dad64f test_endpoints: update to use JWT headers 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
cfc15746a6 keyserver: tests! 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
d19eb16b45 keyserver: add generate key function
The superuser API, initdb, and tests will all need this functionality.
2016-04-29 14:05:16 -04:00
Joseph Schorr
23a8a29654 More tests 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
b0dac1d27e initdb: add unapproved service key 2016-04-29 14:05:16 -04:00
Joseph Schorr
fb1dca4e94 Add API usage tests 2016-04-29 14:05:16 -04:00
Joseph Schorr
11ff3e9b59 keys ui WIP 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
dc593c0197 tests: shell of key server tests 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
bbaeaffbdb run initdb for service keys 2016-04-29 13:38:25 -04:00
josephschorr
9e88b1413d Merge pull request #1325 from coreos-inc/blobuncompressedsize
Fix uncompressed size for blob store and add test
2016-04-28 13:15:33 -04:00
Joseph Schorr
3f8d51ebd7 Fix handling of Clair notifications without New block
Fixes #1398
2016-04-22 13:05:34 -04:00
Joseph Schorr
c604dbd0f6 Fix permissions when converting a user to an org
Fixes #1366
2016-04-14 17:39:45 -04:00
Evan Cordell
7b44beb1fd Fix WWW-Authenticate header on 401 2016-04-13 09:01:42 -04:00
Evan Cordell
b5db41920f Address review comments 2016-04-11 16:34:40 -04:00
Evan Cordell
eba75494d9 Use new error format for auth errors (factor exceptions into module) 2016-04-11 16:22:26 -04:00
Evan Cordell
9c08717173 Return application/problem+json format errors and provide error endpoint
to dereference error codes.
2016-04-11 14:57:24 -04:00
josephschorr
affb600423 Merge pull request #1328 from coreos-inc/queuefilefix
Fix QueueFile to support read-to-end semantics and add some tests
2016-04-08 18:07:06 -04:00
Joseph Schorr
1009362d26 Have recovery auto-verify the user
Fixes #1355
2016-04-08 13:41:16 -04:00
Joseph Schorr
d62ec22fc9 Move security notification work into its own method to allow for return values
Fixes #1302
Fixes #1304
2016-03-31 14:08:33 -04:00
Joseph Schorr
6251e63e0e Fix QueueFile to support read-to-end semantics and add some tests 2016-03-31 12:06:49 -04:00
josephschorr
edb157c5cb Merge pull request #1294 from coreos-inc/partialperms
Change permissions to only load required by default
2016-03-30 16:40:40 -04:00
Joseph Schorr
db6f3691e5 Fix broken test 2016-03-30 16:32:08 -04:00
Joseph Schorr
b5b2df2063 Make test more resilient to changes in IDs 2016-03-30 16:19:15 -04:00
Joseph Schorr
42e934d84f Make notification lookup faster and fix repo pagination on Postgres 2016-03-30 14:46:31 -04:00
Joseph Schorr
0dffdb87c9 Fix uncompressed size for blob store and add test 2016-03-29 14:16:56 -04:00
Joseph Schorr
a3aa4592cf Change permissions to only load required by default
Permissions now load just the namespace and/or repository permissions requested, with a fallback to a full permissions load if necessary.
2016-03-28 16:33:32 -04:00
Joseph Schorr
eab6af2b87 Add mocked unit tests for cloud storage engine 2016-03-23 12:13:54 -04:00
Joseph Schorr
aa5587c93c Fixes and added tests for the security notification worker
Fixes #1301

- Ensures that the worker uses pagination properly
- Ensures that the worker handles failure as expected
- Moves marking the notification as read to after the worker processes it
- Increases the number of layers requested to 100
2016-03-18 20:28:06 -04:00
Jimmy Zelinskie
8af0b887ef fix broken tests 2016-03-18 15:48:41 -04:00
Jimmy Zelinskie
bf477b6b9c add slash_join helper and tests 2016-03-18 14:56:10 -04:00
josephschorr
c1cceb2949 Merge pull request #1295 from coreos-inc/fixeventseverity
Add another test for security notification filtering
2016-03-17 13:00:59 -04:00
Joseph Schorr
6a4584b87a Add another test for security notification filtering 2016-03-17 12:59:27 -04:00
Quentin Machu
d093a7bde5 Merge pull request #1290 from Quentin-M/split_clair_clusters
Split clair clusters
2016-03-15 11:09:51 -04:00
Joseph Schorr
57e5141fb5 Fix link-to-parent-with-different-blob issue and add a test 2016-03-14 15:35:18 -04:00
Quentin Machu
e28d2d7ce8 Fix Clair's testconfig 2016-03-14 14:28:34 -04:00
Jimmy Zelinskie
ba2851c952 Merge pull request #1287 from jzelinskie/namespace-reponame
v2: send proper scopes for authorization failures
2016-03-11 13:46:16 -05:00
Jimmy Zelinskie
ea2e17cc11 v2: send proper scopes for authorization failures
Fixes #1278.
2016-03-11 13:41:38 -05:00
Jimmy Zelinskie
bb46cc933d use kwargs for parse_repository_name 2016-03-09 16:20:28 -05:00
Joseph Schorr
c75fcfbd5e Add body checking to the analyze layer test
Fixes #1272
2016-03-09 11:45:28 -05:00
Jake Moshenko
fe2cd240bc Revert "Remove old search API which is no longer in use" 2016-03-07 10:07:41 -05:00
josephschorr
57430a18b4 Merge pull request #1224 from coreos-inc/removeoldsearch
Remove old search API which is no longer in use
2016-03-04 12:05:07 -05:00
Quentin Machu
f4131d3c8a Enable security notifications in test suite 2016-03-01 16:14:56 -05:00
Jimmy Zelinskie
c7904db30d v2: always send www-authn headers on unauthorized
Fixes #1254.
2016-02-25 17:09:29 -05:00
Joseph Schorr
f498e92d58 Implement against new Clair paginated notification system 2016-02-25 15:58:42 -05:00
Joseph Schorr
c0374d71c9 Refactor the security worker and API calls and add a bunch of tests 2016-02-25 12:29:41 -05:00
josephschorr
6f9fc7fc08 Merge pull request #1225 from coreos-inc/setuptooltest
Add tests for superuser config API calls
2016-02-16 17:01:43 -05:00
josephschorr
81a36ee3b8 Merge pull request #1217 from coreos-inc/v2pagination
Fix V2 catalog and tag pagination
2016-02-16 15:34:49 -05:00
Joseph Schorr
ecaa051791 Fix schema for invoice email updating
Fixes #1209
2016-02-16 11:52:57 -05:00
Joseph Schorr
69262282fe Make sure to encode all V1 metadata strings
Fixes #1239
2016-02-15 10:57:20 -05:00
Jake Moshenko
6454b5aeb7 Update the layer rename PR to preserve the original manifest 2016-02-12 16:25:47 -05:00
Joseph Schorr
abd2e3c234 V1 Docker ID <-> V2 layer SHA mismatch fix
Fix handling of V1 Docker ID <-> V2 layer SHA mismatch by dynamically rewriting the manifest to use new synthesized IDs for all layers above the mismatch. Also adds a bunch of tests for this and other use cases, fixes a bug around manifest digest uniqueness and fixes the 5.5 migration for MySQL.
2016-02-12 17:39:27 +02:00
Jake Moshenko
abb0e9fb88 Fix allocator test 2016-02-11 17:18:19 -05:00
josephschorr
904b2d53d2 Merge pull request #1197 from coreos-inc/webpytest
Tests for endpoints/web and some small fixes
2016-02-11 22:42:43 +02:00
Joseph Schorr
03533db5a3 Add tests for superuser config API calls 2016-02-11 11:04:37 +02:00
Joseph Schorr
1887dc879c Remove old search API which is no longer in use 2016-02-10 15:02:27 +02:00
Joseph Schorr
db0eab0461 Fix V2 catalog and tag pagination 2016-02-10 00:25:33 +02:00
josephschorr
d5920319de Merge pull request #1193 from coreos-inc/keystonetest
Add basic tests for keystone auth
2016-02-05 09:51:05 +02:00
Joseph Schorr
cc677f9824 Add basic tests for keystone auth 2016-02-05 09:50:46 +02:00
Joseph Schorr
6a8331d305 Tests for endpoints/web and some small fixes 2016-02-05 09:45:25 +02:00
Joseph Schorr
534ec9cb2b Add pagination to the repository list API to make it better for public
Fixes #1166
2016-02-01 22:42:44 +02:00
Joseph Schorr
accc576a98 Fix V1 push URL to match Docker and fix registry tests 2016-01-29 16:42:15 +02:00
Joseph Schorr
8cd38569d6 Fix issue with Docker 1.8.3 and pulling public repos with no creds
We now return the valid subset of auth scopes requested.

Adds a test for this case and adds testing of all returned JWTs in the V2 login tests
2016-01-25 15:54:17 -05:00
Joseph Schorr
e4ffaff869 Fix Docker Auth and our V2 registry paths to support library (i.e. namespace-less) repositories.
This support is placed behind a feature flag.
2016-01-22 15:54:06 -05:00
Joseph Schorr
06b0f756bd Fix test to reflect change in the status code returned for no torrent available 2016-01-20 18:35:07 -05:00
Joseph Schorr
7c572fd218 Add support for torrenting verbs
Fixes #1130
2016-01-20 18:15:32 -05:00
josephschorr
3fdadb51b7 Merge pull request #1155 from coreos-inc/torrenttest
Add torrent tests
2016-01-20 13:42:42 -05:00
Joseph Schorr
68c9d5e432 Add torrent tests
Fixes #1128
2016-01-19 17:40:11 -05:00
Jake Moshenko
fe2bdeb6cb Require some data from all models in initdb 2016-01-19 15:30:27 -05:00
Jake Moshenko
909e7d45b7 Add a test for swift path computation 2016-01-15 15:35:04 -05:00
Jake Moshenko
dc23ccce89 Remove the sample data we no longer use 2016-01-15 10:31:52 -05:00
Joseph Schorr
e4da61a05d Fix piece hash calculation 2016-01-12 17:44:19 -05:00
Joseph Schorr
742f3b4fc1 Add GC test for torrent info 2016-01-12 12:15:07 -05:00
Jake Moshenko
77aa58996a Fix the db definition for torrentinfo and add migration 2016-01-06 14:04:03 -05:00
Jake Moshenko
fe87d3c796 Hash and track layer file chunks for torrenting 2016-01-04 16:17:51 -05:00