vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
2398 commits 2 branches 62 tags 205 MiB
Commit graph

785 commits

Author SHA1 Message Date
Joseph Schorr
9d1b6d829a Make sure the external login link for GHE links to the enterprise GitHub and not the hosted version 2014-11-06 20:35:52 -05:00
Joseph Schorr
d5bbb57481 Change registry code to disconnect from the DB before long I/O operations 2014-11-06 18:00:52 -05:00
Joseph Schorr
23d9bd2b42 Change verbs to use short lived database connections 2014-11-06 17:50:48 -05:00
Joseph Schorr
c569299e5c Database optimizations around image creation and logs lookup 2014-11-06 14:48:16 -05:00
Joseph Schorr
3e79379942 - Make the OAuth config system centralized 
- Add support for Github Enterprise login
 2014-11-05 16:43:37 -05:00
Joseph Schorr
a35bc11912 Add perf comments 2014-11-05 12:27:38 -05:00
Joseph Schorr
6adf4644b4 Remove unneeded extra logging call 2014-11-03 17:17:05 -05:00
Joseph Schorr
98602a2d0c Add a new configurable health check, to make sure production instances are not taken down by Redis or non-local DB issues 2014-11-02 15:06:17 -05:00
Joseph Schorr
4eedd54b66 - Make usage language more accurate by stating "repositories" 
- Have usage counter be based on a 4 weeks TTL
- Add a simple usage counter breakage test
 2014-10-30 13:26:02 -04:00
Joseph Schorr
79e4864eb2 Move the the track_and_log code into its own module. This breaks a dependency chain between index -> common -> api -> subscribe -> common again. 2014-10-30 12:49:51 -04:00
Joseph Schorr
c1398c6d2b - Add a log entry for repo verb handling and make the container usage calculation take it into account 
- Move all the repo push/pull/verb logging into a central track_and_log method
- Readd images accidentally deleted in the last CL
- Make the uncompressed size migration script better handle exceptions
 2014-10-29 15:42:44 -04:00
Joseph Schorr
dfe143270c Add is_testing value to the status endpoint 2014-10-29 13:51:05 -04:00
Joseph Schorr
e7f98d991f Fix bug which caused manual builds to always build the master branch 2014-10-29 12:04:34 -04:00
Joseph Schorr
109850b428 Add a basic usage counter for enterprise 2014-10-28 16:33:13 -04:00
Joseph Schorr
93cd7de0e0 Handle email errors in a better manner 2014-10-28 12:10:44 -04:00
Joseph Schorr
fb2470615b Add support for filtering based on tags, in addition to branches 2014-10-23 16:39:10 -04:00
Jake Moshenko
1461310ab8 Merge remote-tracking branch 'origin/master' into nomenclature 
Conflicts:
	endpoints/common.py
	endpoints/notificationhelper.py
	test/data/test.db
	workers/dockerfilebuild.py
 2014-10-23 13:25:37 -04:00
Joseph Schorr
5db9cd948b Add better (jinja-based) messaging to the notifications and add some fixes for the email templates 2014-10-22 19:01:56 -04:00
Joseph Schorr
ea96dbb2ad Remove the pushed_image_count since we aren't getting the data anyway 2014-10-22 15:26:29 -04:00
Joseph Schorr
208c97776f Make sure builds are queued under a transaction. This should prevent a queue item from existing without its repository build object (or vice versa). 2014-10-22 15:20:53 -04:00
Joseph Schorr
8b331b453e Make the contact page dynamic so that enterprise customers can configure it however they like 2014-10-22 14:49:33 -04:00
Joseph Schorr
612a5e5102 Fix the updated_tags information by storing it in the session 2014-10-22 14:14:56 -04:00
Joseph Schorr
0ef17b082b Make sure to disconnect from the database when finished with the processes 2014-10-21 17:40:57 -04:00
Joseph Schorr
47be7cab7a Compute the tarsum only when required. Newer versions of Docker only require the simple SHA256 checksum, so this should save us from writing to a temp file. 2014-10-20 13:11:33 -04:00
Joseph Schorr
5e74edc116 Add extra image check for put_image_checksum 2014-10-17 17:57:10 -04:00
Joseph Schorr
346b594b66 Make the external login error messaging nicer when an email address conflict occurs 2014-10-17 11:44:31 -04:00
Joseph Schorr
a423032e80 - Make sure we log when an exception occurs in the squashing code 
- Have queue file always try to return any remaining data in the buffer
- Remove the raise of the exception when we get an empty tar file
 2014-10-16 12:54:16 -04:00
Joseph Schorr
d43109d7cb - Merge branch 'master' into sha-lom 
- Extract out the tar handling from streamlayerformat into tarlayerformat
- Add a new tarfileappender class to make it easy to append data to gzipped tars
- Fix the gzipwrap to properly close
- Have the .git injection use the new appender
 2014-10-15 15:51:34 -04:00
Joseph Schorr
da28bc4ce9 - Handle missing images properly 
- Add support for deleting directories
- Add a slew of tests for deletion of directories and other kinds of deletion and layering
 2014-10-14 21:40:02 -04:00
Joseph Schorr
c3171a2690 Redo the UI for the trigger setup dialog and add the ability for github triggers to be filtered using a regex on their branch name. 2014-10-14 15:46:35 -04:00
Joseph Schorr
37aa70c28e Notifications must check for the user OR an organization with the namespace name 2014-10-10 19:05:20 -04:00
Joseph Schorr
07f3bd6f8c Add a synthetic .git directory containing the commit sha so that 'git rev-parse HEAD' works from inside builds 2014-10-10 17:20:07 -04:00
Joseph Schorr
adc915a5eb Fix the subprocesses to also use their own storage classes; this fixes a socket error when talking to S3 2014-10-08 16:54:03 -04:00
Joseph Schorr
d16fdde528 Fix bug in dockerloadformat and make sure we handle exceptions properly in the verb call 2014-10-08 13:43:12 -04:00
Jake Moshenko
a0d94f9d59 Merge remote-tracking branch 'origin/laffa' into nomenclature 
Conflicts:
	test/data/test.db
 2014-10-07 15:56:28 -04:00
Joseph Schorr
f38ce51943 Merge master into laffa 2014-10-07 14:03:17 -04:00
Jake Moshenko
ed8bcff39e Merge remote-tracking branch 'origin/master' into nomenclature 
Conflicts:
	test/data/test.db
	workers/dockerfilebuild.py
 2014-10-06 10:29:39 -04:00
Jake Moshenko
6bc982b77b Add a note about updating nginx config when updating the _ping endpoint. 2014-10-06 10:24:40 -04:00
Joseph Schorr
e0993b26af Make query params only read from query params, not JSON as well 2014-10-03 15:05:34 -04:00
Joseph Schorr
e654b2b608 Branch.name has to use a dot accessor, not a dict lookup 2014-10-03 12:35:58 -04:00
Joseph Schorr
1d8ec59362 Merge branch master into bees 2014-10-02 15:08:32 -04:00
Joseph Schorr
c682899861 Add a feature flag to disable user creation 2014-10-02 14:49:18 -04:00
Jake Moshenko
5c18ffe67d Allow the namespace column to be null, and also non-unique. Fix the uncompressed size clobbering the size on the wire field. Add metadata constraints so that foreign key constraints get predictable names. Fix all downgrade migrations. 2014-10-02 10:46:20 -04:00
Jake Moshenko
e8b3d1cc4a Phase 4 of the namespace to user migration: actually remove the column from the db and remove the dependence on serialized namespaces in the workers and queues 2014-10-01 14:23:46 -04:00
Joseph Schorr
d9c7e92637 Add superuser abilities: create user, show logs. Also fix the super users UI to show the user drop down and make all superuser API calls require fresh login 2014-10-01 13:55:09 -04:00
Joseph Schorr
039d53ea6c - Fix initdb 
- Add ability to specific custom fields for manual running of build triggers and add a "branch name" selector for running github builds
 2014-09-30 16:29:32 -04:00
Joseph Schorr
474add0fb1 Have the layer PUT method calculate the uncompressed size in realtime, as trusting the JSON is fraught with complications 2014-09-29 17:00:47 -04:00
Joseph Schorr
dd4c26cb00 Only operations with nicknames can be added to the swagger doc 2014-09-28 15:40:08 -04:00
Jake Moshenko
03190efde3 Phase 2 of migrating repo namespaces to referencing user objects, backfilling the rows without a value for namespace_user, and changing all accesses to go through the namespace_user object. All tests are passing, manual testing still required. 2014-09-24 18:01:35 -04:00
Joseph Schorr
ec484e3efc Move size and checksum updates into the metadata call 2014-09-23 15:49:28 -04:00
Joseph Schorr
86dfca2e3e Add uncompressed size field to the image storage and add a backfill script (which is not yet automatically called) 2014-09-23 14:01:27 -04:00
Joseph Schorr
87bc37f6c8 Merge branch 'sunday' 2014-09-23 11:29:03 -04:00
Joseph Schorr
ba0963a81c Update the worker code to better handle exceptions, fix the utcdate issue and make sure we send the proper retry. Also updates notification workers to send JobExceptions rather than returning true or false 2014-09-23 11:01:50 -04:00
Joseph Schorr
8dd2330ce7 Switch to using straight docker IDs instead of a hashing scheme 2014-09-23 11:01:50 -04:00
Joseph Schorr
9621566d31 Instead of sending DB IDs, send "internal IDs" which are DB IDs hashed. This way, we can still calculate the ancestors without hitting the DB further, but without leaking the size of the images table 2014-09-23 11:01:50 -04:00
Joseph Schorr
f3b03ebc34 Add a feature flag for disabling all emails 2014-09-22 19:11:48 -04:00
Jake Moshenko
3259cda000 The new strategy is to do a three phase migration. This is the first phase: getting the namespace user in the db and written for all new repositories. 2014-09-22 17:27:02 -04:00
Joseph Schorr
dc685b2387 Merge branch 'huggies' 2014-09-22 13:41:29 -04:00
Joseph Schorr
f23038c6ee Update the worker code to better handle exceptions, fix the utcdate issue and make sure we send the proper retry. Also updates notification workers to send JobExceptions rather than returning true or false 2014-09-22 12:52:57 -04:00
Joseph Schorr
e5055763f6 Make the squashed path smaller and handle failure cases on the curl side 2014-09-19 12:54:52 -04:00
Joseph Schorr
e273dca4b4 Change back to using a docker load format 2014-09-19 12:22:54 -04:00
Jake Moshenko
8626d1cd70 Initial changes to move repositories from using a namespace string to referencing a user object. Also stores the user id in the cookie rather than the username, to allow users to be renamed. This commit must not be used unmodified because the database migration is too aggressive for live migration. 2014-09-19 10:17:23 -04:00
Jake Moshenko
11bb8e6448 Actually store the generated image storage in the database, and allow it to be garbage collected when the parent image storage is collected. 2014-09-18 17:26:40 -04:00
Joseph Schorr
a90aab4665 Switch to using straight docker IDs instead of a hashing scheme 2014-09-18 17:16:10 -04:00
Joseph Schorr
b212dbb2ab Merge branch 'master' into better-emails 2014-09-18 13:20:32 -04:00
Joseph Schorr
efc06b54f6 Add a TODO and some slightly better naming 2014-09-16 22:44:45 -04:00
Joseph Schorr
1cfb6fc353 Have the squashing system write the data (via a double queue system and multiprocessing) to both the client and the stack's storage. On subsequent calls, if the synthetic image exists, it will be returned directly instead of being recomputed 2014-09-16 22:43:19 -04:00
Joseph Schorr
5cca609c55 Switch back to send_file and add a bit of gzip buffering 2014-09-16 14:20:42 -04:00
Joseph Schorr
9344839295 Get squashed endpoint for docker import working 2014-09-16 11:53:54 -04:00
Joseph Schorr
e3c52fa0eb Work in progress. This is currently broken! 2014-09-16 00:18:57 -04:00
Jake Moshenko
75d2ef377e Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork 
Conflicts:
	data/model/legacy.py
 2014-09-15 17:52:17 -04:00
Joseph Schorr
913b3e472f Add ability to detach external login services 2014-09-15 12:01:02 -04:00
Joseph Schorr
e8ad01cb41 Lots of small NPE and other exception fixes 2014-09-15 11:27:33 -04:00
Joseph Schorr
10faa7de84 Only allow users matching the team invite to accept, if the invite was specified for a user (rather than an email) 2014-09-12 14:29:01 -04:00
Jake Moshenko
c5ca46a14b Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork 
Conflicts:
	data/model/legacy.py
	static/js/app.js
 2014-09-12 11:03:30 -04:00
Joseph Schorr
8d3ce44682 Address comments on code review 2014-09-11 15:45:41 -04:00
Jake Moshenko
8b3a3178b0 Finish the build logs archiver, add handlers for cloud and local that handle gzip encoded archived content. 2014-09-11 15:33:10 -04:00
Jake Moshenko
539fc04205 Seek the file pointer to zero since we now use multipart for upload of userfiles, which does not seek automatically. 2014-09-10 17:18:49 -04:00
Jake Moshenko
29d40db5ea Add a new RadosGW storage engine. Allow engines to distinguish not only between those that can support direct uploads and downloads, but those that support doing it through the browser. Rename resumeable->resumable. 2014-09-09 15:54:03 -04:00
Joseph Schorr
7c45aca405 Code review changes 2014-09-08 17:20:01 -04:00
Joseph Schorr
63628678b8 Instead of sending DB IDs, send "internal IDs" which are DB IDs hashed. This way, we can still calculate the ancestors without hitting the DB further, but without leaking the size of the images table 2014-09-08 15:02:26 -04:00
Joseph Schorr
3c20402b32 Add a common base email template, translate the emails over to using jinja and add emails when e-mail addresses and passwords are changed. 2014-09-05 19:57:33 -04:00
Jake Moshenko
64480fd4ed Merge remote-tracking branch 'origin/master' into yellowalert 
Conflicts:
	data/migrations/versions/82297d834ad_add_us_west_location.py
	test/data/test.db
 2014-09-05 11:30:30 -04:00
Jake Moshenko
3c57e612b3 Merge remote-tracking branch 'origin/zegooglesdosomething' 2014-09-04 20:10:16 -04:00
Joseph Schorr
987177fd7e Have require_fresh_login not apply if there is no password set for the user 2014-09-04 19:47:12 -04:00
Jake Moshenko
1a230f635a Use datetime.min instead of a fixed span for the last login default time. 2014-09-04 19:15:06 -04:00
Joseph Schorr
e028d4ae0a Merge master into branch 2014-09-04 18:08:18 -04:00
Joseph Schorr
1c2de35f28 Code review fixes 2014-09-04 17:54:51 -04:00
Joseph Schorr
e783df31e0 Add the concept of require_fresh_login to both the backend and frontend. Sensitive methods will now be marked with the annotation, which requires that the user has performed a login within 10 minutes or they are asked to do so in the UI before running the operation again. 2014-09-04 14:24:20 -04:00
Joseph Schorr
1e7e012b92 Add a requirement for the current password to change the user's password or email address 2014-09-03 15:41:25 -04:00
Jake Moshenko
2dcdd7ba5b Add exponential backoff of login attempts. 2014-09-02 15:27:05 -04:00
Joseph Schorr
3b72b26836 Merge branch 'master' into comewithmeifyouwanttowork 2014-08-28 20:50:13 -04:00
Joseph Schorr
ae92098b23 Add invite by email (WIP) 2014-08-28 20:49:11 -04:00
Joseph Schorr
7ca853adee Rename the method so it doesn't shadow the model method. 2014-08-28 18:53:04 -04:00
Joseph Schorr
6f1a4030b6 Add response schema validation (only when in TESTING mode) and add one schema. More will be added in a followup CL 2014-08-27 20:57:46 -04:00
Joseph Schorr
6ec89bb179 Add Slack notification support 2014-08-26 22:09:56 -04:00
Joseph Schorr
d76d4704a0 Add pagination to the notifications API and make the UI only show a maximum of 5 notifications (beyond that, it shows "5+"). 2014-08-26 15:19:39 -04:00
Joseph Schorr
a129aac94b Add ability to regenerate robot account credentials 2014-08-25 17:19:23 -04:00
Joseph Schorr
99d75bede7 Handle error cases better for external services 2014-08-25 15:30:29 -04:00
Joseph Schorr
09a1c4d2b5 Add test fix and make sure Quay ups the connection count in its container 2014-08-25 14:23:21 -04:00