vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
4163 commits 2 branches 62 tags 205 MiB
Commit graph

327 commits

Author SHA1 Message Date
Joseph Schorr
2e840654d3 PR changes 2015-03-05 12:07:39 -05:00
Joseph Schorr
4f04ad2acd Change ImageTree to only use a single loop over the images when building. This should be slightly faster on large image sets 2015-03-04 16:53:22 -05:00
Joseph Schorr
4ca5d9b04b Add support for filtering github login by org 2015-03-03 19:58:42 -05:00
Joseph Schorr
2c662b7861 Make sure to specify a default mail sender when validating emails. Unfortunately for us, flask-mail by default uses the sender from the *global* app instance, rather than the one specified in the Mail(...) call. This was breaking validation. 2015-03-03 13:56:32 -05:00
Jake Moshenko
24ab0ae53a Fix some problems with off by one in the id condition when deleteing temporary access tokens. 2015-02-20 16:23:36 -05:00
Jake Moshenko
f7b5221391 Merge branch 'master' of github.com:coreos-inc/quay 2015-02-20 16:07:34 -05:00
Jake Moshenko
3bbe064291 Add a script for deleting the old temporary access tokens in small batches. 2015-02-20 16:07:31 -05:00
Jimmy Zelinskie
9c6b029f87 cloudwatch: update docs 2015-02-20 16:07:02 -05:00
Jimmy Zelinskie
47f8cb77c4 Merge pull request #11 from coreos-inc/nimbus 
CloudWatch for build job status
 2015-02-18 17:17:28 -05:00
Jimmy Zelinskie
9ab3554226 buildreporter: does not execute in a coroutine! 2015-02-18 17:11:45 -05:00
Jimmy Zelinskie
0d38e0b00b metrics: use config['name'] to get metric conf 2015-02-18 16:05:36 -05:00
Jimmy Zelinskie
f53dea46b7 buildman: address PR #11 comments 2015-02-18 14:13:36 -05:00
Joseph Schorr
c69aea1262 Fix invoice address 2015-02-18 11:57:13 -05:00
Jimmy Zelinskie
ef8d320c95 cloudwatch: global before reading queue 
Technically, it isn't necessary to use the global keyword before reading
a global value, only modifying it. However, in this case it leaves a
pretty annoying log line.
 2015-02-17 13:13:12 -05:00
Jimmy Zelinskie
6a1dd376c2 util: add cloudwatch package 
This isolates the CloudWatch sending thread to it's own package where it
can be shared among any other packages that want to asynchronously send
metrics to CloudWatch.
 2015-02-14 16:30:10 -05:00
Jake Moshenko
2ce6e76d9d Add the required migration for time machine tag lifetimes. 2015-02-13 14:41:08 -05:00
Joseph Schorr
7a199f63eb Various small fixes and add support for subjectAltName to the SSL cert check 2015-02-12 14:00:26 -05:00
Joseph Schorr
f107b50a46 Merge branch 'master' into ackbar 2015-02-12 12:04:45 -05:00
Joseph Schorr
893ae46dec Add an ImageTree class and change to searching *all applicable* branches when looking for the best cache tag. 2015-02-10 21:46:58 -05:00
Joseph Schorr
045614c6c8 Merge branch 'master' into ackbar 2015-02-09 17:16:42 -05:00
Joseph Schorr
cf774e23df Merge branch 'master' into v2 2015-02-05 15:37:14 -05:00
Joseph Schorr
555bd293ea Fix tar layer format comment 2015-02-05 14:40:02 -05:00
Joseph Schorr
400ffa73e6 Add SSL cert and key validation 2015-02-05 13:06:56 -05:00
Joseph Schorr
bfb0784abc Add signing to the ACI converter 2015-02-04 15:29:24 -05:00
Joseph Schorr
84e5c0644e Address comments 2015-02-02 14:07:32 -05:00
Joseph Schorr
30b895b795 Merge branch 'grunt-js-folder' of https://github.com/coreos-inc/quay into ackbar 2015-01-23 17:26:14 -05:00
Joseph Schorr
c8229b9c8a Implement new step-by-step setup 2015-01-23 17:19:15 -05:00
Joseph Schorr
28d319ad26 Add an in-memory superusermanager, which stores the current list of superusers in a process-shared Value. We do this because in the ER, when we add a new superuser, we need to ensure that ALL workers have their lists updated (otherwise we get the behavior that some workers validate the new permission and others do not). 2015-01-20 12:43:11 -05:00
Joseph Schorr
fa55169c35 - Fix bug in tarlayerformat when dealing with larger headers 
- Fix bug around entry points and config in the ACI converter
 2015-01-16 18:23:40 -05:00
Joseph Schorr
53e5fc6265 Have the config setup tool automatically prepare the S3 or GCS storage with CORS config 2015-01-16 16:10:40 -05:00
Jimmy Zelinskie
0da9c5826b Update MixPanel and use BufferedConsumer 2015-01-16 16:04:13 -05:00
Jimmy Zelinskie
f5138792ad Merge branch 'master' of github.com:coreos-inc/quay 2015-01-16 15:32:02 -05:00
Jimmy Zelinskie
fe9e19704b Try/Catch creating connection to CloudWatch 2015-01-16 15:31:40 -05:00
Jimmy Zelinskie
33088f742a Migrate queued metric from processes to threads 2015-01-16 15:30:58 -05:00
Joseph Schorr
1f9479a230 Merge branch 'boxer' 2015-01-16 14:57:22 -05:00
Joseph Schorr
e93544573f Add missing action 2015-01-16 14:57:09 -05:00
Joseph Schorr
5e9d9eb6cd Merge branch 'master' of https://github.com/coreos-inc/quay 2015-01-16 13:44:32 -05:00
Joseph Schorr
b89ba61286 Change to only run the cloud watch reporter in the gunicorn_web 2015-01-16 13:44:29 -05:00
Jimmy Zelinskie
07bb9be603 Some class docs added to queuemetrics classes 2015-01-16 12:14:57 -05:00
Jimmy Zelinskie
0122a6698f Wrap call to CloudWatch in try/catch block. 2015-01-16 12:14:11 -05:00
Joseph Schorr
87a5b17d20 Add inbox actions. Note that this isn't enabled because of issues around DKIM in Gmail with SES. 2015-01-15 17:11:05 -05:00
Joseph Schorr
6ed28930b2 Work in progress: Docker -> ACI conversion 2015-01-13 17:46:11 -05:00
Joseph Schorr
0d2c42ad03 Fix tests 2015-01-09 17:11:51 -05:00
Joseph Schorr
6d604a656a Move config handling into a provider class to make testing much easier 2015-01-09 16:23:31 -05:00
Joseph Schorr
bfd273d16f - Make validation a bit nicer: 
- Add timeout to the DB validation
  - Make DB validation exception handling a bit nicer
  - Move the DB validation error message

- Fix bug around RADOS config default for Is Secure
- Allow hiding of the validation box
 2015-01-08 15:27:49 -05:00
Joseph Schorr
47fb10b79f Merge branch 'master' into ackbar 2015-01-08 13:57:39 -05:00
Joseph Schorr
5ac2c4970a Add Google auth validation and fix the case where no config is specified at all for Google auth or Github auth 2015-01-08 13:56:17 -05:00
Joseph Schorr
5e0ce4eea9 Add validation of github to the config tool 2015-01-08 13:26:24 -05:00
Joseph Schorr
63504c87fb Get end-to-end configuration setup working, including verification (except for Github, which is in progress) 2015-01-07 16:20:51 -05:00
Jimmy Zelinskie
88a60d05b6 fix shadowed variable 
'tag' was being used for both a string value and a boolean value
 2015-01-05 14:51:00 -05:00
Joseph Schorr
219730c341 Better config defaults and remove some unneeded code 2015-01-05 13:01:32 -05:00
Joseph Schorr
40d2b1748f Fix handling of secret key: We now generate it on app startup if it doesn't exist in the config (which it doesn't anymore in the base config.py). 2015-01-05 12:31:02 -05:00
Joseph Schorr
1bf25f25c1 WIP 2015-01-04 14:38:41 -05:00
Joseph Schorr
4ca877c1d4 Add ability to download system logs 2014-12-23 14:01:00 -05:00
Jimmy Zelinskie
6968c148f7 Allow redirects to specific tags 2014-12-18 16:01:59 -05:00
Jimmy Zelinskie
eeeb2e620c move slackwebhook migration from tools to util 
tools isn't shipped inside of the container because it contains private
keys
 2014-12-18 13:22:13 -05:00
Joseph Schorr
c8277b07d9 Merge branch 'master' of https://bitbucket.org/yackob03/quay 2014-12-01 16:19:24 -05:00
Joseph Schorr
1e993b04ef Add a time.sleep(0) to the tight loop in gzipstream to make sure we never use 100% of a machine CPU 2014-12-01 16:19:13 -05:00
Jimmy Zelinskie
f3259c862b Merge branch 'koh' 
Conflicts:
	auth/scopes.py
	requirements-nover.txt
	requirements.txt
	static/css/quay.css
	static/directives/namespace-selector.html
	static/js/app.js
	static/partials/manage-application.html
	templates/oauthorize.html
 2014-12-01 12:30:09 -08:00
Joseph Schorr
72d613614d Merge branch 'bagger' 2014-12-01 12:48:59 -05:00
Joseph Schorr
c266f35648 Fix bug in the changes library 2014-11-29 19:00:48 -05:00
Joseph Schorr
9aa62bd92b Fix dockerfile parsing for unicode 2014-11-28 15:03:04 -05:00
Joseph Schorr
b7a489813a Fix build system to work with Github Enterprise 2014-11-26 12:37:20 -05:00
Joseph Schorr
3a935822fc Fix PhantomJS by always using the local copy of CDN files, and making sure to specify TLS (instead of the default SSLv3, which is now deprecated) 2014-11-25 15:32:10 -05:00
Joseph Schorr
e9cac407df Add a configurable avatar system and add an internal avatar system for enterprise 2014-11-24 19:25:13 -05:00
Jimmy Zelinskie
716d7a737b Strip whitespace from ALL the things. 2014-11-24 16:07:38 -05:00
Jake Moshenko
e863b96166 Tweak the uuid backfill to leave the uuid column nullable. 2014-11-19 15:32:30 -05:00
Jimmy Zelinskie
606ad21bec Apply reviewed changes. 
Adds a length to the UUID field, renames QuayDeferredPermissionUser
parameter id->uuid, adds transactions to backfill script.
 2014-11-19 13:28:16 -05:00
Jimmy Zelinskie
10b627c2ad Add user uuid backfill 2014-11-19 13:28:16 -05:00
Jake Moshenko
17fc72d262 Switch postgres to a non-transactional DDL to allow us to use peewee to modify data in migrations: enterprise customers are running postgres migrations offline already. Move the image backfill script back to a migration since it will now work. Unify the interface to sending a DB URI to env.py for the migration script. 2014-11-18 14:07:33 -05:00
Joseph Schorr
3c990072fd Only run the uncompressed size backfill for images that have a defined size > 0, but no uncompressed size 2014-11-10 19:41:58 -05:00
Joseph Schorr
611bc895e1 Handle invalid tar layers and add tests 2014-11-10 18:26:37 -05:00
Joseph Schorr
9d1b6d829a Make sure the external login link for GHE links to the enterprise GitHub and not the hosted version 2014-11-06 20:35:52 -05:00
Joseph Schorr
d7efb7cf7a Handle missing config in a nicer manner 2014-11-05 17:17:38 -05:00
Joseph Schorr
3e79379942 - Make the OAuth config system centralized 
- Add support for Github Enterprise login
 2014-11-05 16:43:37 -05:00
Joseph Schorr
408fd229a7 Remove the DBURI from the log statement 2014-11-03 21:10:10 -05:00
Joseph Schorr
29c30b336e Move the uncompressed image size migration call outside of alembic, since it will sometimes deadlock with certain kinds of DBs (because alembic is running things inside a transaction) 2014-11-03 21:01:41 -05:00
Joseph Schorr
8bffbde4ae Make sure to log a proper exception when an email error occurs 2014-11-03 17:16:36 -05:00
Joseph Schorr
9aa72c5cc2 Fix migration issues: 
- MySQL 5.5 doesn't support the now() call as a default
  - Postgres migration isn't auto-committed, so we have to check if the table exists first
 2014-11-03 15:25:55 -05:00
Joseph Schorr
c1398c6d2b - Add a log entry for repo verb handling and make the container usage calculation take it into account 
- Move all the repo push/pull/verb logging into a central track_and_log method
- Readd images accidentally deleted in the last CL
- Make the uncompressed size migration script better handle exceptions
 2014-10-29 15:42:44 -04:00
Joseph Schorr
93cd7de0e0 Handle email errors in a better manner 2014-10-28 12:10:44 -04:00
Joseph Schorr
5db9cd948b Add better (jinja-based) messaging to the notifications and add some fixes for the email templates 2014-10-22 19:01:56 -04:00
Joseph Schorr
0ef17b082b Make sure to disconnect from the database when finished with the processes 2014-10-21 17:40:57 -04:00
Joseph Schorr
47be7cab7a Compute the tarsum only when required. Newer versions of Docker only require the simple SHA256 checksum, so this should save us from writing to a temp file. 2014-10-20 13:11:33 -04:00
Joseph Schorr
fc09b8ece8 Switch the tar appender to use the first entry's information, not the last 2014-10-17 16:24:02 -04:00
Joseph Schorr
5c4ad31111 Have the tarfileappender also copy the mode and mtime to the new records 2014-10-17 16:22:14 -04:00
Joseph Schorr
a423032e80 - Make sure we log when an exception occurs in the squashing code 
- Have queue file always try to return any remaining data in the buffer
- Remove the raise of the exception when we get an empty tar file
 2014-10-16 12:54:16 -04:00
Joseph Schorr
ee99e68a67 Fix gzip wrap to keep reading from the buffer once the inner loop has completed 2014-10-15 17:24:17 -04:00
Joseph Schorr
e4d9bacccb Fix other typo 2014-10-15 16:18:34 -04:00
Joseph Schorr
642ebed084 Fix typo 2014-10-15 16:05:51 -04:00
Joseph Schorr
d43109d7cb - Merge branch 'master' into sha-lom 
- Extract out the tar handling from streamlayerformat into tarlayerformat
- Add a new tarfileappender class to make it easy to append data to gzipped tars
- Fix the gzipwrap to properly close
- Have the .git injection use the new appender
 2014-10-15 15:51:34 -04:00
Joseph Schorr
dba75a08af Make sure to handle deleted prefixes properly 2014-10-15 11:57:54 -04:00
Joseph Schorr
da28bc4ce9 - Handle missing images properly 
- Add support for deleting directories
- Add a slew of tests for deletion of directories and other kinds of deletion and layering
 2014-10-14 21:40:02 -04:00
Joseph Schorr
eef7edab49 Fix file deletion skipping 2014-10-14 18:48:26 -04:00
Joseph Schorr
efb3c6c494 Make sure we send the extra zeros in chunks 2014-10-14 15:58:52 -04:00
Jake Moshenko
44637dad96 Merge branch 'master' of bitbucket.org:yackob03/quay 2014-10-14 13:58:14 -04:00
Jake Moshenko
328db8b660 Split the app into separate backends, which can use different worker types and different timeouts. 2014-10-14 13:58:08 -04:00
Joseph Schorr
49f8629566 Make the default mail sender use the Flask mail config value 2014-10-10 13:14:33 -04:00
Joseph Schorr
7ef3be5842 Add the exception's message to the traceback 2014-10-08 14:20:57 -04:00
Joseph Schorr
6007789480 Send the full traceback with any exceptions in the queue process 2014-10-08 14:18:41 -04:00
Joseph Schorr
bb9502ee77 Make sure to raise the exception for WSGI and make the delete method's intent more clear 2014-10-08 13:51:50 -04:00
Joseph Schorr
d16fdde528 Fix bug in dockerloadformat and make sure we handle exceptions properly in the verb call 2014-10-08 13:43:12 -04:00
Joseph Schorr
f4daa5e97b - Update the migrations tool to verify migrations work up and down for both MySQL and PostgresSQL. 
- Add migrations for the squashed image tables and for backfilling the uncompressed sizes
- Make sure gzip stream uses a max length when determining the uncompressed size
 2014-10-07 15:29:56 -04:00
Joseph Schorr
f38ce51943 Merge master into laffa 2014-10-07 14:03:17 -04:00
Jake Moshenko
5c18ffe67d Allow the namespace column to be null, and also non-unique. Fix the uncompressed size clobbering the size on the wire field. Add metadata constraints so that foreign key constraints get predictable names. Fix all downgrade migrations. 2014-10-02 10:46:20 -04:00
Joseph Schorr
474add0fb1 Have the layer PUT method calculate the uncompressed size in realtime, as trusting the JSON is fraught with complications 2014-09-29 17:00:47 -04:00
Joseph Schorr
3d3f2dd6d7 Remove map call and estimate call 2014-09-29 15:33:26 -04:00
Joseph Schorr
746936ce66 - Make the layer going over the estimated size raise an exception 
- Add a heuristic for estimating the layer size if it is 0
- Add a method where we can add a custom defined map of image -> size
 2014-09-29 12:54:22 -04:00
Joseph Schorr
51beddfef1 Add accidentally removed subscription change email contents 2014-09-23 16:28:06 -04:00
Joseph Schorr
e273dca4b4 Change back to using a docker load format 2014-09-19 12:22:54 -04:00
Joseph Schorr
05bb710830 - Add a shared AUFS utility lib and change both changes and streamlayerformat to use it 
- Add UI for selecting whether to pull the tag, the repo, or the squashed tag
 2014-09-18 15:56:59 -04:00
Joseph Schorr
b212dbb2ab Merge branch 'master' into better-emails 2014-09-18 13:20:32 -04:00
Joseph Schorr
1cfb6fc353 Have the squashing system write the data (via a double queue system and multiprocessing) to both the client and the stack's storage. On subsequent calls, if the synthetic image exists, it will be returned directly instead of being recomputed 2014-09-16 22:43:19 -04:00
Joseph Schorr
5cca609c55 Switch back to send_file and add a bit of gzip buffering 2014-09-16 14:20:42 -04:00
Joseph Schorr
9344839295 Get squashed endpoint for docker import working 2014-09-16 11:53:54 -04:00
Joseph Schorr
e3c52fa0eb Work in progress. This is currently broken! 2014-09-16 00:18:57 -04:00
Jake Moshenko
75d2ef377e Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork 
Conflicts:
	data/model/legacy.py
 2014-09-15 17:52:17 -04:00
Joseph Schorr
69c367514c Merge branch 'master' of https://bitbucket.org/yackob03/quay 2014-09-12 14:01:17 -04:00
Joseph Schorr
91b8ecfb63 Fix broken regex for Dockerfile parsing 2014-09-12 14:01:10 -04:00
Jake Moshenko
c5ca46a14b Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork 
Conflicts:
	data/model/legacy.py
	static/js/app.js
 2014-09-12 11:03:30 -04:00
Joseph Schorr
8d3ce44682 Address comments on code review 2014-09-11 15:45:41 -04:00
Jake Moshenko
451e034ca1 Archived logs commit 1. Squash me. 2014-09-08 16:43:17 -04:00
Jake Moshenko
54fbb2a4c0 Rename collections to morecollections to avoid a conflict with the built in module. 2014-09-08 16:42:43 -04:00
Joseph Schorr
3c20402b32 Add a common base email template, translate the emails over to using jinja and add emails when e-mail addresses and passwords are changed. 2014-09-05 19:57:33 -04:00
Jake Moshenko
c7e873366d Inject the tables metadata into the upgrade and downgrade functions. Fix a bunch of the downgrades to actually work. 2014-09-04 20:58:29 -04:00
Jake Moshenko
2dcdd7ba5b Add exponential backoff of login attempts. 2014-09-02 15:27:05 -04:00
Joseph Schorr
3b72b26836 Merge branch 'master' into comewithmeifyouwanttowork 2014-08-28 20:50:13 -04:00
Joseph Schorr
99d75bede7 Handle error cases better for external services 2014-08-25 15:30:29 -04:00
Joseph Schorr
8866b881db Remove all license code 2014-08-21 17:44:56 -04:00
Joseph Schorr
43b6695f9c Get team invite confirmation working and fully tested 2014-08-18 17:24:00 -04:00
Joseph Schorr
56d7a3524d Work in progress: Require invite acceptance to join an org 2014-08-15 17:47:43 -04:00
Jake Moshenko
0372013f70 Merge remote-tracking branch 'origin/redalert' 
Conflicts:
	app.py
 2014-08-04 16:56:34 -04:00
Jake Moshenko
0aa6e92b02 Finish porting the workers over to apscheduler 3.0 2014-08-01 18:38:02 -04:00
Jake Moshenko
09917ff062 Switch unidecode over to the new anunidecode library and write some tests to validate results. 2014-08-01 15:50:25 -04:00
Jake Moshenko
6b38ddb9b6 Remove the gpled loremipsum module. 2014-07-31 16:46:02 -04:00
Joseph Schorr
6b85ee3eb6 Make sure all user emails that can be sent in enterprise properly adjust to the app's URL 2014-07-29 13:47:54 -04:00
Joseph Schorr
34fc279092 Add e-mail authorization to the repository notification flow. Also validates the creation of the other notification methods. 2014-07-28 14:58:12 -04:00
Jake Moshenko
7a5605a568 Add an exact abort and use it to send the expected response for 409s. 2014-06-11 16:55:38 -04:00
Jake Moshenko
0ba4201020 Add a module which will create notifications for all users when the license is at its expiration period, and terminate the process when the license expires. 2014-05-29 11:24:10 -04:00
Jake Moshenko
0683f2657e Rename the email util to not conflict with a builtin library. 2014-05-28 18:22:48 -04:00
Jake Moshenko
0b6552d6cc Fix the metrics so they are usable for scaling the workers down and up. Switch all datetimes which touch the database from now to utcnow. Fix the worker Dockerfile. 2014-05-23 14:16:26 -04:00
Jake Moshenko
f4c488f9b6 Fix the queue query for old jobs which won't run. 2014-05-22 13:50:06 -04:00
Jake Moshenko
f6726bd0a4 Merge branch 'ldapper' 
Conflicts:
	Dockerfile
	app.py
	data/database.py
	endpoints/index.py
	test/data/test.db
 2014-05-22 12:13:41 -04:00
Jake Moshenko
d14798de1d Add a queue capacity reporter plugin to the queue. Move the queue definitions to app. Add a cloudwatch reporter to the dockerfile build queue. 2014-05-21 19:50:37 -04:00
Jake Moshenko
b8466169ac Integrate sentry with the build worker. 2014-05-19 13:50:45 -04:00
Joseph Schorr
cbcff2adee Make Phantomjs use a disk cache for the snapshots 2014-05-19 13:18:37 -04:00
Joseph Schorr
1c0c551d00 Add better logging to the snapshot generator for timing purposes and make sure the PhantomJS script always exists after a maximum of 10 seconds. 2014-05-19 13:11:07 -04:00
Jake Moshenko
11c6c5fa52 Merge remote-tracking branch 'origin/master' into ldapper 
Conflicts:
	app.py
 2014-05-13 16:55:02 -04:00
Jake Moshenko
5fdccfe3e6 Add an alembic migration for the full initial database with the data. Switch LDAP to using bind and creating a federated login entry. Add LDAP support to the registry and index endpoints. Add a username transliteration and suggestion mechanism. Switch the database and model to require a manual initialization call. 2014-05-13 12:17:26 -04:00
Jake Moshenko
bcb993a914 Set up the build logs to use our fake build logs on test and local. 2014-05-09 18:45:11 -04:00