Joseph Schorr
6c59161527
Add V2 storage methods to Swift storage engine
...
Fixes #508
2015-09-28 16:46:19 -04:00
Silas Sewell
9000169b53
Revert "Merge pull request #491 from jakedt/migratebackp2"
...
This reverts commit 7ad2522dbe
, reversing
changes made to a0b191ffa1
.
2015-09-28 16:09:22 -04:00
Jimmy Zelinskie
c5aa3ca4f0
make registry v2 tests pass for GCS
...
Fixes #509 .
2015-09-28 15:42:48 -04:00
josephschorr
7ad2522dbe
Merge pull request #491 from jakedt/migratebackp2
...
Migrate image data back phase 2
2015-09-26 15:11:46 -04:00
Joseph Schorr
a283c8d8ec
Add a check to ensure repository names are valid according to an extended set of rules.
...
Fixes #534
2015-09-24 11:55:08 -04:00
Joseph Schorr
40f3b7137d
Fix dict wrapper access to not raise an exception
2015-09-22 14:18:37 -04:00
Joseph Schorr
bf578420f0
Fix import of Github migration
2015-09-21 16:52:56 -04:00
Joseph Schorr
49b575afb6
Start refactoring of the trigger system:
...
- Move each trigger handler into its own file
- Add dictionary helper classes for easier reading and writing of dict-based data
- Extract the web hook payload -> internal representation building for each trigger system
- Add tests for this transformation
- Remove support for Github archived-based building
2015-09-21 16:36:48 -04:00
Joseph Schorr
1c6933a28d
Fix Github build trigger migration
2015-09-19 14:34:46 -04:00
Jake Moshenko
26cea9a07c
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-09-17 16:16:27 -04:00
Jake Moshenko
a887125c3f
Fixes for backfill_aggregate_size script.
2015-09-17 15:47:18 -04:00
Jake Moshenko
8baacd2741
Migrate old data to new locations, read only new.
2015-09-17 15:47:13 -04:00
Joseph Schorr
eff9ff7a66
Migrate all GitHub build triggers to use deploy keys
2015-09-16 17:55:51 -04:00
Joseph Schorr
6f2271d0ae
Add support for direct download in Swift storage engine
...
Fixes #483
2015-09-14 18:00:03 -04:00
josephschorr
57329b6c78
Merge pull request #475 from coreos-inc/seofix
...
Use a proper HTML parser with BS and catch exceptions
2015-09-14 15:56:03 -04:00
Joseph Schorr
6ca33ca108
Use a proper HTML parser with BS and catch exceptions
...
Fixes #473
2015-09-10 16:14:29 -04:00
Jake Moshenko
9c3ddf846f
Some fixes and tests for v2 auth
...
Fixes #395
2015-09-10 15:38:57 -04:00
Joseph Schorr
fd3a21fba9
Add Kubernetes configuration provider which writes config to a secret
...
Fixes #145
2015-09-10 12:19:59 -04:00
Joseph Schorr
88a04441de
Extract the config provider into its own sub-module
2015-09-10 12:19:59 -04:00
Joseph Schorr
c2fe751d15
Despite being disabled, OAuth config is still read, so switch to .get
2015-09-10 12:09:01 -04:00
Joseph Schorr
c0286d1ac3
Add support for Dex to Quay
...
Fixes #306
- Adds support for Dex as an OAuth external login provider
- Adds support for OIDC in general
- Extract out external logins on the JS side into a service
- Add a feature flag for disabling direct login
- Add support for directing to the single external login service
- Does *not* yet support the config in the superuser tool
2015-09-04 17:05:06 -04:00
Jake Moshenko
210ed7cf02
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-09-04 16:32:01 -04:00
Jake Moshenko
82efc746b3
Make our JWT checking more strict.
2015-09-04 15:18:57 -04:00
Jake Moshenko
8269d4ac90
Checkpoint implementing PATCH according to Docker
2015-09-03 16:26:02 -04:00
Joseph Schorr
b7f487da42
Build the OAuth redirect URL ourselves, rather than relying on undocumented Flask behavior
2015-09-02 13:32:11 -04:00
josephschorr
c693afca6a
Merge pull request #426 from coreos-inc/unicodefix
...
Fix Dockerfile parsing for unicode and add testing
2015-08-31 15:03:01 -04:00
Joseph Schorr
fb86b4bf2c
Fix Dockerfile parsing for unicode and add testing
...
Fixes #423
2015-08-31 14:32:26 -04:00
josephschorr
adc66a2894
Merge pull request #422 from coreos-inc/logsgzipfix
...
Change build logs load to using streaming Gzip
2015-08-31 12:15:30 -04:00
Joseph Schorr
c0c1da3232
Change build logs load to using streaming Gzip
2015-08-28 14:08:13 -04:00
Joseph Schorr
43e77a7a14
Add missing tell()
method to GeneratorFile and add tests
2015-08-28 12:10:03 -04:00
Matt Jibson
4aa5ab88dd
Use real cloudwatch limit
...
Although cloudwatch allows 40KB of data, it may be from no more than 20
different metrics. Until we can do that, limit the total points to 20.
2015-08-26 16:48:48 -04:00
Joseph Schorr
84458811d5
Rename wrap_with_hash to a more generic wrap_with_handler
2015-08-25 15:53:13 -04:00
Joseph Schorr
c07dec4d39
File reader fixes for verbs
...
- Fix local file reader to always read in chunks
- Have gzip stream raise an exception if the full data is requested
2015-08-25 13:49:21 -04:00
Joseph Schorr
84276ee945
Better notifications UI
...
Fixes #369
2015-08-17 17:08:58 -04:00
Joseph Schorr
4625ecf273
Fix tests in response to breakage in #351
2015-08-17 16:26:20 -04:00
Matt Jibson
9a7e5bb35e
Batch cloudwatch puts
2015-08-17 12:03:49 -04:00
Jake Moshenko
e1b3e9e6ae
Another huge batch of registry v2 changes
...
Add patch support and resumeable sha
Implement all actual registry methods
Add a simple database generation option
2015-08-12 16:41:12 -04:00
Matt Jibson
7c3b555ee9
Code review
2015-08-12 16:31:01 -04:00
Matt Jibson
f043bc1379
Don't enable the metric queue if there's no Cloudwatch
2015-08-12 15:14:09 -04:00
Matt Jibson
b483209862
Wrap API and registry requests with common metric timings
...
Record response times, codes, and rollup non-2XX responses.
2015-08-12 12:16:00 -04:00
Matt Jibson
b04c190ca0
Prevent the metric queue from growing unbounded
2015-08-12 12:16:00 -04:00
Matt Jibson
cfb6e884f2
Refactor metric collection
...
This change adds a generic queue onto which metrics can be pushed. A
separate module removes metrics from the queue and adds them to Cloudwatch.
Since these are now separate ideas, we can easily change the consumer from
Cloudwatch to anything else.
This change maintains near feature parity (the only change is there is now
just one queue instead of two - not a big deal).
2015-08-12 12:15:52 -04:00
Jake Moshenko
74d838697f
Fix tarfile to support non-unicode pax fields
2015-08-07 11:56:38 -04:00
Jake Moshenko
18100be481
Refactor the util directory to use subpackages.
2015-08-03 16:04:19 -04:00
Joseph Schorr
26ae629189
Prevent local storage setup on non-mounted paths
...
Fixes #269
2015-07-27 14:32:02 -04:00
Joseph Schorr
52d833b3c6
Fix spacing
2015-07-23 16:00:36 -04:00
Joseph Schorr
c3f269ee23
Add migration for BitBucket web hooks
...
This needs to added only *after* we roll out #255
2015-07-23 14:45:12 -04:00
Joseph Schorr
38a6b3621c
Automatically link the superuser account to federated service for auth
...
When the user commits the configuration, if they have chosen a non-DB auth system, we now auto-link the superuser account to that auth system, to ensure they can login again after restart.
2015-07-22 13:37:23 -04:00
Joseph Schorr
33b54218cc
Refactor the users class into their own files, add a common base class for federated users and add a verify_credentials
method which only does the verification, without the linking. We use this in the superuser verification pass
2015-07-20 11:39:59 -04:00
Joseph Schorr
066637f496
Basic Keystone Auth support
...
Note: This has been verified as working by the end customer
2015-07-20 10:55:21 -04:00
Jake Moshenko
bc29561f8f
Fix and templatize the logic for external JWT AuthN and registry v2 Auth.
...
Make it explicit that the registry-v2 stuff is not ready for prime time.
2015-07-17 11:56:15 -04:00
Jake Moshenko
3efaa255e8
Accidental refactor, split out legacy.py into separate sumodules and update all call sites.
2015-07-17 11:56:15 -04:00
Jake Moshenko
bea8b9ac53
More changes for registry-v2 in python.
...
Implement the minimal changes to the local filesystem storage driver and feed them through the distributed storage driver.
Create a digest package which contains digest_tools and checksums.
Fix the tests to use the new v1 endpoint locations.
Fix repository.delete_instance to properly filter the generated queries to avoid most subquery deletes, but still generate them when not explicitly filtered.
2015-07-17 11:50:41 -04:00
Joseph Schorr
4726559322
The database SSL name needs to be in its own list
...
FIxes #243
2015-07-16 00:49:07 +03:00
Joseph Schorr
4333bb9e14
Implement stream_read_file
for the Swift storage engine
...
Note that Swift doesn't seem to have a file-like interface, so we need to wrap the generator we get back from it.
Fixes #210
2015-07-02 17:52:43 +03:00
Jimmy Zelinskie
756d6784ca
Merge pull request #192 from coreos-inc/sqlssl
...
Allow SSL cert for the database to be configured
2015-06-29 13:33:31 -04:00
Joseph Schorr
dc5af7496c
Allow superusers to disable user accounts
2015-06-29 18:40:52 +03:00
Joseph Schorr
bb07d0965f
Allow SSL cert for the database to be configured
...
This change adds a field for the SSL cert for the database in the setup tool. Fixes #89
2015-06-29 08:08:10 +03:00
Joseph Schorr
07439328a4
Remove user_exists
endpoint from all auth systems
2015-06-23 17:33:51 -04:00
Joseph Schorr
331c300893
Refactor JWT auth to not import app locally
2015-06-17 15:53:21 -04:00
Joseph Schorr
e7fa560787
Add support for custom fields in billing invoices
...
Customers (especially in Europe) need the ability to add Tax IDs, VAT IDs, and other custom fields to their invoices.
Fixes #106
2015-06-12 16:45:01 -04:00
Joseph Schorr
90b4f0a2ed
Fix default log archive location for ER
...
Before this change, the ER was using the default of 'local_us' from the base config, which is incorrect, and caused no logs to be archived.
2015-06-11 13:43:29 -04:00
Joseph Schorr
457ee7306e
Parenthesis fix on the JWT auth error message
2015-06-10 16:00:25 -04:00
Jake Moshenko
2a2414d6af
Merge pull request #60 from coreos-inc/jwtauthentication
...
Add support for an external JWT-based authentication system
2015-06-05 13:37:42 -04:00
Joseph Schorr
8aac3fd86e
Add support for an external JWT-based authentication system
...
This authentication system hits two HTTP endpoints to check and verify the existence of users:
Existance endpoint:
GET http://endpoint/ with Authorization: Basic (username:) =>
Returns 200 if the username/email exists, 4** otherwise
Verification endpoint:
GET http://endpoint/ with Authorization: Basic (username:password) =>
Returns 200 and a signed JWT with the user's username and email address if the username+password validates, 4** otherwise with the body containing an optional error message
The JWT produced by the endpoint must be issued with an issuer matching that configured in the config.yaml, and the audience must be "quay.io/jwtauthn". The JWT is signed using a private key and then validated on the Quay.io side with the associated public key, found as "jwt-authn.cert" in the conf/stack directory.
2015-06-05 13:20:10 -04:00
Joseph Schorr
c0e995c1d4
Merge branch 'master' into nolurk
2015-06-02 13:55:16 -04:00
Joseph Schorr
dd28a845db
Fix NPE in cache control decorator
2015-05-28 13:22:42 -04:00
Joseph Schorr
ac239ec4ee
Make sure to only split into two parts max
2015-05-20 14:54:41 -04:00
Joseph Schorr
54992c23b7
Add a feature flag for disabling unauthenticated access to the registry in its entirety.
2015-05-19 17:52:44 -04:00
Joseph Schorr
2379af71f8
Parse the client secret properly
2015-05-18 15:01:37 -04:00
Joseph Schorr
fb8e718c44
Fix OAuth 2 handler to support retrieving parameters from other places; various OAuth client (such as the Go library) send the values in the request body or even the Auth header
2015-05-18 12:38:39 -04:00
Joseph Schorr
4f2a1b3734
Add setup UI for the new trigger types (bitbucket and gitlab) and add validation
2015-05-03 11:50:26 -07:00
Joseph Schorr
d07f9f04e9
UI and code improvements to make working with the multiple SCMs easier
2015-05-03 10:38:11 -07:00
Jimmy Zelinskie
3ac884beb4
gitlab oauth
2015-05-02 17:54:48 -04:00
Joseph Schorr
c480fb2105
Work in progress: bitbucket support
2015-04-24 15:13:08 -04:00
Jimmy Zelinskie
ba2cb08904
Merge branch 'master' into git
2015-04-16 17:38:35 -04:00
Joseph Schorr
3cd11c8f45
GitHub login fixes:
...
- Allow for case insensitivity in the org name list
- Remove the check for verified email addresses when under Enterprise; it isn't supported there.
2015-04-16 12:17:39 -04:00
Joseph Schorr
036c8e56e0
Add proper error handling when the config volume is mounted in a read-only state.
2015-04-02 18:54:09 -04:00
Joseph Schorr
5cd500257d
Merge branch 'master' into orgview
2015-04-01 13:56:49 -04:00
Joseph Schorr
27a9b84587
Switch avatars to be built out of CSS and only overlayed with the gravatar when a non-default exists
2015-03-30 17:55:04 -04:00
Joseph Schorr
aaf1b23e98
Address CL concerns and switch to a real encryption system
2015-03-26 15:10:58 -04:00
Joseph Schorr
85d6500daa
Merge resistanceisfutile into master
2015-03-23 15:39:08 -04:00
Jimmy Zelinskie
f6f93e9079
consolidate everything into one GitHub trigger
2015-03-19 17:12:27 -04:00
Jimmy Zelinskie
5a29218c5c
Merge branch 'master' into git
2015-03-19 12:10:34 -04:00
Jimmy Zelinskie
288f847e9a
util.ssh: reorder return args
2015-03-18 17:32:59 -04:00
Joseph Schorr
b8d88c0f4e
Add aggregate size column and a migration to backfill it
2015-03-16 18:03:17 -04:00
Joseph Schorr
360aa69d92
Fix LDAP error and url handling to be more clear for the end user
2015-03-16 14:33:53 -04:00
Jimmy Zelinskie
c9d955e432
util.ssh: generate ssh key method
2015-03-16 13:37:27 -04:00
Jimmy Zelinskie
47675b88f5
analytics: fix misspelled class name
2015-03-06 12:02:13 -05:00
Joseph Schorr
2e840654d3
PR changes
2015-03-05 12:07:39 -05:00
Joseph Schorr
4f04ad2acd
Change ImageTree to only use a single loop over the images when building. This should be slightly faster on large image sets
2015-03-04 16:53:22 -05:00
Joseph Schorr
4ca5d9b04b
Add support for filtering github login by org
2015-03-03 19:58:42 -05:00
Joseph Schorr
2c662b7861
Make sure to specify a default mail sender when validating emails. Unfortunately for us, flask-mail by default uses the sender from the *global* app instance, rather than the one specified in the Mail(...) call. This was breaking validation.
2015-03-03 13:56:32 -05:00
Jake Moshenko
24ab0ae53a
Fix some problems with off by one in the id condition when deleteing temporary access tokens.
2015-02-20 16:23:36 -05:00
Jake Moshenko
f7b5221391
Merge branch 'master' of github.com:coreos-inc/quay
2015-02-20 16:07:34 -05:00
Jake Moshenko
3bbe064291
Add a script for deleting the old temporary access tokens in small batches.
2015-02-20 16:07:31 -05:00
Jimmy Zelinskie
9c6b029f87
cloudwatch: update docs
2015-02-20 16:07:02 -05:00
Jimmy Zelinskie
47f8cb77c4
Merge pull request #11 from coreos-inc/nimbus
...
CloudWatch for build job status
2015-02-18 17:17:28 -05:00
Jimmy Zelinskie
9ab3554226
buildreporter: does not execute in a coroutine!
2015-02-18 17:11:45 -05:00
Jimmy Zelinskie
0d38e0b00b
metrics: use config['name'] to get metric conf
2015-02-18 16:05:36 -05:00
Jimmy Zelinskie
f53dea46b7
buildman: address PR #11 comments
2015-02-18 14:13:36 -05:00
Joseph Schorr
c69aea1262
Fix invoice address
2015-02-18 11:57:13 -05:00
Jimmy Zelinskie
ef8d320c95
cloudwatch: global before reading queue
...
Technically, it isn't necessary to use the global keyword before reading
a global value, only modifying it. However, in this case it leaves a
pretty annoying log line.
2015-02-17 13:13:12 -05:00
Jimmy Zelinskie
6a1dd376c2
util: add cloudwatch package
...
This isolates the CloudWatch sending thread to it's own package where it
can be shared among any other packages that want to asynchronously send
metrics to CloudWatch.
2015-02-14 16:30:10 -05:00
Jake Moshenko
2ce6e76d9d
Add the required migration for time machine tag lifetimes.
2015-02-13 14:41:08 -05:00
Joseph Schorr
7a199f63eb
Various small fixes and add support for subjectAltName to the SSL cert check
2015-02-12 14:00:26 -05:00
Joseph Schorr
f107b50a46
Merge branch 'master' into ackbar
2015-02-12 12:04:45 -05:00
Joseph Schorr
893ae46dec
Add an ImageTree class and change to searching *all applicable* branches when looking for the best cache tag.
2015-02-10 21:46:58 -05:00
Joseph Schorr
045614c6c8
Merge branch 'master' into ackbar
2015-02-09 17:16:42 -05:00
Joseph Schorr
cf774e23df
Merge branch 'master' into v2
2015-02-05 15:37:14 -05:00
Joseph Schorr
555bd293ea
Fix tar layer format comment
2015-02-05 14:40:02 -05:00
Joseph Schorr
400ffa73e6
Add SSL cert and key validation
2015-02-05 13:06:56 -05:00
Joseph Schorr
bfb0784abc
Add signing to the ACI converter
2015-02-04 15:29:24 -05:00
Joseph Schorr
84e5c0644e
Address comments
2015-02-02 14:07:32 -05:00
Joseph Schorr
30b895b795
Merge branch 'grunt-js-folder' of https://github.com/coreos-inc/quay into ackbar
2015-01-23 17:26:14 -05:00
Joseph Schorr
c8229b9c8a
Implement new step-by-step setup
2015-01-23 17:19:15 -05:00
Joseph Schorr
28d319ad26
Add an in-memory superusermanager, which stores the current list of superusers in a process-shared Value. We do this because in the ER, when we add a new superuser, we need to ensure that ALL workers have their lists updated (otherwise we get the behavior that some workers validate the new permission and others do not).
2015-01-20 12:43:11 -05:00
Joseph Schorr
fa55169c35
- Fix bug in tarlayerformat when dealing with larger headers
...
- Fix bug around entry points and config in the ACI converter
2015-01-16 18:23:40 -05:00
Joseph Schorr
53e5fc6265
Have the config setup tool automatically prepare the S3 or GCS storage with CORS config
2015-01-16 16:10:40 -05:00
Jimmy Zelinskie
0da9c5826b
Update MixPanel and use BufferedConsumer
2015-01-16 16:04:13 -05:00
Jimmy Zelinskie
f5138792ad
Merge branch 'master' of github.com:coreos-inc/quay
2015-01-16 15:32:02 -05:00
Jimmy Zelinskie
fe9e19704b
Try/Catch creating connection to CloudWatch
2015-01-16 15:31:40 -05:00
Jimmy Zelinskie
33088f742a
Migrate queued metric from processes to threads
2015-01-16 15:30:58 -05:00
Joseph Schorr
1f9479a230
Merge branch 'boxer'
2015-01-16 14:57:22 -05:00
Joseph Schorr
e93544573f
Add missing action
2015-01-16 14:57:09 -05:00
Joseph Schorr
5e9d9eb6cd
Merge branch 'master' of https://github.com/coreos-inc/quay
2015-01-16 13:44:32 -05:00
Joseph Schorr
b89ba61286
Change to only run the cloud watch reporter in the gunicorn_web
2015-01-16 13:44:29 -05:00
Jimmy Zelinskie
07bb9be603
Some class docs added to queuemetrics classes
2015-01-16 12:14:57 -05:00
Jimmy Zelinskie
0122a6698f
Wrap call to CloudWatch in try/catch block.
2015-01-16 12:14:11 -05:00
Joseph Schorr
87a5b17d20
Add inbox actions. Note that this isn't enabled because of issues around DKIM in Gmail with SES.
2015-01-15 17:11:05 -05:00
Joseph Schorr
6ed28930b2
Work in progress: Docker -> ACI conversion
2015-01-13 17:46:11 -05:00
Joseph Schorr
0d2c42ad03
Fix tests
2015-01-09 17:11:51 -05:00
Joseph Schorr
6d604a656a
Move config handling into a provider class to make testing much easier
2015-01-09 16:23:31 -05:00
Joseph Schorr
bfd273d16f
- Make validation a bit nicer:
...
- Add timeout to the DB validation
- Make DB validation exception handling a bit nicer
- Move the DB validation error message
- Fix bug around RADOS config default for Is Secure
- Allow hiding of the validation box
2015-01-08 15:27:49 -05:00
Joseph Schorr
47fb10b79f
Merge branch 'master' into ackbar
2015-01-08 13:57:39 -05:00
Joseph Schorr
5ac2c4970a
Add Google auth validation and fix the case where no config is specified at all for Google auth or Github auth
2015-01-08 13:56:17 -05:00
Joseph Schorr
5e0ce4eea9
Add validation of github to the config tool
2015-01-08 13:26:24 -05:00
Joseph Schorr
63504c87fb
Get end-to-end configuration setup working, including verification (except for Github, which is in progress)
2015-01-07 16:20:51 -05:00
Jimmy Zelinskie
88a60d05b6
fix shadowed variable
...
'tag' was being used for both a string value and a boolean value
2015-01-05 14:51:00 -05:00
Joseph Schorr
219730c341
Better config defaults and remove some unneeded code
2015-01-05 13:01:32 -05:00
Joseph Schorr
40d2b1748f
Fix handling of secret key: We now generate it on app startup if it doesn't exist in the config (which it doesn't anymore in the base config.py).
2015-01-05 12:31:02 -05:00
Joseph Schorr
1bf25f25c1
WIP
2015-01-04 14:38:41 -05:00
Joseph Schorr
4ca877c1d4
Add ability to download system logs
2014-12-23 14:01:00 -05:00
Jimmy Zelinskie
6968c148f7
Allow redirects to specific tags
2014-12-18 16:01:59 -05:00
Jimmy Zelinskie
eeeb2e620c
move slackwebhook migration from tools to util
...
tools isn't shipped inside of the container because it contains private
keys
2014-12-18 13:22:13 -05:00
Joseph Schorr
c8277b07d9
Merge branch 'master' of https://bitbucket.org/yackob03/quay
2014-12-01 16:19:24 -05:00
Joseph Schorr
1e993b04ef
Add a time.sleep(0) to the tight loop in gzipstream to make sure we never use 100% of a machine CPU
2014-12-01 16:19:13 -05:00
Jimmy Zelinskie
f3259c862b
Merge branch 'koh'
...
Conflicts:
auth/scopes.py
requirements-nover.txt
requirements.txt
static/css/quay.css
static/directives/namespace-selector.html
static/js/app.js
static/partials/manage-application.html
templates/oauthorize.html
2014-12-01 12:30:09 -08:00
Joseph Schorr
72d613614d
Merge branch 'bagger'
2014-12-01 12:48:59 -05:00
Joseph Schorr
c266f35648
Fix bug in the changes library
2014-11-29 19:00:48 -05:00
Joseph Schorr
9aa62bd92b
Fix dockerfile parsing for unicode
2014-11-28 15:03:04 -05:00
Joseph Schorr
b7a489813a
Fix build system to work with Github Enterprise
2014-11-26 12:37:20 -05:00
Joseph Schorr
3a935822fc
Fix PhantomJS by always using the local copy of CDN files, and making sure to specify TLS (instead of the default SSLv3, which is now deprecated)
2014-11-25 15:32:10 -05:00
Joseph Schorr
e9cac407df
Add a configurable avatar system and add an internal avatar system for enterprise
2014-11-24 19:25:13 -05:00
Jimmy Zelinskie
716d7a737b
Strip whitespace from ALL the things.
2014-11-24 16:07:38 -05:00
Jake Moshenko
e863b96166
Tweak the uuid backfill to leave the uuid column nullable.
2014-11-19 15:32:30 -05:00
Jimmy Zelinskie
606ad21bec
Apply reviewed changes.
...
Adds a length to the UUID field, renames QuayDeferredPermissionUser
parameter id->uuid, adds transactions to backfill script.
2014-11-19 13:28:16 -05:00
Jimmy Zelinskie
10b627c2ad
Add user uuid backfill
2014-11-19 13:28:16 -05:00
Jake Moshenko
17fc72d262
Switch postgres to a non-transactional DDL to allow us to use peewee to modify data in migrations: enterprise customers are running postgres migrations offline already. Move the image backfill script back to a migration since it will now work. Unify the interface to sending a DB URI to env.py for the migration script.
2014-11-18 14:07:33 -05:00
Joseph Schorr
3c990072fd
Only run the uncompressed size backfill for images that have a defined size > 0, but no uncompressed size
2014-11-10 19:41:58 -05:00
Joseph Schorr
611bc895e1
Handle invalid tar layers and add tests
2014-11-10 18:26:37 -05:00
Joseph Schorr
9d1b6d829a
Make sure the external login link for GHE links to the enterprise GitHub and not the hosted version
2014-11-06 20:35:52 -05:00
Joseph Schorr
d7efb7cf7a
Handle missing config in a nicer manner
2014-11-05 17:17:38 -05:00
Joseph Schorr
3e79379942
- Make the OAuth config system centralized
...
- Add support for Github Enterprise login
2014-11-05 16:43:37 -05:00
Joseph Schorr
408fd229a7
Remove the DBURI from the log statement
2014-11-03 21:10:10 -05:00
Joseph Schorr
29c30b336e
Move the uncompressed image size migration call outside of alembic, since it will sometimes deadlock with certain kinds of DBs (because alembic is running things inside a transaction)
2014-11-03 21:01:41 -05:00
Joseph Schorr
8bffbde4ae
Make sure to log a proper exception when an email error occurs
2014-11-03 17:16:36 -05:00
Joseph Schorr
9aa72c5cc2
Fix migration issues:
...
- MySQL 5.5 doesn't support the now() call as a default
- Postgres migration isn't auto-committed, so we have to check if the table exists first
2014-11-03 15:25:55 -05:00
Joseph Schorr
c1398c6d2b
- Add a log entry for repo verb handling and make the container usage calculation take it into account
...
- Move all the repo push/pull/verb logging into a central track_and_log method
- Readd images accidentally deleted in the last CL
- Make the uncompressed size migration script better handle exceptions
2014-10-29 15:42:44 -04:00
Joseph Schorr
93cd7de0e0
Handle email errors in a better manner
2014-10-28 12:10:44 -04:00
Joseph Schorr
5db9cd948b
Add better (jinja-based) messaging to the notifications and add some fixes for the email templates
2014-10-22 19:01:56 -04:00
Joseph Schorr
0ef17b082b
Make sure to disconnect from the database when finished with the processes
2014-10-21 17:40:57 -04:00
Joseph Schorr
47be7cab7a
Compute the tarsum only when required. Newer versions of Docker only require the simple SHA256 checksum, so this should save us from writing to a temp file.
2014-10-20 13:11:33 -04:00
Joseph Schorr
fc09b8ece8
Switch the tar appender to use the first entry's information, not the last
2014-10-17 16:24:02 -04:00
Joseph Schorr
5c4ad31111
Have the tarfileappender also copy the mode and mtime to the new records
2014-10-17 16:22:14 -04:00
Joseph Schorr
a423032e80
- Make sure we log when an exception occurs in the squashing code
...
- Have queue file always try to return any remaining data in the buffer
- Remove the raise of the exception when we get an empty tar file
2014-10-16 12:54:16 -04:00
Joseph Schorr
ee99e68a67
Fix gzip wrap to keep reading from the buffer once the inner loop has completed
2014-10-15 17:24:17 -04:00
Joseph Schorr
e4d9bacccb
Fix other typo
2014-10-15 16:18:34 -04:00
Joseph Schorr
642ebed084
Fix typo
2014-10-15 16:05:51 -04:00
Joseph Schorr
d43109d7cb
- Merge branch 'master' into sha-lom
...
- Extract out the tar handling from streamlayerformat into tarlayerformat
- Add a new tarfileappender class to make it easy to append data to gzipped tars
- Fix the gzipwrap to properly close
- Have the .git injection use the new appender
2014-10-15 15:51:34 -04:00
Joseph Schorr
dba75a08af
Make sure to handle deleted prefixes properly
2014-10-15 11:57:54 -04:00
Joseph Schorr
da28bc4ce9
- Handle missing images properly
...
- Add support for deleting directories
- Add a slew of tests for deletion of directories and other kinds of deletion and layering
2014-10-14 21:40:02 -04:00
Joseph Schorr
eef7edab49
Fix file deletion skipping
2014-10-14 18:48:26 -04:00
Joseph Schorr
efb3c6c494
Make sure we send the extra zeros in chunks
2014-10-14 15:58:52 -04:00
Jake Moshenko
44637dad96
Merge branch 'master' of bitbucket.org:yackob03/quay
2014-10-14 13:58:14 -04:00
Jake Moshenko
328db8b660
Split the app into separate backends, which can use different worker types and different timeouts.
2014-10-14 13:58:08 -04:00
Joseph Schorr
49f8629566
Make the default mail sender use the Flask mail config value
2014-10-10 13:14:33 -04:00
Joseph Schorr
7ef3be5842
Add the exception's message to the traceback
2014-10-08 14:20:57 -04:00
Joseph Schorr
6007789480
Send the full traceback with any exceptions in the queue process
2014-10-08 14:18:41 -04:00
Joseph Schorr
bb9502ee77
Make sure to raise the exception for WSGI and make the delete method's intent more clear
2014-10-08 13:51:50 -04:00
Joseph Schorr
d16fdde528
Fix bug in dockerloadformat and make sure we handle exceptions properly in the verb call
2014-10-08 13:43:12 -04:00
Joseph Schorr
f4daa5e97b
- Update the migrations tool to verify migrations work up and down for both MySQL and PostgresSQL.
...
- Add migrations for the squashed image tables and for backfilling the uncompressed sizes
- Make sure gzip stream uses a max length when determining the uncompressed size
2014-10-07 15:29:56 -04:00
Joseph Schorr
f38ce51943
Merge master into laffa
2014-10-07 14:03:17 -04:00
Jake Moshenko
5c18ffe67d
Allow the namespace column to be null, and also non-unique. Fix the uncompressed size clobbering the size on the wire field. Add metadata constraints so that foreign key constraints get predictable names. Fix all downgrade migrations.
2014-10-02 10:46:20 -04:00
Joseph Schorr
474add0fb1
Have the layer PUT method calculate the uncompressed size in realtime, as trusting the JSON is fraught with complications
2014-09-29 17:00:47 -04:00
Joseph Schorr
3d3f2dd6d7
Remove map call and estimate call
2014-09-29 15:33:26 -04:00
Joseph Schorr
746936ce66
- Make the layer going over the estimated size raise an exception
...
- Add a heuristic for estimating the layer size if it is 0
- Add a method where we can add a custom defined map of image -> size
2014-09-29 12:54:22 -04:00
Joseph Schorr
51beddfef1
Add accidentally removed subscription change email contents
2014-09-23 16:28:06 -04:00
Joseph Schorr
e273dca4b4
Change back to using a docker load format
2014-09-19 12:22:54 -04:00
Joseph Schorr
05bb710830
- Add a shared AUFS utility lib and change both changes and streamlayerformat to use it
...
- Add UI for selecting whether to pull the tag, the repo, or the squashed tag
2014-09-18 15:56:59 -04:00
Joseph Schorr
b212dbb2ab
Merge branch 'master' into better-emails
2014-09-18 13:20:32 -04:00
Joseph Schorr
1cfb6fc353
Have the squashing system write the data (via a double queue system and multiprocessing) to both the client and the stack's storage. On subsequent calls, if the synthetic image exists, it will be returned directly instead of being recomputed
2014-09-16 22:43:19 -04:00
Joseph Schorr
5cca609c55
Switch back to send_file and add a bit of gzip buffering
2014-09-16 14:20:42 -04:00
Joseph Schorr
9344839295
Get squashed endpoint for docker import
working
2014-09-16 11:53:54 -04:00
Joseph Schorr
e3c52fa0eb
Work in progress. This is currently broken!
2014-09-16 00:18:57 -04:00
Jake Moshenko
75d2ef377e
Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork
...
Conflicts:
data/model/legacy.py
2014-09-15 17:52:17 -04:00
Joseph Schorr
69c367514c
Merge branch 'master' of https://bitbucket.org/yackob03/quay
2014-09-12 14:01:17 -04:00
Joseph Schorr
91b8ecfb63
Fix broken regex for Dockerfile parsing
2014-09-12 14:01:10 -04:00
Jake Moshenko
c5ca46a14b
Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork
...
Conflicts:
data/model/legacy.py
static/js/app.js
2014-09-12 11:03:30 -04:00
Joseph Schorr
8d3ce44682
Address comments on code review
2014-09-11 15:45:41 -04:00
Jake Moshenko
451e034ca1
Archived logs commit 1. Squash me.
2014-09-08 16:43:17 -04:00
Jake Moshenko
54fbb2a4c0
Rename collections to morecollections to avoid a conflict with the built in module.
2014-09-08 16:42:43 -04:00
Joseph Schorr
3c20402b32
Add a common base email template, translate the emails over to using jinja and add emails when e-mail addresses and passwords are changed.
2014-09-05 19:57:33 -04:00
Jake Moshenko
c7e873366d
Inject the tables metadata into the upgrade and downgrade functions. Fix a bunch of the downgrades to actually work.
2014-09-04 20:58:29 -04:00
Jake Moshenko
2dcdd7ba5b
Add exponential backoff of login attempts.
2014-09-02 15:27:05 -04:00
Joseph Schorr
3b72b26836
Merge branch 'master' into comewithmeifyouwanttowork
2014-08-28 20:50:13 -04:00
Joseph Schorr
99d75bede7
Handle error cases better for external services
2014-08-25 15:30:29 -04:00
Joseph Schorr
8866b881db
Remove all license code
2014-08-21 17:44:56 -04:00
Joseph Schorr
43b6695f9c
Get team invite confirmation working and fully tested
2014-08-18 17:24:00 -04:00
Joseph Schorr
56d7a3524d
Work in progress: Require invite acceptance to join an org
2014-08-15 17:47:43 -04:00
Jake Moshenko
0372013f70
Merge remote-tracking branch 'origin/redalert'
...
Conflicts:
app.py
2014-08-04 16:56:34 -04:00
Jake Moshenko
0aa6e92b02
Finish porting the workers over to apscheduler 3.0
2014-08-01 18:38:02 -04:00
Jake Moshenko
09917ff062
Switch unidecode over to the new anunidecode library and write some tests to validate results.
2014-08-01 15:50:25 -04:00
Jake Moshenko
6b38ddb9b6
Remove the gpled loremipsum module.
2014-07-31 16:46:02 -04:00
Joseph Schorr
6b85ee3eb6
Make sure all user emails that can be sent in enterprise properly adjust to the app's URL
2014-07-29 13:47:54 -04:00
Joseph Schorr
34fc279092
Add e-mail authorization to the repository notification flow. Also validates the creation of the other notification methods.
2014-07-28 14:58:12 -04:00
Jake Moshenko
7a5605a568
Add an exact abort and use it to send the expected response for 409s.
2014-06-11 16:55:38 -04:00
Jake Moshenko
0ba4201020
Add a module which will create notifications for all users when the license is at its expiration period, and terminate the process when the license expires.
2014-05-29 11:24:10 -04:00
Jake Moshenko
0683f2657e
Rename the email util to not conflict with a builtin library.
2014-05-28 18:22:48 -04:00
Jake Moshenko
0b6552d6cc
Fix the metrics so they are usable for scaling the workers down and up. Switch all datetimes which touch the database from now to utcnow. Fix the worker Dockerfile.
2014-05-23 14:16:26 -04:00
Jake Moshenko
f4c488f9b6
Fix the queue query for old jobs which won't run.
2014-05-22 13:50:06 -04:00
Jake Moshenko
f6726bd0a4
Merge branch 'ldapper'
...
Conflicts:
Dockerfile
app.py
data/database.py
endpoints/index.py
test/data/test.db
2014-05-22 12:13:41 -04:00
Jake Moshenko
d14798de1d
Add a queue capacity reporter plugin to the queue. Move the queue definitions to app. Add a cloudwatch reporter to the dockerfile build queue.
2014-05-21 19:50:37 -04:00
Jake Moshenko
b8466169ac
Integrate sentry with the build worker.
2014-05-19 13:50:45 -04:00
Joseph Schorr
cbcff2adee
Make Phantomjs use a disk cache for the snapshots
2014-05-19 13:18:37 -04:00
Joseph Schorr
1c0c551d00
Add better logging to the snapshot generator for timing purposes and make sure the PhantomJS script always exists after a maximum of 10 seconds.
2014-05-19 13:11:07 -04:00
Jake Moshenko
11c6c5fa52
Merge remote-tracking branch 'origin/master' into ldapper
...
Conflicts:
app.py
2014-05-13 16:55:02 -04:00
Jake Moshenko
5fdccfe3e6
Add an alembic migration for the full initial database with the data. Switch LDAP to using bind and creating a federated login entry. Add LDAP support to the registry and index endpoints. Add a username transliteration and suggestion mechanism. Switch the database and model to require a manual initialization call.
2014-05-13 12:17:26 -04:00
Jake Moshenko
bcb993a914
Set up the build logs to use our fake build logs on test and local.
2014-05-09 18:45:11 -04:00
Jake Moshenko
4556ff52b3
Change the request ids to use urns. Add the logger name to the log formatter.
2014-05-02 13:43:57 -04:00
Jake Moshenko
a5a61576ae
Revamp the logging a bit. Not quite done yet.
2014-05-01 19:44:28 -04:00
Jake Moshenko
450928674b
Use a new caching algorithm which can limit the size for the build nodes. Stop treating public images as special. Add a new phase to the builder for pulling.
2014-04-30 18:48:36 -04:00
Jake Moshenko
fe665118bb
Add sentry exception monitoring.
2014-04-28 18:59:22 -04:00
jakedt
189903ffe9
Send an email automatically when a payment fails.
2014-04-22 13:56:34 -04:00
jakedt
5a2a64074f
Make the command portion of the dockerfile parser case insensitive.
2014-04-17 16:49:04 -04:00
jakedt
a0cbead5aa
Update the subscription change webhook to be more friendly and not send emails for payments.
2014-04-17 16:18:37 -04:00
jakedt
0a9ee6c49f
Bust the dockerfile build cache across repository lines.
2014-04-16 15:45:41 -04:00
jakedt
d95c321e28
Respond to subscription change events so I can stop polling the stripe event list.
2014-04-15 17:00:32 -04:00
jakedt
0827e0fbac
Merge remote-tracking branch 'origin/master' into ncc1701
...
Conflicts:
endpoints/web.py
static/directives/signup-form.html
static/js/app.js
static/js/controllers.js
static/partials/landing.html
static/partials/view-repo.html
test/data/test.db
2014-04-14 19:37:22 -04:00
jakedt
724fec1b74
Test third party repo images for public-ness in the builder. Always clean up private images that we dont know about before build. Pull the base image to refresh before every build.
2014-04-14 18:54:39 -04:00
jakedt
52fdd60779
Merge remote-tracking branch 'origin/detective'
...
Conflicts:
static/partials/repo-admin.html
2014-04-14 16:15:32 -04:00
Joseph Schorr
0e54b0501c
Return the reason a username validation failed and add tests to verify we are sending the reason to client
2014-04-07 20:37:02 -04:00
jakedt
8e9faf6121
Toward running quay in a docker container.
2014-04-07 01:20:09 -04:00
Joseph Schorr
7c466dab7d
- Add an analyze method on triggers that, when given trigger config, will attempt to analyze the trigger's Dockerfile and determine what pull credentials, if any, are needed and available
...
- Move the build trigger setup UI into its own directive (makes things cleaner)
- Fix a bug in the entitySearch directive around setting the current entity
- Change the build trigger setup UI to use the new analyze method and flow better
2014-04-02 23:33:58 -04:00
Joseph Schorr
9a79d1562a
Change to store the pull robot on the repository build and only add the credentials to the queue item. This prevents the credentials from being exposed to the end user. Also fixes the restart build option
2014-04-01 21:49:06 -04:00
jakedt
3525e383df
Merge remote-tracking branch 'origin/master' into pullinprivate
...
Conflicts:
test/data/test.db
2014-04-01 18:28:48 -04:00
jakedt
7c44932c87
Use safer tar extraction. Handle error messages in the build process more intelligently.
2014-04-01 13:46:41 -04:00
jakedt
d67a1cddc2
Merge remote-tracking branch 'origin/master' into pullinprivate
...
Conflicts:
workers/dockerfilebuild.py
2014-03-31 18:10:34 -04:00
Joseph Schorr
2006917e03
Add support for pull credentials on builds and build triggers
2014-03-27 18:33:13 -04:00
jakedt
910fabe103
Disable that pesky browser cache in the ways that matter.
2014-03-26 18:36:59 -04:00
jakedt
5d2274fb05
Add CORS headers to all error responses.
2014-03-25 15:38:31 -04:00
jakedt
092e236694
Write a flask-restful version of cache-control. Remove the comments to add back in post methods.
2014-03-14 18:39:31 -04:00
jakedt
e74eb3ee87
Add scope ordinality and translations. Process oauth tokens and limit scopes accordingly.
2014-03-12 16:31:37 -04:00
Joseph Schorr
61ca29de04
Move the auth context methods into their own file so that we don't have auth trying to import itself
2014-02-25 15:07:24 -05:00
Joseph Schorr
a120f6c64a
Make sure all aborts have message information
2014-02-25 14:15:12 -05:00
yackob03
a6f98570a1
Send everything in production through the gunicorn logger (allows rotation using USR1). Add a gunicorn logger that emits the logstash format. Move the gunicorn config to the conf subdir. Update the postrotate script to hopefully work.
2014-02-04 15:08:49 -05:00
yackob03
0e2adf7a30
Empty tar files should be interepreted as no diffs.
2014-01-31 11:29:55 -05:00
Joseph Schorr
392e0d7c58
Change docs url
2014-01-29 15:38:25 -05:00
Joseph Schorr
3dc3af9eb2
The CLI can be… odd… when handling errors. Make them nicer for the CLI
2014-01-29 14:08:14 -05:00
Joseph Schorr
2b134158f5
Add issue URLs to most errors. The corresponding issue pages will be checked into the public docs repo
2014-01-28 18:29:45 -05:00
Joseph Schorr
c7e616edb9
Add the request URL to the reported error info
2014-01-24 20:40:22 -05:00
Joseph Schorr
aef719c71a
Add the status code and message to the mix panel reported error
2014-01-24 20:37:44 -05:00
Joseph Schorr
b1ec9c4469
Typo fix
2014-01-24 20:34:18 -05:00
Joseph Schorr
c18212f477
Meant update, not copy
2014-01-24 20:33:42 -05:00
Joseph Schorr
9650c1867b
Merge branch 'better-error' of https://bitbucket.org/yackob03/quay into better-error
...
Conflicts:
util/http.py
2014-01-24 20:32:14 -05:00
Joseph Schorr
8d074d8f3a
Add mix panel reporting to http error codes
2014-01-24 20:29:25 -05:00
yackob03
f585430399
Make abort return a json block, which is apparently what the client expects. Remove unused imports. Fix line length and kwarg problems.
2014-01-24 17:00:42 -05:00
Joseph Schorr
0d84cfdf17
Add more default messages
2014-01-24 15:26:32 -05:00
Joseph Schorr
374754c9c9
Handle if message is a non-string
2014-01-24 15:24:26 -05:00
Joseph Schorr
4cea2a6449
Nicer error formatting
2014-01-24 15:17:00 -05:00
Joseph Schorr
229eeec1be
Don’t have a headers var anymore
2014-01-24 15:12:11 -05:00
Joseph Schorr
fd4ac04b5a
HTTPException import
2014-01-24 15:10:51 -05:00
Joseph Schorr
efcbe4421d
Fix logger
2014-01-24 15:09:35 -05:00
Joseph Schorr
77f2706a19
Properly use the auth’ed objects
2014-01-24 15:06:35 -05:00
Joseph Schorr
30a26d099f
Have the index use the same abort method
2014-01-24 15:01:40 -05:00
Joseph Schorr
a363ada41c
Add ability to view and change an account’s email address
2014-01-17 17:04:05 -05:00
yackob03
8e4bbdf1dd
Add no-cache as a response header.
2014-01-02 18:01:34 -05:00
yackob03
52bee66c9f
Update the checksums to a version that supports empty layers.
2013-12-06 14:23:26 -05:00
yackob03
354a03c499
Remove the logger basicConfig from the seo module.
2013-12-02 15:23:15 -05:00
Joseph Schorr
43f2dd80a0
Merge branch 'master' into looksirdroids
2013-11-22 18:22:29 -05:00
yackob03
026ed7ffb4
Start fleshing out support for robots.
2013-11-20 16:13:03 -05:00
yackob03
f5e6af37cf
Fix the nginx config to work with the new snapshots. Fix the snapshot creator to correctly create the request path.
2013-11-18 19:17:58 -05:00
yackob03
6355b4a217
Fix some things with the seo snapshots and use the pep8 style guite.
2013-11-18 18:42:27 -05:00
Joseph Schorr
738973cf39
Add the snapshot endpoint to web.py and have the phantomjs running only load the page's HTML once there are no further pending XHR requests
2013-11-18 17:11:06 -05:00
Joseph Schorr
4c6012f756
Add ability to download receipts in PDF form
2013-11-18 14:49:54 -05:00
Joseph Schorr
6a2b439863
Merge branch 'master' of https://bitbucket.org/yackob03/quay
2013-11-15 15:31:11 -05:00
Joseph Schorr
04780cb3fa
Fix token bug in stripe payments
2013-11-15 15:31:05 -05:00
Joseph Schorr
457b619647
Add receipt/invoice email support and option to Quay
2013-11-15 14:42:31 -05:00
yackob03
96921c2a29
Update the username validation to match the Docker CLI rules.
2013-11-15 12:31:39 -05:00
yackob03
13b457c440
Add caching to the changes api since it is so expensive and can return large results.
2013-10-20 01:18:31 -04:00
yackob03
4679b0dfcc
Fix bugs and clean up data in the way diffs are generated.
2013-10-18 20:19:05 -04:00
yackob03
63ffa52245
Make the diffs absolute and sort them.
2013-10-18 17:05:51 -04:00
yackob03
a1164269be
Complete the diff generating functionality.
2013-10-18 14:31:14 -04:00
yackob03
93b856bdb3
First few changes for the image diffs feature.
2013-10-17 18:25:19 -04:00
yackob03
4c15072c5a
Move signin to use AJAX. Render all flask templates with the common header. Move the header to a partial. Add account recovery.
2013-10-14 17:50:07 -04:00
yackob03
16ee147eae
Add a form for changing the password and prompt the user to do so when there is no password on the account.
2013-10-10 13:44:34 -04:00
yackob03
0c9937ec5e
Forgot the new analytics library.
2013-10-03 16:21:37 -04:00
yackob03
ee41f79bcc
Switch the confirmation email from fluxmonkey to quay.
2013-10-01 15:48:53 -04:00
yackob03
a0adffdc8e
PEP8 fixes.
2013-09-27 20:03:07 -04:00
yackob03
32581c0621
Handle the confirmation codes to validate user emails.
2013-09-27 19:55:04 -04:00
yackob03
99341f7d53
Send a confirmation email when an account is created. Links don't do anything yet.
2013-09-27 19:29:01 -04:00
yackob03
87dc3b6344
Add checks for username and passwords, move checks to model.
2013-09-27 18:38:41 -04:00
yackob03
1574be3c07
Make sure usernames and emails are unique and message is sent to user.
2013-09-27 18:16:26 -04:00
yackob03
730f4f2d66
Update landing page to use gravatar.
2013-09-27 18:15:31 -04:00
yackob03
deee70d53b
Add the registry code.
2013-09-25 17:50:03 -04:00
yackob03
08446ef59e
Fix some stuff with logins and permissions, add tags to the mode.
2013-09-25 16:46:28 -04:00