Commit graph

262 commits

Author SHA1 Message Date
Evan Cordell
f42200a303 allow empty conf/stack in nginx config generation
needed for QE users that may not have one yet
2017-02-23 18:29:02 -05:00
Evan Cordell
eac9927414 Add FEATURE_SIGNING flag and refactor nginx_conf_create.sh 2017-02-23 14:38:16 -05:00
Evan Cordell
16ec19d356 Add dnsmasq so nginx will allow an upstream service to not block startup 2017-02-23 14:38:16 -05:00
Evan Cordell
9affe193db Add support for tuf metadata endpoints 2017-02-23 14:38:16 -05:00
Joseph Schorr
3d09d64421 Make certs_install not fail if the extra_ca_certs dir is empty
Stupid `cp` will fail if the source dir is empty
2017-01-26 15:17:18 -05:00
Jimmy Zelinskie
64421db0a3 MAINTAINERS: init owners to subpkgs 2017-01-23 17:46:34 -05:00
josephschorr
96b9d702fe Merge pull request #2180 from coreos-inc/requests-ssl
Have certs_install install all custom certs for requests as well
2016-12-05 13:03:54 -05:00
Joseph Schorr
009c1f7a5f Have certs_install install all custom certs for requests as well
Also supports `extra_ca_certs` being a single file, which is useful for the Kubernetes configmap case

Fixes https://www.pivotaltracker.com/story/show/134302623
2016-11-30 14:04:26 -05:00
Jake Moshenko
51ba68d135 Configure nginx to gzip our svg and js files. 2016-11-29 09:30:52 -05:00
Joseph Schorr
2726405ea5 Enable full debuggable logs on non-proxy protocol nginx config
Fixes #2037
2016-11-28 16:29:35 -05:00
Brad Ison
31c6628e74 Don't dump core when killing buildmanager 2016-11-17 14:31:11 -08:00
Brad Ison
ebf80bdd13 Dump core when killing buildmanager from monit 2016-11-17 10:20:03 -08:00
Joseph Schorr
5f99448adc Add a chunk cleanup queue for async GC of empty chunks
Instead of having the Swift storage engine try to delete the empty chunk(s) synchronously, we simply queue them and have a worker come along after 30s to delete the empty chunks. This has a few key benefits: it is async (doesn't slow down the push code), helps deal with Swift's eventual consistency (less retries necessary) and is generic for other storage engines if/when they need this as well
2016-11-15 15:07:41 -05:00
Joseph Schorr
5aef4f5ee7 Remove trollies debug now that we have the proper stack traces 2016-11-02 14:42:59 -04:00
Joseph Schorr
72fdf93d29 Add monit-based monitoring of build manager
Should catch when the build manager freezes and restart it
2016-11-02 14:14:07 -04:00
Joseph Schorr
5109f4a04e Change read timeout on WAMP to 5 min 2016-11-01 16:07:17 -04:00
Joseph Schorr
854c739417 Enable trollius debug in buildman in prod 2016-10-31 13:37:25 -04:00
Joseph Schorr
460137779f Switch proxy resolver to use the local resolv.conf values 2016-09-29 11:13:41 +02:00
Joseph Schorr
dd2e086a20 Add feature flag to force all direct download URLs to be proxied
Fixes #1667
2016-09-29 11:13:41 +02:00
Joseph Schorr
d34650976a Set the proxy_read_timeout for the builder web socket to be much higher
We rarely send data from the build manager to the builder, so this should make sure nginx doesn't accidentally kill the connection

Fixes #1782
2016-09-27 12:37:26 +02:00
josephschorr
ad4efba802 Merge pull request #1830 from coreos-inc/superuser-dashboard
Add prometheus stats to enable better dashboarding
2016-09-26 17:19:22 +02:00
Joseph Schorr
c7beea2032 Fix handling of custom LDAP cert
This change moves the LDAP cert installation into a common script and reorganizes the startup scripts for creating and installing these certs

Fixes #1846
2016-09-19 17:55:08 -04:00
Joseph Schorr
7506471a82 Add missing service def for globalpromstats worker 2016-09-16 16:28:09 -04:00
Jimmy Zelinskie
e54d729a84 init: add logrotate.conf
logrotate was broken due to phusion/baseimage-docker#338
This changes logrotate to use the root user which has the proper
permissions on /var/log.
2016-09-08 13:27:37 -04:00
Jimmy Zelinskie
46e11894d7 nginx: fix paths to stack 2016-08-13 13:53:04 -04:00
Jimmy Zelinskie
6a681bb748 move nginx 2016-08-10 16:14:54 -04:00
Joseph Schorr
a1009af61c Move aggregator into its own repo and add it to the image 2016-07-05 15:39:51 -04:00
Jimmy Zelinskie
2b84888c2f syslog: have syslog generate timestamps (#1585)
This is the more elegant solution to #1579.
2016-06-27 14:42:44 -04:00
Jimmy Zelinskie
a40b065bd3 syslog: fix timestamp (#1579)
Previously the timestamp was locked to the time at which the logger
process started. This change parses messages in bash and then calls the
logger once for each message ignoring newlines (read -r) in order to
guarantee the timestamp is correct.
2016-06-24 15:46:58 -04:00
josephschorr
7173d53030 Merge pull request #1549 from coreos-inc/certs
Switch to install custom LDAP cert by name
2016-06-21 15:13:44 -04:00
Joseph Schorr
66ec1d81ce Switch to install custom LDAP cert by name 2016-06-21 15:10:26 -04:00
Jake Moshenko
a1cf12e460 Add a sitemap.txt for popular public repos
and reference it from the robots.txt
2016-06-17 14:34:20 -04:00
Jimmy Zelinskie
d599406140 nginx: use upstream ubuntu package (#1546)
Ubuntu 16.04 LTS has a newer version than what we compile.
2016-06-16 13:51:04 -04:00
Jimmy Zelinskie
a33a70a419 init: supress sv check output (#1545) 2016-06-15 17:57:27 -04:00
Jake Moshenko
746728ba24 Remove escaped_fragment snapshot rendering. 2016-06-14 12:53:10 -04:00
Jimmy Zelinskie
40e3a95868 runit: wait for syslog-ng before starting loggers (#1537) 2016-06-10 20:29:45 -04:00
Jimmy Zelinskie
2464e007d8 runit: add dependencies to loggers (#1515)
This guarantees that the logger starts after syslog and the process it's
logging.
2016-06-03 15:32:15 -04:00
Joseph Schorr
5746b42c69 Add a cleanup worker for the queue item table
Fixes #784
2016-06-02 15:00:44 -04:00
Jimmy Zelinskie
5568cc77b8 remove all default keys (#1485)
This change:
- Generates a new BitTorrent pepper by default
- Generates a new pagination key by default
- Changes the pagination key format to base64
- Removes selfsigned JWT certs
- Moves test keys to test/data
2016-05-23 16:00:48 -04:00
Jake Moshenko
17536e66dc Change our jwt signing key to actually be self signed. 2016-05-23 15:07:33 -04:00
Joseph Schorr
2cbdecb043 Implement setup tool support for Clair
Fixes #1387
2016-05-04 13:40:50 -04:00
Evan Cordell
53ce4de6aa Merge pull request #1426 from ecordell/wait-for-jwtproxy-config
Don't start jwtproxy if conf is not created yet
2016-05-03 13:20:36 -05:00
Evan Cordell
8da0ba37ea jwtproxy run: sleep between retries 2016-05-03 13:09:34 -05:00
Evan Cordell
ed96c9ec85 Don't print 'waiting' message when jwtproxy is restarting 2016-05-03 10:47:19 -05:00
Evan Cordell
612c546d16 Don't start jwtproxy if conf is not created yet 2016-05-02 17:10:56 -05:00
Jake Moshenko
1dd978aa76 Fix copy pasta 2016-05-02 12:00:26 -04:00
Jake Moshenko
cc8e58e7f4 Split secscan endpoints into a new process 2016-05-02 11:38:00 -04:00
Quentin Machu
1207a71308 Allow adding extra CA certificates to the system 2016-04-29 17:25:45 -04:00
Evan Cordell
a6f6a114c2 service key worker to refresh automatic keys 2016-04-29 14:10:33 -04:00
Evan Cordell
c766727d1d address review comments
- more inline documentation
 - don't explicitly specify audience
 - approver is optional in `generate_key`
 - ADD -> RUN for better caching of jwtproxy
2016-04-29 14:10:33 -04:00
Evan Cordell
9df650688b Install jwtproxy in /usr/local/bin 2016-04-29 14:10:33 -04:00
Evan Cordell
97ad9684d7 Use jwtproxy binary from github 2016-04-29 14:10:33 -04:00
Evan Cordell
4d0627f83d Turn down logging on jwtproxy 2016-04-29 14:10:33 -04:00
Evan Cordell
9ffc32f680 Generate preshared key on boot 2016-04-29 14:10:33 -04:00
Evan Cordell
668ce2c7cd Generate private key on startup 2016-04-29 14:10:33 -04:00
Evan Cordell
85667a9cf6 Creat mitm certs on boot 2016-04-29 14:10:33 -04:00
Evan Cordell
492dcf4781 Verify that jwt was issued by clair 2016-04-29 14:10:33 -04:00
Evan Cordell
118f2d0ce5 Add mitm certs to jwtproxy 2016-04-29 14:10:33 -04:00
Evan Cordell
9e7a501dae Authenticate in the other direction with jwtproxy 2016-04-29 14:10:33 -04:00
Evan Cordell
da0a988650 Configure jwtproxy from stack/conf yaml 2016-04-29 14:10:33 -04:00
Evan Cordell
adc86456b5 Secure the correct endpoint 2016-04-29 14:10:33 -04:00
Evan Cordell
8c8ee9c2be Add jwtproxy and configure verifier for /secscan/notify 2016-04-29 14:10:33 -04:00
Joseph Schorr
1264c6330e Increase read timeout on V2 to match V1
Fixes #1377
2016-04-19 17:52:54 -04:00
Jake Moshenko
0fdbf8a210 Trust upstream proxies to specify https scheme 2016-02-03 13:08:43 -05:00
Joseph Schorr
e7842a2a49 Add 502 page 2016-02-01 15:07:50 +02:00
Jimmy Zelinskie
e1f955a3f6 add a log rotation worker
Fixes #609.
2015-12-16 17:22:28 -05:00
Joseph Schorr
dd344aba81 Add request time and upstream request time to the nginx logs
Fixes #1026
2015-12-16 14:08:07 -05:00
Joseph Schorr
a25572f2b3 Enable HTTP2 under proxy protocol 2015-12-08 15:36:26 -05:00
Joseph Schorr
769ec4c2a3 Enable http2 in nginx 2015-12-04 17:06:55 -05:00
Silas Sewell
8781cf6e11 Increase nginx proxy timeout and close db before storage operation 2015-12-03 11:19:39 -05:00
Jimmy Zelinskie
87a4e1f417 404 on v2 routes for the hostname v1.quay.io
This also copies v2 into its own separate location directive because you
cannot have nested location directives. Also, the `if` directive can be
very tricky and should only be used to return response codes.
2015-11-24 17:02:09 -05:00
Jake Moshenko
4c0e215c2f Silence boto logs when running locally 2015-11-18 19:04:26 -05:00
Jake Moshenko
30bb97a04d Remove the Transfer Encoding directive from v2 headers 2015-11-18 17:23:30 -05:00
Jake Moshenko
d6c5fc5d1b Stop clobbering our proxy_set_header directives 2015-11-18 16:00:23 -05:00
Jake Moshenko
ad273eb002 Re-seed crypto random on all forks 2015-11-17 12:23:10 -05:00
Jake Moshenko
0459c3bc54 Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-11-16 14:22:54 -05:00
Joseph Schorr
49ab87bab4 Fix log permissions 2015-11-12 22:45:52 -05:00
Joseph Schorr
7816b0c657 Merge master into vulnerability-tool 2015-11-12 21:52:47 -05:00
Jake Moshenko
ab340e20ea Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-11-11 16:41:40 -05:00
Jimmy Zelinskie
5655c08467 fix security worker service permissions 2015-11-10 15:22:36 -05:00
Jimmy Zelinskie
270010105d add security notification worker to init 2015-11-10 15:22:30 -05:00
Silas Sewell
e826b14ca4 Merge pull request #725 from coreos-inc/setup-tool-georeplication
superuser: add storage replication config
2015-11-09 17:43:38 -05:00
Silas Sewell
5000b1621c superuser: add storage replication config 2015-11-09 17:34:22 -05:00
Jake Moshenko
c2fcf8bead Merge remote-tracking branch 'upstream/phase4-11-07-2015' into python-registry-v2 2015-11-06 18:18:29 -05:00
Quentin Machu
f59e35cc81 Add support for Quay's vulnerability tool 2015-11-06 15:22:18 -05:00
Quentin Machu
c1fa22d9b0 Define nginx v2 vhost & properly set 404 status code
Fixes #777
2015-11-04 14:56:18 -05:00
Silas Sewell
49b395ba4e Disable diffsworker 2015-11-03 23:59:38 -05:00
Quentin Machu
3f35265858 Merge pull request #683 from Quentin-M/whoops-404
Add 404 page
2015-10-30 14:30:20 -04:00
Jake Moshenko
e7a6176594 Merge remote-tracking branch 'upstream/v2-phase4' into python-registry-v2 2015-10-22 16:59:28 -04:00
Quentin Machu
adb744089e Add 404 page
Fixes coreos-inc/quay#677
2015-10-21 18:40:15 -04:00
Jimmy Zelinskie
069ab0c644 Merge pull request #658 from Quentin-M/nginx_semicolon
Add missing semicolon in nginx conf
2015-10-16 17:25:17 -04:00
Quentin Machu
18a7caf474 Add missing semicolon in nginx conf 2015-10-16 13:55:16 -04:00
Silas Sewell
9c866eac4b nginx: add www redirect
Fixes #452
2015-10-07 11:17:07 -04:00
Joseph Schorr
acac893495 Crypto's Random needs to be reset after forks, otherwise it exceptions 2015-09-28 15:45:01 -04:00
Jake Moshenko
26cea9a07c Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-17 16:16:27 -04:00
Silas Sewell
386c017d99 Add quay releases 2015-09-16 17:18:46 -04:00
Jake Moshenko
210ed7cf02 Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-04 16:32:01 -04:00
Quentin Machu
8a4c5a5491 Add newline char in syslog-ng config 2015-09-02 10:07:34 -04:00
josephschorr
62ea4a6cf4 Merge pull request #191 from coreos-inc/carmen
Add automatic storage replication
2015-09-01 15:04:36 -04:00
Joseph Schorr
724b1607d7 Add automatic storage replication
Adds a worker to automatically replicate data between storages and update the database accordingly
2015-09-01 14:53:32 -04:00