Commit graph

297 commits

Author SHA1 Message Date
Joseph Schorr
f79542fefb Enable connection pooling in the registry 2017-07-27 14:00:23 -04:00
josephschorr
78652de3ee Merge pull request #2766 from coreos-inc/joseph.schorr/QUAY-634/buildlogsarchiver-data-interface
Change buildlogsarchiver to use a data model interface
2017-07-19 16:40:05 -04:00
josephschorr
9bd4cee029 Merge pull request #2765 from coreos-inc/joseph.schorr/QUAY-629/globalprom-data-interface
Switch globalpromstats worker to use a data interface
2017-07-19 16:39:36 -04:00
Erica
6576965647 Merge pull request #2780 from coreos-inc/FIX-teamsync-logger
fix(init/service/teamsyncworker/log/run): log correct worker
2017-07-12 23:38:44 -04:00
josephschorr
fdb21aa5dc Merge pull request #2777 from coreos-inc/joseph.schorr/QUAY-618/notificationworker-data-interface
Change notificationworker to use data interface
2017-07-13 00:23:15 +03:00
josephschorr
2206c81a95 Merge pull request #2776 from coreos-inc/joseph.schorr/QUAY-652/servicekeyworker-data-interface
Change service key worker to use a data interface
2017-07-13 00:22:49 +03:00
EvB
67abfe7483 fix(init/service/teamsyncworker/log/run): log correct worker 2017-07-12 13:52:22 -04:00
Joseph Schorr
fbfd78532c Move notification worker to its own package 2017-07-12 17:35:09 +03:00
Joseph Schorr
3b496e2759 Move serverkeyworker into its own package 2017-07-12 15:57:02 +03:00
Joseph Schorr
e2cf2d6f2b Move teamsyncworker into its own package 2017-07-12 15:53:01 +03:00
josephschorr
dc6c6b30fc Merge pull request #2768 from coreos-inc/joseph.schorr/QUAY-653/blobuploadcleanupworker-data-interface
Change blobuploadcleanupworker to use a data interface
2017-07-12 00:32:09 +03:00
Evan Cordell
8d07bbc7af Remove volumes 2017-07-11 10:54:56 -04:00
Joseph Schorr
b87415129f Move blobuploadcleanupworker into its own package 2017-07-11 15:38:10 +03:00
Joseph Schorr
22f088d90a Move buildlogsarchiver worker to its own package 2017-07-11 14:42:18 +03:00
Joseph Schorr
265520d071 Move globalpromstats worker into its own package 2017-07-11 13:52:15 +03:00
Evan Cordell
dacb0131a5 symlink all files from /conf/stack into QUAYCONF/stack 2017-07-10 22:34:21 -04:00
Antoine Legrand
cdb3722c17 Use $QUAYPATH and $QUAYDIR in conf and init files 2017-07-05 16:23:54 +02:00
josephschorr
a96555511b Merge pull request #2743 from coreos-inc/joseph.schorr/QUAY-663/gcworker-interface
Change GC worker to use new data interface style
2017-06-29 20:54:04 +03:00
Joseph Schorr
76c9339453 Rename GC worker package to gc 2017-06-29 09:37:32 +03:00
Joseph Schorr
38f1752a2d Move gcworker into its own package 2017-06-28 15:04:10 +03:00
Joseph Schorr
1ddb09ac11 Change security worker to use data interface 2017-06-28 14:50:52 +03:00
Antoine Legrand
3bd2148abd gunicorn-conf: add quay directory to syspath 2017-06-08 20:50:42 +02:00
Antoine Legrand
3c99928a27 Add log JSON formatter 2017-06-07 00:02:52 +02:00
Evan Cordell
653cd997a1 fixes install of certs 2017-05-25 18:06:20 -04:00
Evan Cordell
20da91d879 Add tests for providers and update install script 2017-05-23 15:43:21 -04:00
Evan Cordell
0c05958739 Update cert install scripts to read prefixed names 2017-05-23 13:59:09 -04:00
Joseph Schorr
bf51ec20e8 Disable gzip on HEAD requests in v2 endpoints
nginx's gzip module will ignore the content-length header on the HEAD request and try to gzip the body.... but there is no body, so it simply writes no header at all.

Code to turn this off was based off of https://trac.nginx.org/nginx/ticket/261
2017-05-03 18:27:45 -04:00
Joseph Schorr
c22dc5ebae Add missing team sync worker service 2017-04-10 20:11:57 -04:00
Joseph Schorr
da8032fe61 Fix SSL custom certs installation file for bash shell scripting bug
The missing quotes caused the script to fail with a bash error
2017-03-24 16:39:28 -04:00
Jimmy Zelinskie
f6a785c1b5 conf/nginx: add cnr path 2017-03-23 13:06:22 -04:00
Joseph Schorr
dd35677712 Add configurable maximum layer size in nginx 2017-03-21 13:14:11 -04:00
Joseph Schorr
e25c989fef Add a cleanup worker for blob uploads 2017-03-16 13:36:59 -04:00
Jimmy Zelinskie
850c32ebfb Merge pull request #2298 from jzelinskie/maintainers
MAINTAINERS: init owners to subpkgs
2017-03-09 17:30:38 -05:00
Evan Cordell
41033ae05d fix typo 2017-02-23 19:03:26 -05:00
Evan Cordell
ecd441269b Pass host to apostille (required for k8s ingress) 2017-02-23 18:29:02 -05:00
Evan Cordell
f42200a303 allow empty conf/stack in nginx config generation
needed for QE users that may not have one yet
2017-02-23 18:29:02 -05:00
Evan Cordell
eac9927414 Add FEATURE_SIGNING flag and refactor nginx_conf_create.sh 2017-02-23 14:38:16 -05:00
Evan Cordell
16ec19d356 Add dnsmasq so nginx will allow an upstream service to not block startup 2017-02-23 14:38:16 -05:00
Evan Cordell
9affe193db Add support for tuf metadata endpoints 2017-02-23 14:38:16 -05:00
Joseph Schorr
3d09d64421 Make certs_install not fail if the extra_ca_certs dir is empty
Stupid `cp` will fail if the source dir is empty
2017-01-26 15:17:18 -05:00
Jimmy Zelinskie
64421db0a3 MAINTAINERS: init owners to subpkgs 2017-01-23 17:46:34 -05:00
josephschorr
96b9d702fe Merge pull request #2180 from coreos-inc/requests-ssl
Have certs_install install all custom certs for requests as well
2016-12-05 13:03:54 -05:00
Joseph Schorr
009c1f7a5f Have certs_install install all custom certs for requests as well
Also supports `extra_ca_certs` being a single file, which is useful for the Kubernetes configmap case

Fixes https://www.pivotaltracker.com/story/show/134302623
2016-11-30 14:04:26 -05:00
Jake Moshenko
51ba68d135 Configure nginx to gzip our svg and js files. 2016-11-29 09:30:52 -05:00
Joseph Schorr
2726405ea5 Enable full debuggable logs on non-proxy protocol nginx config
Fixes #2037
2016-11-28 16:29:35 -05:00
Brad Ison
31c6628e74 Don't dump core when killing buildmanager 2016-11-17 14:31:11 -08:00
Brad Ison
ebf80bdd13 Dump core when killing buildmanager from monit 2016-11-17 10:20:03 -08:00
Joseph Schorr
5f99448adc Add a chunk cleanup queue for async GC of empty chunks
Instead of having the Swift storage engine try to delete the empty chunk(s) synchronously, we simply queue them and have a worker come along after 30s to delete the empty chunks. This has a few key benefits: it is async (doesn't slow down the push code), helps deal with Swift's eventual consistency (less retries necessary) and is generic for other storage engines if/when they need this as well
2016-11-15 15:07:41 -05:00
Joseph Schorr
5aef4f5ee7 Remove trollies debug now that we have the proper stack traces 2016-11-02 14:42:59 -04:00
Joseph Schorr
72fdf93d29 Add monit-based monitoring of build manager
Should catch when the build manager freezes and restart it
2016-11-02 14:14:07 -04:00
Joseph Schorr
5109f4a04e Change read timeout on WAMP to 5 min 2016-11-01 16:07:17 -04:00
Joseph Schorr
854c739417 Enable trollius debug in buildman in prod 2016-10-31 13:37:25 -04:00
Joseph Schorr
460137779f Switch proxy resolver to use the local resolv.conf values 2016-09-29 11:13:41 +02:00
Joseph Schorr
dd2e086a20 Add feature flag to force all direct download URLs to be proxied
Fixes #1667
2016-09-29 11:13:41 +02:00
Joseph Schorr
d34650976a Set the proxy_read_timeout for the builder web socket to be much higher
We rarely send data from the build manager to the builder, so this should make sure nginx doesn't accidentally kill the connection

Fixes #1782
2016-09-27 12:37:26 +02:00
josephschorr
ad4efba802 Merge pull request #1830 from coreos-inc/superuser-dashboard
Add prometheus stats to enable better dashboarding
2016-09-26 17:19:22 +02:00
Joseph Schorr
c7beea2032 Fix handling of custom LDAP cert
This change moves the LDAP cert installation into a common script and reorganizes the startup scripts for creating and installing these certs

Fixes #1846
2016-09-19 17:55:08 -04:00
Joseph Schorr
7506471a82 Add missing service def for globalpromstats worker 2016-09-16 16:28:09 -04:00
Jimmy Zelinskie
e54d729a84 init: add logrotate.conf
logrotate was broken due to phusion/baseimage-docker#338
This changes logrotate to use the root user which has the proper
permissions on /var/log.
2016-09-08 13:27:37 -04:00
Jimmy Zelinskie
46e11894d7 nginx: fix paths to stack 2016-08-13 13:53:04 -04:00
Jimmy Zelinskie
6a681bb748 move nginx 2016-08-10 16:14:54 -04:00
Joseph Schorr
a1009af61c Move aggregator into its own repo and add it to the image 2016-07-05 15:39:51 -04:00
Jimmy Zelinskie
2b84888c2f syslog: have syslog generate timestamps (#1585)
This is the more elegant solution to #1579.
2016-06-27 14:42:44 -04:00
Jimmy Zelinskie
a40b065bd3 syslog: fix timestamp (#1579)
Previously the timestamp was locked to the time at which the logger
process started. This change parses messages in bash and then calls the
logger once for each message ignoring newlines (read -r) in order to
guarantee the timestamp is correct.
2016-06-24 15:46:58 -04:00
josephschorr
7173d53030 Merge pull request #1549 from coreos-inc/certs
Switch to install custom LDAP cert by name
2016-06-21 15:13:44 -04:00
Joseph Schorr
66ec1d81ce Switch to install custom LDAP cert by name 2016-06-21 15:10:26 -04:00
Jake Moshenko
a1cf12e460 Add a sitemap.txt for popular public repos
and reference it from the robots.txt
2016-06-17 14:34:20 -04:00
Jimmy Zelinskie
d599406140 nginx: use upstream ubuntu package (#1546)
Ubuntu 16.04 LTS has a newer version than what we compile.
2016-06-16 13:51:04 -04:00
Jimmy Zelinskie
a33a70a419 init: supress sv check output (#1545) 2016-06-15 17:57:27 -04:00
Jake Moshenko
746728ba24 Remove escaped_fragment snapshot rendering. 2016-06-14 12:53:10 -04:00
Jimmy Zelinskie
40e3a95868 runit: wait for syslog-ng before starting loggers (#1537) 2016-06-10 20:29:45 -04:00
Jimmy Zelinskie
2464e007d8 runit: add dependencies to loggers (#1515)
This guarantees that the logger starts after syslog and the process it's
logging.
2016-06-03 15:32:15 -04:00
Joseph Schorr
5746b42c69 Add a cleanup worker for the queue item table
Fixes #784
2016-06-02 15:00:44 -04:00
Jimmy Zelinskie
5568cc77b8 remove all default keys (#1485)
This change:
- Generates a new BitTorrent pepper by default
- Generates a new pagination key by default
- Changes the pagination key format to base64
- Removes selfsigned JWT certs
- Moves test keys to test/data
2016-05-23 16:00:48 -04:00
Jake Moshenko
17536e66dc Change our jwt signing key to actually be self signed. 2016-05-23 15:07:33 -04:00
Joseph Schorr
2cbdecb043 Implement setup tool support for Clair
Fixes #1387
2016-05-04 13:40:50 -04:00
Evan Cordell
53ce4de6aa Merge pull request #1426 from ecordell/wait-for-jwtproxy-config
Don't start jwtproxy if conf is not created yet
2016-05-03 13:20:36 -05:00
Evan Cordell
8da0ba37ea jwtproxy run: sleep between retries 2016-05-03 13:09:34 -05:00
Evan Cordell
ed96c9ec85 Don't print 'waiting' message when jwtproxy is restarting 2016-05-03 10:47:19 -05:00
Evan Cordell
612c546d16 Don't start jwtproxy if conf is not created yet 2016-05-02 17:10:56 -05:00
Jake Moshenko
1dd978aa76 Fix copy pasta 2016-05-02 12:00:26 -04:00
Jake Moshenko
cc8e58e7f4 Split secscan endpoints into a new process 2016-05-02 11:38:00 -04:00
Quentin Machu
1207a71308 Allow adding extra CA certificates to the system 2016-04-29 17:25:45 -04:00
Evan Cordell
a6f6a114c2 service key worker to refresh automatic keys 2016-04-29 14:10:33 -04:00
Evan Cordell
c766727d1d address review comments
- more inline documentation
 - don't explicitly specify audience
 - approver is optional in `generate_key`
 - ADD -> RUN for better caching of jwtproxy
2016-04-29 14:10:33 -04:00
Evan Cordell
9df650688b Install jwtproxy in /usr/local/bin 2016-04-29 14:10:33 -04:00
Evan Cordell
97ad9684d7 Use jwtproxy binary from github 2016-04-29 14:10:33 -04:00
Evan Cordell
4d0627f83d Turn down logging on jwtproxy 2016-04-29 14:10:33 -04:00
Evan Cordell
9ffc32f680 Generate preshared key on boot 2016-04-29 14:10:33 -04:00
Evan Cordell
668ce2c7cd Generate private key on startup 2016-04-29 14:10:33 -04:00
Evan Cordell
85667a9cf6 Creat mitm certs on boot 2016-04-29 14:10:33 -04:00
Evan Cordell
492dcf4781 Verify that jwt was issued by clair 2016-04-29 14:10:33 -04:00
Evan Cordell
118f2d0ce5 Add mitm certs to jwtproxy 2016-04-29 14:10:33 -04:00
Evan Cordell
9e7a501dae Authenticate in the other direction with jwtproxy 2016-04-29 14:10:33 -04:00
Evan Cordell
da0a988650 Configure jwtproxy from stack/conf yaml 2016-04-29 14:10:33 -04:00
Evan Cordell
adc86456b5 Secure the correct endpoint 2016-04-29 14:10:33 -04:00
Evan Cordell
8c8ee9c2be Add jwtproxy and configure verifier for /secscan/notify 2016-04-29 14:10:33 -04:00
Joseph Schorr
1264c6330e Increase read timeout on V2 to match V1
Fixes #1377
2016-04-19 17:52:54 -04:00
Jake Moshenko
0fdbf8a210 Trust upstream proxies to specify https scheme 2016-02-03 13:08:43 -05:00
Joseph Schorr
e7842a2a49 Add 502 page 2016-02-01 15:07:50 +02:00