Joseph Schorr
009c1f7a5f
Have certs_install install all custom certs for requests as well
...
Also supports `extra_ca_certs` being a single file, which is useful for the Kubernetes configmap case
Fixes https://www.pivotaltracker.com/story/show/134302623
2016-11-30 14:04:26 -05:00
Joseph Schorr
5f99448adc
Add a chunk cleanup queue for async GC of empty chunks
...
Instead of having the Swift storage engine try to delete the empty chunk(s) synchronously, we simply queue them and have a worker come along after 30s to delete the empty chunks. This has a few key benefits: it is async (doesn't slow down the push code), helps deal with Swift's eventual consistency (less retries necessary) and is generic for other storage engines if/when they need this as well
2016-11-15 15:07:41 -05:00
Joseph Schorr
5aef4f5ee7
Remove trollies debug now that we have the proper stack traces
2016-11-02 14:42:59 -04:00
Joseph Schorr
72fdf93d29
Add monit-based monitoring of build manager
...
Should catch when the build manager freezes and restart it
2016-11-02 14:14:07 -04:00
Joseph Schorr
854c739417
Enable trollius debug in buildman in prod
2016-10-31 13:37:25 -04:00
Joseph Schorr
460137779f
Switch proxy resolver to use the local resolv.conf values
2016-09-29 11:13:41 +02:00
josephschorr
ad4efba802
Merge pull request #1830 from coreos-inc/superuser-dashboard
...
Add prometheus stats to enable better dashboarding
2016-09-26 17:19:22 +02:00
Joseph Schorr
c7beea2032
Fix handling of custom LDAP cert
...
This change moves the LDAP cert installation into a common script and reorganizes the startup scripts for creating and installing these certs
Fixes #1846
2016-09-19 17:55:08 -04:00
Joseph Schorr
7506471a82
Add missing service def for globalpromstats worker
2016-09-16 16:28:09 -04:00
Jimmy Zelinskie
e54d729a84
init: add logrotate.conf
...
logrotate was broken due to phusion/baseimage-docker#338
This changes logrotate to use the root user which has the proper
permissions on /var/log.
2016-09-08 13:27:37 -04:00
Jimmy Zelinskie
6a681bb748
move nginx
2016-08-10 16:14:54 -04:00
Joseph Schorr
a1009af61c
Move aggregator into its own repo and add it to the image
2016-07-05 15:39:51 -04:00
Jimmy Zelinskie
2b84888c2f
syslog: have syslog generate timestamps ( #1585 )
...
This is the more elegant solution to #1579 .
2016-06-27 14:42:44 -04:00
Jimmy Zelinskie
a40b065bd3
syslog: fix timestamp ( #1579 )
...
Previously the timestamp was locked to the time at which the logger
process started. This change parses messages in bash and then calls the
logger once for each message ignoring newlines (read -r) in order to
guarantee the timestamp is correct.
2016-06-24 15:46:58 -04:00
josephschorr
7173d53030
Merge pull request #1549 from coreos-inc/certs
...
Switch to install custom LDAP cert by name
2016-06-21 15:13:44 -04:00
Joseph Schorr
66ec1d81ce
Switch to install custom LDAP cert by name
2016-06-21 15:10:26 -04:00
Jake Moshenko
a1cf12e460
Add a sitemap.txt for popular public repos
...
and reference it from the robots.txt
2016-06-17 14:34:20 -04:00
Jimmy Zelinskie
d599406140
nginx: use upstream ubuntu package ( #1546 )
...
Ubuntu 16.04 LTS has a newer version than what we compile.
2016-06-16 13:51:04 -04:00
Jimmy Zelinskie
a33a70a419
init: supress sv check
output ( #1545 )
2016-06-15 17:57:27 -04:00
Jimmy Zelinskie
40e3a95868
runit: wait for syslog-ng before starting loggers ( #1537 )
2016-06-10 20:29:45 -04:00
Jimmy Zelinskie
2464e007d8
runit: add dependencies to loggers ( #1515 )
...
This guarantees that the logger starts after syslog and the process it's
logging.
2016-06-03 15:32:15 -04:00
Joseph Schorr
5746b42c69
Add a cleanup worker for the queue item table
...
Fixes #784
2016-06-02 15:00:44 -04:00
Evan Cordell
53ce4de6aa
Merge pull request #1426 from ecordell/wait-for-jwtproxy-config
...
Don't start jwtproxy if conf is not created yet
2016-05-03 13:20:36 -05:00
Evan Cordell
8da0ba37ea
jwtproxy run: sleep between retries
2016-05-03 13:09:34 -05:00
Evan Cordell
ed96c9ec85
Don't print 'waiting' message when jwtproxy is restarting
2016-05-03 10:47:19 -05:00
Evan Cordell
612c546d16
Don't start jwtproxy if conf is not created yet
2016-05-02 17:10:56 -05:00
Jake Moshenko
1dd978aa76
Fix copy pasta
2016-05-02 12:00:26 -04:00
Jake Moshenko
cc8e58e7f4
Split secscan endpoints into a new process
2016-05-02 11:38:00 -04:00
Quentin Machu
1207a71308
Allow adding extra CA certificates to the system
2016-04-29 17:25:45 -04:00
Evan Cordell
a6f6a114c2
service key worker to refresh automatic keys
2016-04-29 14:10:33 -04:00
Evan Cordell
c766727d1d
address review comments
...
- more inline documentation
- don't explicitly specify audience
- approver is optional in `generate_key`
- ADD -> RUN for better caching of jwtproxy
2016-04-29 14:10:33 -04:00
Evan Cordell
9df650688b
Install jwtproxy in /usr/local/bin
2016-04-29 14:10:33 -04:00
Evan Cordell
4d0627f83d
Turn down logging on jwtproxy
2016-04-29 14:10:33 -04:00
Evan Cordell
85667a9cf6
Creat mitm certs on boot
2016-04-29 14:10:33 -04:00
Evan Cordell
118f2d0ce5
Add mitm certs to jwtproxy
2016-04-29 14:10:33 -04:00
Evan Cordell
9e7a501dae
Authenticate in the other direction with jwtproxy
2016-04-29 14:10:33 -04:00
Evan Cordell
8c8ee9c2be
Add jwtproxy and configure verifier for /secscan/notify
2016-04-29 14:10:33 -04:00
Jimmy Zelinskie
e1f955a3f6
add a log rotation worker
...
Fixes #609 .
2015-12-16 17:22:28 -05:00
Joseph Schorr
49ab87bab4
Fix log permissions
2015-11-12 22:45:52 -05:00
Joseph Schorr
7816b0c657
Merge master into vulnerability-tool
2015-11-12 21:52:47 -05:00
Jimmy Zelinskie
5655c08467
fix security worker service permissions
2015-11-10 15:22:36 -05:00
Jimmy Zelinskie
270010105d
add security notification worker to init
2015-11-10 15:22:30 -05:00
Silas Sewell
e826b14ca4
Merge pull request #725 from coreos-inc/setup-tool-georeplication
...
superuser: add storage replication config
2015-11-09 17:43:38 -05:00
Silas Sewell
5000b1621c
superuser: add storage replication config
2015-11-09 17:34:22 -05:00
Quentin Machu
f59e35cc81
Add support for Quay's vulnerability tool
2015-11-06 15:22:18 -05:00
Silas Sewell
49b395ba4e
Disable diffsworker
2015-11-03 23:59:38 -05:00
Silas Sewell
386c017d99
Add quay releases
2015-09-16 17:18:46 -04:00
Quentin Machu
8a4c5a5491
Add newline char in syslog-ng config
2015-09-02 10:07:34 -04:00
josephschorr
62ea4a6cf4
Merge pull request #191 from coreos-inc/carmen
...
Add automatic storage replication
2015-09-01 15:04:36 -04:00
Joseph Schorr
724b1607d7
Add automatic storage replication
...
Adds a worker to automatically replicate data between storages and update the database accordingly
2015-09-01 14:53:32 -04:00
Jake Moshenko
3a0d28653b
Stop logging user and messages files in syslog
...
They contained duplicates of all of our app logs.
2015-09-01 11:44:15 -04:00
Joseph Schorr
5bdd7ba990
Add support for custom favicon in ER
...
Fixes #340
2015-08-10 13:39:39 -04:00
Joseph Schorr
70de107268
Make GC of repositories fully async for whitelisted namespaces
...
This change adds a worker to conduct GC on repositories with garbage every 10s.
Fixes #144
2015-07-28 15:30:04 -04:00
Joseph Schorr
784a45372d
Make the doupdatelimits script optional
...
Without the `privileged` flag or the proper kernel capability, this command can fail the start of the container. With this change, we still print the error message, but don't fail container start. The downside of this command not running is a lower maximum connection count (128), which should be okay for most of our enterprise customers.
2015-07-01 15:13:36 +03:00
Joseph Schorr
191f84fd0b
Add support for custom robots.txt in conf/stack
...
Fixes #115
2015-06-11 12:33:21 -04:00
Joseph Schorr
3f1e8f3c27
Add a RepositoryActionCount table so we can use it (instead of LogEntry) when scoring repo search results
2015-04-13 13:31:07 -04:00
Jake Moshenko
24cf27bd12
Route all of the logging through syslog-ng. Add the ability to specify extra syslog-ng config. Simplify the Dockerfile.
2015-03-26 09:22:47 -04:00
Jake Moshenko
11562a74de
Remove the old builder infrastructure.
2015-01-29 11:03:23 -05:00
Jake Moshenko
2b0fc9087f
Performance improvements meant to help get jobs off of the load balancer more quickly.
2014-12-18 11:53:10 -05:00
Joseph Schorr
72d613614d
Merge branch 'bagger'
2014-12-01 12:48:59 -05:00
Joseph Schorr
279e4b5069
Fix the preplogsdir script to work after a restart of the container
2014-11-29 14:33:55 -05:00
Joseph Schorr
9d675b51ed
- Change SSL to only be enabled via an environment variable. Nginx will be terminating SSL for the ER.
...
- Add the missing dependencies to the requirements.txt
- Change the builder ports to non-standard locations
- Add the /b1/socket and /b1/controller endpoints in nginx, to map to the build manager
- Have the build manager start automatically.
2014-11-25 18:08:18 -05:00
Jake Moshenko
17fc72d262
Switch postgres to a non-transactional DDL to allow us to use peewee to modify data in migrations: enterprise customers are running postgres migrations offline already. Move the image backfill script back to a migration since it will now work. Unify the interface to sending a DB URI to env.py for the migration script.
2014-11-18 14:07:33 -05:00
Joseph Schorr
29c30b336e
Move the uncompressed image size migration call outside of alembic, since it will sometimes deadlock with certain kinds of DBs (because alembic is running things inside a transaction)
2014-11-03 21:01:41 -05:00
Jake Moshenko
328db8b660
Split the app into separate backends, which can use different worker types and different timeouts.
2014-10-14 13:58:08 -04:00
Jake Moshenko
511ee12a58
Redirect stderr to stdout for the buildlogsarchiver.
2014-09-12 14:46:29 -04:00
Jake Moshenko
353da13b16
Set up the service monitoring for the log migration script. Add a database migration to add the logs_archived column.
2014-09-12 11:57:47 -04:00
Joseph Schorr
67905c277e
Remove webhook worker
2014-08-25 19:13:40 -04:00
Joseph Schorr
09a1c4d2b5
Add test fix and make sure Quay ups the connection count in its container
2014-08-25 14:23:21 -04:00
Joseph Schorr
49801bc2c4
- Add web hook queue code back in. We'll remove it and turn it off after this CL goes to prod
...
- Make notification lookup always be by repo and its UUID, rather than the internal DB ID
- Add the init script for the notification worker
2014-07-31 13:30:54 -04:00
Jake Moshenko
870cc9ce0b
Only keep around 4 days worth or 400MB of logs.
2014-06-23 11:25:50 -04:00
Jake Moshenko
f6726bd0a4
Merge branch 'ldapper'
...
Conflicts:
Dockerfile
app.py
data/database.py
endpoints/index.py
test/data/test.db
2014-05-22 12:13:41 -04:00
Jake Moshenko
707bca892e
Rename the nginx configuration to remove the word enterprise
2014-05-18 17:28:51 -04:00
Jake Moshenko
212a4650f4
Rework the config to use runit logging.
2014-05-18 17:19:14 -04:00
Jake Moshenko
cc47e77156
Upgrade to the 0.11.1 tutum version of docker. Package it as a Dockerfile using Docker in Docker. Add a status server option to the workers to utilize the new termination signal and status features of gantry.
2014-05-16 18:31:24 -04:00
Jake Moshenko
f049f738da
Run the db migrations on container start unless we're running against Sqlite.
2014-05-13 15:20:17 -04:00
jakedt
576fbe4f0d
Switch over to phusion baseimage. Prevent everything from daemonizing and start it with runit under phusion. Make workers trap and handle sigint and sigterm. Extend the reservation to 1hr for dockerfilebuild. Update nginx to remove the dependency on libgd. Merge the requirements and requirements enterprise files.
2014-04-11 13:32:45 -04:00