Commit graph

9228 commits

Author SHA1 Message Date
Joseph Schorr
ba69b9c002 Add tests for the logs model interface 2019-02-13 14:45:39 -05:00
Kenny Lee Sin Cheong
035541c6f2 Implement logs model using Elasticsearch
- Implement logs model using Elasticsearch with tests
- Implement transition model using both elasticsearch and database model
- Add LOGS_MODEL configuration to choose which to use.

Co-authored-by: Sida Chen <sidchen@redhat.com>
Co-authored-by: Kenny Lee Sin Cheong <kenny.lee@redhat.com>
2019-02-12 16:33:42 -05:00
Sida Chen
5808a84279
Merge pull request #3358 from KeyboardNerd/fix_license
Fix python license checking and uninstall docutils on build
2019-02-12 13:32:32 -05:00
Sida Chen
8434b65fcb Fix python license checking and uninstall docutils on build 2019-02-12 12:50:35 -05:00
Joseph Schorr
0b706bc7c4 Make sure images in the process of being uploaded are not scanned by Clair 2019-02-08 16:20:44 -05:00
Joseph Schorr
8d722dee81
Merge pull request #3356 from quay/tag-backfill-skip-improvements
Tag backfill improvements
2019-02-07 13:52:32 -05:00
Joseph Schorr
0bfbccdd44 Add a handler for broken tags in the tag backfill system
This will generate a tag pointing to an empty manifest; the tag will be broken, but as it is *already* broken, at least the backfill can complete
2019-02-07 13:36:47 -05:00
Joseph Schorr
bbaee0d3f2 Add additional logging for missing parent images 2019-02-06 17:34:18 -05:00
Joseph Schorr
606b08906c Fix typo 2019-02-06 11:41:50 -05:00
Joseph Schorr
097311f360 Handle additional error cases in the manifest schema 1 implementation 2019-02-06 11:41:50 -05:00
Joseph Schorr
82897a2bba Add a log for an integrity error in writing a manifest 2019-02-06 11:41:50 -05:00
Joseph Schorr
f8dd8b2494 Have tag backfill worker continue processing ranges that have errors 2019-02-06 11:41:50 -05:00
thomasmckay
c8bb6e0ec5
Merge pull request #3355 from thomasmckay/1316-exportactionlogsworker
1316 - exportactionlogsworker period to 60sec
2019-02-06 08:17:48 -05:00
Tom McKay
56ceef823a 1316 - exportactionlogsworker period to 60sec 2019-02-05 16:06:29 -05:00
thomasmckay
39097b2318
Merge pull request #3347 from thomasmckay/1290-nginx-vts
1290 - remove nginx vts
2019-02-05 14:14:23 -05:00
Joseph Schorr
9863930ab7
Merge pull request #3354 from quay/skip-database-check
Skip the database check filtering if not present, such as on warnings
2019-02-04 16:37:52 -05:00
Joseph Schorr
fcfc81806c Skip the database check filtering if not present, such as on warnings 2019-02-04 16:36:02 -05:00
Joseph Schorr
18e5550f74
Merge pull request #3353 from quay/healthcheck-improvements
Improvements to health checks
2019-02-04 16:22:48 -05:00
Tom McKay
b38c31464f 1290 - remove nginx vts 2019-02-04 13:19:48 -05:00
Joseph Schorr
cac5f44d15 Improvements to health checks
- Adds a warning endpoint for warning-only checks
- Changes the default for the disk space check to 1%, instead of 10%
- Removes instance services from the overall health check endpoint
2019-02-04 13:17:59 -05:00
thomasmckay
6661ee8119
Merge pull request #3351 from thomasmckay/1311-remove-monit
1311 - remove buildmanager monit
2019-02-04 11:23:41 -05:00
Tom McKay
1ee2f0e3b2 1311 - remove buildmanager monit 2019-02-01 12:59:18 -05:00
Joseph Schorr
40c0352dd1
Merge pull request #3350 from quay/small-manifest-opt
Small optimization in manifest loading in the OCI data model
2019-01-29 13:16:37 -05:00
Joseph Schorr
0d1ac7a4bb Small optimization in manifest loading in the OCI data model 2019-01-29 13:11:48 -05:00
Joseph Schorr
6c3eab47e0
Merge pull request #3348 from quay/optimize-log-listing
Optimize log listing
2019-01-29 11:25:32 -05:00
Joseph Schorr
ec9913326a Change bulk logs downloader to use datetime ranges as well 2019-01-28 18:15:55 -05:00
Joseph Schorr
d4c74bc1d3 Optimize listing of logs by changing to bucket by datetime, over which we have an index 2019-01-28 17:45:45 -05:00
Joseph Schorr
9f09d68ad8
Merge pull request #3323 from quay/joseph.schorr/QUAY-1282/log-interfacing
Interface out all action log data model operations
2019-01-28 15:09:25 -05:00
Joseph Schorr
9e6f2c5be9 Interface out the clients test in prep for supporting other kinds of clients 2019-01-25 17:28:35 -05:00
Joseph Schorr
b773a18ed8 Interface out all action log data model operations
This will allow us to reimplement the logs data model against a non-database system in the near future
2019-01-25 15:52:22 -05:00
Kenny Lee Sin Cheong
95d92f3a3b
Update dockerclient test boxes (#3316)
* Update boxes Check for vagrant and vagrant-scp

* Email flag is deprecated on after 17.0.6
2019-01-25 13:49:19 -05:00
Kenny Lee Sin Cheong
1b1c9cd1cf
Add V1_ONLY_DOMAIN to internal properties (#3345) 2019-01-23 18:01:15 -05:00
Joseph Schorr
6d7b113ca0
Merge pull request #3344 from quay/tag-backfill-legacy-tags-fix
Fix a bug in the tag backfill that prevented copying lifetime_start_ts when it is zero
2019-01-23 15:31:04 -05:00
Joseph Schorr
4e9949f69d Fix a bug in the tag backfill that prevented copying lifetime_start_ts when it is zero 2019-01-23 15:28:05 -05:00
Joseph Schorr
a4cd7fbc0d
Merge pull request #3343 from quay/fix-backfill-verify-tool
Add a check for missing tag manifests
2019-01-23 15:12:40 -05:00
Joseph Schorr
987ffe5614 Add a check for missing tag manifests
This can occur if a manifest was moved to a different tag
2019-01-23 15:09:17 -05:00
Joseph Schorr
61d00d0c9e
Merge pull request #3342 from quay/backfill-verify-tool
Add a tool for verifying the tag backfill on a namespace
2019-01-23 14:52:05 -05:00
Joseph Schorr
ef6a809ef3 Add a tool for verifying the tag backfill on a namespace 2019-01-23 13:29:31 -05:00
Joseph Schorr
4e4f7ccf88
Merge pull request #3341 from quay/lazy-load-manifests
Switch loading of the manifest contents to be lazy
2019-01-22 17:04:49 -05:00
Joseph Schorr
c0e6660ac9 Switch loading of the manifest contents to be lazy
We don't need the manifest returned by the tags API except for manifest lists, so just load it lazily
2019-01-22 16:50:42 -05:00
Joseph Schorr
110138c305
Merge pull request #3340 from quay/manifest-lookup-optimization
Fix bug where we weren't properly caching enum values and we were always looking up the media type when constructing manifests
2019-01-22 15:58:26 -05:00
Joseph Schorr
bbfb0211dc Fix bug where we weren't properly caching enum values and we were always looking up the media type when constructing manifests 2019-01-22 14:46:38 -05:00
Joseph Schorr
ed75daf3d9
Merge pull request #3339 from quay/fix-backfill-bug
Fix a bug in manifest backfill that made it only work on active tags
2019-01-21 17:06:39 -05:00
Joseph Schorr
0035a1ec6d Fix a bug in manifest backfill that made it only work on active tags 2019-01-21 16:59:37 -05:00
Joseph Schorr
cb637e1233
Merge pull request #3338 from quay/oci-whitelist
Allow namespaces to be whitelisted for OCI, but not V22
2019-01-21 15:13:25 -05:00
Joseph Schorr
7c241a93f6 Allow namespaces to be whitelisted for OCI, but not V22 2019-01-21 14:40:26 -05:00
Joseph Schorr
92808e8d3f
Merge pull request #3337 from quay/additional-validation-test
Add an additional validation test to the tag backfill worker tests
2019-01-21 14:17:47 -05:00
Joseph Schorr
74bc161b2e Add an additional validation test to the tag backfill worker tests 2019-01-21 14:06:30 -05:00
Joseph Schorr
171e2ec39b
Merge pull request #3336 from quay/small-cleanup
Some small cleanup to the tag backfill worker
2019-01-21 13:43:46 -05:00
Joseph Schorr
c1ac12ff8d Some small cleanup to the tag backfill worker 2019-01-21 13:37:11 -05:00