Security scanner flow changes and auto-retry

Changes the security scanner code to raise exceptions now for non-successful operations. One of the new exceptions raised is MissingParentLayerException, which, when raised, will cause the security worker to perform a full rescan of all parent images for the current layer, before trying once more to scan the current layer. This should allow the system to be "self-healing" in the case where the security scanner engine somehow loses or corrupts a parent layer.

workers/securityworker.py

@@ -9,7 +9,7 @@ from data.database import UseThenDisconnect
 from data.model.image import (get_images_eligible_for_scan, get_max_id_for_sec_scan,
                               get_min_id_for_sec_scan, get_image_id)
 from util.secscan.api import SecurityConfigValidator
-from util.secscan.analyzer import LayerAnalyzer
+from util.secscan.analyzer import LayerAnalyzer, PreemptedException
 from util.migrate.allocator import yield_random_entries
 from endpoints.v2 import v2_bp
 
@@ -48,8 +48,9 @@ class SecurityWorker(Worker):
     with UseThenDisconnect(app.config):
       for candidate, abt in yield_random_entries(batch_query, get_image_id(), BATCH_SIZE, max_id,
                                                  self._min_id):
-        _, continue_batch = self._analyzer.analyze_recursively(candidate)
-        if not continue_batch:
+        try:
+          self._analyzer.analyze_recursively(candidate)
+        except PreemptedException:
           logger.info('Another worker pre-empted us for layer: %s', candidate.id)
           abt.set()