Add a chunk cleanup queue for async GC of empty chunks
Instead of having the Swift storage engine try to delete the empty chunk(s) synchronously, we simply queue them and have a worker come along after 30s to delete the empty chunks. This has a few key benefits: it is async (doesn't slow down the push code), helps deal with Swift's eventual consistency (less retries necessary) and is generic for other storage engines if/when they need this as well
This commit is contained in:
parent
59cb6bd216
commit
5f99448adc
12 changed files with 191 additions and 59 deletions
|
@ -14,7 +14,7 @@ logger = logging.getLogger(__name__)
|
|||
|
||||
|
||||
class LocalStorage(BaseStorageV2):
|
||||
def __init__(self, metric_queue, storage_path):
|
||||
def __init__(self, context, storage_path):
|
||||
super(LocalStorage, self).__init__()
|
||||
self._root_path = storage_path
|
||||
|
||||
|
|
Reference in a new issue