Commit graph

703 commits

Author SHA1 Message Date
Troels Thomsen
28be207bc0 Pass through known errors
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2016-04-29 23:34:24 +02:00
Richard Scothern
c0dcad3a48 Merge pull request #1677 from RichardScothern/tonyhb-fix-s3-gc-error
Move GC into storage package and add tests
2016-04-28 14:09:58 -07:00
Richard Scothern
ea4f065672 Merge pull request #1665 from andrewhsu/middleware-redirect
add middleware storage driver for redirect
2016-04-27 15:05:52 -07:00
Richard Scothern
63d28d3b81 Add a test with a missing _manifests directory
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-04-27 13:34:25 -07:00
Richard Scothern
3a034b477e Move garbage collect code into storage package
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-04-27 13:34:25 -07:00
Tony Holdstock-Brown
898fdb48a1 Ensure GC continues marking if _manifests is nonexistent
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-04-27 13:33:36 -07:00
Richard Scothern
cd4f752e49 Merge pull request #1675 from sergeyfd/master
Fix wording for dry-run flag in usage message for garbage collector.
2016-04-27 10:08:53 -07:00
Serge Dubrouski
8775da93d6 Fix wording for dry-run flag in useage message for garbage collector.
Signed-off-by: Serge Dubrouski <sergeyfd@gmail.com>
2016-04-26 19:44:23 -06:00
Andrew Hsu
54edbdfee6 separate the go/non-go imports and reorder
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-26 14:33:54 -07:00
Richard Scothern
51972682c7 Merge pull request #1660 from jhaohai/cn-north-1-fix
Add cn-north-1 to valid check
2016-04-25 16:07:54 -07:00
Richard Scothern
f91dc3ca51 Merge pull request #1666 from sergeyfd/master
Add blobWrtiter.Close() call into blobWriter.Commit()
2016-04-25 16:02:48 -07:00
Richard Scothern
e6ad3a227a Merge pull request #1670 from vadmeste/fix_parts_sorting
s3 driver: Sorting completed parts by part number for a better accordance with S3 spec
2016-04-25 16:02:17 -07:00
Andrew Hsu
3336cc13e4 modify redirect test to include port
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:46 -07:00
Andrew Hsu
fba2e3a206 scheme and host mandatory in baseurl
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:25 -07:00
Andrew Hsu
cec7248bd1 separate the go/non-go imports and reorder
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:03 -07:00
Anis Elleuch
d11a979591 Sorting completed parts by part number for a better accordance with the S3 spec
Signed-off-by: Anis Elleuch <vadmeste@gmail.com>
2016-04-23 22:36:04 +01:00
Serge Dubrouski
6615b77a09 Add blobWrtiter.Close() call into blobWriter.Commit()
Signed-off-by: Serge Dubrouski <sergeyfd@gmail.com>
2016-04-22 19:23:17 -06:00
Andrew Hsu
a691d82aee add middleware storage driver for redirect
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-21 16:02:52 -07:00
jhaohai
fdb0fb77df add cn-north-1 to valid check
Signed-off-by: jhaohai <jhaohai@foxmail.com>
2016-04-21 11:51:34 +08:00
Richard Scothern
406d03bdb0 Merge pull request #1604 from ArdaXi/custom-s3-skip-region-check
Only check validity of S3 region if not using custom endpoint
2016-04-08 15:38:24 -07:00
Richard Scothern
ede3c5033e Merge pull request #1605 from majewsky/swift/deal-with-outdated-container-listings
registry/storage/swift: detect and fix outdated container listings
2016-04-08 15:38:06 -07:00
Stefan Majewsky
84aa48b56c detect outdated container listings during Stat() and getAllSegments()
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-04-06 15:21:27 +02:00
Arien Holthuizen
c655241209 Only check validity of S3 region if not using custom endpoint
Signed-off-by: Arien Holthuizen <aholthuizen@schubergphilis.com>
2016-04-06 13:38:09 +02:00
Richard Scothern
4e45a3a903 Merge pull request #1597 from tonyhb/log-upload-copy-errors
Ensure we log io.Copy errors and bytes copied/total in uploads
2016-04-05 09:46:12 -07:00
Tony Holdstock-Brown
86ca50dfe5 Ensure we log io.Copy errors and bytes copied/total in uploads
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-04-04 17:21:36 -07:00
Richard Scothern
2a3d48fb82 Merge pull request #1578 from majewsky/do-not-swallow-error
don't swallow errors in Swift driver's GetContent()
2016-03-30 17:46:27 -07:00
Richard Scothern
bf182a4d3f Merge pull request #1560 from RichardScothern/gc
Fix signature handling with GC.
2016-03-30 17:46:04 -07:00
Richard Scothern
3d4b652b58 Update the gc documentation.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-30 16:57:21 -07:00
Richard Scothern
31ece3d3b6 Fix signature handling with GC.
If a schema 1 manifest is uploaded with the `disablesignaturestore` option set
to true, then no signatures will exist.  Handle this case.

If a schema 1 manifest is pushed, deleted, garbage collected and pushed again, the
repository will contain signature links from the first version, but the blobs will
not exist.  Disable the signature store in the garbage-collect command so
signatures are not fetched.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-03-30 16:10:27 -07:00
Richard Scothern
15e3ffb3f2 Add a --dry-run flag. If enabled this will print the mark and sweep process
with removing any files.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-30 16:10:27 -07:00
Richard Scothern
2f2875e8db Merge pull request #1582 from aaronlehmann/garbagecollect-errors
garbagecollect: Clean up errors
2016-03-30 14:29:27 -07:00
Aaron Lehmann
59ef6d2d40 garbagecollect: Clean up errors
- Clean up error messages

- Add a missing error check on the result of blobService.Enumerate.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-03-30 11:35:24 -07:00
Stefan Majewsky
b015bf0676 don't swallow errors in Swift driver's GetContent()
In 000dec3c6f, which was only intended to
be a refactoring commit, the behavior of this block subtly changed so
that unknown types of errors would be swallowed instead of propagated.

I noticed this while investigating an error similar to #1539 aka
docker/docker#21290. It appears that during GetContent() for a
hashstate, the Swift proxy produces an error. Since this error was
silently swallowed, an empty []byte is used to restart the hash, then
producing the digest of the empty string instead of the layer's digest.

This PR will not fix the issue, but it should make the actual error more
visible by propagating it into `blobWriter#resumeDigest' and
'blobWriter#validateBlob', respectively.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-03-30 16:11:29 +02:00
Stephen Day
432889df67 Merge pull request #1546 from arschles/doc-storage
Add documentation for how to register new StorageDrivers
2016-03-29 15:58:51 -07:00
Aaron Schlesinger
091ad89197 Remove the example
Instead, direct users to the one in the factory package

Signed-off-by: Aaron Schlesinger <aschlesinger@deis.com>
2016-03-29 14:42:28 -07:00
Aaron Schlesinger
0f09bcd16a Add documentation for how to register new StorageDrivers
This commit adds context-specific documentation on StorageDriver,
StorageDriverFactory, and the factory’s Register func, explaining how
the internal registration mechanism should be used.

This documentation follows from the thread starting at
https://github.com/deis/builder/pull/262/files#r56720200.

cc/ @stevvooe

Signed-off-by: Aaron Schlesinger <aschlesinger@deis.com>
2016-03-29 14:42:19 -07:00
Tony Holdstock-Brown
d52cbf923c utulize config log format within gc
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-03-24 14:29:43 -07:00
Richard Scothern
0ef5587b76 Merge pull request #1522 from RichardScothern/tag-events
Send tag events to notification listeners
2016-03-23 16:41:29 -07:00
Stephen Day
0b372266b4 Merge pull request #1491 from RichardScothern/relative-url
Return relative URLs
2016-03-23 16:19:39 -07:00
Richard Scothern
3dd506d896 Enable URLs returned from the registry to be configured as relative.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-23 15:14:32 -07:00
Richard Scothern
f93d166068 Propogate tag as a functional argument into the notification system to attach
tags to manifest push and pull event notifications.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-23 14:57:52 -07:00
Richard Scothern
96c57c4725 Merge pull request #1547 from jzwlqx/issue1535
fix manifest revision search, closes #1535
2016-03-21 16:01:16 -07:00
姜继忠
5f38f0b1fe fix manifest revision search, closes #1535
Signed-off-by: 姜继忠 <jizhong.jiangjz@alibaba-inc.com>
2016-03-19 19:11:25 +08:00
Derek McGowan
71d808de07 Merge pull request #1542 from runcom/typed-errors
registry: client: auth: type errors
2016-03-18 10:37:55 -07:00
Antonio Murdaca
e6b317f94f registry: client: auth: type errors
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-18 09:34:50 +01:00
Richard Scothern
461816a8d2 Merge pull request #1531 from aaronlehmann/empty-errors-slice
Don't return empty errcode.Errors slices
2016-03-17 10:46:51 -07:00
Richard Scothern
22cab6d148 Merge pull request #1538 from runcom/body-leaks
registry: client: repository: close response body
2016-03-16 15:58:22 -07:00
Antonio Murdaca
20bba4025a registry: client: repository: close response body
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 19:53:17 +01:00
Richard Scothern
e7ad3ced86 Merge pull request #1532 from RichardScothern/azure-error-types
Update missing blob error checking with latest Azure API
2016-03-15 10:08:36 -07:00
Aaron Lehmann
9638c7644e Include status code in UnexpectedHTTPResponseError
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-03-15 09:03:56 -07:00