097fce3bb2
This changeset addresses intermittent internal server errors encountered during pushes. The root cause has been isolated to layers that result in identical, empty filesystems but may have some path declarations (imaginge "./"), resulting in different tarsums. The main error message reported during these upload problems was a 500 error, which was not correct. Further investigation showed the errors to be rooted in digest verification when finishing uploads. Inspection of the surrounding code also identified a few issues. PutLayerChunk was slightly refactered into PutLayerUploadComplete. Helper methods were avoided to make handler less confusing. This simplification leveraged an earlier change in the spec that moved non-complete chunk uploads to the PATCH method. Simple logging was also added in the unknown error case that should help to avoid mysterious 500 errors in the future. At the same time, the glaring omission of a proper layer upload cancel method was rectified. This has been added in this change so it is not missed in the future. In the future, we may want to refactor the handler code to be more straightforward, hopefully letting us avoid these problems in the future. Added test cases that reproduce these errors and drove these changes include the following: 1. Push a layer with an empty body results in invalid blob upload. 2. Push a layer with a different tarsum (in this case, empty tar) 3. Deleting a layer upload works. 4. Getting status on a deleted layer upload returns 404. Common functionality was grouped into shared functions to remove repitition. The API tests will still require future love. Signed-off-by: Stephen J Day <stephen.day@docker.com> |
||
|---|---|---|
| api/v2 | ||
| auth | ||
| client | ||
| cmd | ||
| configuration | ||
| digest | ||
| doc | ||
| Godeps | ||
| manifest | ||
| open-design | ||
| project | ||
| registry | ||
| storage | ||
| storagedriver | ||
| testutil | ||
| .drone.yml | ||
| .gitignore | ||
| .mailmap | ||
| AUTHORS | ||
| circle.yml | ||
| CONTRIBUTING.md | ||
| doc.go | ||
| Dockerfile | ||
| LICENSE | ||
| MAINTAINERS | ||
| Makefile | ||
| README.md | ||
Notice: This repository hosts experimental components that are currently under heavy and fast-paced development, not-ready for public consumption. If you are looking for the stable registry, please head over to docker/docker-registry instead.
Distribution
The Docker toolset to pack, ship, store, and deliver content.
Planned content for this repository:
- Distribution related specifications
- Image format
- JSON registry API
- Registry implementation: a Golang implementation of the JSON API
- Client libraries to consume conforming implementations of the JSON API
Ongoing open sprint
What is an open sprint?
The open sprint is a focused effort of a small group of people to kick-off a new project, while commiting to becoming maintainers of the resulting work.
Having a dedicated team work on the subject doesn't mean that you, the community, cannot contribute! We need your input to make the best use of the sprint, and focus our work on what matters for you. For this particular topic:
- Come discuss on IRC: #docker-distribution on FreeNode
- Come read and participate in the Google Groups
- Submit your ideas, and upvote those you think matter the most on Google Moderator
Goal of the distribution sprint
Design a professional grade and extensible content distribution system, that allow users to:
- Enjoy an efficient, secured and reliable way to store, manage, package and exchange content
- Hack/roll their own on top of healthy open-source components
- Implement their own home made solution through good specs, and solid extensions mechanism.
Schedule and expected output
The Open Sprint will start on Monday December 29th, and end on Friday January 16th.
What we want to achieve as a result is:
-
Tactical fixes of today's frustrations in the existing Docker codebase
- This includes a throrough review of docker/docker#9784 by core maintainers
-
Laying the base of a new distribution subsystem, living independently, and with a well defined group of maintainers. This is the purpose of this repository, which aims at hosting:
- A specification of the v2 image format
- A specification of the JSON/HTTP protocol
- Server-side Go implementation of the v2 registry
- Client-side Go packages to consume this new API
- Standalone binaries providing content distribution functionalities outside of Docker
-
Constraints for interface provided by Distribution to Core:
- The distribution repository is a collection of tools for packaging and shipping content with Docker
- All tools are usable primarily as standalone command-line tools. They may also expose bindings in one or more programming languages. Typically the first available language is Go, but that is not automatically true and more languages will be supported over time
- The distribution repository is still under incubation, any code layout, interface and name may change before it gets included in a stable release of Docker
How will this integrate with Docker engine?
Building awesome, independent, and well maintained distribution tools should give Docker core maintainers enough incentive to switch to the newly develop subsystem. We make no assumptions on a given date or milestone as urgency should be fixed through docker/docker#9784, and in order to maintain focus on producing a top quality alternative.