with a new `proxy` section in the configuration file.
Create a new registry type which delegates storage to a proxyBlobStore
and proxyManifestStore. These stores will pull through data if not present
locally. proxyBlobStore takes care not to write duplicate data to disk.
Add a scheduler to cleanup expired content. The scheduler runs as a background
goroutine. When a blob or manifest is pulled through from the remote registry,
an entry is added to the scheduler with a TTL. When the TTL expires the
scheduler calls a pre-specified function to remove the fetched resource.
Add token authentication to the registry middleware. Get a token at startup
and preload the credential store with the username and password supplied in the
config file.
Allow resumable digest functionality to be disabled at runtime and disable
it when the registry is a pull through cache.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Storage drivers can implement a method called URLFor which can return a direct
url for a given path. The functionality allows the registry to direct clients
to download content directly from the backend storage. This is commonly used
with s3 and cloudfront. Under certain conditions, such as when the registry is
not local to the backend, these redirects can hurt performance and waste
incoming bandwidth on pulls. This feature addition allows one to disable this
feature, if required.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Conflicts:
configuration/configuration.go
registry/handlers/app.go
registry/storage/catalog_test.go
registry/storage/manifeststore_test.go
registry/storage/registry.go
Implement the delete API by implementing soft delete for layers
and blobs by removing link files and updating the blob descriptor
cache. Deletion is configurable - if it is disabled API calls
will return an unsupported error.
We invalidate the blob descriptor cache by changing the linkedBlobStore's
blobStatter to a blobDescriptorService and naming it blobAccessController.
Delete() is added throughout the relevant API to support this functionality.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
This PR is for issue of "email after registry webapp panic" #41, improving my
previous design (closed).
It use self setting up hooks, to catch panic in web application.
And, send email in hooks handle directly, to no use new http server and
handler.
Signed-off-by: xiekeyang <keyangxie@126.com>
Allow to use a unix socket as a listener.
To specify an endpoint type we use an optional configuration
field 'net', as there's no way to distinguish a relative
socket path from a hostname.
Signed-off-by: Anton Tiurin <noxiouz@yandex.ru>
This allows one to better control the usage of the cache and turn it off
completely. The storage configuration module was modified to allow parameters
to be passed to just the storage implementation, rather than to the driver.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Redis has been integrated with the web application for use with various
services. The configuraiton exposes connection details, timeouts and pool
parameters. Documentation has been updated accordingly.
A few convenience methods have been added to the context package to get loggers
with certain fields, exposing some missing functionality from logrus.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
You shouldn't have to import both:
github.com/docker/distribution/context
golang.org/x/net/context
just to use the distribution tools and implement the distribution interfaces.
By pulling the Context interface from golang.org/x/net/context into the
context package within the distribution project, you no longer have to import
both packages.
Note: You do not have to change anything anywhere else yet! All current uses
of both packages together will still work correctly because the Context
interface from either package is identical.
I've also made some other minor changes:
- Added a RemoteIP function. It's like RemoteAddr but discards the port suffix
- Added `.String()` to the response duration context value so that JSON log
formatting shows human-parseable duration and not just number of nano-seconds
- Added WithMapContext(...) to the context package. This is a useful function
so I pulled it out of the main.go in cmd/registry so that it can be used
elsewhere.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
To allow flexibility in log message context information, this changeset
provides the ability to configure static fields that are included in the
context. Such fields can be set via configuration or environment variables.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This changeset simply adds hooks into the configuration system to support
multiple different kinds of output formats. These formatters are provided by
logrus and include options such as "text" and "json". The configuraiton
documentation has been updated accordingly.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Convert middleware in the config to be a map of type->[]Middleware
Add support for registry & repository middleware.
Some naming updates as well.
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
middleware concept.
This also breaks the dependency the storage package had on goamz
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
Endpoints are now created at applications startup time, using notification
configuration. The instances are then added to a Broadcaster instance, which
becomes the main event sink for the application. At request time, an event
bridge is configured to listen to repository method calls. The actor and source
of the eventBridge are created from the requeest context and application,
respectively. The result is notifications are dispatched with calls to the
context's Repository instance and are queued to each endpoint via the
broadcaster.
This commit also adds the concept of a RequestID and App.InstanceID. The
request id uniquely identifies each request and the InstanceID uniquely
identifies a run of the registry. These identifiers can be used in the future
to correlate log messages with generated events to support rich debugging.
The fields of the app were slightly reorganized for clarity and a few horrid
util functions have been removed.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
If configured, a debug http server will be started to serve default registered
endpoints, such as pprof and expvar. The endpoint should be secured carefully
and not available to external traffic. It is disabled by default but the
development config has been modified to make it available on localhost.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This changeset provides simple tls support for a registry instance. Simply
providing a cert and key file are enough to get a tls registry running. If the
certs are trusted by the client, tls can be used throughout the push and pull
process.
If more complex TLS options are required, it is recommend that a proxy be used.
Contributions will be accepted to add more features, if necessary.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
To support clustered registry, upload UUIDs must be recognizable by
registries that did not issue the UUID. By creating an HMAC verifiable
upload state token, registries can validate upload requests that other
instances authorized. The tokenProvider interface could also use a redis
store or other system for token handling in the future.
This changeset integrates the AccessController into the main registry app. This
includes support for configuration and a test implementation, called "silly"
auth. Auth is only enabled if the configuration is present but takes measure to
ensure that configuration errors don't allow the appserver to start with open
access.
Most conditional parsing code has been moved into UnmarshalYAML
functions for simplicity
Uses the BrianBland fork of goyaml in configuration.go temporarily until
fix https://github.com/go-yaml/yaml/pull/52 is merged in
