quay

⚠️ The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order to get stable binaries.

Quay is project to build, store, and distribute container images.

High-level features include:

• Docker Registry Protocol v1, v2
• Docker Manifest Schema v2.1
• AppC Image Discovery via on-demand transcoding
• Image Squashing via on-demand transcoding
• Authentication provided by LDAP, Keystone, Dex, Google, GitHub
• ACLs, team management, and auditability logs
• Geo-replicated storage provided by local filesystems, S3, GCS, Swift, Ceph
• Continuous Integration integrated with GitHub, Bitbucket, GitLab, and git
• Security Vulnerability Analysis via Clair
• Swagger-compliant HTTP API

Table of Contents

1. Getting Started
2. macOS
3. Linux
4. Development
5. PivotalTracker Integration
6. Running and Testing
7. Test Data
8. Local Scripts
9. Development inside Docker
10. Adding a Python Dependency
11. Adding a Yarn Dependency
12. Running the Build System
13. To run individual tests 1. Pytest 2. Tox
14. Running Migrations
15. How to run a build with tests for a push or merge
16. Documentation
17. Architecture at a Glance
18. Terminology 1. Organizations 2. Concepts 3. Software

Getting Started

macOS

macOS developers will need:

• command line tools or xcode
• brew

# Download the code
git clone git@github.com:coreos-inc/quay.git && cd quay

# Install the system dependencies
brew install libevent libmagic postgresql gpgme pyenv pyenv-virtualenv node yarn

# Create a default virtualmachine for docker
docker-machine create -d virtualbox default

# Add these to ~/.bashrc or ~/.zshrc
eval "$(pyenv virtualenv-init -)"
eval "$(pyenv init -)"
eval $(/usr/local/bin/docker-machine env default)
export PYTHONPATH="."

# Some installs don't have /usr/include, required for finding SASL header files
# This command might fail because of the rootfs is read-only. Refer to the following:
# http://apple.stackexchange.com/questions/196224/unix-ln-s-command-not-permitted-in-osx-el-capitan-beta3
if [ ! -e /usr/include ]; then sudo ln -s `xcrun --show-sdk-path`/usr/include /usr/include; fi

# Install the Python dependencies
pyenv install 2.7.12
pyenv virtualenv 2.7.12 quay
pyenv activate quay
pyenv local quay

# Some packages may fail to build with clang (which now defaults to C11).
# If you're getting errors trying running again with CFLAGS='std=c99'.
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install -r requirements-test.txt

# Setup a local config
git clone git@github.com:coreos-inc/quay-config.git ../quay-config
ln -s ../../quay-config/local conf/stack

# Install Node Dependencies
yarn install

# Link Typescript
yarn link typescript

Third Party Docs

• docker
• docker-machine
• pyenv
• pyenv-virtualenv

Docker Compose

You'll need Docker and Docker Compose installed. If you're on macOS, Docker for Mac should include both tools. Otherwise, follow the docs for your platform.

You'll also need Node.js and NPM if you want to interact with the frontend code outside a container.

Finally, you'll need a recent Go version for the builder.

To start Quay locally:

# Clone the various repos you'll need:
git clone https://github.com/coreos-inc/quay.git
git clone https://github.com/coreos-inc/quay-config-local.git
git clone https://github.com/coreos-inc/quay-builder.git

# Build the builder:
cd quay-builder
make build GOOS=linux

# Install NPM modules:
cd ../quay
npm install

# Build or pull images and start all Quay components:
docker-compose up

Third Party Docs

• Docker Compose
• Docker for Mac

Linux

Do you use Linux? Send us a PR! Or use docker-compose!

Development

JIRA

The Quay backlog can be found in JIRA: https://jira.coreos.com/projects/QUAY

Running and Testing

Test Data

A SQLite database full of test data is committed to this git repository at test/data/test.db. This database is generated by executing python initdb.py. The username and password of the admin test account is devtable and password, respectively.

Local Scripts

Running the web server locally requires goreman:

go get github.com/mattn/goreman

• local-run runs the web server for testing
• local-test runs the unit test suite
• yarn run build builds front-end dependencies
• yarn run watch builds and watches front-end dependencies
• yarn test runs front-end test suite

Development inside Docker

To build and run a development container, pass one argument to local-docker.sh:

• buildman: run the buildmanager
• dev: run web server on port 5000
• initdb: clear and initialize the test database
• notifications: run the notification worker
• test: run the unit test suite

Adding a Python Dependency

# Create a new virtualenv and activate it
pyenv virtualenv 2.7.12 quay-deps
pyenv activate quay-deps

# Install unversioned dependencies with your changes
pip install -r requirements-nover.txt

# Run the unit test suite
./local-test.sh

# Freeze the versions of all of the dependencies
pip freeze > requirements.txt

# Delete the virtualenv
pyenv uninstall quay-deps

Adding a Yarn Dependency

We use Yarn for frontend dependency management. The yarn.lock file ensures that we get consistant version installs using the yarn install command. However, new dependencies should be added using yarn add <npm package>. This will add an entry to package.json and yarn.lock.

Occassionally there will be merge conflicts with yarn.lock. To resolve them, use the following (taken from here).

git rebase origin/master
git checkout origin/master -- yarn.lock
yarn install
git add yarn.lock
git rebase --continue

Running the Build System

TODO

# Run an instance of redis
docker run -d -p 6379:6379 quay.io/quay/redis

To run individual tests

# To run a specific suite
TEST=true python -m test.test_api_usage -f

# To run a specific test in a suite
TEST=true python -m test.test_api_usage -f SuiteName

Pytest

# To run all tests
TEST=true PYTHONPATH="." py.test --verbose

# To run a specific test module
TEST=true PYTHONPATH="." py.test --verbose test/registry_tests.py

# To run a specific test unique test
TEST=true PYTHONPATH="." py.test --verbose test/test_api_usage.py::TestDeleteNamespace

# To retry only last failed (--lf):
TEST=true PYTHONPATH="." py.test --verbose --lf

# To start pdb on failure:
TEST=true PYTHONPATH="." py.test --verbose --pdb

# To run a coverage report (html pages in ./htmlcov):
TEST=true PYTHONPATH="." py.test --cov="." --cov-report=html --cov-report=term-missing  --cov-config=.coverage.ini --verbose

# Don't capture stdout (-s)
TEST=true PYTHONPATH="." py.test --verbose -s

Tox

To create a virtualenv to run the tests. It allows to test the code on multiple env like python2.x and python3.x or different library versions

# Test all tox env:
tox

# Add extra parameters to the pytest command:
# tox -- [pytest ARGS]
tox -- -x

# build a single env with -e:
tox -e py27-api

Running migrations

# To create a new migration with this description.
# Note there might be some errors about unique id being to long
# That's okay as long as the migration file is created
./data/migrations/migration.sh "Description goes here"

# To test the up and down of the migration
./data/migrations/migration.sh # without params

# Migrations get run when you create a docker image or you can run them
# manually with the following command.
PYTHONPATH=. alembic upgrade head

# You can also rebuild your local sqlite db image from initdb.py using
# And once you have a migration you should do this and check in the
# changes to share your migration with others.
rm test/data/test.db
python initdb.py

Running tests for migrations

Use AWS/RDS to create a test image. To create a new database from a snapshot to test against see this.

Then point the migrations to the new instance using quay-config/local/config.yaml

Remember to run this from the root of the quay directory and to set your python environment first.

PYTHONPATH=. alembic upgrade head

How to run a build with tests for a push or merge

# Inside the quay directory.
export QUAY_TAG=quay.io/quay/quay:localtest
docker build -t $QUAY_TAG --build-arg RUN_TESTS=true .

Documentation

• Quay Enterprise Documentation
• Quay.io Documentation

Architecture at a Glance

Edit chart on Google Docs at Architecture at a Glance.

Terminology

Organizations

• AppC: a standards body responsible for a Runtime and Image Format superseded by the Open Container Initiative
• Open Container Initiative: a standards body responsible for a Runtime specification and an Image Format
• Docker: a company that builds a platform that has its own Image Formats, Build System, Container Runtime, and Container Orchestration

Concepts

• Image: an archive containing all of the contents necessary to execute a container
• Image Format: a specification for the structure of an Image
• Image Layer: an Image that may depend on being applied to other Images to generate a final Image
• Image Squashing: the process of compressing an Image into a single Layer
• Manifest: a text file containing metadata for a particular Image
• Tag: a human-friendly named, mutable pointer to a particular set of Images
• Build System: a program used to generate Images
• Registry: a program that speaks one or more standard protocols to store and receive Images
• Repository: a collection of related Tags organized by a Registry
• Push: the act of uploading an Image to a Registry
• Pull: the act of downloading an Image from a Registry
• Container: an Image and its execution environment
• Container Runtime: a program that can transform an Image into a Container by executing it
• Container Orchestration: a program or set of programs that provides a framework for deploying Containers

Software

• Quay.io: CoreOS's hosted Registry
• Quay: CoreOS's enterprise-grade Registry product
• quayctl: an open source program that implements alternative methods for pulling Images from Quay
• Clair: an open source static analysis tool used to detect vulnerability in Images
• Quay Security Scanning: the integration between Clair and Quay
• Kubernetes: an open source program implementing Container Orchestration
• Docker Hub: Docker's hosted Registry
• Docker Trusted Registry: Docker's enterprise-grade Registry product
• Notary: an open source implementation of the TUF protocol used in Docker Content Trust
• Docker Content Trust: the integration between Notary and Docker Trusted Registry
• Docker Engine: a program used to interact with all aspects of the Docker platform
• Swarm: a program implementing Container Orchestration for the Docker platform