Skip to content

Add Docker builds

Patrick Godwin requested to merge patrick.godwin/gwcelery:docker_builds into main

This adds a Dockerfile to build Docker containers, depending on python 3.9 (same as the deployment).

Also adds a CI job that publishes images to containers.ligo.org based off of tags and branches, and creates/updates a 'latest' image when a new version is released.

The Dockerfile uses a 3 stage build to minimize the image size:

  1. Build stage: Copy local repo, build wheel and export requirements via poetry.
  2. Install stage: Install packages into a virtualenv
  3. App stage: Copy over virtualenv, define relevant env variables, set up non-root user, add gwcelery entrypoint.

The dev environment is configured by default, but can be changed by overriding $CELERY_CONFIG_MODULE.

Example:

$ docker build . -t gwcelery
...
Successfully tagged localhost/gwcelery:latest
ec5b704473c7dcff1e70ff13405593bc1f33b3d6228c7478ecf263ae5ed85dca

$ docker run -it --rm gwcelery:latest --help
Usage: gwcelery [OPTIONS] COMMAND [ARGS]...

  Celery command entrypoint.

Options:
  -A, --app APPLICATION
  -b, --broker TEXT
  --result-backend TEXT
  --loader TEXT
  --config TEXT
  --workdir PATH
  -C, --no-color
  -q, --quiet
  --version
  --skip-checks          Skip Django core checks on startup. Setting the
                         SKIP_CHECKS environment variable to any non-empty
                         string will have the same effect.
  --help                 Show this message and exit.

Commands:
  amqp     AMQP Administration Shell.
  beat     Start the beat periodic task scheduler.
  call     Call a task by name.
  condor   Shortcuts for HTCondor commands to manage deployment of...
  control  Workers remote control.
  events   Event-stream utilities.
  flask    Flask web application for manually triggering certain tasks.
  graph    The ``celery graph`` command.
  inspect  Inspect the worker at runtime.
  list     Get info from broker.
  logtool  The ``celery logtool`` command.
  migrate  Migrate tasks from one broker to another.
  multi    Start multiple worker instances.
  nagios   A Nagios plugin for monitoring GWCelery.
  purge    Erase all messages from all known task queues.
  report   Shows information useful to include in bug-reports.
  result   Print the return value for a given task id.
  shell    Start shell session with convenient access to celery symbols.
  status   Show list of workers that are online.
  upgrade  Perform upgrade between versions.
  worker   Start worker instance.

Merge request reports