Skip to content

WIP: Move to pinned packages for production deployment

Duncan Macleod requested to merge (removed):package-pinning into master

This MR is my first cut at moving from a free-for-all of packages to a production-ready pinned set of packages.

The plan

  • all LSCSoft packages are fixed to a specific build for each of linux and osx platforms

    • each package is defined in a <package>.yaml file that outlines the name, version, and build_string, as well as any skips, and tests
  • all upstream packages are fixed to a specific version

    • the upstream set is declared in a single upstream.yaml file that resembles a conda environment file
  • CI/CD will render that set of package files into the following set of environments

    • stable (fully pinned)
    • proposed (approved changes to the stable environment)
    • testing (requested changes to the stable environment)
    • unstable (no pins at all)

The request workflow

  • open a MR that updates the .yaml file for one or more packages
  • this should be automatically merged onto the testing branch, and a new MR opened to merge from testing to proposed
  • the SCCB will then consider the request and merge the request onto proposed when approved
  • when 'Tuesday maintenance' rolls around, proposed will be merged into stable/master and tagged with a datestamp

Required development

  • set up the necessary branches to support the different regimes: stable/proposed/testing
  • add .yaml files for all controlled packages
  • add one or more .yaml files that list out the upstream packages that we don't control
  • update CI to do all the things for us:
    • render package files into environment files
    • build environments for testing
    • run integration tests
    • build latest singularity image to unpack into /cvmfs (only: [branches@lscsoft/conda])
    • build <tag> singularity image to unpack into /cvmfs (only: [tags@lscsoft/conda])
    • create documentation page that includes the rendered environment files for both platforms
Edited by Duncan Macleod

Merge request reports