tips for installing to a virtualenv

Some notes on the procedure @james-clark has found helpful for developing python code in a controlled environment.

Installing bilby to a virtualenv

Create virtualenv:

[jclark@ldas-pcdev6] $ virtualenv --python=/usr/bin/python3.6 $HOME/virtualenvs/bilby_pipe
Running virtualenv with interpreter /usr/bin/python3.6
Using base prefix '/usr'
New python executable in /home/jclark/virtualenvs/bilby_pipe/bin/python3.6
Also creating executable in /home/jclark/virtualenvs/bilby_pipe/bin/python
Installing setuptools, pip, wheel...done.

Activate and install bilby_pipe:

[jclark@ldas-pcdev6] $ source virtualenvs/bilby_pipe/bin/activate
(bilby_pipe) [jclark@ldas-pcdev6] $ pip install --upgrade bilby_pipe

Installing a fork to a virtualenv

Clone a fork:

➜  lscsoft git clone git@git.ligo.org:james-clark/bilby_pipe.git
Cloning into 'bilby_pipe'...
remote: Enumerating objects: 1945, done.
remote: Counting objects: 100% (1945/1945), done.
remote: Compressing objects: 100% (605/605), done.
remote: Total 1945 (delta 1333), reused 1925 (delta 1313)
Receiving objects: 100% (1945/1945), 327.73 KiB | 276.00 KiB/s, done.
Resolving deltas: 100% (1333/1333), done.

Ensure the fork is up to date:

➜  lscsoft cd bilby_pipe
➜  bilby_pipe git:(master) git remote add upstream git@git.ligo.org:lscsoft/bilby_pipe.git
➜  bilby_pipe git:(master) git fetch upstream
remote: Enumerating objects: 52, done.
remote: Counting objects: 100% (52/52), done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 52 (delta 30), reused 38 (delta 23)
Unpacking objects: 100% (52/52), done.
From git.ligo.org:lscsoft/bilby_pipe
 * [new branch]      42-expose-bilby-data-setting-methods             -> upstream/42-expose-bilby-data-setting-methods
 * [new branch]      colm-hack                                        -> upstream/colm-hack
 * [new branch]      colm_add_tests                                   -> upstream/colm_add_tests
 * [new branch]      convert-underscores-to-hyphens                   -> upstream/convert-underscores-to-hyphens
 * [new branch]      make-new-priors                                  -> upstream/make-new-priors
 * [new branch]      make-output-html-summary-navigate-to-run-results -> upstream/make-output-html-summary-navigate-to-run-results
 * [new branch]      master                                           -> upstream/master
 * [new branch]      rework-singularity                               -> upstream/rework-singularity
 * [new branch]      streamline-data-aquisition                       -> upstream/streamline-data-aquisition
➜  bilby_pipe git:(master) git merge upstream/master
Updating 087ed28..90a8b23
Fast-forward
 bilby_pipe/create_injections.py |  9 ++++++---
 bilby_pipe/data_analysis.py     | 18 +++++++++++-------
 bilby_pipe/data_generation.py   | 13 ++++++++-----
 bilby_pipe/main.py              | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++----------------
 bilby_pipe/parser.py            | 17 ++++++++++++-----
 tests/create_injection_test.py  | 61 +++++++++++++++++++++++++++++++++----------------------------
 tests/dag_test.py               |  1 +
 tests/data_generation_test.py   | 12 ++++++++----
 tests/example_prior.prior       |  3 +--
 9 files changed, 130 insertions(+), 70 deletions(-)
➜  bilby_pipe git:(master) git push
Enumerating objects: 56, done.
Counting objects: 100% (56/56), done.
Delta compression using up to 4 threads
Compressing objects: 100% (44/44), done.
Writing objects: 100% (44/44), 5.75 KiB | 840.00 KiB/s, done.
Total 44 (delta 36), reused 0 (delta 0)
To git.ligo.org:james-clark/bilby_pipe.git
   087ed28..90a8b23  master -> master

Public Repository

Repeat the procedure above to create a virtualenv but now install directly from your fork:

[jclark@ldas-pcdev6] $ virtualenv --python=/usr/bin/python3.6 $HOME/virtualenvs/bilby_pipe
[jclark@ldas-pcdev6] $ pip install --upgrade git+https://git.ligo.org/james-clark/bilby_pipe.git#egg=bilby_pipe

I've never figured out how to make this work with private gitlab repos so:

Private Repository

Similar but we need to use a local checkout:

(bilby_pipe_dev) [jclark@ldas-pcdev6] $ git clone git@git.ligo.org:james-clark/bilby_pipe.git bilby_pipe_dev
Cloning into 'bilby_pipe_dev'...
X11 forwarding request failed on channel 0
remote: Enumerating objects: 1989, done.
remote: Counting objects: 100% (1989/1989), done.
remote: Compressing objects: 100% (649/649), done.
remote: Total 1989 (delta 1371), reused 1915 (delta 1313)
Receiving objects: 100% (1989/1989), 337.71 KiB | 1.60 MiB/s, done.
Resolving deltas: 100% (1371/1371), done.
(bilby_pipe_dev) [jclark@ldas-pcdev6] $ pip install --upgrade git+file://${HOME}/src/lscsoft/bilby_pipe_dev
Collecting git+file:///home/jclark/src/lscsoft/bilby_pipe_dev
  Cloning file:///home/jclark/src/lscsoft/bilby_pipe_dev to /local/jclark/pip-vjue8a0r-build
<snip>

Add dependencies (e.g., pip install bilby) according to taste)