dynesty-parallization.md

@@ -2,6 +2,20 @@ Overview of a study into the parallelization scaling of dynesty
 
 ## Quick test v1
 
+### Overview
+I've set up a simple `bilby_pipe` job which runs a fiducial BBH injection and recovery. To speed things up, I've reduced the model to a non-spinning system, used all available marginalizations, and reduced the sampler settings from our reviewed settings (`nact=5`). The reviewed settings (`nact=10`) are highly conservative, but the `nact=5` case produces results in about half the time and are sufficient for testing purposes (they are sufficiently close to what the outputs should be).
+
+To run the job, copy the `ini` and `prior` files (see the drop-down's below) and run
+
+```
+$ bilby_pipe config.ini --request-cpus 16 --outdir OUTDIR
+```
+
+Note, the command line arguments overrides the `request-cpus=` and `outdir=` given in the `config.ini` file itself and makes it easy to script running several different jobs. During the setup, the `request-cpus=16` ensures that (a) the HTCondor submit files contain `request-cpus=16` and (b) that the sampler is instructed to use 16 cores (specifically, in the file `OUTDIR/test_config_complete.ini`, the lines `sampler-kwargs = {npool: 16}`). 
+
+If you want to submit the files at the same time, add `--submit` to the command line call. For further details on bilby_pipe ini files, see [the documentation](https://lscsoft.docs.ligo.org/bilby_pipe/master/ini_file.html). 
+
+### Files
 <p>
 <details>
 <summary> <code>config.ini</code> a modified version of the 4s review test</summary>
@@ -194,4 +208,14 @@ phase = Uniform(name='phase', minimum=0, maximum=2 * np.pi, boundary='periodic')
 </code></pre>
 
 </details>
-</p>
\ No newline at end of file
+</p>
+
+### Scaling test
+I ran this bash script using the files above
+```
+for i in {1..20};
+do
+    bilby_pipe config.ini --request-cpus $i --outdir outdir_${i}
+done
+```
+For the cases `request_cpus > 16`, I manually modified the `.submit` files to only request 16 cpus. In this sense, everything above `16` is instructing the sampler to use more cores than there are available.
\ No newline at end of file