... | ... | @@ -2,6 +2,20 @@ Overview of a study into the parallelization scaling of dynesty |
|
|
|
|
|
## Quick test v1
|
|
|
|
|
|
### Overview
|
|
|
I've set up a simple `bilby_pipe` job which runs a fiducial BBH injection and recovery. To speed things up, I've reduced the model to a non-spinning system, used all available marginalizations, and reduced the sampler settings from our reviewed settings (`nact=5`). The reviewed settings (`nact=10`) are highly conservative, but the `nact=5` case produces results in about half the time and are sufficient for testing purposes (they are sufficiently close to what the outputs should be).
|
|
|
|
|
|
To run the job, copy the `ini` and `prior` files (see the drop-down's below) and run
|
|
|
|
|
|
```
|
|
|
$ bilby_pipe config.ini --request-cpus 16 --outdir OUTDIR
|
|
|
```
|
|
|
|
|
|
Note, the command line arguments overrides the `request-cpus=` and `outdir=` given in the `config.ini` file itself and makes it easy to script running several different jobs. During the setup, the `request-cpus=16` ensures that (a) the HTCondor submit files contain `request-cpus=16` and (b) that the sampler is instructed to use 16 cores (specifically, in the file `OUTDIR/test_config_complete.ini`, the lines `sampler-kwargs = {npool: 16}`).
|
|
|
|
|
|
If you want to submit the files at the same time, add `--submit` to the command line call. For further details on bilby_pipe ini files, see [the documentation](https://lscsoft.docs.ligo.org/bilby_pipe/master/ini_file.html).
|
|
|
|
|
|
### Files
|
|
|
<p>
|
|
|
<details>
|
|
|
<summary> <code>config.ini</code> a modified version of the 4s review test</summary>
|
... | ... | @@ -194,4 +208,14 @@ phase = Uniform(name='phase', minimum=0, maximum=2 * np.pi, boundary='periodic') |
|
|
</code></pre>
|
|
|
|
|
|
</details>
|
|
|
</p> |
|
|
\ No newline at end of file |
|
|
</p>
|
|
|
|
|
|
### Scaling test
|
|
|
I ran this bash script using the files above
|
|
|
```
|
|
|
for i in {1..20};
|
|
|
do
|
|
|
bilby_pipe config.ini --request-cpus $i --outdir outdir_${i}
|
|
|
done
|
|
|
```
|
|
|
For the cases `request_cpus > 16`, I manually modified the `.submit` files to only request 16 cpus. In this sense, everything above `16` is instructing the sampler to use more cores than there are available. |
|
|
\ No newline at end of file |