parallel_bilby
is incompatible with schwimmbad
0.4.0 and above because the schwimmbad._VERBOSE
private variable was removed.
dynesty: 2.1.2
bilby: 2.2.3
bilby_pipe: 1.3.0
parallel_bilby: 2.0.2
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/duncan/opt/mambaforge/envs/parallel_bilby/lib/python3.11/site-packages/parallel_bilby/analysis/__init__.py", line 3, in <module>
from .main import analysis_runner, main
File "/home/duncan/opt/mambaforge/envs/parallel_bilby/lib/python3.11/site-packages/parallel_bilby/analysis/main.py", line 19, in <module>
from ..schwimmbad_fast import MPIPoolFast as MPIPool
File "/home/duncan/opt/mambaforge/envs/parallel_bilby/lib/python3.11/site-packages/parallel_bilby/schwimmbad_fast.py", line 8, in <module>
from schwimmbad import _VERBOSE, MPIPool, log
ImportError: cannot import name '_VERBOSE' from 'schwimmbad' (/home/duncan/opt/mambaforge/envs/parallel_bilby/lib/python3.11/site-packages/schwimmbad/__init__.py)
conda create -n parallel_bilby parallel-bilby "schwimmbad>=0.4.0"
conda activate parallel_bilby
python3 -c "import parallel_bilby.analysis"
As part of bilby_pipe!583, I added a flag to allow the user to pass a custom generation input class. This MR allows parallel_bilby_generation
to also allow for a custom generation input class.
Once bilby_pipe!583 has been merged, and a release subsequently made, I will update the bilby_pipe
version requirement.
As part of bilby_pipe!583, I added a flag to allow the user to pass a custom generation executable parser (analogous to the existing --analysis-executable-parser
flag, see https://lscsoft.docs.ligo.org/bilby_pipe/master/user-interface.html#job-submission-arguments). This MR allows parallel_bilby_generation
to also allow for a custom generation executable parser.
Once bilby_pipe!583 has been merged, and a release subsequently made, I will update the bilby_pipe
version requirement.
The option rejection-sample-posterior
is present in the arguments of analysis_runner
, but it is not supplied to format_result
when called.
As a result, this user-input of this option will not be respected. This MR fixes that.
Avi Vajpeyi (c25759c8) at 31 Jan 22:19
Merge branch 'lscsoft-fix3' into 'master'
... and 1 more commit
The option rejection-sample-posterior
is present in the arguments of analysis_runner
, but it is not supplied to format_result
when called.
As a result, this user-input of this option will not be respected. This MR fixes that.
This MR fixes the current extension of filename
(of AnalysisNode
) does not respect the user-input result-format
, which will end up causing a File Not Found
error when merging results.
Avi Vajpeyi (0abbf920) at 31 Jan 22:18
Merge branch 'lscsoft-fix1' into 'master'
... and 1 more commit
This MR fixes the current extension of filename
(of AnalysisNode
) does not respect the user-input result-format
, which will end up causing a File Not Found
error when merging results.
This adds a new user option for slurm, to tell pbilby to insert sleep
commands between the sbatch
instances for parallel analysis jobs (n_parallel>1
). This can be useful when the cluster one's working on is a bit busy filesystem-wise: e.g. I've recently had several times jobs fail due to resource loading conflicts for either matplotlib caches or distance marginalization .npz files. I've also decided to put in verbose echo
statements so that the user isn't confused when the submit script seems to "hang" after adding this option. Also, if the option is not set, nothing should change at all.
Avi Vajpeyi (d140b8eb) at 31 Jan 22:17
Merge branch 'slurm-sleep-between-submit' into 'master'
... and 1 more commit
The two MPI-related options: fast-mpi
and mpi-timing
are boolean options, which will be written as a flag in the slurm file later on.
However, the current parser expects them to have one argument, which causes error during generation.
Avi Vajpeyi (8e13cd19) at 31 Jan 22:16
Merge branch 'lscsoft-fix2' into 'master'
... and 1 more commit
The option rejection-sample-posterior
is present in the arguments of analysis_runner
, but it is not supplied to format_result
when called.
As a result, this user-input of this option will not be respected. This MR fixes that.
This MR fixes the current extension of filename
(of AnalysisNode
) does not respect the user-input result-format
, which will end up causing a File Not Found
error when merging results.
The two MPI-related options: fast-mpi
and mpi-timing
are boolean options, which will be written as a flag in the slurm file later on.
However, the current parser expects them to have one argument, which causes error during generation.
possible fix: add conversion function here
pB doesn't really generate samples for parameters specified in generation-function argument in post-processing stage as bilby_pipe does.
@avi.vajpeyi it took me a bit to realise that so far it wasn't testing the master bash_label.sh
script at all yet, but now it should, let me know if the (very basic) implementation makes sense. The remaining pipeline failure is independent: #72.
The latest pipeline on master https://git.ligo.org/lscsoft/parallel_bilby/-/jobs/2953120 failed with this, and I'm also hitting it on !132 :
FAILED tests/analysis/test_readwrite.py::test_readwrite - TypeError: cdata 'void *' does not support iteration