... | @@ -258,7 +258,7 @@ Snapshots show that the dispersion in task length is high at the beginning of ea |
... | @@ -258,7 +258,7 @@ Snapshots show that the dispersion in task length is high at the beginning of ea |
|
|
|
|
|
This problem must be fixed if the code if efficient scaling is desired beyond 512 cores. In this study, we have attempted to reduce the load imbalance by overloading the number of tasks onto the workers. While this did not improve overall runtime, the task length dispersion was reduced at the beginning of each run. Based on these findings, adaptively varying the pool size may be an effective optimisation. A promising idea is to start by overloading the workers, then gradually reducing the number of tasks as the completion times converge. There is currently no simple method of implementing this, since modifications to the dynesty library are required.
|
|
This problem must be fixed if the code if efficient scaling is desired beyond 512 cores. In this study, we have attempted to reduce the load imbalance by overloading the number of tasks onto the workers. While this did not improve overall runtime, the task length dispersion was reduced at the beginning of each run. Based on these findings, adaptively varying the pool size may be an effective optimisation. A promising idea is to start by overloading the workers, then gradually reducing the number of tasks as the completion times converge. There is currently no simple method of implementing this, since modifications to the dynesty library are required.
|
|
|
|
|
|
A more advanced optimisation is to allow iterations to progress without waiting for workers to return. This may reduce the wasted time by ignoring points that are not necessary for an iteration to progress. Tasks performed by some workers will be wasted, but it should be investigated (mathematically) whether the skipped results can still be reused in the next iteration if they lie within the reduced volume, further recuperating some of the wasted time. This will require modifications to the dynesty library and the Schwimmbad library.
|
|
A more advanced optimisation is to allow iterations to progress without waiting for workers to return. This may reduce the wasted time by ignoring points when the majority of workers have finished. Tasks performed by the remaining workers will be wasted, but it should be investigated (mathematically) whether the skipped results can still be reused in the next iteration if they lie within the reduced volume, further recuperating some of the wasted time. This will require modifications to the dynesty library and the Schwimmbad library.
|
|
|
|
|
|
## Reducing serial Bilby overheads
|
|
## Reducing serial Bilby overheads
|
|
|
|
|
... | | ... | |