Skip to content

Set the concurrency of the main celery queue to 32 (down from 64).

Roberto DePietri requested to merge roberto.depietri/gwcelery:concurrency-32-celery into main

The emfollow machine runs out of memory ~once per day. The current allocated memory is 64GB and the concurrency of the main celery queue is set to 64. To mitigate the effect (since it seams that in MDC we never have more that ~20 concurrent active processes in the queue here we set the concurrency of the main celery queue to 32 (down from 64) instead of increasing the memory of the virtuaa machine.

Memory usage dropped from ~64 GByte to ~15 GByte at 5:50 Pacific time (12:50UTC) for no apparent reason. Now memory is rumping up.

Edited by Roberto DePietri

Merge request reports