When directories fill up Diskcache should quickly locate the new directory and mark it as hot
If you have a hot directory, such as /kafkacache/K1/K-K1_llhoft-139733. After all 139733 frames are finished the 139734 directory is created and filled. However the new directory is not hot and it takes a iteration of the full mount points before the new directory is located and marked hot.
In discussions with Ed, one way to deal with this, is that if a hot directory is not written to, to rescan the parent. This would make it easy to discover quickly when frame filling moves to a new directory.
The idea would be:
- If the scan of the current hot directory results in no change
- Check if the parent directory has changed and has new directories
- If the parent directory has changed and has new directores
- Update parent metadata
- Add new directories to cache
- Add any new directories to the list of hot directories