Improve caching in django
The settings exist (https://git.ligo.org/lscsoft/gracedb/blob/master/config/settings/base.py#L242) in gracedb's configuration for memcached caching, which is a quick and easy way to cache webpage views. This will be particularly helpful when new events come in and hundreds of users start hitting the website. Even some modest caching (I don't know what that means yet? 10 seconds? 30 seconds?) would greatly reduce server load and prevent django from making db queries and rendering new templates every time someone goes to the website or hits "reload".
A couple of issues:
- The configuration is set up for
memcachedcaching in memory, but the
memcacheddaemon isn't actually running or installed on any of the development machines or AWS containers. I installed it manually on
gracedb-dev2and it started memcaching almost immediately. Almost.
config/settings/base.pyneeds to be edited to look like this:
# List of middleware classes to use. MIDDLEWARE = [ 'core.middleware.maintenance.MaintenanceModeMiddleware', 'events.middleware.PerformanceMiddleware', 'core.middleware.accept.AcceptMiddleware', 'core.middleware.api.ClientVersionMiddleware', 'core.middleware.api.CliExceptionMiddleware', 'django.middleware.cache.UpdateCacheMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.cache.FetchFromCacheMiddleware', 'core.middleware.proxy.XForwardedForMiddleware', 'user_sessions.middleware.SessionMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'ligoauth.middleware.ShibbolethWebAuthMiddleware', 'ligoauth.middleware.ControlRoomMiddleware', ]
Note that the order apparently matters.
One other complication: it occurred to me that we're running a docker swarm of nodes, and yeah, each one has plenty of memory. However, they won't be able to access each others' local memory cache. Hmmm. I can run some tests and monitor the memory and caching on each node, but it doesn't seem efficient.
Amazon offers something called memcached "Elasticache", which appears to be a shared memory cache for different nodes:
It seems to be what we're looking for. Also this requires a new django backend:
So I'm guessing the process is going to look like:
- Log into AWS and find out how to make a new elasticache partition for each one of the different tiers. This can probably be automated with ansible, but at first I'll just click through the web interface like a caveman.
config/settings/container/base.pyto include the elasticache stuff under
CACHES. The address is going to be different, but that can be automated with a deployment environment variable in the docker swarm deployment yml.
MIDDLEWAREto include the django-elasticache middleware. I'm not sure what this will look like exactly, but it should probably model the block that I pasted up there.