The solution described in this article is in use on the production servers powering mmogle.com where it ensures page response times below 100ms for cached content.

Let me summarize the concept for readers not familiar with my first article: The goal we want to achieve is to greatly decrease the load on the application server (container) by storing the raw HTML output of rendered pages in a (distributed-) cache to be picked up by a Frontend Web Server without even bothering the application server behind it for the current request. Since we would need stuff like Edge Side Includes (ESI) for dealing with personalized pages, we are going limit ourselves to anonymous users.

This picture below shows the server setup:

Nginx will act as frontend web server. This is where all requests will initially arrive. Requests for static resources will always be handled by nginx. Dynamic requests will be scrutinized and once it is determined that a request could be cached, a cache key is computed based on request URI, cookies etc. Nginx will then proceed with querying memcached if one of the application servers has deposited content under that key. If the answer is positive the response from memcached is directly returned to the client – the application servers are bypassed. Perhaps it is worth noting that the application server (tomcat) is only responsible for storing the content, whereas nginx – acting as frontend server – is responsible for retrieving it. This decoupling is what sets this technique apart from the Grails Cache Filter Plugin which both stores and retrieves cached content.

Let’s continue with the actual implementation. When I decided to port the technique I described in the aforementioned article from Python/Django to Groovy/Grails, I thought it was going to be a very easy task. All that’s needed is to grab the rendered page contents within a Grails afterView Filter, put it into memcached and be done with it, right? Wrong! Imagine my surprise when I noticed the rendered output never had a layout applied. After lots of trying I gave up, realising that there is simply no way to get my hands on the complete HTML (emphasis on complete) for a rendered Grails page from within the application itself. The reason for this is Grails layouts are based on Sitemesh which is implemented as a ServletFilter. Grails applies the Layout to pages through its GrailsPageFilter ServletFilter which is executed further down in the pipeline after any Grails Filters.

That left me with only one option: a custom ServletFilter that executes before GrailsPageFilter. So I checked out the source of the Grails Cache Filter Plugin, stripped it off anything not necessary for our purpose and born was MemcachedFilter. You can download the source here.

MemcachedFilter is written in Groovy and it needs to be configured to execute before GrailsPageFilter. That means that it’s filter mapping element must appear before the filter mapping for GrailsPageFilter in web.xml. Here’s a sample web.xml snippet:

<filter>
  <filter-name>memcached</filter-name>
  <filter-class>com.banshee.servlet.MemcachedFilter</filter-class>
</filter>
 
<!-- memcached filter -->
<filter-mapping>
  <filter-name>memcached</filter-name>
  <url-pattern>/*</url-pattern>
</filter-mapping>
 
 
<filter-mapping>
  <filter-name>charEncodingFilter</filter-name>
  <url-pattern>/*</url-pattern>
</filter-mapping>
 
<filter-mapping>
  <filter-name>sitemesh</filter-name>
  <url-pattern>/*</url-pattern>
</filter-mapping>

MemcachedFilter has a dependency on a bean named “memcachedClient”. Please refer to this article for configuration instructions.

Now that the filter is configured and active we actually want to put it to use – which is quite easy. The filter scans all Responses it encounters for the presence of two special headers: “X-Memcached-Filter-Cache-Key” and “X-Memcached-Filter-Cache-Timeout” (the latter is optional). If the “X-Memcached-Filter-Cache-Key” is detected the filter will store the response content in memcached using the header value as key. The expiration timeout will be either a default value or the value of the “X-Memcached-Filter-Cache-Timeout” header. It’s that simple. Here’s a usage example:

class FooController
{
  def index =
  {
    if(!SecurityUtils.subject.principal)      // store response only for anonymous users (using Shiro plugin)
    {
      response.setHeader(MemcachedFilter.MEMCACHED_FILTER_X_CACHE_KEY, ConfigurationHolder.config.app_prefix + (request.forwardURI - request.contextPath))
      response.setHeader(MemcachedFilter.MEMCACHED_FILTER_X_CACHE_TIMEOUT, 120)         // cache for two minutes
    }
  }
}

This will store the response content for FooController’s ‘index’ action in memcached for two minutes using the context relative request URI as cache key as long as the requesting user is not logged in. It is of paramount importance that you never ever do this for authenticated users or bad things may happen. You don’t want John Doe to access information intended for the boss, only because the boss happened to be the first user accessing a certain page, do you? To make sure that our Frontend server will try fetch content from the cache for authenticated users we need a way for it to tell if a request is from an anonymous or authenticated user. To do this I wrote a simple Grails Filter that ensures that authenticated users are tagged with a cookie:

class CacheFilters
{
  CacheService questionCacheService
 
  final static String MEMCACHED_USER_IS_AUTH_INDICATOR_COOKIE = 'user_id'
 
  def filters =
  {
    /** Sets or deletes a cookie that signals nginx that the current user is authenticated - not used for permission checks */
    setNginxUserIsAuthenticatedIndicator(controller: '*', action: '*')
    {
      afterView =
      {
        def cookie = request.cookies.find { it.name == MEMCACHED_USER_IS_AUTH_INDICATOR_COOKIE }
 
        // is the user authenticated (by login or cookie?)
        if(SecurityUtils.subject.principal)
        {
          // yes, set the cookie if needed
          if(!cookie)
          {
            cookie = new Cookie(MEMCACHED_USER_IS_AUTH_INDICATOR_COOKIE, User.get(SecurityUtils.subject.principal).id.toString())
            cookie.setPath(request.getContextPath())
            response.addCookie(cookie)
          }
        }
 
        else
        {
          // delete the cookie if necessary
          if(cookie)
          {
            cookie.setMaxAge(0)
            cookie.setPath(request.getContextPath())
            response.addCookie(cookie)
          }
        }
      }
    }
  }
}

Rest assured that the sole purpose of the cookie is to signal the frontend server that a user is authenticated in order to disable any cache retrieval attempts. No security checks are ever performed against the cookie.

Now that we’ve taken care of storing the content, someone has to retrieve it. As already mentioned we are using Nginx as Frontend server. I’m a big fan of it because of its small footprint, great performance and integrated support for memcached. Below is a Nginx virtual host configuration that implements the remaining bits and pieces:

upstream tomcats
{
  server 127.0.0.1:8080 weight=1;
}
 
server
{
  listen 80;
  server_name example.com;
  access_log /var/log/nginx/example.log;
 
  include ua_ban_list.conf;
 
  proxy_set_header Host $http_host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
  default_type text/html;
  charset utf-8;
 
  proxy_redirect http://tomcats/ http://example.com/;
 
  location /
  {
    # only consider GET requests for caching
    if ($request_method != GET)
    {
      proxy_pass http://tomcats;
      break;
    }
 
    # never cache auth requests
    if ($request_uri ~* (^/auth/.*$))
    {
      proxy_pass http://tomcats;
      break;
    }
 
    # detect cookies that indicate an authenticated user
    if ($http_cookie ~* "(rememberMe|user_id)")
    {
      # don't try cache lookup for authenticated user - NEVER
      proxy_pass http://tomcats;
      break;
    }
 
    # compute cache key from app prefix + request_uri
    # if they computed key does not match the key computed by the application
    # server when storing content we will get nothing but memcached misses
    # and NO speedup
    set $memcached_key example_$request_uri;
    memcached_pass memcached;
    error_page 404 = @cache_miss;
  }
 
  location @cache_miss
  {
    internal;
    proxy_pass http://tomcats;
  }
}

Please pay special attention to the line where the cache key is computed. A key prefix “example_” is used. To get the example to work you would have to add a config variable ‘app_prefix’ to Config.groovy which needs to have the value “example_”.

Finally I should point out that this approach isn’t tied to Grails applications. Basically any Java Web application could use it if someone would port the MemcachedFilter from Groovy to Java which should be a piece of cake considering that it was developed from Java Sources in the first place.

That’s it for now. I hope you enjoyed the ride.

My first problem was to come up with a viable cache key scheme since simply using the full request URI as suggested in the article wouldn’t work for me because my site renders a different version of a navigation menu depending on the authentication state of current the user. After weighing in the advantages and disadvantages between the super clean variant of factoring the session cookie and all other cookies into the memcached key and a less heavy weight method that would only append a server supplied abstract “page version” field to the request URI, I went for the latter. My resulting nginx virtual host config was looking like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

# define application servers
upstream backend
{
  server 127.0.0.1:8080 weight=1;
}
 
server
{
  listen 80;
  server_name domain.com;
  access_log /var/log/nginx/domain.com.log;
 
  location /
  {
    # we never cache post requests
    if ($request_method = POST)
    {
      proxy_pass http://backend;
      break;
    }
 
    # extract cache key args and compute cache key
    if ($http_cookie ~* "pv=([^;]+)(?:;|$)")
    {
      set $page_version $1;
    }
    set $memcached_key $request_uri&pv=$page_version;
 
    # Check if local memcached server can answer this request
    default_type text/html;
    memcached_pass 127.0.0.1:11211;
 
    # Send to app. server if Memcached could not answer the request
    error_page 404 = @cache_miss;
  }
 
  location @cache_miss
  {
    proxy_pass http://backend;
  }
}

The important lines are line 23-27. Line 23 tests the cookies header of the current request for the presence of a cookie named ‘pv’ using a regular expression and line 25 stores the extracted value of this cookie in a temporary variable. Later, in line 27 we combine the value of that variable with the request URI to form the final memcached key. For example if the request uri would be /foo and the headers would contain a cookie pv=acme123, then the cache key would be /foo&pv=acme123.

Line 31 is where the actual memcached lookup happens. If the computed key is present in the cache, the cached page is returned immediately – completely bypassing the backend. If the page (or the specific version of the page) is not present in the cache, then the alternate branch at line 37 is taken which ultimately forwards the request to our load balanced cluster of application servers in line 39. The actual servers making up the cluster is defined at line 2. There’s currently only one lonely application server defined there and that’s a apache 2.2 listening on localhost port 8080 running our django application through mod_wsgi.

So far I have explained how the front line request processing in nginx works but how is invalidation handled? To tackle that problem I wrote a django middleware that can be applied to arbitrary django views using a decorator:

1
2
3
4
5
6
7
8
9
10
11
12

@cache_page_nginx(3600*6, compute_common_page_version)
def index(request):
  return render_to_response('index.html')
 
def compute_common_page_version(request):
  '''This method is called by the nginx_cache decorator
  The method is supposed to return some value that can be used to distinguish
  different cached versions of the same page. For now it is sufficient to differentiate
  between authenticated and anonymous users'''
  if request.user and request.user.is_authenticated():
    return 1
  return 0

Some readers will notice that the decorator looks very similar to Django’s built-in cache_page decorator. And it actually works similar but with several important differences:

• It totally relies on nginx doing the actual cache lookup. The decorator is only responsible for storing the rendered content of page in the cache.
• It bypasses the Django cache framework and talks to memcached directly using cmemcached or python-memcached. This was done because there’s reallly no point in supporting an additional level of abstraction if the consumer of the cached content won’t look for it anywhere but in memcached.

A word about the compute_common_page_version function. What does it do? The purpose of this method is to ensure that the cache will contain at maximum two distinct versions of the home page. Once for authenticated users and one for anonymous users. For more complex scenarios, the complexity of this method would also increase, resulting on more distinct versions of the same page. Obviously the difficulty lies in maintaining data integrity for every user while not producing to many cached versions of a page, thus increasing memory pressure on the server and lowering cache hit ratio.


The decorator:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

from django.utils.decorators import decorator_from_middleware
from django.conf import settings
from django.core.cache import cache
from django.utils.encoding import smart_unicode, smart_str
from utils import kdebug
 
try:
    import cmemcache as memcache
except ImportError:
    try:
        import memcache
    except:
        raise InvalidCacheBackendError("Memcached cache backend requires either the 'memcache' or 'cmemcache' library")
 
class UpdateCacheMiddleware(object):
  """
  Updates memcached with the response of the request. It is of _paramount_
  importance that the generated cache_key matches exactly the key generated
  by your web to lookup the page from the cache.
 
  This class talks to memcached, bypassing Djangos cache backend because
  it is only meant to talk to memcached and nothing else.
 
  Must the first piece of middleware in settings.MIDDLEWARE_CLASSES so
  that it'll get called last during the response phase.
  """
  def __init__(self, cache_timeout, page_version_method):
    '''timeout - is the timeout in seconds after which the cached response expires in memcached
    page_version_method - is called during response processing and must return an arbitrary value
    that can be used to distinguish different cached versions of the same page '''
    self.cache_timeout = cache_timeout
    self.page_version_method = page_version_method
    self.cache = memcache.Client(settings.NGINX_MEMCACHED_SERVERS)
    self.cookie_name = 'pv'
 
  def process_response(self, request, response):
    """Sets the cache, if needed."""
    if not settings.NGINX_MEMCACHED_ENABLE or request.method != 'GET' or not response.status_code == 200:
      # because of interactions between this middleware and the
      # HTTPMiddleware, which throws the body of a HEAD-request
      # away before this middleware gets a chance to cache it.
      return response
 
    self.cache_response(request, response)
    return response
 
  def cache_response(self, request, response):
    '''Manually insertion of cached version of page into the cache'''
    # retrieve page version
    pv = self.page_version_method(request)
 
    # compute key that's follows the same naming convention as specified in nginx configuration:
    # set $memcached_key $request_uri&sid=$session_id;
    cache_key = self.compute_cache_key_from_request(request, pv)
 
    # update cache
    self.cache.set(cache_key, response._get_content(), self.cache_timeout)
 
    # communicate the page version to the browser using cookie
    response.set_cookie(self.cookie_name, pv)
 
  def compute_cache_key_from_request(self, request, page_version):
    '''Computes the cache key for the specified page and version of the page'''
    return self.compute_cache_key(request.get_full_path(), page_version)
 
  def compute_cache_key(self, page, page_version):
    '''Computes the cache key for the specified page and version of the page'''
    return smart_str("%s&%s=%s" % (page, self.cookie_name, page_version))
 
  def invalidate_page_version_from_request(self, request, page_version):
    '''Removes the specified version of the page requested by 'request' from the cache'''
    cache_key = self.compute_cache_key_from_request(request, page_version)
    self.cache.delete(cache_key)
 
  def invalidate_page_version(self, page, page_version):
    '''Removes the specified version of the page requested by 'request' from the cache'''
    cache_key = self.compute_cache_key(request, page_version)
    self.cache.delete(cache_key)
 
# decorator
cache_page_nginx = decorator_from_middleware(UpdateCacheMiddleware)