Out of Bandwidth Cache Expiry

August 29th, 2007

So, you’ve been merrily caching away and using the cache plugin I wrote and you’re still getting some heat about your site sometimes “hanging”. I reckon the problem might be that the person complaining is getting stuck behind a slow running process that is re-caching a page on your site. So what to do?

What I’ve done is started an extra mongrel in my pack but not included it in the standard mongrel pack, my (cut down) mongrel config looks like this:

1
2
3
4
5
6

--- 
port: "2222"
environment: production
address: 127.0.0.1
pid_file: log/mongrel.pid
servers: 7

As you can see there is 7 mongrels there. Next, setup a second cluster that contains only your 7th mongrel

1
2
3

<Proxy balancer://expirecluster>
  BalancerMember http://127.0.0.1:2228
</Proxy>

Next, tell apache to redirect any of your magic expiration URLs to that cluster

1
2

  RewriteCond  %{QUERY_STRING} magic_url_expire_param=1$
  RewriteRule ^/(.*)$ balancer://expirecluster%{REQUEST_URI} [P,QSA,L]

Restart apache and you’re done. Any cron triggered expire requests will now go through the out of bandwith mongrel and not slow down visitors to you site.

This method can actually be used to segregate any URLs off to different clusters, I have a project coming up that will have a lot of slow processes that rely on an external SOAP service running. They’ll all be farmed off to a cluster running on a totally different machine with trimmed down, DB free mongrels.

1 Response to “Out of Bandwidth Cache Expiry”

  1. Tim Lucas Says:
    September 5th, 2007 at 06:10 PM Really good idea about moving the long-running mongrels out into a different cluster. Mebe a good idea to chuck an IP filter on the magic_url_expire_param RewriteCond...

Sorry, comments are closed for this article.