“That solution doesn’t scale”

I posted yesterday about a poor man’s distributed caching solution using Windows Azure queues and ASP.NET cache. I’ve got an interesting comment in twitter:

image

My short answer is that there aren’t solutions that scale or doesn’t scale. The scalability is a quality attribute of a system and it varies depending on the context. A good system is the one that is easily adaptable to new contexts and a good solution is the one that is the most convenient in a local context.

Putting things in context

So here is the analysis of the solution proposed and the context where it might applies.

I used the following parameters in the calculation:

  • Time to dequeue 32 messages at a time: 1600 milliseconds (source)
  • Time spent notifying ASP.NET cache the dependency changed: 300 milliseconds (this is a worst case scenario, it is actually in the nanosecs scale)

image

Some conclusions you can take from this

  • From 1 to 1000 items updated per second (i.e. items invalidated) there is a penalty of 3 minutes (in the worst case) to invalidate the cache of those 1000 items. We are talking about items invalidated PER SECOND. Even having 1 update per second is a lot, in certain systems, so this seems to be good enough for many applications.
  • Passing the 1000 items barrier, the time to invalidate all the cache items could be unacceptable (e.g.: you don’t want to wait hours to invalidate the cache). However, if you have more than 1000 updates per seconds, you are probably having other problems ;)

So what is the drawback compared to a distributed cache?

If you compare this solution to something like memcached, the main difference resides when you have lots of web servers in a farm. This is because memcached will replicate and keep synchronized the cache items between nodes. So when you insert an item in memcached it is available to all the web servers almost immediately. However, when using the ASP.NET Cache and the cache dependency mechanism, when the item is invalidated, EACH web server will have to retrieve the item again and insert it in the cache. Retrieving the resource is the expensive operation, but not invalidating the cache.

I hope this clarifies a bit what I meant by “poor man’s memcached” :)

Published: June 27 2010

blog comments powered by Disqus