Although migrating to Discount ASP was straightforward, things are never as easy as first impressions would suggest. A few code changes had to be made in order to get some of the basic functionality working.
I'm paranoid about performance, so I've integrated a cache system into the core of the design. Whenever the framework requests a file from an external source, the result is always stored close to hand. This cuts down on network traffic, and ensures quick access to data retrieved by previous processes.
The original framework design used the HttpRuntime Cache built into ASP.NET. This in-memory cache exists for the lifetime of the Application object. Data can be stored here and in theory should stick around for a while, or at least until it is set to expire. This was ideal on the old server, as I had few other options available. Nevertheless, this didn't work as expected on the new server.
Running a few tests, I discovered that items stored in the Cache didn't persist for the hours I was expecting, and were instead purged within about ten minutes. Nothing specifically wrong with this - the HttpRuntime Cache is supposed to be temporary after all - but it's not exactly perfect for medium-length data storage. Ideally, I'd want to cache reasonably static data for 24 hours at a time, so a better solution needed to manifest itself.
To this end I created a very simple Cache table in SQL Server 2008; consisting of a unique ID, Expiry Time, Binary field, and a separate Length field, allowing the total size of the cache to be determined with a simple SUM(binaryLength). Stored Procedures were coded to allow Inserting, Querying, Retrieving and Purging of cached data, and the existing cache code modified to follow a simple three step pattern:
- Check the HttpRuntime Cache first. If it's there, return it.
- If not, then check the Database Cache for a non-expired copy. If it's there, write it to the HttpRuntime Cache and return it.
- If not, then and only then attempt to retrieve from an external source. Write it to the Database Cache and HttpRuntime Cache and return it.
(A future step would be to fall back to expired data if the external attempt fails. This will require a buffer zone between cache Expiry Time and the time it is actually deleted from the cache table. The Purge Procedure can be tweaked for this.)
The cache code is sufficiently flexible to allow any binary stream to be Inserted and Retrieved, and not just data from external sources. This ensures that generated graphics are not recreated upon each call. Instead they are set to Expire at the same time as the source data used in construction.
I chose the database route instead of going with a filesystem based cache as cached items are intended to be small. The largest item should be no more than 100kb, and most data sources have been substantially smaller - less than 1kb from sites that publish proper APIs. There will be a database overhead for checking file metadata anyway, so storage may as well be in the same process. It works for now, but I'll be keeping an eye on things. If it ultimately proves more performance-effective to use the filesystem, it wouldn't be too hard to change.
It would be nice to implement the ASP.NET Provider Model (as used with User and Role management), and create a pluggable cache architecture. This will allow a standard interface for calling cache functionality from various developments. The benefit is to ease the integration of third party components such as Flickr.NET and especially the various BlogEngine Extensions. These implement their own mechanisms for caching external data, either using the Filesystem or HttpRuntime. For consistency's sake, it would be nice to have all cache requests go through a single chunk of code.
It's probably outside the scope of the framework I'm developing, but it's something that I could get round to implementing. The Database Cache code is currently sat in a separate web application from the Blog code, so cannot be called directly. Once things are stable enough; it can be compiled together, slapped in the Blog's /bin directory, and the Extensions tweaked accordingly. A bit of a diversion, but simple enough to do.
In theory.
Subscribe