Caching Data Fragments

Audience Level

Advanced

Summary

A set of functions to completely handle the caching and retrieval of String data to and from a custom file store. Cached data can be repeatedly retrieved whilst it is still ‘fresh’ and regenerated when it is stale. This is implemented below as a series of JScript ASP functions. A function is included to check whether cached content has become stale or not. A worked example that saves and re-uses the result from a XML/XSLT transformation is described after the source code. All functions are fault-tolerant, e.g. if a cached file goes missing (say it is deleted accidentally) then the functions won't fail catastrophically.

The idea for caching files is that computationally expensive processes (e.g. results from XSLT transformations, certain database queries, etc.) can be saved and re-used for as long as they are still up-to-date (‘fresh’) — this can signficantly save webserver/database resources (reducing CPU load and database connections for example.)

Source Code

/*
Function: getCachedContent()
Description:
Returns:
History:
20040324 2159 UTC   v1  Andrew Urquhart     Created
*/

function getCachedContent(strCacheId, blnIsASPPage) {
    try {
        if (!strCacheId) {
            throw new Error(1, "Required parameter \"strCacheId\" was not defined");
        }

        var strCacheId_internal = "CACHE_" + strCacheId.replace(/\W/ig, "x");
        var strCacheFilepath    = "/cache/" + strCacheId_internal + (blnIsASPPage ? ".asp" : ".dat");
        var blnSuccessStatus    = null;
        try {
            Server.Execute(strCacheFilepath);
            blnSuccessStatus = true;
        }
        catch (err) {
            blnSuccessStatus = false;
        }
        return blnSuccessStatus;
    }
    catch (err) {
        throw new Error(err.number, "Function getCachedContent() failed with parameters strCacheId=\"" + strCacheId + "\". Message=\r\n" + err.description);
    }
}


/*
Function: getCachedContentIfModified()
Description:
Returns:
History:
20040320 2009 UTC   v1      Andrew Urquhart     Created
20040910 0106 BST   v1.1    Andrew Urquhart     Fixed bug '!+objDateLastCached' had been written as '+objDateLastCached' so was always reading date from file via FSO on every request :-(
*/

function getCachedContentIfModified(strCacheId, objDateLastModified, blnIsASPPage) {
    try {
        if (!strCacheId) {
            throw new Error(1, "Required parameter \"strCacheId\" was not defined");
        }
        if (!objDateLastModified) {
            throw new Error(2, "Required parameter \"objDateLastModified\" was not defined");
        }

        var strCacheId_internal = "CACHE_" + strCacheId.replace(/\W/ig, "x"); // The true cacheId

        // Get cache date from memory
        var objDateLastCached = new Date(Application(strCacheId_internal));
        if (!+objDateLastCached) {
            // No cache date from memory, get it from file last modified and then add to memory for next invocation
            try {
                var strCacheFilepath    = "/cache/" + strCacheId_internal + (blnIsASPPage ? ".asp" : ".dat"); // The relative path to the cache file
                objDateLastCached = getFileLastModifiedDate(Server.MapPath(strCacheFilepath));
                Application(strCacheId_internal) = objDateLastCached.valueOf(); // Save this in memory for the next query to save the costly use of FSO
            }
            catch (err) {
                // nothing
            }
        }
        var blnSuccessStatus = false;
        if (+objDateLastCached >= new Date(objDateLastModified).valueOf()) {
            blnSuccessStatus = getCachedContent(strCacheId, blnIsASPPage);
        }
        return blnSuccessStatus;
    }
    catch (err) {
        throw new Error(err.number, "Function getCachedContentIfModified() failed with parameters strCacheId=\"" + strCacheId + "\", objDateLastModified=\"" + objDateLastModified + "\". Message=\r\n" + err.description);
    }
}


/*
Function: putCachedContent()
Description:
Returns:
History:
20040320 2009 UTC   v1      Andrew Urquhart     Created
20040730 1754 BST   v1.1    Converted to use ADODB.Stream and the default characterset set for the global application (strCharSet)
*/

function putCachedContent(strCacheId, strContent, blnIsASPPage) {
    try {
        if (!strCacheId) {
            throw new Error(1, "Required parameter \"strCacheId\" was not defined");
        }
        var strCacheId_internal = "CACHE_" + strCacheId.replace(/\W/ig, "x");
        var strCacheFilepath    = "/cache/" + strCacheId_internal + (blnIsASPPage ? ".asp" : ".dat");

        doSaveFile(Server.MapPath(strCacheFilepath), "UTF-8", strContent, true);
        Application(strCacheId_internal) = (new Date()).valueOf(); // Only update the memory state after a successful attempt to save the cached data
    }
    catch (err) {
        throw new Error(err.number, "Function putCachedContent() failed with parameters strCacheId=\"" + strCacheId + "\", strContent=\"" + strContent + "\". Message=\r\n" + err.description);
    }
}

I appreciate that it might be difficult to see what the above does, so I'll try to explain how I use it in this website:

Note 1: The source code requires that there be a directory named 'cache' in the root of the website — this is where all of the cache files will be saved to and read from.

Note 2: The “doSaveFile()” method mentioned has it's own page: Load And Save File Helper Functions and the “getFileLastModifiedDate()” function is listed below.

Download

Download the source directly.

Example Usage

Here's an example based upon my own sitemap. The sitemap is generated from an XML file of navigation data and an XSLT file. Since the XML file and XSLT file change so infrequently it is a waste of server resources to perform the XML-XSLT transformation on every request for the sitemap page. The solution is to perform the XSLT transformation just once, save the result of the transformation and keep re-using it until the XML (and/or XSLT) file is next updated (or the cache file gets deleted). Here's the small piece of script from the sitemap page that handles all of this procedure:

try {
    // The date of the last time the XML was modified (function contents listed later on)
    var objDateTOCModified = getFileLastModifiedDate("/xml/toc.xml");

     // Attempt to write the cached file contents direct to the output stream
    var blnSuccess = getCachedContentIfModified("SITEMAP", objDateTOCModified);
    if (!blnSuccess) {
        // Couldn't output cached file for some reason (it might not exist, or the XML file may have
        // been modified at some point after the cached file was created).
        // So regenerate and re-save the cached file. FYI: Function doTransformXMLWithAddParam() is
        // described in another scripting example elsewhere in this website. It's returning a big string.
        var strContent = doTransformXMLWithAddParam(strTocXMLFile, Server.MapPath("/xsl/sitemap.xsl"));

        // OK, have now generated the new content, now save it to a cache file:
        putCachedContent("SITEMAP", strContent);

        // Now write the content to the output stream.
        Response.Write(strContent);

        // Now that the cached file has been saved we won't need to regenerate the content until either the
        // XML file is modified again or the cached file goes missing - e.g. gets deleted for whatever reason.
    }
}
catch (err) {
    // Do something with this error
    Response.Write("Error: \"" + err.number + "\"" + err.description);
}

Now for the explanation: "objDateTOCModified" is the modified date of the XML file - it was obtained by invoking the following function:

/*
Function: getFileLastModifiedDate()
Description: Returns the last modified date of a given file
Returns:
History:
20040319 1050UTC    v1  Andrew Urquhart     Created
*/

function getFileLastModifiedDate(strAbsoluteFilePath) {
    try {
        if (!strAbsoluteFilePath) {
            throw new Error(1, "Required parameter \"strAbsoluteFilePath\" was not defined");
        }
        var objFSO = Server.CreateObject("Scripting.FileSystemObject");
        return new Date(objFSO.GetFile(strAbsoluteFilePath).DateLastModified);
    }
    catch (err) {
        throw new Error(err.number, "Function getFileLastModifiedDate() failed with parameter strAbsoluteFilePath=\"" + strAbsoluteFilePath + "\". Message=\r\n" + err.description);
    }
}

Function "getCachedContentIfModified()" (described above) then takes this 'last-modified' date of the XML file along with some unique identifier for the cached file - in this case the word ‘SITEMAP’. Function "getCachedContentIfModified()" compares this date with the date that the cached file was last saved. If the date on the cached file is older than the 'last-modified' date of the XML file, then "getCachedContentIfModified()" quits without fetching the cached content since the content is stale and no longer useful. If the modified date of the XML file was as fresh as the date on the cached file, then we write the cached file to the output stream directly via function "getCachedContent()" and "Server.Execute()".

The "if" statement that follows the call to "getCachedContentIfModified()" checks the status of our attempt to fetch the cached content. If the value is true then the content has been fetch from the file cache successfully. If it's false then we haven't retrieved the cached file for some reason, so we'd better do the expensive XSLT transformation and then re-save it into the cache for the next time. So we do our XSLT transformation with function "doTransformXMLWithAddParam()" (more details of that function at the "XSL Transformation With Parameters" page). The resultant string from this is then passed to function "putCachedContent()", which saves the string as a file. We then just "Response.Write()" the already generated XSLT transformation result to the output stream.

On subsequent requests then for the sitemap page we'll do the above routine again except that the routine will find a previous cached copy of the sitemap XSLT transformation and just write this cached copy direct to the output stream. The web server would thus be saved from having to perform the expensive XSLT tranformation unnecessarily.

Advertisement

Feedback

Voting Panel
Is this useful?
or
Did you find any bugs?
or
Did it solve your programming problem?
or
Rate this script: (0=poor, 5=very good)
Answers are anonymous, only the combined totals are stored. Uses cookies.