servd

Caching Craft CMS

Posted: 18th May 2021

Numerous studies in recent years have demonstrated that high page load time is a killer for both user engagement and online revenue.

Shopify compiled several studies and found that a one second decrease in page load time can add an additional 11% to your site's pageviews, and 7% to an ecom store's revenue.

This effect is compounded on an increasingly mobile-first web, where network connections are often slower and less stable than on desktop.

So page load time should be a key part of any website's optimisation strategy, but how can we achieve that with Craft CMS? There are a few solutions, but non are more effective than caching.

What is Caching?

Caching can be applied in many ways and at many different places within our Craft projects, but all of these methods have a single goal in mind.

Caching is the storage of a 'result' so that we don't need to re-do the 'work' every time it's required.

This 'result' can be the output of a function, the response from an API call or the entire HTML of a webpage. Each of these can be stored so that the next time they are requested we can quickly grab it, rather than re-running whatever process is needed to generate it every time.

Caches are normally stored temporarily, allowing us to re-run the 'work' on a regular basis in order to make sure our cached version is kept up to date. However, this re-run should be much less frequent than the rate at which our application needs to actually fetch the result.

This allows us to vastly reduce the amount of resources we spend on doing the work, with the trade-off of sometimes having an out of date result.

The Methods

We're talking about Craft CMS, so we'll focus on methods of caching which we can use in the context of web requests, reducing load times for our end users.

We'll begin at the lowest level and build up from there. So here's what we'll cover:

  • Data caching
  • Query caching
  • Twig {% cache %} tags
  • Caching with plugins
  • Web server caching
  • CDN Edge caching

That's a lot of potential cache.

Data Caching

Craft is built on top of a framework called Yii2 which provides a ready-made mechanism for caching arbitrary data directly from our project's PHP code. This is known as the Data Cache and can store anything which PHP is able to convert to a string (via serialization or otherwise).

The cached data is stored in whatever storage device the cache component in your project has been configured with. By default this is the filesystem and your cached data will simply be stored in files on the hard disk, but you can also swap that out for Redis, memcached or even your existing database (but don't use the database, it'll be slow).

Craft already uses the data cache to store several pieces of information which it needs on a regular basis, but you can also add and fetch your own data too.

Because the Data Cache is primarily accessed via raw PHP code, it's most applicable in the context of custom plugins or modules.

Here's an example:

$myKey = 'a-unique-key';
$myValue = 'Cache Me!';

// Set the cached value for 60 seconds
\Craft::$app->cache->set($myKey, $myValue, 60);

// Get the cached value
$cachedValue = \Craft::$app->cache->get($myKey);

Normally we'd wrap some expensive work in a check to see if the cache already exists and only perform that work if it doesn't.

There's a nice helper function to perform this logic for us:

$myKey = 'a-unique-key';

// Get the cached value, but if it doesn't exist, re-do the work
// and store it for 60 seconds
$result = \Craft::$app->cache->getOrSet($myKey, function(){
    // Some expensive work goes in here
    return 1 + 2;
}, 60);

echo $result; // 3

You can also access the data cache directly from your twig templates. This is useful if you need to cache the result of someone else's plugin and you can't change their source code:

{% set interestGroups = craft.app.cache.get('mailchimp_groups') %}

{% if interestGroups is empty %}

    {% set interestGroups = craft.mailchimpSubscribe.getInterestGroups('audience-id') %}
    {% set success = craft.app.cache.set('mailchimp_groups', interestGroups, 60 * 60 * 24) %}

{% endif %}

You can read some more about that here too.

Possible Uses

The data cache can be used in any plugin or module code which generates the same result each time it is executed, or where we can use the same result for a period of time without worrying about how it has changed.

Some prime examples include:

  • The result of an outbound API request which rarely changes
  • The output of a particularly complex PHP function call

Query Caching

Yii also has the ability to cache the results of database queries on your behalf and then recall them whenever you run the same query again in the future. This can significantly improve response times for requests which perform heavy database queries.

The easiest way to make use of this in your Craft codebase is by calling the cache function on an element query:

$cachedBlogs = \Craft::$app->entries->section('blog')->status(null)->cache(60)->all();

And you can do the same in your twig templates too:

{% set blogs = craft.entries().section('blog').status(null).cache(60).all() %}

You might have noticed that we've added status(null) into those queries. That's because, by default, Craft will filter Entries based on their postDate by including the current date and time in the database query. The query itself is used as the cache key for this form of caching, so if the query contains the current time, the cache keys will never match and the query will never return a cached result! By setting status(null) we tell Craft to not include this time check, and we could add our own .postDate("<= NOW()") to compensate if desired.

Keep in mind that there's no cache invalidation with this system though. Once you set a cache for a specific query, you'll get the same result until the cache expires, even if the database data changes. This can have some unexpected results when working with relationships between different objects, so use this technique sparingly, if at all.

Twig {% cache %} Tags

{% cache %} tags are the caching mechanism that you're likely to run into first whilst building a Craft site. They can be added directly within your twig templates and have a simple syntax which makes them very accessible, even if you don't know exactly what's going on under the hood.

These tags simply cache the output of any code which sits within them, for example:

{% cache %}
  {% for blog in craft.entries.section('blogs').all() %}
    <h2>{{ blog.title }}</h2>
  {% endfor %}
{% endcache %}

The first time that this code is executed, and no existing cache is available, it'll hit the database to grab the entries and output their titles within the loop. Craft will then cache the HTML output of the block ready for the next time it is required.

This can vastly improve rendering times for twig templates which perform a lot of database queries, or perform complex operations in order to organise their data.

Behind the scenes Craft is making use of PHP's output buffering to capture the generated HTML. This is then stored in the Data Cache (which we learnt about earlier).

As well as the HTML output, Craft can also keep track of the elements and sections which were used to populate the content. These relationships between the HTML and the objects within Craft allow automated cache invalidation to occur - when you edit an entry in Craft, all of the cached HTML within which that entry was used will be cleared causing them to regenerate.

This sounds like the perfect solution to our caching needs, but it needs to be added to your templates carefully. A good understanding on how the cache tag generates its keys is required in order to make sure you aren't causing more harm than good. You can read some more about appropriate use of cache tags in section 9 of our 10 Things To NOT Do In Your Craft CMS Twig Templates article.

Caching With Plugins

Everything that we've covered so far has been a core part of the Craft codebase, so you can make use of them without adding anything extra. However, there are also several Craft plugins which can help us by adding additional caching functionality. The most popular of which is Blitz.

These plugins can provide several different extensions to Craft's built in caching mechanisms, including:

  • Integration with web servers
  • Integration with CDNs
  • Automatic cache warming
  • Additional cache storage locations
  • Additional cache configuration options

Integrating this extra functionality into your project can involve a range of different things. For some elements it's enough to simply install and activate the plugin. For others you might need to add function calls to your twig templates. These integrations are usually simple, but can result in your project's code becoming interwoven with references to specific plugins. Refactoring your codebase at a later date to remove or update the plugin can become a challenge.

The majority of caching plugins will perform full-page caching (capturing the entire HTML of a page), rather than only caching individual pieces of data or sections of a twig template like the other methods we've covered. This provides less flexibility, but can offer a significant benefit. By capturing an entire page's HTML there's no longer any need to perform any processing in PHP at all - we can simply send back the fully cached HTML to the user!

But as usual, there's a caveat. The logic needed to decide when and how to send the cached page lives within the plugin, and the plugin relies on PHP, Yii and Craft to do its thing. So even though a page might be fully cached, we still need to:

  1. Send the request to a PHP worker
  2. Boot Yii + Craft, including all of their components - including several database queries
  3. Run all installed plugin(s) initialisation code
  4. Run the logic for finding and sending the cached content
  5. Shut down

This process can still take a while to complete. In our very unscientific local testing on a powerful machine we found the just Craft with no plugins installed takes an average of 25ms to send an empty page, even though it's the performing the bare minimum amount of work. This could easily end up causing problems under a moderate traffic load.

It would be much better if we could avoid sending the request to PHP in the first place and just stream the raw HTML for the page back to the user.

Web Server Caching

Whenever a user sends a requests to Craft it first passes through a web server. The most common web servers in use with Craft are Apache and nginx. These pieces of software will process the incoming request and decide what to do with it. If the request is for an image, the web server will look for the image on the hard disk and send its contents back to the user if the file exists. If the request needs to be processed by Craft, it'll forward the request to the PHP process.

Web servers are extremely quick at doing their job (nginx more so than apache) and can send a small static file back to the user in less than a millisecond. If we compare this to the minimum processing time that we found for PHP + Craft earlier (25ms) we can see that we'd be able to process upwards of 25x more traffic if the webserver was handling everything, and that's without any additional plugins making the PHP requests slower.

There are a couple of ways that we can make this dream a reality:

  • Integrations between caching plugins and the web server
  • Caching functionality within the webserver itself

Plugin <-> Webserver Integration

Some Craft caching plugins (notably Blitz) can integrate directly with the web server in order to ensure it has the opportunity to send cached pages directly, instead of forwarding the request to PHP. They achieve this by writing the cached pages onto the hard disk. The web server can then be configured to check for the existence of these files, and send them directly, before falling back to forwarding the request to PHP.

This can provide the benefit of web-server only performance whilst also maintaining the additional functionality which might be offered by caching plugins.

There's no such thing as a free lunch though! This setup requires custom configuration of the webserver, which can be a challenge if you aren't a regular server tinkerer. It also prevents some more advanced server configurations from being used, including load balancing and horizontal scaling, as it relies on there only ever being a single filesystem to act as the cache store.

If you're happy editing web server config files and you don't expect to be distributing the project over multiple servers in the future, this can be a good solution to offer maximum performance with a relatively small effort.

Direct Webserver Caching

Both Nginx and Apache contain optional features (fastcgi_cache and mod_cache respectively) which allow them to cache full-page HTML without requiring any plugins in Craft at all. This can provide the same performance benefits as a Plugin <-> Webserver integration, but without making your webserver configuration dependent on the plugins installed in your Craft project.

Setting up caching within the webserver often requires a little more knowledge of the server's configuration files, especially if they already contain any tweaks to adjust their behaviour.

However, the benefit is that only a single component (the web server) is managing and using the cache, and this can therefore be done more efficiently and cover more edge cases.

It also provides a better separation of concerns. Your Craft application can remain completely unaware that any caching is taking place and therefore won't be polluted by code related to caching. The web server can just cache everything that passes through it, without caring about the specifics of the PHP application that it's forwarding traffic to.

How To Choose

If you'd like to use the extended functionality offered by a caching plugin, use a caching plugin and try to integrate it with your web server.

If you're comfortable editing web server config files and you'd like to keep your Craft project and the server itself agnostic of each other, then use direct web-server caching.

CDN Edge Caching

There's not much more we can do to reduce the page load time beyond caching the raw HTML output and sending it back to the client as quickly as possible, with one exception: moving the cache physically closer to the user.

Storing the cache closer to the user means that their requests don't have to travel as far down the internet pipes, making the entire exchange take less time.

We can achieve this by using a caching CDN service, such as CloudFlare. Caching CDNs allow you to proxy all of your project's incoming traffic through their servers. They are then able to capture the response from your 'origin' server and store a copy on their own servers.

Subsequent requests will hit the CDN servers, which are close by, first and just get a cached copy directly from them in no time at all. Rather than a request having to travel from Sydney to New York and back again, taking upwards of 250ms, it might hit a server only a few miles away, taking 30ms instead.

If you have a page containing a lot of resources, this difference can certainly add up!

CloudFlare itself has over 300 'POPs' (Point of Presence) around the world, meaning that there's always at least one close to populated areas.

These services do have two primary downsides though.

1. The cache is no longer directly editable by your project, so clearing the cache or invalidating individual pages can become difficult. Most CDN services do offer APIs to help with this, and many Craft plugins (Blitz, Upper, CloudFlare) integrate directly with these to help to automate things, but there's no guarantees that a cache purge will succeed - in our experience they often don't.

2. The cache is now distributed, meaning there are multiple copies of the cache spread around and maintained independently and are not accessible centrally. This means that every POP will need to make a request back to the origin server to get its own copy of the HTML for a page. Unlike our other forms of caching, where the code is only ever executed once to generate the cache, CDNs will require the code to run once per CDN POP. As CloudFlare has 300+ POPs, this means your cache misses could be up to 300 times larger compared to a centralised cache.

It's important to take these into consideration when deciding which caching strategy you should use. A globally distributed audience would benefit from CDN Edge Caching as the content will be held closer to them, but your origin server also needs to be up to the task due to the increase in cache misses!

Caching on Servd

It wouldn't be a Servd article without a little explanation of how Servd does things a little differently!

The Servd platform contains its own caching mechanism which is a based on Direct Webserver Caching, but also includes some of the benefits of Plugin Based Caching, combining raw speed with extended functionality.

It's the easiest solution to configure, simply enable it in the Servd dashboard with a single switch and then add any pages which you do not want to cache. With those changes in place your project will automatically begin caching pages and serving them to your users.

But the fun doesn't end there! The Servd plugin can interact with the cache in a few useful ways by:

  • Automatically clearing the cache for pages which have had their content changed
  • Purging specific URLs from the cache directly from the Craft Control Panel
  • Injecting CSRF tokens into your HTML to ensure <forms>s still work, even when cached
  • Flagging specific sections of your pages as dynamic, so that they remain outside of the cache

You can read some more about how to set up and configure static caching on Servd in our documentation.

In our next article we'll be discussing some of the ways to mix static and dynamic content to keep your content user-specific whilst still benefiting from the additional speed that caching provides.

We'll also be introducing a new feature of the Servd Plugin which makes this a lot more fun.

If the speed of your Craft site is an issue, Servd might be exactly what you need with its static caching functionality deeply integrated with Craft. Give it a try with a free trial.

Start A Free Trial

More Blog Posts

Our New Ingress Platform

US-EAST-4 Outage and Data Centre Fire

Asset Platform V3