Since 2017 I’ve worked as the Technical Lead for a Sydney Media publisher. My team and I build and manage over 70 commercial WordPress websites generating over 1M page views per month and regularly sending content to over 500,000 subscribers.
While I wouldn’t necessarily call this massive scale, it’s definitely required a lot of on-going work and optimisation to ensure everything runs smoothly.
In this post I wanted to discuss a few of the things I’ve learnt running WordPress sites at this kind of scale. WordPress is a very capable platform and can handle almost everything you can throw at it, but at some point you’re going to hit the ceiling of what a ’normal’ WordPress installation can handle.
I’ll be assuming your WordPress site is running in some kind of Cloud environment where you can customise the infrastructure to meet demand. If you’re running in a traditional shared hosting environment there will likely be a limit to what you can do, and in this case I recommend you contact an expert for help. Feel free to send me a message if I can be of assistance.
I’m also assuming you have Administrative access to your WordPress installation.
Two-tier WordPress
One of the things I’ve found is when a site is under heavy load is the WordPress database is one of the first things to start struggling. Many WordPress sites when they are first setup start with the database running on the same ‘physical’ server as the WordPress installation and when the WordPress site is under load it can have negative affects on both the database and the Web server.
Here’s an example screenshot of a WordPress site under heavy load. You can see in the highlighted portion both CPU cores running at 100% and the top process using all the resources is MySQL, which is the WordPress database.
When this happens you’ll start to experience degraded performance on your WordPress site and may even see errors such as a Timeout.
Moving the database to an external database server can help resolve some issues because it gives the database room to grow without affecting the other processes, such as Apache, on the main Web server. Having an external database is also the first step towards being able to load balance the traffic across multiple WordPress web servers.
Caching
When you request a Webpage, the request reaches your Web server, which is commonly Apache, and then returns the requested resource. If the resource you’re requesting is just a normal HTML page, Apache will fetch that page from the filesystem and send it straight back to the browser as is.
In the case of a dynamic Web application such as WordPress, there’s a few more steps in between Apache receiving the request and returning the response. WordPress is written in a programming language called PHP which works alongside a database such as MySQL, to essentially ‘create’ your webpage each time it is requested. Apache receives the request and hands it over to PHP. PHP starts to generate the webpage and usually has to fetch data from the database in order to complete the final page. This data could be WordPress posts, comments, user profile information, even settings that plugins or themes might store in the database, and when PHP has all that information then it creates a HTML page to send back to the browser.
All this additional processing that WordPress, PHP and MySQL have to do adds additional load to the server.
One way to ease that load is to incorporate caching. In a very simplistic explanation, you can think of caching as a pre-created ‘screenshot’ of the Webpage. In exactly the same way that WordPress creates the page described above, rather than the page being sent to the browser and then lost forever, a cache stores that page for re-use. Next time someone asks Apache for the page, instead of generating it all over again and fetching more stuff from the database, Apache first looks to the cache of already created pages. If the page exists no more work is required and the page is sent to the browser.
There’s a number of ways to set up caching on a WordPress site, but a common method is to use a plugin such as W3 Total Cache. This plugin has extensive settings that allow you optimise many parts of the WordPress site, in particular caching for pages.
Offload your media
Similarly to how I mentioned above where you can separate your WordPress database, you can also separate your uploaded media such as images. By default, WordPress stores any images you upload to your site on the web server file system, which not only takes up disk space but it requires WordPress and Apache to worry about sending them to the browser. If you have lots of images this can take up a lot of resources for your server trying to send them all to your visitors.
By moving them off your Web server you ease that resource burden, increasing the performance of your website.
One way to do this is by using the brilliant plugin WP Offload Media by Delicious Brains. WP Offload Media can store your images in an external location for example, Amazon S3, and will take care of not only offloading the media for you, but also rewriting all the media attachment URLs in your posts.
Server scaling
If your WordPress site is running on a very small server instance it might be worth increasing this to a larger size to allow your site to accommodate increased traffic. In AWS you can create a snapshot of your running server and then ‘spin up’ a larger server based on that snapshot. This has the benefit of not requiring re-configuration of the server and the application and can allow you to move your site to a larger web server fairly painlessly. This is known as vertical scaling, meaning you’re scaling your server UP to a larger size.
An alternative to increasing the size of the server is to scale your WordPress site horizontally. This basically means adding multiple servers to handle the traffic rather than relying on one large server. I discussed above separating your WordPress database to an external database server, so the next step in vertical scaling is to allow your WordPress site to run across two or more Web servers and using a Load Balancer to manage the traffic. It’s beyond the scope of this article to discuss load balancing and horizontal scaling and should be something discussed with an experienced Cloud Administrator.
These are just a few of the options to consider when your WordPress site starts to grow. An increase in traffic can mean your WordPress site starts to experience performance issues, your site will be slow to load and if you are trying to serve customers to your business it could mean loss in sales or frustrated users.
A WordPress site requires constant care and attention to allow it to continue growing and serving your business.