So, we are building this great community portal for all the world to see. We thought about all aspects that would make
this project a huge success. We are heavily in control. And finally the day has come: our portal hits the worldwide
web. And so it seems, the website is well received! A little too well received. Soon traffic rates hit the ceiling. And
so does our carefully configured web server. End of this little story. We’ve become the victim of our own success.
Performance testing is all about being prepared
OK, so perhaps this tale is a little bit over dramatized. Still, in my daily work I see this kind of misery
happening a lot. To be honest… we tend to step into this pitfall as well from time to time. And we shouldn’t have to.
Testing the performance of your site means that you can actually prepare for large amounts of traffic on your site. You can be predictable.
If we really wanted this site to be a success, we might have had to prepare better for that success.
This is where performance testing comes in. We simply want to know how our website is performing once it hits higher
What are we testing anyway?
But what do we actually test? What about busy periods? Does the caching do its job? Or have we configured things incorrectly?
And when we really tighten the screw, what will happen? Will it bring down our site? What about peak times, just
after our project went viral? And are we fully utilizing the server capacity?
We use a load test to closely monitor the use of the site. This way we can make statements about the performance.
But… it comes down to the same problems anyway?
Without performance testing we can look at common causes that can give rise to performance problems.
Wrongly deployed caching is obvious. And perhaps you simply load too much data on a page, stressing the database.
If you are an experienced Drupal developer, there is a fair chance that guess right we can and sleep peacefully again. In many cases, the real problem is something you did not think of.
There are many different types of performance tests. We will discuss some.
With a load test we mainly look at the expected load on the site. So: if we build an intranet, we want to know what happens when, for example, 60% of the company simultaneously uses the intranet intensively.
With stress testing we look more at deviant situations. An exceptional load. In the example of our fansite: we appear in the news and therefore suddenly have a multitude of visitors.
Our example is of course exaggerated. It is difficult to anticipate an unexpected and absurdly high number of visitors. But what we do want to know: where is the border?
With an endurance test, we test the site measured over a longer period, to investigate whether the site will perform worse over time, for example due to the increase of logged-in users or content.
With peak testing we mainly look at intensity peaks. In the example of the intranet you can think of a peak in the morning when everyone arrives at the office.
Load testing: 5 steps
Because we want to know if our site can handle the desired number of visitors, and we also want to know how far we can go, we focus on load and stress testing.
The steps are the following.
1. Analysis of any existing data
Simply retrieve from existing statistics, for example visitor numbers. This helps to create realistic scenarios in our test preparation.
In the analysis we can fall back on tools such as Google Analytics. But … if the site is new, we have to make statements about traffic in a different way. Often the customer can share some insights about this. And you might be building a new version of the same site so you can still say a lot about expected visits.
2. Think of some scenarios
Make sure you will test with realistic scenarios. Prepare the test well! One scenario is not enough. We want to simulate that a large number of visitors, simultaneously, click on several pages, download documents, log in, send forms, etc. Therefore, make sure you have enough CPU on your machine to perform the tests. And … determine limit values for the test, to assign a score to the performance.
The APDEX may be of help here. The APDEX is an industry standard to make statements about satisfaction with performance based on tests. Actually, we can thus determine whether the performance meets the expectations!
Response times alone do not say much. We can interpret this with the APDEX. The APDEX is a score based on satisfaction surrounding the performance.
A lot of tools, such as New Relic and Jmeter, therefore use the APDEX.
The APDEX calculates a score based on measurements:
Satisfying, Tolerating and Frustrating.
We will apply a time aspect here, for example:
Satisfying: 0-1.5 seconds;
Tolerating: 1.5-6 seconds;
Frustrating: 6 seconds or more.
- Satisfying results count for 100% (this is great)
- Tolerating results count for 50% (this is ok)
- Frustrating results do not yield a score (this is not good)
And then it is time to do some calculations.
APDEX = Satisfied + (Tolerated / 2) / Total samples
3. Pick the right tool
Determine which tooling we can use to carry out the test. And, well ok, there is a lot to be found out there. Let’s try to highlight some.
- AB - Apache Bench - What can we say, this is very easy. It proves some low level local testing is not that hard.
- Gatling - A paid (and great) alternative for powerful open source tooling like JMeter. Check out this extensive comparison.
- Locust: Define user behavior with Python code.
- Siege - Similar to AB, but more extensive, useful to perform a quick load test.
- Blazemeter - Simple characterization: a hosted version of Jmeter. Less simple characterization: testing on steroids. Offering A LOT when it comes to testing. Test from all over the world, very large numbers, hosted Selenium, Gatling, Locust and more…
- Jmeter - Open source, many possibilities, the old, the proven solution. Still goin’ strong.
We should point out that Blazemeter stands out as a modern, versatile cloud test platform (not only load testing!).
- Easy / quick boarding
- Drupal (D7 + D8) stable module available
- Important advantages with respect to JMeter: geography is not limited to your own local environment and processing is not limited to your own machine.
Yes, a con: It is a commercial solution, it will cost you money…
4. Carry out the test
In part 2 of this series we will actually carry out a test in Siege and Jmeter.
After running the test we will analyze the results. Do we learn more about problems in our application?
In part 3 of this series, we will look at profiling and analyzing.
Coming up in Load testing in Drupal (part 2):
- Performing load tests to your Drupal application in Siege and JMeter.
Coming up in Load testing in Drupal (part 3):
- Profiling and monitoring your Drupal application with XHProf / XHGui and New Relic.