Skip to main content

Magento 2.2 benchmark

· 5 min read

Few weeks ago, one of our customers asked to benchmark one of our extensions. To do that, we first needed to perform a benchmark on a clean Magento installation – we chose Magento 2.2.4.

We created our benchmark test using Apache Jmeter. Jmeter is a very handy tool to perform benchmarks and load tests. It is very easy to create a test with Jmeter – it has a recorder tool that lets you record your test by listening to a browser. After recording the test, the Jmeter lets you run it and choose the load of your test (the “Ramp-up Period” and “Number of Threads” – those definitions are explained later on in the article).

The Goal of the Benchmark​

In our benchmark we wanted to check how many concurrent buying customers can servers with different capabilities handle within specific period of time.

Therefore, we run our Jmeter test on different combinations of the following factors – server capabilities, the “Ramp-up Period” and “Number of Threads”.

The Test Structure​

The test performed the following actions:

  • Go to the category page
  • Add product #1 to the cart
  • Add product #2 to the cart
  • Add product #3 to the cart
  • Go to the cart page
  • Go to the checkout page
  • Send checkout address & shipping information
  • Send payment information and place the order

The Servers​

We used 3 Digital Ocean droplets (servers) to perform our tests. We used droplets with the following specs:

  • 2 vCPUs / 4GB RAM
  • 4 vCPUs / 8GB RAM
  • 6 vCPUs / 16GB RAM

Magento Installation Specs​

Magento version: 2.2.4 CE Mode: Production Cache: FPC Database: MySQL 14.14

The Results​

Magento 2.2 benchmark results

Understanding the “Number of Threads” and the “Ramp-up Period”​

The load of the tests is defined by the two parameters – the “Number of Threads” and the “Ramp-up Period”. The number of threads is the number of tests that you want to run, and the ramp-up period is the period in which all the tests must be started.

For example, if our test is 150/300, ie 150 number of threads and 300 seconds ramp up period, it means that all 150 tests (where a test is the sequence of – go to the category page, add product #1 to the cart and so on) must start within 300 seconds.

Another way of looking at 150/300 test is as “1 purchase session every 2 seconds”, ie every 2 seconds we have a customer coming to perform a purchase in our website.

Results Analysis​

As expected, the results show that more powerful servers can handle higher loads. By handling we mean the number of completed checkouts and the speed of the (add to cart and place order) requests completion by the server.

For example, we can see that the 2 vCPUs / 4GB RAM completes only 104 out of 150 checkouts (in 150/300 test), with average speed of 13670ms for add to cart requests, and 49081ms for place order. On the other hand, the 4 vCPUs / 8GB RAM doesn’t fail at all on 150/300 load, and the add to cart request takes only 836ms, while a place order requests takes only 1654ms.

Why do some requests fail?​

The exception log indicates that the failures happened due to MySQL time outs (the error in the log is “General error: 1205 Lock wait timeout exceeded; try restarting transaction”). Therefore, we believe that if we would increase the MySQL time out, all the requests would eventually complete (although it might take “forever”). However, we don’t know for sure, since we haven’t tried it out.

Using the benchmark results to choose the right server for your website​

You can use the benchmark results to choose the right server for your website. To do this, you need to find out what is the maximal load that your website is expected to go through. After that, you can estimate what is the number of threads and the ramp up period that matches your load.

Example #1:​

Let’s assume that your maximal load is 1 checkout session every 2 seconds, and the usual time that such load lasts is 2 minutes – than the test that matches your load is 60/120. 120 represents the 2 minutes (that is, 120 seconds), in which the maximal load takes place. 60 is because within those 120 seconds, there is a checkout session happening every 2 seconds.

In this case, the best server that suits your needs is 4 vCPUs / 8GB RAM although the 2 vCPUs / 4GB RAM would also handle the load, but with significantly slower response times.

Example #2:​

Let’s assume that your maximal load is one checkout sessions every second, and the usual time that such load lasts is 5 minutes – than the test that matches your load is 300/300.

In this case, the best server that suits your needs is 6 vCPUs / 16GB RAM.

Summing up​

It is very important to choose a server with the right capabilities for your Magento 2 store. The server should be able to handle the maximal expected loads. Otherwise, the user may experience slow response times, or even response timeouts, causing website abandonment and revenue loss.

We hope that our benchmark and our guide of how to choose the right server using the benchmark, helped you, or at least shed some light on the Magento 2.2 performance on the different servers.