In our previous blog post we compared the performance between Magento CE 1.9 and Magento CE 2.0. The results of the performance test indicate that Magento 2 is half as fast Magento 1. In addition, Magento 2 can be 7 times more expensive to host due to the need for additional CPU resources.
We’ve received a lot of feedback from the Magento Community and are happy to see so much interest and focus on our analysis. This truly indicates how much excitement surrounds the Magento 2 project.
This blog post continues our series of articles about Magento 2 and concentrates on the performance analysis related to PHP 7 specifically.
While using PHP 7, the distance between the systems was reduced allowing Magento 2 to process many more requests comparing to PHP 5 and only 24% less requests in a given load scenario comparing to Magento 1 (28,751 total number of requests for Magento 2 comparing to 37,604 total requests for Magento 1). Details of this test are shown on Graph 2. Magento 1 and Magento 2 total requests comparison when running PHP 7 below.
The discrepancy between Magento 1 and Magento 2 is still significant when we look at it from a user experience perspective. As we can see in the paragraph ” M1 vs M2 server response time with PHP 7 ” , Magento 2 took twice as long to respond to a user request compared to Magento 1.
It is worth mentioning that the Magento 2 project was initially based on PHP 5 but then announced PHP 7 support. During the comparison process we compared both Magento 1 and Magento 2 using PHP 5.5 version, as well as analyzing their performance on PHP 7.0.3.
Before the load test, all indexes were generated/refreshed, static content was deployed, cache was cleared, code compiled, JS, and CSS minified.
Hardware, software, and testing methodology
The test was performed on Amazon EC2 instance.
- EC2 C4.8XLARGE (CPU 36 cores, RAM 60 Gb)
We used the same software and testing methodology during our previous round of analysis with the exception of PHP version where we decided to make it variable.
Magento 1 vs Magento 2 performance analysis while running PHP 5 and PHP 7
Magento instances were tested using PHP 5 with a goal to identify major bottlenecks making Magento 2 a slower solution.
Graph 1. Magento 1 and Magento 2 total requests comparison when running PHP 5
The graph shows that Magento 2 can only process half as many requests compared to Mageno 1 on PHP 5. Knowing that PHP 7 significantly improves PHP processing, we applied several patches to Magento 1 and ran the system on the latest version of PHP7.
Note: Magento 1 is not officially supported with PHP 7. However, it only took us a few hours to apply publicly available community patches and additional fixes to make it work. We are planning to provide additional details on that in one of our upcoming blog articles.
Graph 2. Magento 1 and Magento 2 total requests comparison when running PHP 7
This graph shows that Magento 1 performed a little bit better (~16%) with PHP 7, but Magento 2 performance improved significantly, allowing it to process almost double the amount of requests that could be done by the server without a failure.
Graph number 3 below compares the results of running Magento 2 on both PHP 5 and PHP 7.
Graph 3. Magento 2 total requests comparison when running PHP 5 and PHP 7
Magento 1 vs. Magento 2 Server Response Time With PHP 7
Server response time affects user experience, especially when a site has a large number of concurrent users. In our next comparison we find that Magento 2 fails to respond to user requests in an acceptable time much more often then Magento 1.
On a product page the M1 server response time was 250 or less ms (milliseconds) in 90% of requests as shown in the graph number 4.
Graph 4. Magento 1 product page response time distribution when running PHP7
On M2 less than 25% of requests were able to achieve the same 250 ms response time. As shown on the graph below, the majority of requests to the server resulted in a response time of 600 ms and more.
Graph 5. Magento 2 product page response time distribution when running PHP7
The server response time differs between the systems depending on the specific page. In general, we found catalog pages took twice as long in M2 (response of 500 ms or more) for the majority of requests, while in M1 the server responds in 300 ms and less for the majority of requests.
Additional details are available in the following Gatling reports:
Load Test Results With PHP 7 On Different Server Types
In order to determine minimum server environment configuration recommended for production deployment we repeated the same tests using Magento CE 1.9 and Magento CE 2.0 on the following environments:
Additional details about EC2 instances are available on https://aws.amazon.com/ec2/instance-types/
Results for Magento CE 1.9 on all tested servers are nearly identical. Under actual load Magento 1 has no visible performance bottlenecks. At the same time, results for Magento CE 2.0 are only the same between c4.8xlarge and c4.4xlarge while c4.2xlarge infrastructure is not as fast. Based on this analysis the minimum server type for Magento CE 2.0 to handle 20 concurrent requests per second is c4.4xlarge ($453.33 per month, 1 year term) while Magento CE 1.9 a c4.xlarge ($113.15 per month, 1 year term) is enough.
Official information about PHP 7 from Zend website:
Introducing PHP 7 – a revolution in the way we deliver applications that power everything from websites and mobile to enterprises and the cloud. This is the most important change for PHP since the release of PHP 5 in 2004, bringing explosive performance improvements, drastically reduced memory consumption, and a host of brand-new language features to make your apps soar.
Thanks to the new Zend Engine 3.0, your apps see up to 2x faster performance and 50% better memory consumption than PHP 5.6, allowing you to serve more concurrent users without adding any hardware. Designed and refactored for today’s workloads, PHP 7 is the ultimate choice for web developers today.
The difference is especially noticeable on the modern system with a flexible architecture such as Magento 2. At the same time, we should still remember that Magento 2 consumes much more CPU resources when compared to Magento 1 no matter which PHP version. Magento 2 also delivers pages with a longer delay (page response time distribution) comparing to Magento 1 while under load.
Based on these test results PHP 7 does help Magento 2 acheive better performance, as well as Magento 1. We believe Magento will address performance issues and improve the new product at some point in the future. We also hope Magento will enable PHP7 support for Magento 1 merchants as well.
About the Author
As the former lead Architect of the Magento project, Dima was responsible for planning and executing technical delivery for all the Magento Inc. products and making sure Magento delivers high preforming scalable product to its customers. In addition, Dima’s leadership and involvement in countless complex implementation projects made him one of the most authoritative and respected figures in the Magento community.
Following his successful career at Magento, Dima joined forces with Yoav Kutner and Jary Carter to found Oro Inc. As the CTO of Oro Inc. and its service arm MageCore, Dima is leading the development and technical execution of highly scalable open-source business applications and continues to leads complex Magento implementations projects.