Magento 2 Load Testing Tools Compared: JMeter, k6, Gatling, Siege and VoltTest

Magento 2 load testing has been a JMeter job for a decade, and that was fair: Adobe shipped a JMeter plan in the core repository and every agency runbook pointed at it. That plan no longer ships with the codebase. The store you are testing probably runs a front-end architecture the original JMeter plan predates, such as Hyvä or PWA Studio. And the people asked to run the test are PHP developers, not performance engineers.

This post compares the five load testing tools for Magento that an Adobe Commerce or open-source Magento team hears about when it asks "what should we load test with": Apache JMeter with the Performance Toolkit, k6 with the community Magento toolkit, Gatling, Siege and VoltTest. It ends with a guest checkout scenario in PHP, because checkout is often where Magento's uncached, write-heavy workload exposes capacity limits first, and no tool gives it to you for free. One disclosure: I build VoltTest, and I ran the Laravel and Symfony benchmarks on this blog the same way. Where it falls short for Magento I say so, and there are no benchmark numbers on this page; the measured results are coming in a follow-up.

Why Magento 2 Load Testing Is Different

A tools comparison only makes sense against the thing being tested, and Magento performance testing has three properties that disqualify most generic advice.

Full page cache hides the real capacity. With Varnish in front, an anonymous visit to the home page, a category or a product that hits full page cache never touches PHP. A load test that only browses cached catalogue pages is therefore closer to a Varnish benchmark than a Magento one. The moment a shopper adds to cart, logs in or reaches checkout, every request goes to php-fpm, MySQL and Redis. You can tell the two apart from outside: Varnish sets an Age header above zero on a hit, and in developer mode Magento adds X-Magento-Cache-Debug: HIT or MISS.

Checkout is write-heavy and uncached. Creating a cart inserts into quote, and every item change recalculates totals. Estimating shipping runs the carrier collectors. Placing the order writes sales_order, its items and addresses, and, on a store with the Inventory Management modules installed, an inventory_reservation row per product, all in one transaction. None of it is cacheable. A test that never reaches payment-information has skipped one of the most expensive parts of the purchase flow.

The first ceiling is often not raw web-server throughput. PHP worker capacity, CPU, memory, MySQL, Redis, OpenSearch and third-party integrations can each become the limiting factor. Which one goes first depends on the store. Two are easy to miss. pm.max_children is often bounded by RAM before the cores are busy, and once workers run out requests queue in nginx and time out at Varnish. Search and layered navigation go to OpenSearch, a separate service with its own ceiling.

So the requirements for a Magento 2 load testing tool are concrete. It has to hold a cart id per virtual user and reuse it, because Magento hands you a masked id you did not choose. It has to weight scenarios so checkout is a realistic fraction of traffic. And it has to run from more than one machine when the target is bigger than the box generating load.

Magento 2 Load Testing Tools Compared

Locust and Artillery can also script a Magento checkout flow; they are left out of this comparison only because they are less common choices in Magento-focused teams, not because they cannot do the job.

JMeter with the Performance Toolkit

For most of Magento 2's life, "Magento load testing" meant one file: setup/performance-toolkit/benchmark.jmx. It shipped in the core repository up to and including 2.4.5, and it is a serious piece of work. The 2.4.5 plan is over seven megabytes of XML with nine thread-group pools: a frontend pool that browses, searches, adds simple and configurable products to cart and runs checkout through shipping, payment and place order, plus admin, customer-service, REST API, GraphQL and combined pools. It needs the JMeter JSON plugins, and its README tells you to clear active quotes in MySQL before every run.

From 2.4.6 onward the directory holds only the fixture profiles and the setup:perf:generate-fixtures command, which is still the best way to get a realistic catalogue for any tool on this page: the small profile generates roughly 800 simple products, 30 categories and 200 customers. Adobe's data generation docs cover the profiles; the JMeter plan you now pull from the 2.4.5 tag and adapt.

That is the case for JMeter: the scenarios were written by the people who wrote the checkout, and an Adobe Commerce support engineer will recognise the output. The case against it is the tooling. The plan is XML you edit in a Java desktop GUI, it is too large to review in a pull request, and a seven-megabyte file nobody on the team wrote is the definition of a test that rots. It is a QA department's tool, and if that is who owns performance at your company it is the right one. The VoltTest vs JMeter page covers the developer-experience differences in detail.

k6 with the community Magento toolkit

k6 is the tool most developers reach for today: a single Go binary, tests in JavaScript, thresholds that fail the CI job. Each virtual user gets its own cookie jar and reading a value out of a JSON response is one line, so a Magento checkout is entirely scriptable in k6.

Be clear about what the community toolkit gives you, though. The most visible "Magento k6 performance toolkit" on GitHub is a single script that hits one URL with a configurable number of users, appends a per-request query string to bust full page cache when asked, and reports p95. The cache-busting trick is worth stealing, because it is the simplest way to measure the store instead of Varnish. But there is no category walk, no add to cart and no checkout in it. The correlation work, from the masked cart id through the shipping and payment calls, is yours to write and maintain in JavaScript.

That is the real cost for a PHP team: the test lives in a second language, and distributed runs typically mean operating the k6 Kubernetes infrastructure yourself or using Grafana Cloud k6. The VoltTest vs k6 comparison covers where k6's broader protocol support wins, which it does for WebSocket or gRPC work that Magento does not need.

Gatling

Gatling has particularly strong built-in reporting, and its ecosystem has historically been less natural for PHP-heavy Magento teams. Scenarios are written with official SDKs for Java, Kotlin, Scala, JavaScript or TypeScript, so it is no longer a JVM-only tool. Correlation is first class, so a checkout is no harder than in k6, and the HTML report after every run can go in front of a client unedited. But few Magento teams already have a Gatling project, there is no Magento-specific starting point, and distributed runs need Gatling Enterprise. With a JVM or TypeScript team next door it is a fine choice; otherwise it is a second stack to learn for one job.

Siege

Siege is a C command-line tool that takes a URL or a file of URLs, a concurrency and a duration, and reports throughput and response time. It is the right tool for two Magento jobs: warming Varnish after a deploy, and checking that a category page still returns 200 under a few hundred concurrent hits. It is not a load testing tool for Magento 2 in the sense this article means, because it cannot read a response and reuse a value, so no cart id, no add to cart, no order. If a vendor shows you a Siege number for a Magento store, it is a Varnish number.

VoltTest

VoltTest is the tool I build, and its pitch to a Magento team is simple: the load test is a PHP file, it lives in the store's repository next to the code it protects, and it runs locally or in the cloud from several regions without changing a line. Steps are HTTP calls with headers and bodies. Extractors pull values from JSON, headers, cookies, HTML and regex into variables the next step interpolates as ${var}. Each virtual user gets its own cookie jar when you ask for it, and scenarios are weighted with setWeight() so checkout can be 5% of traffic while browsing is 80%, the mix the e-commerce load testing page describes.

Within those limits it correlates the masked cart id, weights scenarios and runs distributed by default. The rest of this post shows the correlation part, because that is where every Magento checkout script, in any tool, either works or does not.

A Guest Checkout Scenario in PHP

Luma's checkout is a JavaScript front end over five REST calls, and the same five calls work from any HTTP client. Hyvä Theme with the Luma checkout fallback makes the same calls; Hyvä Checkout does not, as noted above. In order: create a guest cart, add an item, estimate shipping methods, set shipping information, set payment information. The last one places the order. Adobe's order tutorial walks the same sequence with curl. The script below, the Magento 2.4.8 store it was validated against and the Docker stack that runs both are at volt-test/magento-load-testing, with the SDK in the store's own composer.json.

Two things trip up every first attempt. First, POST /rest/V1/guest-carts does not return an object. It returns a bare JSON string, the masked quote id, and so does the customer token from integration/customer/token. The order id from payment-information is a bare scalar too, a quoted string on some installs and a plain number on others, which is why the regex below makes the quotes optional. A JSON path extractor expects an object at the root, so those responses are read with a regex. Second, the store needs a payment method and a shipping method with no third party behind them. The example deliberately uses Magento's offline Check / Money Order method, checkmo, together with flatrate, both built in and both enabled on a default install, so the benchmark exercises Magento's own checkout without adding an external payment gateway as another dependency, and another potential bottleneck, to the measurement. If your store has switched them off, turn them back on for the test environment.

composer require volt-test/php-sdk

use VoltTest\DataSourceConfiguration;
use VoltTest\VoltTest;
 
$base = rtrim(getenv('TARGET_URL') ?: 'https://magento.test', '/');
 
$test = (new VoltTest('Magento guest checkout'))
    ->target($base)
    ->setVirtualUsers((int) (getenv('VUS') ?: 20))
    ->setDuration(getenv('DURATION') ?: '2m')
    ->setRampUp('30s');
 
$address = '{"firstname":"Load","lastname":"Test","street":["1 Test St"],'
    . '"city":"Los Angeles","region":"CA","region_id":12,"region_code":"CA",'
    . '"country_id":"US","postcode":"90001","telephone":"5555555555","email":"${email}"}';
 
$checkout = $test->scenario('Guest checkout via REST')
    ->setWeight(5)
    ->setThinkTime('2s')
    ->autoHandleCookies()
    ->setDataSourceConfiguration(
        new DataSourceConfiguration(__DIR__.'/data/skus.csv', 'random', true)
    );
 
$checkout->step('Create guest cart')
    ->post("$base/rest/V1/guest-carts")
    ->header('Content-Type', 'application/json')
    ->validateStatus('cart created', 200)
    ->extractFromRegex('cart_id', '^"([A-Za-z0-9]+)"$');
 
$checkout->step('Add item')
    ->post("$base/rest/V1/guest-carts/\${cart_id}/items",
        '{"cartItem":{"sku":"${sku}","qty":1,"quote_id":"${cart_id}"}}')
    ->header('Content-Type', 'application/json')
    ->validateStatus('item added', 200);
 
$checkout->step('Estimate shipping')
    ->post("$base/rest/V1/guest-carts/\${cart_id}/estimate-shipping-methods",
        '{"address":{"country_id":"US","region_id":12,"postcode":"90001"}}')
    ->header('Content-Type', 'application/json')
    ->validateStatus('shipping estimated', 200);
 
$checkout->step('Set shipping information')
    ->post("$base/rest/V1/guest-carts/\${cart_id}/shipping-information",
        '{"addressInformation":{"shipping_address":'.$address.',"billing_address":'.$address.','
        . '"shipping_carrier_code":"flatrate","shipping_method_code":"flatrate"}}')
    ->header('Content-Type', 'application/json')
    ->validateStatus('shipping set', 200);
 
$checkout->step('Place order')
    ->post("$base/rest/V1/guest-carts/\${cart_id}/payment-information",
        '{"email":"${email}","paymentMethod":{"method":"checkmo"},"billing_address":'.$address.'}')
    ->header('Content-Type', 'application/json')
    ->validateStatus('order placed', 200)
    ->extractFromRegex('order_id', '^"?([0-9]+)"?$');
 
$test->run(true);

Repeated checkmo orders consume real fixture inventory. Every placed order takes stock, and once a SKU runs out the failures start: a cart that already holds the last unit gets 400 Some of the products are out of stock. from payment-information, and a fresh cart gets 400 Product that you are trying to add is not available. from items. The wording is the same with and without the Inventory Management modules. On a dashboard that looks exactly like a Magento capacity failure. Either generate fixtures with a large enough quantity or enable backorders for the benchmark, or reset the relevant order, quote and inventory state between runs. The Performance Toolkit README's advice to deactivate open quotes before each JMeter run is the same operational idea, but clearing quotes alone does not put stock back.

The skus.csv file has sku and email columns: export the SKUs from catalog_product_entity after generating fixtures. ${email} goes into the guest address and the final payment call because a guest order needs one; it comes from the CSV so that generated orders are easy to identify and trace in Magento Admin. A unique generated email is useful for benchmark traceability, but Magento does not require it to be unique to accept the guest address. The regex on the first step captures the id inside the quotes Magento returns, and from then on ${cart_id} is interpolated into every later URL. In a PHP double-quoted string the dollar sign is escaped as \${cart_id} so PHP leaves it for the engine; the single-quoted JSON bodies need no escaping. The $base prefix is repeated on purpose: the SDK validates every step URL as an absolute URL and the engine sends it as given, so target($base) identifies the target for the run but does not prefix step paths.

Every step asserts a 200 with validateStatus(). Magento answers 400 with a JSON message on validation failures and 404 when the masked id is wrong, so a broken correlation shows up as an error-rate spike on that step, not a silently green run. The shipping-information response is a real object, {"payment_methods":[...],"totals":{...}}, and it is where a JSON path extractor works if you need something from it; this example does not, because the payment method is fixed.

To turn this into a store test, add a browse scenario at setWeight(80) that walks home, category, product and search, and a cart scenario at 15 that stops after items. A 200-user run then sends about ten shoppers through checkout at a time and reports p95 and error rate per step. Customer checkout is the same sequence with a bearer token from integration/customer/token and /rest/V1/carts/mine, fed from a CSV of fixture customers in unique mode so no two virtual users share a session.

Which Magento 2 Load Testing Tool Should You Use?

Which load testing tool is best for Magento 2?

Of all the Magento load testing tools, the best is the one that can script checkout and that the team will still run after the launch. That means JMeter with the Performance Toolkit if a QA group owns the test and wants Adobe's own scenarios, k6 if your team is comfortable in JavaScript and will run its own load generators, and VoltTest if you want the test in PHP with the infrastructure handled.

• A QA team owns it, or Adobe support is involved. JMeter. Pull the 2.4.5 plan, point it at small profile fixtures, accept the GUI.
• Your developers write JavaScript and already run Grafana. k6. Use the toolkit's cache-busting trick for browse, then write the five REST calls above in JavaScript.
• Your developers write PHP and the store is one repository. VoltTest. The scenario above is the whole checkout, and it runs in CI with an exit code and in the cloud from several regions.
• You have a JVM or TypeScript team next door. Gatling, mostly for the reports.
• You need to warm Varnish or check one page. Siege, and do not call the result a load test.

Whichever you choose, generate fixtures with the Performance Toolkit so the catalogue is realistic, script checkout through to payment-information rather than stopping at add to cart, and weight checkout at the fraction your analytics show. A test that is half checkout finds a saturation point real traffic never reaches, and a test with no checkout misses the one that matters.

What Comes Next

This is the tools half. The measured half is a Magento 2.4 store with MySQL, OpenSearch, Redis and Varnish, loaded with the Performance Toolkit small profile and pushed from 50 to 800 concurrent users with the scenario above at 5% of traffic: warm versus cold Varnish on the same box, anonymous versus logged-in shoppers, and the user count where checkout starts returning errors. There are no numbers here because those runs are not finished, and I would rather publish them once than revise them. When the follow-up is live it will be linked from this page and from the e-commerce load testing hub.

Resources

• volt-test/magento-load-testing: the Magento 2.4.8 project, Docker stack and checkout script from this post, with the raw REST responses captured during validation
• Generate data for performance testing: Adobe's fixture profiles and the setup:perf:generate-fixtures command
• Performance Toolkit at Magento 2.4.5: the last release that includes benchmark.jmx
• Adobe Commerce REST order tutorial: the guest cart, items, shipping and payment calls used above
• Configure and use Varnish: full page cache setup and how to tell a hit from a miss
• Apache JMeter, k6, Gatling, Siege
• VoltTest documentation, steps and extractors, scenarios and data sources
• Related: PHP load testing, Laravel load testing, Symfony load testing

VoltTest's PHP SDK is open source on GitHub. If this comparison saved you a week in a JMeter GUI, a star helps the next Magento developer find it.