Three Things That Silently Break When You Migrate a Performance Test

A migrated load test that fails is a good outcome. You see the error, you fix it, you move on.

The dangerous outcome is the one that works. The script runs, every request returns 200, the report is green, and the numbers are meaningless — because the test is no longer doing what the original did.

I have hit three of these. They share a shape: valid output, clean lint, no warning, wrong test. None of them is specific to any tool. If you hand-port a JMeter plan to k6, you have to remember all three yourself; a converter just automates whichever one you would have got wrong.

Here they are, with the check for each.

1. The session that isn’t

JMeter maintains no cookies unless you add an HTTP Cookie Manager. k6 and Gatling give every virtual user a cookie jar automatically and use it without being asked.

That asymmetry is the problem. Nothing in a k6 script says “keep a session” — it just happens. So when the script becomes a JMeter plan, there is nothing to translate, and the resulting plan quietly has no session handling at all.

What that looks like in practice: the login request succeeds and returns a session cookie. JMeter throws it away. Every subsequent request runs unauthenticated. Against most applications you do not get errors — you get 200s, from a login page, redirect or friendly “please sign in” screen. Response times look better than production, because you are load testing the least expensive page on the site.

The check. Run one iteration with a single user and look at the second request after login. Not the status code — the response body. If it contains your login form, your test has been measuring the login page.

In JMeter, confirm there is an HTTP Cookie Manager and that it sits high enough in the tree to cover every sampler that needs it. Going the other way, be aware you get session handling for free whether you asked for it or not, which can mask an authentication bug rather than expose it.

2. Headers that leak across journeys

JMeter scopes a Header Manager to the subtree it sits in. One at the plan level applies everywhere. One inside a thread group applies to that group only. One attached to a sampler applies to that request alone. Where two overlap, the innermost wins.

Flatten that hierarchy — which is easy to do accidentally, and easy for a tool to do on your behalf — and two things go wrong at once.

The first is a correctness bug: an override stops overriding. If your plan sets Content-Type: application/json globally and text/xml for one group, flattening can leave the outer value winning everywhere, and the group posting XML now claims to post JSON.

The second is worse, because the headers people scope to a single journey are almost always credentials. An Authorization header, an API key, a tenant identifier. Flatten the scope and one journey’s credential is attached to every other journey’s requests — including ones pointing at a different host. Nothing errors. The receiving service either ignores the extra header or, occasionally, honours it.

The check. Take any plan with more than one Header Manager and inspect the actual headers on one request from each thread group. In JMeter, View Results Tree shows the request headers as sent. What you are looking for is a header from group A appearing on a request in group B. If your suite has one Header Manager, this cannot bite you; if it has several, it is worth ten minutes.

3. Disabled elements that come back to life

Disabling a sampler rather than deleting it is normal practice. You are debugging, you want to skip a step, you untick it and move on. JMeter does not run it, and disabling a controller disables everything beneath it.

That “enabled” flag is a property of the file, and it is easy to lose in translation. When it goes, a request you deliberately parked becomes a live request — so the converted script sends traffic the original never sent. If you disabled a step because it posts to a payment endpoint, or because it was hammering a service that asked you to stop, the migrated script does not know that.

This one is the most likely to bite mid-project, because that is exactly when half a plan is switched off.

The check. Count the requests. Run one iteration of the original and one of the migration, and compare the number of samples and the distinct endpoints hit. They should match. If the migration makes more requests than the original, something disabled has woken up.

Why these are all the same bug

Each of these produces output that is syntactically valid and semantically wrong. That combination defeats most of the checks people rely on.

A linter cannot help. I ran a broken conversion and a fixed conversion through a linter and got identical scores — A, 95/100, zero errors, both times. The broken one contained six URLs with no host and two undefined variables. https:///api/orders is a valid JavaScript string containing a plausible URL. ${username} with no declaration is valid syntax. The linter had nothing to complain about.

A smoke run does not help either, because the failure mode is a successful run.

What does help is asserting on behaviour rather than absence of errors: does the session persist, do the headers match per journey, is the request count the same. All three checks above take longer to describe than to perform.

How I know

I found all three in my own converter, which is why I can be this specific.

They shipped for months. Nobody had downloaded the affected releases, so no customer received bad output — but that was luck rather than process, and it is the reason I now test conversions by asserting on the generated text rather than by checking they convert without error. A test that says “it produced a file” tells you nothing about whether the file does what the source did.

The fixes are in the current release, and the tests that would have caught each one are in the repo. The more useful outcome is the checklist above, which applies whatever you migrate with — including by hand.

If you want to try it on your own script

The converter is free in the browser at tools.martkos-it.co.uk. It converts between JMeter, k6 and Gatling in any direction, and it runs entirely inside your tab — your script is never uploaded, and the page has no way to send it anywhere.

It will also tell you what it could not convert, which is the part worth reading. Anything it flags is work you would have had to do by hand, and anything a converter has to skip is usually the same thing that would have caught you out doing it manually.