Correlation in JMeter: Finding and Extracting Dynamic Values

When automating the process of test assets there are often dynamic values that need to be handled through correlation. While LoadRunner provides auto-correlation and Blazemeter offers SmartJMX auto-correlation via proxy, standard JMeter requires manual setup using its extraction processors.

Understanding Correlation

Correlation is the process of capturing dynamic values from server responses and using them in subsequent requests. Common examples include:

• Session IDs
• CSRF tokens
• Dynamic identifiers
• Authentication tokens

Finding What Needs Correlating

Configuring an extractor is the easy half. The half that costs an afternoon is working out which values are dynamic in the first place, because a recorded script replays them happily and fails in ways that look like something else.

Three approaches, cheapest first:

Record the same journey twice and compare. Any value that differs between two recordings of identical steps is generated by the server, and every one of them is a correlation candidate. This finds almost everything and needs no tooling.

Work backwards from the first failure. Run the script and find the earliest request that fails or returns the wrong page. The value it sent that it should not have — a session ID, a token, a row identifier — came from a response earlier in the flow. Correlate that one first; failures downstream often disappear with it, because they were all carrying the same stale value.

Search the recording for the suspect value. Once you have a candidate, search the whole recording for it. The response that contains it first is the one to extract from. This matters: extracting from a later response works in a smoke test and breaks under load, when the earlier request is still in flight.

Values that are almost always dynamic: session identifiers, CSRF and anti-forgery tokens, ViewState, OAuth and bearer tokens, anything named nonce, order and transaction references, and any identifier created by the step before.

Boundary Extractor Method

The Boundary Extractor allows you to extract values by specifying the text that appears before and after the target value.

For example, to extract a value like d41d8cd98f00b204e9800998ecf8427e from a string containing blog-collection-list-, configure the extractor as follows:

Setting	Value
Left Boundary	blog-collection-list-
Right Boundary	-
Match Number	-1

Using match number -1 captures all matching instances as separate variables:

• blogcollectionlist_1
• blogcollectionlist_2
• etc.

A count variable is also created to track the total number of matches.

Regular Expression Extractor Method

An alternative approach uses regular expressions for more flexible pattern matching.

For the same extraction, use:

Setting	Value
Regular Expression	blog-collection-list-([a-z0-9]+)
Template	$1$
Match Number	-1

This generates similarly-indexed variables:

• blogcollectionlist_1_g1
• blogcollectionlist_2_g1

The g1 suffix indicates the capture group assignment.

JSON Extractor Method

Most APIs answer in JSON, where boundaries and regular expressions are the wrong tool — a reformatted response breaks them, and JSON has no reliable left-and-right text anchor. Use the JSON Extractor, which addresses values by path:

Setting	Value
Names of created variables	authToken
JSON Path expressions	$.data.token
Match No.	1
Default Values	TOKEN_NOT_FOUND

The path syntax is JSONPath: $ is the root, $.items[0].id takes the first element, $..id finds every id at any depth.

Match Numbers, and the Variables You Get

Match No. is the setting that catches people out, because all three extractors share it and the values are not intuitive:

Match No.	Behaviour
1 (or any positive N)	The Nth match. What you want most of the time
0	A match chosen at random on every iteration
-1	Every match, as indexed variables

0 deserves care. It is genuinely useful for picking a random product from a listing page, and a genuine bug when someone reaches for it expecting “the first one”.

With -1 you get name_1, name_2 … plus name_matchNr holding the count. Read ${name_matchNr} before looping over the results, or you will reference variables that were never created.

Where the Extractor Goes

An extractor is a post-processor, and placement decides which responses it reads. As a child of one sampler, it runs against that sampler’s response only — which is almost always what you want. Placed directly under the thread group, it runs against every sampler in it, and will happily overwrite your variable with a match from an unrelated response later in the flow.

Variables are per-thread. Each virtual user extracts and holds its own session ID, which is exactly right for correlation, and the reason correlated values must never be moved into a User Defined Variable to “share” them.

Using Extracted Values

Both methods enable subsequent test steps to reference extracted values via JMeter variable syntax:

${blogcollectionlist_1}

Or for regex extractions:

${blogcollectionlist_1_g1}

Proving the Correlation Worked

A correlation that silently fails is worse than one that errors, because the run still produces a green report. Three habits make failure loud:

Set a Default Value. Give every extractor one that could never occur naturally — SESSION_NOT_FOUND rather than a blank. A blank default makes a failed extraction indistinguishable from an empty match, and the request goes out with an empty parameter.

Assert on it. Add a Response Assertion that fails when the response contains your default value. Now a broken extractor fails the sample instead of quietly sending nonsense.

Look at the variables. Add a Debug Sampler and open it in View Results Tree: it lists every JMeter variable with its current value, which answers “did it extract, and what exactly did it get” in one place.

The failure to watch for is the run that passes while testing nothing — every request returning 200, because every one of them was served the login page.

Choosing Between Methods

• Boundary Extractor: simplest when you know the text either side, and the cheapest to read six months later. Start here
• Regular Expression Extractor: for complex patterns, or when you need several capture groups from one match
• JSON Extractor: for any JSON response — do not pattern-match JSON with a regex

Further Reading

• Apache JMeter Boundary Extractor
• Apache JMeter Regular Expression Extractor
• Apache JMeter JSON Extractor

Related

For what this costs on a real deadline — and what it takes to automate — see The Correlation Problem: A JMeter Story.

Correlation is one way a test can pass while testing nothing. There are others — see Three things that silently break when you migrate a performance test, particularly the session-handling failure, which produces a run full of 200s from a login page.