Splitting JTL files

During a typical performance test, traffic will be loaded gradually before reaching the desired steady state. In Apache JMeter the ramp-up and steady state data points would be captured in the same JTL file. When generating the HTML report this can make it hard to isolate the data you want to target.

Since a JTL file is a csv file it can be altered. To identify the data that we wish to extract we simply need to target the epoch time range. This is the data captured in the timeStamp column of the JTL file.

timeStamp,elapsed,label,responseCode,responseMessage,threadName,success,failureMessage,bytes,grpThreads,allThreads,Latency,Hostname,Connect
1638354259479,0,A2-001,200,OK,Test Sample 1-18,true,,4260,46,46,0,2f917425da58,0
1638354260479,1,A1-001,200,OK,Test Sample 1-2,true,,4291,46,46,0,2f917425da58,0

To ease this process we have created a BASH shell script which will extract data, removing any ramp-up and ramp-down data points. It can also break up the data into the required duration.

To use the script you will need BASH with bc installed. Then copy the code to a file called split-jtl.sh and make sure it is executable.

The script expects input values from the command line. These are ramp-up, ramp-down, hold and filename. If any of the values are not entered you will be prompted with a help dialogue. This example has a 600 second ramp-up, no ramp-down and we want to generate reports for 1800 second periods.

./split-jtl.sh 600 0 1800 dummy.jtl

---

#!/bin/bash

rampup=$1
rampdown=$2
hold=$3
filename=$4
JMETER_HOME="./apache-jmeter-5.5/bin"

if ! [[ -n $rampup && -n $rampdown && -n $hold && -f $filename ]]; then
    echo "$0 600 0 600 kpi-out.jtl"
    exit 1
else
    echo "Deleting any old report and temp files/folders"
    rm -Rf $(ls | grep -E 'report[0-9]+|temp') > /dev/null
    
    header=`grep timeStamp $filename | head -n1`
    columns=`echo $header | awk -F "," ' { print NF-1 } '`
    startDate=`sed '1d' $filename  | head -n1 | cut -c1-10`
    endDate=`cut -d, -f1 $filename | tail -n1 | cut -c1-10`
    durationSec=$(echo "$endDate - $startDate" | bc)
    testDuration=$(echo "$durationSec - $rampup - $rampdown" | bc)
    loops=$(echo "$testDuration / $hold" | bc)

    echo "### startDate=$(date -d @$startDate) | endDate=$(date -d @$endDate) | duration=$durationSec | Report=$loops ###"
    echo "Spliting into individual report jtls and generating HTML reports"
    
    echo "### Generating Full Report  -->  report0.jtl  Generating HTML report0 ###"
    awk -v columns=$columns -F "," ' { if ( NF-1 == columns ) print $0 } ' $filename | tee report0.jtl | sed '1d' | sort -k1 > temp0.jtl && filename=temp0.jtl
    (exec ${JMETER_HOME}/jmeter -g report0.jtl -o report0)
    
    for (( i=1 ; i<=$loops ; i++ ));
    do
        if [ $i -eq 1 ]; then
            Start=$(echo $startDate + $rampup + 1 | bc)
        else
            Start=$(echo $startDate + 1 | bc)
        fi
        End=$(echo $Start + $hold | bc)
        echo "### $(date -d @$Start) - $(date -d @$End)  -->|Generating report$i.jtl|Generating HTML report$i ###" | column -s "|"  -t
        sed -n "/^$Start/,/^$End/{p;/^$End/q}" $filename | awk -v columns=$columns -F "," ' { if ( NF-1 == columns ) print $0 } ' > report"$i".jtl
        sed -i "1i $header"  report"$i".jtl
        (exec ${JMETER_HOME}/jmeter -g report"$i".jtl -o report"$i")
        startDate=$End
    done
fi

---

Example Output

./split-jtl.sh 600 0 1800 dummy.jtl 
Deleting any old report and temp files/folders
### startDate=Wed  1 Dec 10:24:19 GMT 2021 | endDate=Wed  1 Dec 11:35:59 GMT 2021 | duration=4300 | Report=2 ###
Spliting into individual report jtls and generating HTML reports
### Generating Full Report  -->  report0.jtl  Generating HTML report0 ###
### Wed  1 Dec 10:34:20 GMT 2021 - Wed  1 Dec 11:04:20 GMT 2021  -->  Generating report1.jtl  Generating HTML report1 ###
### Wed  1 Dec 11:04:21 GMT 2021 - Wed  1 Dec 11:34:21 GMT 2021  -->  Generating report2.jtl  Generating HTML report2 ###

FURTHER READING

split-jtl repo

bc manual