Jmeter HTTP request is giving a 400 when using data extracted from a CSV but works fine when manually passing the data on the request body - groovy

I have two HTTP GET requests in Jmeter. The first one calls to a server and gets a CSV that holds some user data. Using a JSR223 Post processor, I am , mapping that data onto a JSON and assigning the values to three variables to be passed onto the second request. The script for doing that is below.
import org.apache.commons.io.IOUtils
import java.nio.charset.StandardCharsets
import groovy.json.JsonBuilder
import groovy.json.JsonSlurper
import groovy.json.JsonOutput
def response = prev.getResponseDataAsString()
def lines = response.split('\n')
def userData = []
for (int i = 1; i < lines.length; i++) {
def line = lines[i]
def tokens = line.split(',')
userData << [login_type: tokens[0], username: tokens[1], password: tokens[2]]
}
def jsonString = JsonOutput.toJson(userData)
def jsonSlurper = new JsonSlurper()
def jsonMap = jsonSlurper.parseText(jsonString)
for (int i = 1; i <= Math.min(jsonMap.size(), Integer.parseInt("${__P(threads)}")); i++) {
if(i < jsonMap.size()){
vars.put("login_type" , jsonMap[Integer.parseInt("${__threadNum}")-1].login_type)
vars.put("username" , jsonMap[Integer.parseInt("${__threadNum}")-1].username)
vars.put("password" , jsonMap[Integer.parseInt("${__threadNum}")-1].password)
}
}
I pass these three variables in the next request body as {"login_type":"${login_type}","username":"${username}","password":"${password}"}
When running the script i'm getting the response as 400 for second request even though i can see the data is getting passed.
POST data:
{"login_type":"data","username":"data","password":"data"}
I tried the second request by manually giving the login data instead of the variables and it works.
{"login_type":"EMAIL","username":"username","password":"pass"}
The only difference i see on both attempts is in the request header where the Content-Length: 83 is shown for when manually sending data and Content-Length: 84 is shown for when passing the data from the groovy script. Though i don't think that's what causing the issue. Can anyone explain as to why this is happening and how to fix this.
I looked into the requests and the POST request body coming from the groovy script has a line break at the end.
{"login_type":"login_type","username":"username","password":"password
"}
Hence causing the request to throw a 400. How can i send the body data in one line?

Maybe there is some invisible character like whitespace or line break which is expected by your system under test and which you're not sending when you're populating variables in Groovy.
Use a sniffer tool like Wireshark or Fiddler to compare both requests byte by byte and amend your JMeter configuration or Groovy code to 100% match the "manual" request.
Also regarding your usage of JMeter Functions in the script, according to JSR223 Sampler documentation:
The JSR223 test elements have a feature (compilation) that can significantly increase performance. To benefit from this feature:
Use Script files instead of inlining them. This will make JMeter compile them if this feature is available on ScriptEngine and cache them.
Or Use Script Text and check Cache compiled script if available property.
When using this feature, ensure your script code does not use JMeter variables or JMeter function calls directly in script code as caching would only cache first replacement. Instead use script parameters.
So replace:
${__P(threads)} with props.get('threads')
${__threadNum} with ctx.getThreadNum()
See Top 8 JMeter Java Classes You Should Be Using with Groovy for more information on what do these props and ctx guys mean.

Related

how to use jmeter JSR223 PreProcessor for reading each line from csv file

My requirement is to send (all the files bodies at once) the multiple JSON body to the API using POST method from JMeter.
I have a CSV file with all the file path in json_test_plan.csv file. And in each of the four files I have the JSON body. I used
In each line, I have mentioned a file path which contains the JSON body.
D:\jmeter_tests\plan1.json
D:\jmeter_tests\plan2.json
D:\jmeter_tests\plan3.json
D:\jmeter_tests\plan4.json
created a HTTP Request sampler with the Body data as mentioned below:
{__FileToString(${JSON},,)}
Added JSR223 PreProcessor as a child of the HTTP Request sampler which I use for sending the JSON input
Put the following code into Script area
new File("D:/jmeter_tests/json_test_plan.csv").readLines().each { line ->
def builder = new StringBuilder()
builder.append(new File(line).text).append(System.getProperty('line.separator'))
sampler.getArguments().removeAllArguments()
sampler.addNonEncodedArgument('', builder.toString(), '')
sampler.setPostBodyRaw(true)
}
Added a http header manager, added content-type and application/json as values for Name and Value field respectively
Added a CSV Dataset Config mentioning the CSV data source "D:/jmeter_tests/json_test_plan.csv"
Now the issue is, each time i run a jmeter test, the above just reads the last line(row) from CSV file "D:\jmeter_tests\plan4.json" and gives the required output. think there is some indexing issue in the code. Could someone pls help me to rea each row/line from the CSV and process.
thanks
It happens because you're calling sampler.getArguments().removeAllArguments() for each file. Just remove it and your code should start working more or less okayish.
You can also think about enhancing it to use Arguments class
def data = new org.apache.jmeter.config.Arguments()
def builder = new StringBuilder()
new File("D:/jmeter_tests/json_test_plan.csv").readLines().each { line ->
builder.append(new File(line).text).append(System.getProperty('line.separator'))
}
def body = new org.apache.jmeter.protocol.http.util.HTTPArgument('', builder.toString(), '', false)
body.setAlwaysEncoded(false)
data.addArgument(body)
sampler.setArguments(data)
More information on Groovy scripting in JMeter: Apache Groovy: What Is Groovy Used For?

how to replace blank values with null in http post body in Jmeter while reading data from csv for multiple iterations?

I am able to use below code to replace blank value with null in http post request body
def body = sampler.getArguments().getArgument(0).getValue().replaceAll('""','null')
sampler.getArguments().removeAllArguments()
sampler.addNonEncodedArgument('', body,'')
sampler.setPostBodyRaw(true)
But, I get an error for multiple iterations.
javax.script.ScriptException: javax.script.ScriptException: java.lang.NullPointerException: Cannot invoke method getValue() on null object
I suspect the removeAllArguments call is affecting subsequent calls to the first line. If it works the first time and fails all subsequent calls then it's probably that. Try commenting that line out and see if it continues to happen:
def body = sampler.getArguments().getArgument(0).getValue().replaceAll('""','null')
sampler.getArguments().removeAllArguments() // I bet this line affects all invocations of the script.
sampler.addNonEncodedArgument('', body,'')
sampler.setPostBodyRaw(true)
Wouldn't that be easier to go for __strReplace() function instead (can be installed as a part of Custom JMeter Functions bundle using JMeter Plugins Manager)?
Whatever, if you like Groovy and copy-pasting the code from Internet without understanding what it's doing I think you need to amend "your" code to look like:
def data = new org.apache.jmeter.config.Arguments()
def body = new org.apache.jmeter.protocol.http.util.HTTPArgument('',sampler.getArguments().getArgument(0).getValue().replaceAll('""','null'),'',false)
data.addArgument(body)
sampler.setArguments(data)

How do i iterate through CSV in jmeter creating new thread

I am trying to come up with jmeter setup in which i want to read entire csv file that has 200000 rows and i want iterate through each rows creating new thread since i am using JSR223 pre-processor that requires new thread for removing empty parameters from the request body. For some reason when i use while loop then only first test passes and rest of the tests fails as JSR223 pre-processor keeps on reading previous thread. I have also un-checked cached compiled script if available but still no luck. I also want to add that when i explicitly specify the number of threads as 100 out of 200000 then all of my 100 test passes as it reads new thread each time. Below is the screenshot of my set up:
This Fails -
This Passes -
JSR223 Pre-Processor Script that i am using:
def request = new groovy.json.JsonSlurper().parseText(sampler.getArguments().getArgument(0).getValue())
def newRequest = evaluate(request.inspect())
request.body.each { entry ->
if (entry.getValue().equals('')) {
newRequest.body.remove(entry.getKey())
}
}
sampler.getArguments().removeAllArguments()
sampler.addNonEncodedArgument('', new groovy.json.JsonBuilder(newRequest).toPrettyString(), '')
sampler.setPostBodyRaw(true)
Console log when using while controller
Replace this :
sampler.getArguments().removeAllArguments()
By:
def arguments = new org.apache.jmeter.config.Arguments();
sampler.setArguments(arguments);
If you're looking to learn jmeter correctly, this book will help you.
It is not possible to provide a comprehensive answer without seeing:
Your While Controller condition stanza
First 3 lines of your CSV file
Your CSV Data Set Config setup
Your HTTP Request sampler parameters or body
Double check the following:
Compare HTTP Request sampler body for 1st and 2nd requests under the While Controller
JMeter Variables originating from the CSV Data Set Config (you can inspect them using Debug Sampler and View Results Tree listener combination)
Enable debug logging for the While Controller, it might be the case it is not doing what you expect, it can be done by adding the next line to log4j2.xml file:
<Logger name="org.apache.jmeter.control.WhileController" level="debug" />

Not able to pass variable value in sampler's HTTP request which is created using groovy scripting in JSR223 Preprocessor in Jmeter

I'm trying to create a hash using groovy script in JSR223 Preprocessor and then pass it to the parent HTTP request's body data, but i'm not sure where I'm doing it wrong. When I have one HTTP request in the thread group then it doesn't work but if it has two HTTP requests then it works for the second request.
Note: Variable HASH is being set after the first request. Though I've tried preprocessor to be the child of first HTTP request or putting it into thread group before HTTP request but nothing works.
Script data:
import java.security.MessageDigest
def requestBody = sampler.getArguments().getArgument(0).getValue()
def data = new XmlParser().parseText(requestBody)
String method=data.method.text()
String token=data.token.text()
String time=data.time.text()
String xyz ='method'+method+'token'+token+'time'+time+'3VDEY-6ZHLH-D27C0-T2ALI'
String hash = MessageDigest.getInstance("MD5").digest(xyz.bytes).encodeHex().toString()
vars.put("HASH", hash)
HTTP request body data:
<?xml version="1.0" encoding="UTF-8"?>
<root>
<method>ping</method>
<token>-</token>
<time>1575542978</time>
<signature>${HASH}</signature>
<params/>
</root>
Test plan Images
https://drive.google.com/file/d/1vGbwyDo8eWUDL7EDZlK8bdbApxtFpWGT/view?usp=sharing
https://drive.google.com/file/d/1Us6tSuRw6MJ__YrnX0tTCKL8S9QSgp-v/view?usp=sharing
https://drive.google.com/file/d/10hobprJcga6_y23VWd5U3X1aaSEXzGkb/view?usp=sharing
It sounds like a bug in JMeter, I would suggest raising an issue via JMeter Bugzilla
In the meantime you can proceed by replacing the JMeter Variable directly in the request body and substituting the "old" request body with the newly generated hash.
Suggested code amendment:
import java.security.MessageDigest
def requestBody = sampler.getArguments().getArgument(0).getValue()
def data = new XmlParser().parseText(requestBody)
String method=data.method.text()
String token=data.token.text()
String time=data.time.text()
String xyz ='method'+method+'token'+token+'time'+time+'3VDEY-6ZHLH-D27C0-T2ALI'
String hash = MessageDigest.getInstance("MD5").digest(xyz.bytes).encodeHex().toString()
requestBody = requestBody.replace('${HASH}', hash)
def args = new org.apache.jmeter.config.Arguments()
sampler.setArguments(args)
sampler.addNonEncodedArgument('', requestBody, '')
sampler.setPostBodyRaw(true)
More information on Groovy scripting in JMeter: Apache Groovy - Why and How You Should Use It
It might be helpful to provide a screenshot or textual representation of your test plan tree since I suspect that you are having an issue with scope here.
As a workaround you could replace the JSR223 PreProcessor with a JSR223 Sampler and place it in the correct order into your test plan. To avoid that sampler showing up in your results you can attach a JSR223 PostProcessor with prev.setIgnore()

How to use XmlSlurper in soapUI

I have the below groovy script which I run in groovyconsole and it runs just fine. I'm finding the number of child nodes for a particular node in my xml response and printing out required values for each child node.
def path = new XmlSlurper().parse(new File('C://SoapUI//ResponseXML/Response.xml'))
NumberOfPositions = path.Body.GetPositionsAggregateResponse.GetPositionsAggregateResult.AccountPositions.Securities.Positions.children().size()
for(def i=0; i<NumberOfPositions; i++){
println i
println path.Body.GetPositionsAggregateResponse.GetPositionsAggregateResult.AccountPositions.Securities.Positions.PositionSummary[i].Legs[0].PositionAggregate[0].PositionID[0].text()
println path.Body.GetPositionsAggregateResponse.GetPositionsAggregateResult.AccountPositions.Securities.Positions.PositionSummary[i].Legs[0].PositionAggregate[0].AccountID[0].text()
}
I want to perform the same task in soapUI, but couldn't get it working using groovyutils as mentioned here : http://www.soapui.org/Scripting-Properties/tips-a-tricks.html
1) How do I parse the xml response from my request to xmlSlurper?
def path = new XmlSlurper().parse (?)
2) Would I be able to use the same code above in soapUI too?
Any help is appreciated. Thanks!
(1)
For parsing the response message you could try the following:
def response = context.expand( '${TestRequest#Response}' )
def xml = new XmlSlurper().parseText(response)
TestRequest represents the name of your test step which is sending the SOAP request message.
(2)
Yes, soapUI should be able to handle any Groovy code.
You can directly use normal groovy script in SoapUI. Check this link, it might help you. But, remember that instead of "println' You need to use "log.info" while scripting in SoapUI.

Resources