How to obtain virtual user id/details in gatling? - performance-testing

I am new to Gatling and Scala and I need your advice.
I would like to obtain load test for n-users. Each user have to send request for creating different accounts. This is obtained by sending json file with appropriate array of objects ('entries' in our case).
Each single user must send different login as our backend system is checking if username is unique. Somehow we have to be sure that gatling is sending different data for each virtual user and also for each entries as well.
We noticed that there us session element which represents virtual user's state. Problem is that code showed below will not work as Exec structure used with expression function does not send any request.
There is section that could work but I do not know how to determine third parameter to distinguish virtual user id. Please find below simple json file structure used for this test
{
"entries": [
{
"userName": "some user name",
"password": "some password"
}
}
and scala code with my comments
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class UserCreationTest extends Simulation {
val profilesNumber = 2
val virtualUsers = 2
val httpConf = http
.baseURL("some url")
.acceptHeader("application/json")
.basicAuth("username", "password")
// This method will multiply 'entries' section in JSON 'entriesNumber' times
def createJsonUserEntries(entriesNumber: Int, users: List[String], userId : Long): String = {
val header = """{"entries": ["""
val footer = """]}"""
val builder = StringBuilder.newBuilder
for (i <- 0 until entriesNumber) {
val userIndex = (userId.toInt - 1) * entriesNumber + i
val userName = users(userIndex).get
val apiString =
s"""{
"userName": "${userName}"
"password": "password"
}"""
builder.append(apiString)
if (i != entriesNumber) {
builder.append(",")
}
}
header + builder.toString() + footer
}
// We do have method for generating user names based on profilesNumber and virtualUsers variables
// but for sake of this example lets hardcode 4 (profilesNumber * virtualUsers) user names
val usersList = List("user-1", "user-2", "user-3", "user-4")
//This will throw exception as no request was send. According to documentation function block is used to debugging/editing session
val scn = scenario("Create WiFi User Profile")
.exec(session => {
http("CreateUserProfile")
.post("/userProfiles/create/")
.body(StringBody(
createJsonUserEntries(profilesNumber, userslList, session.userId).toString
)
).asJSON
session})
// This exec block will send a request but I do not know how to determine third param that should be virtual user Id
// To run this section please comment previous whole scenario block
/*
val scn = scenario("")
.exec(http("CreateUserProfile")
.post("/userProfiles/create/")
.body(StringBody(
createJsonUserEntries(profilesNumber, emailList, ???).toString
)
).asJSON
)
*/
setUp(scn.inject(atOnceUsers(virtualUsers)).protocols(httpConf))
}
Can you help me on that please? Is there any other way to do that in gatling? Thank you very much in advance

so you are trying to have each user have a unique userId?
you could create a feeder that does this
var userIdFeeder = (1 to 999999).toStream.map(i => Map("userId" -> i)).toIterator
val scn = scenario("")
.feed(userIdFeeder)
.exec(http("CreateUserProfile")
.post("/userProfiles/create/")
.body(StringBody(
createJsonUserEntries(profilesNumber, emailList, "${userId}").toString
)
).asJSON
)

Related

Gatling Rest API Testing - retrieve a value from json response and add it to the list, iterate through list

I am new to Gatling, I am trying to do the performance testing for couple of rest calls. In my scenario I need to extract a value from json response of the 1st call and add those values to the list after looping for few times. Again after looping for few times and adding the values into the list, I want to reuse each value in my next rest call by iterating over the values in the list. Can anyone please suggest on how to implement this. I tried something as below,
var datasetIdList = List.empty[String]
val datasetidsFeeder = datasetIdList.map(datasetId => Map("datasetId" -> datasetId)).iterator
def createData() = {
repeat(20){
feed("").exec(http("create dataset").post("/create/data").header("content-type", "application/json")
.body(StringBody("""{"name":"name"}"""))
.asJson.check(jsonPath("$.id").saveAs("userId"))))
.exec(session => { var usrid = session("userId").as[String].trim
datasetIdList:+= usrid session})
}}
def upload()= feed(datasetidsFeeder).exec(http("file upload").post("/compute-metaservice/datasets/${datasetId}/uploadFile")
.formUpload("File","./src/test/resources/data/File.csv")
.header("content-type","multipart/form-data")
.check(status is 200))
val scn = scenario("create data and upload").exec(createData()).exec(upload())
setUp(scn.inject(atOnceUsers(1))).protocols(httpConf)
}
I am seeing an exception that ListFeeder is empty when trying to run above script. Can someone please help
Updated Code:
class ParallelcallsSimulation extends Simulation{
var idNumbers = (1 to 50).iterator
val customFeeder = Iterator.continually(Map(
"name" -> ("test_gatling_"+ idNumbers.next())
))
val httpConf = http.baseUrl("http://localhost:8080")
.header("Authorization","Bearer 6a4aee03-9172-4e31-a784-39dea65e9063")
def createDatasetsAndUpload() = {
repeat(3) {
//create dataset
feed(customFeeder).exec(http("create data").post("/create/data").header("content-type", "application/json")
.body(StringBody("""{ "name": "${name}","description": "create data and upload file"}"""))
.asJson.check(jsonPath("$.id").saveAs("userId")))
.exec(session => {
val name = session("name").asOption[String]
println(name.getOrElse("COULD NOT FIND NAME"))
val userId = session("userId").as[String].trim
println("%%%%% User ID ====>"+userId)
val datasetIdList = session("datasetIdList").asOption[List[_]].getOrElse(Nil)
session.set("datasetIdList", userId :: datasetIdList)
})
}
}
// File Upload
def fileUpload() = foreach("${datasetIdList}","datasetId"){
exec(http("file upload").post("/uploadFile")
.formUpload("File","./src/test/resources/data/File.csv")
.header("content-type","multipart/form-data")
.check(status is 200))
}
def getDataSetId() = foreach("${datasetIdList}","datasetId"){
exec(http("get datasetId")
.get("/get/data/${datasetId}")
.header("content-type","application/json")
.asJson.check(jsonPath("$.dlp.dlp_job_status").optional
.saveAs("dlpJobStatus")).check(status is 200)
).exec(session => {
val datastId = session("datasetId").asOption[String]
println("request for datasetId >>>>>>>>"+datastId.getOrElse("datasetId not found"))
val jobStatus = session("dlpJobStatus").asOption[String]
println("JOB STATUS:::>>>>>>>>>>"+jobStatus.getOrElse("Dlp Job Status not Found"))
println("Time: >>>>>>"+System.currentTimeMillis())
session
}).pause(10)
}
val scn1 = scenario("create multiple datasets and upload").exec(createDatasetsAndUpload()).exec(fileUpload())
val scn2 = scenario("get datasetId").pause(100).exec(getDataSetId())
setUp(scn1.inject(atOnceUsers(1)),scn2.inject(atOnceUsers(1))).protocols(httpConf)
}
I see below error when I try to execute above script
[ERROR] i.g.c.s.LoopBlock$ - Condition evaluation crashed with message 'No attribute named 'datasetIdList' is defined', exiting loop
var datasetIdList = List.empty[String] defines a mutable variable pointing to a immutable list.
val datasetidsFeeder = datasetIdList.map(datasetId => Map("datasetId" -> datasetId)).iterator uses the immutable list. Further changes to datasetIdList is irrelevant to datasetidsFeeder.
Mutating a global variable with your virtual user is usually not a good idea.
You can save the value into the user's session instead.
In the exec block, you can write:
val userId = session("userId").as[String].trim
val datasetIdList = session("datasetIdList").asOption[List[_]].getOrElse(Nil)
session.set("datasetIdList", userId :: datasetIdList)
Then you can use foreach to iterate them all without using a feeder at all.
foreach("${datasetIdList}", "datasetId") {
exec(http("file upload")
...
}
You should put more work in your question.
Your code is not syntax-highlighted, and is formatted poorly.
You said "I am seeing an exception that ListFeeder is empty" but the words "ListFeeder" are not seen anywhere.
You should post the error message so that it's easier to see what went wrong.
In the documentation linked, there is a Warning. Quoted below:
Session instances are immutable!
Why is that so? Because Sessions are messages that are dealt with in a multi-threaded concurrent way, so immutability is the best way to deal with state without relying on synchronization and blocking.
A very common pitfall is to forget that set and setAll actually return new instances.
This is why the code in the updated question doesn't update the list.
session => {
...
session.set("datasetIdList", userId :: datasetIdList)
println("%%%% List =====>>>" + datasetIdList.toString())
session
}
The updated session is simply discarded. And the original session is returned in the anonymous function.

Mockito returning null value for Future

I am unit-testing a controller in a play-framework application.
The controller uses a repository and I am mocking the repository as follows
val mockUserRepository = mock(classOf[UsersRepository])
when(mockUserRepository.findOne(userKeys)).thenReturn(Future{Some(user)})
when(mockUserRepository.save(user)).thenReturn(Future(Some(user)))
Then I run the following test. In the test, controller.signupUser(request) calls the findOne method of the repository as follows
val findUserFuture: Future[Option[User]] = userRepo.findOne(userKeys)
println("user future is ",findUserFuture)
findUserFuture.flatMap { (userOption: Option[User]) => //this crashes because findUserFuture is null)
But findOne returns a null instead of a dummy Future{user}
"User signup request with body" should {
"return OK and user profile if the signup profile data is correct" in {
val jsonBody = Json.parse(
"""
{
"external-profile":{
"email":"test#test.com",
"firstname":"fn",
"lastname":"ln",
"password":"aA1!1111"
}
}
""")
//val jsonBody = Json.toJson(signupInfo)
val request: Request[AnyContentAsJson] = FakeRequest("POST", "ws/users/signup",Headers(("someH"->"someV")),new AnyContentAsJson(jsonBody))
println("sending sign up request ", request)
//request.body = signupInfo
val response: Future[Result] = controller.signupUser(request)
val responseBodyAsJsValue:JsValue = contentAsJson(response)
println("received response of sign up ", responseBodyAsJsValue)
}
}
error
received profile
UserProfile(None,ExternalUserProfile(test#test.com,fn,ln,Some(aA1!1111)))
checking if the user with the following details exists LoginInfo(credentials,test#test.com)
returning id 116 for name test#test.com
(user future is ,null)
java.lang.NullPointerException was thrown.
java.lang.NullPointerException
at controllers.UserController.$anonfun$signupUser$1(UserController.scala:116)
What I might be doing wrong?
The issue apparently was the I wasn't using when correctly.
I read that "
Mockito allows to configure the return values of its mocks via a fluent API. Unspecified method calls return "empty" values:
null for objects
0 for numbers
false for boolean
empty collections for collections
Mocks can return different values depending on arguments passed into a method. The when(…​.).thenReturn(…​.) method chain is used to specify a a return value for a method call with pre-defined parameters.
"
when expects the method as well as the exact arguments. So if I want to call findUser(userkey) where userkey's value is say 1 in the actual call then I need to write when(findUser(1)) or userKey=1; findUser(userKey)).
In my wrong implementation, I had set userkey as
UserKeys(1,"test#test.com",loginInfo,"","")
but the call to findUser was with value
UserKeys(116,"d#d.com",loginInfo,"fn","ln")
I changed the userkey value in test and it worked
val userKeys = UserKeys(utilities.bucketIDFromEmail(email)/*returns 116*/,"d#d.com",loginInfo,"fn","ln")
when(mockUserRepository.findOne(userKeys)).thenReturn(Future{Some(user)})
when(mockUserRepository.save(user)).thenReturn(Future(Some(user)))

FakeRequest seem to be Null when passed to a controller in unit test

Getting Null Pointer error when unit testing controller. The issue seem to be in the line
def signupUser = Action.async{
implicit request => { //requeust seem to be null
I suspect so because the stacktrace from previous tests point to implicit request line. But I don’t know what could be wrong in this because I am using FakeRequest like so val request = FakeRequest("POST", "/ws/users/signup").withJsonBody(Json.parse("""{"bad": "field"}"""))
Following is a snippet of a controller I want to unit-test
class UserController #Inject()(userRepo: UsersRepository,cc: ControllerComponents, silhouette: Silhouette[JWTEnv])(implicit exec: ExecutionContext) extends AbstractController(cc){
def signupUser = Action.async{
implicit request => {...}
}
I only want to test that the controller returns an error when it gets a request without json body. Thus I don't need Silhouette and I want to mock it. But I am getting null pointer error.
Following is the way I have written my unit test case is
class UserControllerUnitSpec extends PlaySpec with MockitoSugar {
"User signup request with non-JSON body" should {
"return 400 (Bad Request) and the validation text 'Incorrect body type. Body type must be JSON'" in {
val email = "d#d.com"
val loginInfo = LoginInfo(CredentialsProvider.ID, email);
val passwordInfo = PasswordInfo("someHasher","somePassword",Some("someSalt"))
val internalUserProfile = InternalUserProfile(loginInfo,true,Some(passwordInfo))
val externalUserProfile = ExternalUserProfile(email,"d","d",Some("somePassword"))
val userProfile = UserProfile(Some(internalUserProfile),externalUserProfile)
val user = User(UUID.randomUUID(),userProfile)
println("testing with mocked User value",user);
val mockUserRepository = mock[UsersRepository]
when(mockUserRepository.findUser(loginInfo)).thenReturn(Future(Some(user)))
when(mockUserRepository.saveUser(user)).thenReturn(Future(Some(user)))
val mockSilhouette = mock[Silhouette[JWTEnv]] //I am probably not doing this correctly
val mockControllerComponents = mock[ControllerComponents] //I am not sure if this is correct either
val controller = new UserController(mockUserRepository,mockControllerComponents,mockSilhouette)
val result:Future[Result] = controller.signupUser(FakeRequest())
(result.map(response => {
println("response: ",response)
response mustBe BadRequest
}))
}
}
}
Regarding mockControllerComponents, Helpers.stubControllerComponents can be used instead of a mock:
val mockControllerComponents = Helpers.stubControllerComponents()
Regarding mockSilhouette, you have to setup the mock using when(...).thenReturn(...) similarly to how you have done it formockUserRepository, that is, inspect all the usages of silhouette inside signupUser and provide the appropriate method stubs:
val mockSilhouette = mock[Silhouette[JWTEnv]]
when(mockSilhouette.foo(...)).thenReturn(...)
when(mockUserRepository.bar(...)).thenReturn(...)
...
(Posted solution on behalf of the question author).
Here is the answer which worked. Thanks Mario.
class UserControllerUnitSpec extends PlaySpec /*with MockitoSugar*/ {
"User signup request with non-JSON body" should {
"return 400 (Bad Request) and the validation text 'Incorrect body type. Body type must be JSON'" in {
val email = "d#d.com"
val loginInfo = LoginInfo(CredentialsProvider.ID, email);
val passwordInfo = PasswordInfo("someHasher","somePassword",Some("someSalt"))
val internalUserProfile = InternalUserProfile(loginInfo,true,Some(passwordInfo))
val externalUserProfile = ExternalUserProfile(email,"d","d",Some("somePassword"))
val userProfile = UserProfile(Some(internalUserProfile),externalUserProfile)
val user = User(UUID.randomUUID(),userProfile)
println("testing with mocked User value",user);
val mockUserRepository = mock(classOf[UsersRepository])
// when(mockUserRepository.findUser(loginInfo)).thenReturn(Future(Some(user)))
// when(mockUserRepository.saveUser(user)).thenReturn(Future(Some(user)))
// val mockSilhouette = mock(classOf[Silhouette[JWTEnv]])
val mockControllerComponents = Helpers.stubControllerComponents()//mock(classOf[ControllerComponents])
/*
The controller needs Silhouette. Using Silhouette's test kit to create fake instances.
If you would like to test this controller, you must provide an environment that can handle your Identity and Authenticator implementation.
For this case Silhouette provides a FakeEnvironment which automatically sets up all components needed to test your specific actions.
You must only specify one or more LoginInfo -> Identity pairs that should be returned by calling request.identity in your action and
the authenticator instance that tracks this user.
*/
//User extends Identity trait
/*
Under the hood, the environment instantiates a FakeIdentityService which stores your given identities and returns it if needed.
It instantiates also the appropriate AuthenticatorService based on your defined Authenticator type. All Authenticator services are real
service instances set up with their default values and dependencies.
*/
implicit val sys = ActorSystem("MyTest")
implicit val mat = ActorMaterializer()
implicit val env = FakeEnvironment[JWTEnv](Seq(loginInfo->user))
val defaultParser = new mvc.BodyParsers.Default()
val securedAction = new DefaultSecuredAction(new DefaultSecuredRequestHandler(new DefaultSecuredErrorHandler(stubMessagesApi())),defaultParser)
val unsecuredAction = new DefaultUnsecuredAction(new DefaultUnsecuredRequestHandler(new DefaultUnsecuredErrorHandler(stubMessagesApi())),defaultParser)
val userAware = new DefaultUserAwareAction(new DefaultUserAwareRequestHandler(),defaultParser)
val mockSilhouette = new SilhouetteProvider[JWTEnv](env,securedAction,unsecuredAction,userAware)
val controller = new UserController(mockUserRepository,mockControllerComponents,mockSilhouette)
val request = FakeRequest("POST","ws/users/signup")
println("sending request",request)
//val result = controller.someMethod()
val result:Future[Result] = controller.signupUser(request)
status(result) mustBe BAD_REQUEST
}
}
}

Finding all properties for a schema-less vertex class

I have a class Node extends V. I add instances to Node with some set of document type information provided. I want to query the OrientDB database and return some information from Node; to display this in a formatted way I want a list of all possible field names (in my application, there are currently 115 field names, only one of which is a property used as an index)
To do this in pyorient, the only solution I found so far is (client is the name of the database handle):
count = client.query("SELECT COUNT(*) FROM Node")[0].COUNT
node_records = client.query("SELECT FROM Node LIMIT {0}".format(count))
node_key_list = set([])
for node in node_records:
node_key_list |= node.oRecordData.keys()
I figured that much out pretty much through trial and error. It isn't very efficient or elegant. Surely there must be a way to have the database return a list of all possible fields for a class or any other document-type object. Is there a simple way to do this through either pyorient or the SQL commands?
I tried your case with this dataset:
And this is the structure of my class TestClass:
As you can see from my structure only name, surname and timeStamp have been created in schema-full mode, instead nameSchemaLess1 and nameSchemaLess1 have been inserted into the DB in schema-less mode.
After having done that, you could create a Javascript function in OrientDB Studio or Console (as explained here) and subsequently you can recall it from pyOrient by using a SQL command.
The following posted function retrieves all the fields names of the class TestClass without duplicates:
Javascript function:
var g = orient.getGraph();
var fieldsList = [];
var query = g.command("sql", "SELECT FROM TestClass");
for (var x = 0; x < query.length; x++){
var fields = query[x].getRecord().fieldNames();
for (var y = 0; y < fields.length; y++) {
if (fieldsList == false){
fieldsList.push(fields[y]);
} else {
var fieldFound = false;
for (var z = 0; z < fieldsList.length; z++){
if (fields[y] == fieldsList[z]){
fieldFound = true;
break;
}
}
if (fieldFound != true){
fieldsList.push(fields[y]);
}
}
}
}
return fieldsList;
pyOrient code:
import pyorient
db_name = 'TestDatabaseName'
print("Connecting to the server...")
client = pyorient.OrientDB("localhost", 2424)
session_id = client.connect("root", "root")
print("OK - sessionID: ", session_id, "\n")
if client.db_exists(db_name, pyorient.STORAGE_TYPE_PLOCAL):
client.db_open(db_name, "root", "root")
functionCall = client.command("SELECT myFunction() UNWIND myFunction")
for idx, val in enumerate(functionCall):
print("Field name: " + val.myFunction)
client.db_close()
Output:
Connecting to the server...
OK - sessionID: 54
Field name: name
Field name: surname
Field name: timeStamp
Field name: out_testClassEdge
Field name: nameSchemaLess1
Field name: in_testClassEdge
Field name: nameSchemaLess2
As you can see all of the fields names, both schema-full and schema-less, have been retrieved.
Hope it helps
Luca's answer worked. I modified it to fit my tastes/needs. Posting here to increase the amount of OrientDB documentation on Stack Exchange. I took Luca's answer and translated it to groovy. I also added a parameter to select the class to get fields for and removed the UNWIND in the results. Thank you to Luca for helping me learn.
Groovy code for function getFieldList with 1 parameter (class_name):
g = orient.getGraph()
fieldList = [] as Set
ret = g.command("sql", "SELECT FROM " + class_name)
for (record in ret) {
fieldList.addAll(record.getRecord().fieldNames())
}
return fieldList
For the pyorient part, removing the database connection it looks like this:
node_keys = {}
ret = client.command("SELECT getFieldList({0})".format("'Node'"))
node_keys = ret[0].oRecordData['getFieldList']
Special notice to the class name; in the string passed to client.command(), the parameter must be encased in quotes.

Multiple key search in CouchDB

Given the following object structure:
{
key1: "...",
key2: "...",
data: "..."
}
Is there any way to get this object from a CouchDB by quering both key1 and key2 without setting up two different views (one for each key) like:
select * from ... where key1=123 or key2=123
Kind regards,
Artjom
edit:
Here is a better description of the problem:
The object described above is a serialized game state. A game has exactly one creator user (key1) and his opponent (key2). For a given user I would like to get all games where he is involved (both as creator and opponent).
Emit both keys (or only one if equal):
function(doc) {
if (doc.hasOwnProperty('key1')) {
emit(doc.key1, 1);
}
if (doc.hasOwnProperty('key2') && doc.key1 !== doc.key2) {
emit(doc.key2, 1);
}
}
Query with (properly url-encoded):
?include_docs=true&key=123
or with multiple values:
?include_docs=true&keys=[123,567,...]
UPDATE: updated to query multiple values with a single query.
You could create a CouchDB view which produces output such as:
["key1", 111],
["key1", 123],
["key2", 111],
["key2", 123],
etc.
It is very simple to write a map view in javascript:
function(doc) {
emit(["key1", doc["key1"]], null);
emit(["key2", doc["key2"]], null);
}
When querying, you can query using multiple keys:
{"keys": [["key1", 123], ["key2", 123]]}
You can send that JSON as the data in a POST to the view. Or preferably use an API for your programming language. The results of this query will be each row in the view that matches either key. So, every document which matches on both key1 and key2 will return two rows in the view results.
I also was struggling with simular question, how to use
"select * from ... where key1=123 or key2=123".
The following view would allow you to lookup customer documents by the LastName or FirstName fields:
function(doc) {
if (doc.Type == "customer") {
emit(doc.LastName, {FirstName: doc.FirstName, Address: doc.Address});
emit(doc.FirstName, {LastName: doc.LastName, Address: doc.Address});
}
}
I am using this for a web service that queries all my docs and returns every doc that matches both the existence of a node and the query. In this example I am using the node 'detail' for the search. If you would like to search a different node, you need to specify.
This is my first Stack Overflow post, so I hope I can help someone out :)
***Python Code
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import httplib, json
from tornado.options import define,options
define("port", default=8000, help="run on the given port", type=int)
class MainHandler(tornado.web.RequestHandler):
def get(self):
db_host = 'YOUR_COUCHDB_SERVER'
db_port = 5984
db_name = 'YOUR_COUCHDB_DATABASE'
node = self.get_argument('node',None)
query = self.get_argument('query',None)
cleared = None
cleared = 1 if node else self.write('You have not supplied an object node.<br>')
cleared = 2 if query else self.write('You have not supplied a query string.<br>')
if cleared is 2:
uri = ''.join(['/', db_name, '/', '_design/keysearch/_view/' + node + '/?startkey="' + query + '"&endkey="' + query + '\u9999"'])
connection = httplib.HTTPConnection(db_host, db_port)
headers = {"Accept": "application/json"}
connection.request("GET", uri, None, headers)
response = connection.getresponse()
self.write(json.dumps(json.loads(response.read()), sort_keys=True, indent=4))
class Application(tornado.web.Application):
def __init__(self):
handlers = [
(r"/", MainHandler)
]
settings = dict(
debug = True
)
tornado.web.Application.__init__(self, handlers, **settings)
def main():
tornado.options.parse_command_line()
http_server = tornado.httpserver.HTTPServer(Application())
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
if __name__ == '__main__':
main()
***CouchDB Design View
{
"_id": "_design/keysearch",
"language": "javascript",
"views": {
"detail": {
"map": "function(doc) { var docs = doc['detail'].match(/[A-Za-z0-9]+/g); if(docs) { for(var each in docs) { emit(docs[each],doc); } } }"
}
}
}

Resources