I am building a service method that suppose to build a CSV file. The titles of the file and results are coming from different threads.
def buildCsv(template: Template) : Future[TemporaryFile] = {
val schemaFuture = dbViewSchemaRepository.findOneByTemplateId(template.id)
val resultsFuture = checklistResultRepository.findAllByTemplateId(template.id)
schemaFuture flatMap { optSchema =>
val schema = optSchema match {
case Some(sch : Schema) => sch
case _ => throw UnexpectedException("Schema not found")
}
//get the titles
val titles = buildTitles(schema)
// create temp file
val tempFile = TemporaryFile("test", ".csv")
logger.info("Absolute path: " + tempFile.file.getAbsolutePath)
// start writing results
resultsFuture map { results =>
results foreach { result =>
val resultRow = buildResultRow(result, schema)
tempFile.file.writeCsv(List(resultRow), ',', titles)
}
tempFile
}
}
}
I've built a pretty simple test:
var dbViewSchemaRepo = mock[DbViewSchemaRepository]
doReturn(Future(schema)).when(dbViewSchemaRepo).findOneByTemplateId(schema.templateId)
var checklistResultRepo = mock[ChecklistResultRepository]
doReturn(Future(List(result))).when(checklistResultRepo).findAllByTemplateId(schema.templateId)
val template = mock[Template]
template.id returns schema.templateId
var srv = new ChecklistResultsExportService(dbViewSchemaRepo, checklistResultRepo)
When I run it I get an error:
[error] services.data.model.ChecklistSchema$Schema cannot be cast
to scala.Option (ChecklistResultsExportService.scala:38) [error]
services.data.ChecklistResultsExportService$$anonfun$buildCsv$1.apply(ChecklistResultsExportService.scala:38)
Line 38 is this:
schemaFuture flatMap { optSchema
What am I missing?
Thanks.
My problem was the way I was mocking the response from the repository objects. I should have done it like this:
doReturn(Future.successful(Some(schema))).when(dbViewSchemaRepo).findOneByTemplateId(schema.templateId)
doReturn(Future.successful(List(result))).when(checklistResultRepo).findAllByTemplateId(schema.templateId)
Seem to be working now.
Thanks.
Related
I'm using Node.js I have data like this:
const data= [
{"id":"1","date":"2022-09-07T15:56:32.279Z","req_id":"98"},
{"id":"2","date":"2022-09-08T15:48:19.075Z","req_id":"97"},
{"id":"3","date":"2022-09-06T15:48:19.073Z","req_id":"96"}
{"id":"4","date":"2022-09-06T15:48:19.073Z","req_id":"96"}
]
I want data in this format:
expected Output:
"2022-09-06":[
{"id":"4","date":"2022-09-06T15:48:19.073Z","req_id":"96"},
{"id":"3","date":"2022-09-06T15:48:19.073Z","req_id":"96"}
]
"2022-09-08":[
{"id":"2","date":"2022-09-08T15:48:19.075Z","req_id":"97"}
]
"2022-09-07":[
{"id":"1","date":"2022-09-07T15:56:32.279Z","req_id":"98"}
]
Assuming the dates are always in the same format, I would do something like this:
function mapData(data){
// returns the given date as an string in the "%dd-%mm-%yyyy" format
const getDateWithoutTime = (dateString) => dateString.split("T")[0];
const mappedData = [];
for(const req of data){
const formattedDate = getDateWithoutTime(req.date);
// if the key already exists in the array, we add a new item
if(mappedData[formattedDate]) mappedData[formattedDate].push(req);
// if the key doesn't exist, we create an array with that single item for that key
else mappedData[formattedDate] = [req];
}
return mappedData;
}
Straightforward solution using regex:
let result = new Map();
for (const item of data) {
let date = item['date'].match(/\d{4}-\d{2}-\d{2}/)[0];
let items = result.get(date) || [];
items.push(item);
result.set(date, items)
}
I am trying to add some brocolli to my basket in the svelte store I have created. My code adds the brocooli to the basket but then duplicates the baskets and adds a whole new basket to my store. Not sure if the problem is caused by my lack of understanding of javascript or svelte.
Desired result
Basket 1 OrangePineapple Basket 2 BananaApplePlumwalnuthazelnutnutmegbroccoli
ACTUAL RESULT
Basket 1 OrangePineapple Basket 2 BananaApplePlumwalnuthazelnutnutmeg Basket 2 BananaApplePlumwalnuthazelnutnutmegbroccoli
Link to svelte codebox where you can view and run code
https://svelte.dev/repl/80d428000a3f425da798cec3450a59d4?version=3.46.2
if you click the button you see that my basket is duplicating. I am just trying to add the brocooli to the basket.
code below
import { writable } from 'svelte/store';
export const storeBaskets = writable([
{
"name": "Basket 1",
"items": ["Orange", "Pineapple"]
},
{
"name": "Basket 2",
"items": ["Banana", "Apple","Plum","walnut","hazelnut","nutmeg"]
}
])
//Local functions
export const add = (item,basketIndex) => {
storeBaskets.update(val => {
const newItems = [...val[basketIndex].items, item]
const newBasket = {'name':val[basketIndex].name,'items':newItems}
val = [...val,newBasket]
return val
})
val = [...val,newBasket]
With this line you're copying the previous store value and adding the newBasket "on top". That's how the spread operator works with arrays
let arr = [1,2,3]
let n = 4
let arr2 = [...arr, n]
console.log(arr2) // [ 1 , 2 , 3 , 4 ]
I wonder if you might have thought of the different behaviour when spreading an object, where an already existing entry might be overriden if the key already exists
let obj = {key: 'value'}
let key = 'newValue'
let obj2 = {...obj, key}
console.log(obj2) // { key: "newValue" }
To make your code working you could replace the line by val[basketIndex] = newBasket
export const add = (item,basketIndex) => {
storeBaskets.update(val => {
const newItems = [...val[basketIndex].items, item]
const newBasket = {'name':val[basketIndex].name,'items':newItems}
val[basketIndex] = newBasket
return val
})
}
Or, instead of spreading, simply push the new value directly to the according nested array in just one line
export const add = (item,basketIndex) => {
storeBaskets.update(val => {
val[basketIndex].items.push(item)
return val
})
}
You might not need to spread, because it's an array, you'r spreading the existing items of the array and then adding the new basket to it. You can map and replace by basketIndex, like:
export const add = (item,basketIndex) => {
storeBaskets.update(val => {
const newItems = [...val[basketIndex].items, item]
const newBasket = {'name':val[basketIndex].name,'items':newItems}
return val.map((basket, i) => i === basketIndex ? newBasket : basket)
})
}
(Working example)
I am new to Gatling, I am trying to do the performance testing for couple of rest calls. In my scenario I need to extract a value from json response of the 1st call and add those values to the list after looping for few times. Again after looping for few times and adding the values into the list, I want to reuse each value in my next rest call by iterating over the values in the list. Can anyone please suggest on how to implement this. I tried something as below,
var datasetIdList = List.empty[String]
val datasetidsFeeder = datasetIdList.map(datasetId => Map("datasetId" -> datasetId)).iterator
def createData() = {
repeat(20){
feed("").exec(http("create dataset").post("/create/data").header("content-type", "application/json")
.body(StringBody("""{"name":"name"}"""))
.asJson.check(jsonPath("$.id").saveAs("userId"))))
.exec(session => { var usrid = session("userId").as[String].trim
datasetIdList:+= usrid session})
}}
def upload()= feed(datasetidsFeeder).exec(http("file upload").post("/compute-metaservice/datasets/${datasetId}/uploadFile")
.formUpload("File","./src/test/resources/data/File.csv")
.header("content-type","multipart/form-data")
.check(status is 200))
val scn = scenario("create data and upload").exec(createData()).exec(upload())
setUp(scn.inject(atOnceUsers(1))).protocols(httpConf)
}
I am seeing an exception that ListFeeder is empty when trying to run above script. Can someone please help
Updated Code:
class ParallelcallsSimulation extends Simulation{
var idNumbers = (1 to 50).iterator
val customFeeder = Iterator.continually(Map(
"name" -> ("test_gatling_"+ idNumbers.next())
))
val httpConf = http.baseUrl("http://localhost:8080")
.header("Authorization","Bearer 6a4aee03-9172-4e31-a784-39dea65e9063")
def createDatasetsAndUpload() = {
repeat(3) {
//create dataset
feed(customFeeder).exec(http("create data").post("/create/data").header("content-type", "application/json")
.body(StringBody("""{ "name": "${name}","description": "create data and upload file"}"""))
.asJson.check(jsonPath("$.id").saveAs("userId")))
.exec(session => {
val name = session("name").asOption[String]
println(name.getOrElse("COULD NOT FIND NAME"))
val userId = session("userId").as[String].trim
println("%%%%% User ID ====>"+userId)
val datasetIdList = session("datasetIdList").asOption[List[_]].getOrElse(Nil)
session.set("datasetIdList", userId :: datasetIdList)
})
}
}
// File Upload
def fileUpload() = foreach("${datasetIdList}","datasetId"){
exec(http("file upload").post("/uploadFile")
.formUpload("File","./src/test/resources/data/File.csv")
.header("content-type","multipart/form-data")
.check(status is 200))
}
def getDataSetId() = foreach("${datasetIdList}","datasetId"){
exec(http("get datasetId")
.get("/get/data/${datasetId}")
.header("content-type","application/json")
.asJson.check(jsonPath("$.dlp.dlp_job_status").optional
.saveAs("dlpJobStatus")).check(status is 200)
).exec(session => {
val datastId = session("datasetId").asOption[String]
println("request for datasetId >>>>>>>>"+datastId.getOrElse("datasetId not found"))
val jobStatus = session("dlpJobStatus").asOption[String]
println("JOB STATUS:::>>>>>>>>>>"+jobStatus.getOrElse("Dlp Job Status not Found"))
println("Time: >>>>>>"+System.currentTimeMillis())
session
}).pause(10)
}
val scn1 = scenario("create multiple datasets and upload").exec(createDatasetsAndUpload()).exec(fileUpload())
val scn2 = scenario("get datasetId").pause(100).exec(getDataSetId())
setUp(scn1.inject(atOnceUsers(1)),scn2.inject(atOnceUsers(1))).protocols(httpConf)
}
I see below error when I try to execute above script
[ERROR] i.g.c.s.LoopBlock$ - Condition evaluation crashed with message 'No attribute named 'datasetIdList' is defined', exiting loop
var datasetIdList = List.empty[String] defines a mutable variable pointing to a immutable list.
val datasetidsFeeder = datasetIdList.map(datasetId => Map("datasetId" -> datasetId)).iterator uses the immutable list. Further changes to datasetIdList is irrelevant to datasetidsFeeder.
Mutating a global variable with your virtual user is usually not a good idea.
You can save the value into the user's session instead.
In the exec block, you can write:
val userId = session("userId").as[String].trim
val datasetIdList = session("datasetIdList").asOption[List[_]].getOrElse(Nil)
session.set("datasetIdList", userId :: datasetIdList)
Then you can use foreach to iterate them all without using a feeder at all.
foreach("${datasetIdList}", "datasetId") {
exec(http("file upload")
...
}
You should put more work in your question.
Your code is not syntax-highlighted, and is formatted poorly.
You said "I am seeing an exception that ListFeeder is empty" but the words "ListFeeder" are not seen anywhere.
You should post the error message so that it's easier to see what went wrong.
In the documentation linked, there is a Warning. Quoted below:
Session instances are immutable!
Why is that so? Because Sessions are messages that are dealt with in a multi-threaded concurrent way, so immutability is the best way to deal with state without relying on synchronization and blocking.
A very common pitfall is to forget that set and setAll actually return new instances.
This is why the code in the updated question doesn't update the list.
session => {
...
session.set("datasetIdList", userId :: datasetIdList)
println("%%%% List =====>>>" + datasetIdList.toString())
session
}
The updated session is simply discarded. And the original session is returned in the anonymous function.
\n54766392632990,178.32.243.13,wfsdsfsdfs23432,\n54766393632990,178.32.243.13,
Above u can see example of string which I want to parse.. I want to get array if numbers which exist between (\n....,178.32.243.13) .. In this example it will be smth like :
[54766392632990,54766393632990] - how to make it
Please run this script it full file your requirement
var ss = "\n54766392632990,178.32.243.13,wfsdsfsdfs23432,\n54766393632990,178.32.243.13,"
var ddd = ss.split(",")
console.log(ddd)
var dfd = []
ddd.forEach(function(res){
if(res.startsWith("\n"))
{
dfd.push(res.replace("\n",""))
}
})
console.log(dfd)
Result [ '54766392632990', '54766393632990' ]
"\n54766392632990,178.32.243.13,wfsdsfsdfs23432,\n54766393632990,178.32.243.13,"
.split("\n")
.filter((n)=> n!== "")
.map(n=> parseInt(n.split(",")[0]))
You can do something like this to parse this string
let s = "\n54766392632990,178.32.243.13,wfsdsfsdfs23432,\n54766393632990,178.32.243.13,"
s = s.split("\n");
let array = [];
for(let i=0;i<s.length;i++) {
let v = s[i].split(",178.32.243.13,");
for(let j=0;j<v.length;j++) {
if(!isNaN(parseInt(v[j]))) {
array.push(v[j]);
}
}
}
console.log(array);
Working on an internal website where the URL contains the source reference from other systems. This is a business requirement and cannot be changed.
i.e. "http://localhost:9000/source.address.com/7808/project/repo"
"http://localhost:9000/build.address.com/17808/project/repo"
I need to remove these strings from the "project/repo" string/variables using a trait so this can be used natively from multiple services. I also want to be able to add more sources to this list (which already exists) and not modify the method.
"def normalizePath" is the method accessed by services, 2 non-ideal but reasonable attempts so far. Getting stuck on a on using foldLeft which I woudl like some help with or an simpler way of doing the described. Code Samples below.
1st attempt using an if-else (not ideal as need to add more if/else statements down the line and less readable than pattern match)
trait NormalizePath {
def normalizePath(path: String): String = {
if (path.startsWith("build.address.com/17808")) {
path.substring("build.address.com/17808".length, path.length)
} else {
path
}
}
}
and 2nd attempt (not ideal as likely more patterns will get added and it generates more bytecode than if/else)
trait NormalizePath {
val pattern = "build.address.com/17808/"
val pattern2 = "source.address.com/7808/"
def normalizePath(path: String) = path match {
case s if s.startsWith(pattern) => s.substring(pattern.length, s.length)
case s if s.startsWith(pattern2) => s.substring(pattern2.length, s.length)
case _ => path
}
}
Last attempt is to use an address list(already exists elsewhere but defined here as MWE) to remove occurrences from the path string and it doesn't work:
trait NormalizePath {
val replacements = (
"build.address.com/17808",
"source.address.com/7808/")
private def remove(path: String, string: String) = {
path-string
}
def normalizePath(path: String): String = {
replacements.foldLeft(path)(remove)
}
}
Appreciate any help on this!
If you are just stripping out those strings:
val replacements = Seq(
"build.address.com/17808",
"source.address.com/7808/")
replacements.foldLeft("http://localhost:9000/source.address.com/7808/project/repo"){
case(path, toReplace) => path.replaceAll(toReplace, "")
}
// http://localhost:9000/project/repo
If you are replacing those string by something else:
val replacementsMap = Seq(
"build.address.com/17808" -> "one",
"source.address.com/7808/" -> "two/")
replacementsMap.foldLeft("http://localhost:9000/source.address.com/7808/project/repo"){
case(path, (toReplace, replacement)) => path.replaceAll(toReplace, replacement)
}
// http://localhost:9000/two/project/repo
The replacements collection can come from elsewhere in the code and will not need to be redeployed.
// method replacing by empty string
def normalizePath(path: String) = {
replacements.foldLeft(path){
case(startingPoint, toReplace) => startingPoint.replaceAll(toReplace, "")
}
}
normalizePath("foobar/build.address.com/17808/project/repo")
// foobar/project/repo
normalizePath("whateverPath")
// whateverPath
normalizePath("build.address.com/17808build.address.com/17808/project/repo")
// /project/repo
A very simple replacement could be made as follows:
val replacements = Seq(
"build.address.com/17808",
"source.address.com/7808/")
def normalizePath(path: String): String = {
replacements.find(path.startsWith(_)) // find the first occurrence
.map(prefix => path.substring(prefix.length)) // remove the prefix
.getOrElse(path) // if not found, return the original string
}
Since the expected replacements are very similar, have you tried to generalize them and use regex matching?
There are a million and one ways to extract /project/repo from a String in Scala. Here are a few I came up with:
val list = List("build.address.com/17808", "source.address.com/7808") //etc
def normalizePath(path: String) = {
path.stripPrefix(list.find(x => path.contains(x)).getOrElse(""))
}
Output:
scala> normalizePath("build.address.com/17808/project/repo")
res0: String = /project/repo
val list = List("build.address.com/17808", "source.address.com/7808") //etc
def normalizePath(path: String) = {
list.map(x => if (path.contains(x)) {
path.takeRight(path.length - x.length)
}).filter(y => y != ()).head
}
Output:
scala> normalizePath("build.address.com/17808/project/repo")
res0: Any = /project/repo
val list = List("build.address.com/17808", "source.address.com/7808") //etc
def normalizePath(path: String) = {
list.foldLeft(path)((a, b) => a.replace(b, ""))
}
Output:
scala> normalizePath("build.address.com/17808/project/repo")
res0: String = /project/repo
Depends how complicated you want your code to look (or how silly you want to be), really. Note that the second example has return type Any, which might not be ideal for your scenario. Also, these examples aren't meant to be able to just take the String out of the middle of your path... they can be fairly easily modified if you want to do that though. Let me know if you want me to add some examples just stripping things like build.address.com/17808 out of a String - I'd be happy to do so.