I have a set of functions which using FSharp.Data.TypeProviders makes several calls to the database via sql to retrieve data.
The code looks like the following:
let query_one (firstName:string) (lastName:string) =
query { for a in db.Names do
where (a.firstname = FirstName && a.secondname = LastName)
select a } |> Seq.last
let query_two (jobTitle:string) =
query { for a in db.Jobs do
where (a.jobId = jobTitle)
select a } |> Seq.last
let detailsList = new List<('a*'b)>()
let queries =
workerDetails
|> Array.Parallel.map (fun (firstn, lastn, jobt) ->
let j1 = query_one firstn lastn
let j2 = query_two jobt
detailsList.add (j1,j2) )
When I run this I get the following:
A first chance exception of type 'System.ArgumentException' occurred
in FSharp.Core.dll
Additional information: An item with the same key has already been
added.
To be more exact:
System.ArgumentException: An item with the same key has already been
added. at
Microsoft.FSharp.Linq.RuntimeHelpers.LeafExpressionConverter.EvaluateQuotation(FSharpExpr
e) at
Microsoft.FSharp.Linq.QueryModule.EvalNonNestedInner(CanEliminate
canElim, FSharpExpr queryProducingSequence) at
Microsoft.FSharp.Linq.QueryModule.EvalNonNestedOuter(CanEliminate
canElim, FSharpExpr tm) at
Microsoft.FSharp.Linq.QueryModule.clo#1737-1.Microsoft-FSharp-Linq-ForwardDeclarations-IQueryMethods-Execute[a,b](FSharpExpr`1
) ... ... at
Microsoft.FSharp.Collections.ArrayModule.Parallel.Map#714-3.Invoke(Int32
obj) at
System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.b__c()
at System.Threading.Tasks.Task.InnerInvoke() at
System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask) at
System.Threading.Tasks.Task.<>c__DisplayClass11.b__10(Object
param0)<---
I also tried running it from C# using Parallel.For().. I am probably missing something basic. Is there an easy way to make a function thread-safe? or a query expression?
Thanks for the help!
Possible option to resolve:
When I included the call for the database context in the function in resolved the parralel exception. Such as:
let query_one (firstName:string) (lastName:string) =
let db = temp_schema.GetDataContext()
query { for a in db.Names do
where (a.firstname = FirstName && a.secondname = LastName)
select a } |> Seq.last
Related
I am trying to pull some documents from Sharepoint using the CSDOM Microsoft.Sharepoint.Client SDK.
I have a basic query set up like this:
let uri = #"XXXXXX"
let userName = XXXXXX
let password = XXXXXX
let networkCredential = new NetworkCredential(userName, password)
let context = new ClientContext(uri)
context.Credentials <- networkCredential
let list = context.Web.Lists.GetByTitle("XXXXXX")
let listItemCollection = list.GetItems(CamlQuery.CreateAllItemsQuery())
context.Load(listItemCollection)
context.ExecuteQuery()
I am getting this error message on the load method
error FS0193: internal error: GenericArguments[0],
'Microsoft.SharePoint.Client.ListItemCollection', on 'Void Load[T](T,
System.Linq.Expressions.Expression1[System.Func2[T,System.Object]][])'
violates the constraint of type 'T'.
I think I have to pass in the linq expression also? This seems like a lot of unneeded steps as all I want to do is get a list of documents from a folder to iterate.
Anyone have any alternative code?
OK, I was able to get this to work using this answer. Here's my working code:
open System
open System.Linq.Expressions
type Expr =
static member Quote(e:Expression<System.Func<_, _>>) = e
// ...
// Authentication logic goes here. Personally, I had to use
// SharePointOnlineCredentials instead of NetworkCredential.
// ...
let items =
let list = context.Web.Lists.GetByTitle("Documents")
list.GetItems(CamlQuery())
let getDisplayName =
Expr.Quote(
fun (item : ListItem) ->
item.DisplayName :> obj)
context.Load(
items,
fun (items : ListItemCollection) ->
items.Include(getDisplayName) :> obj)
context.ExecuteQuery()
for item in items do
printfn "%s" item.DisplayName
It ain't pretty, but it gets the job done.
I am new to Gatling, I am trying to do the performance testing for couple of rest calls. In my scenario I need to extract a value from json response of the 1st call and add those values to the list after looping for few times. Again after looping for few times and adding the values into the list, I want to reuse each value in my next rest call by iterating over the values in the list. Can anyone please suggest on how to implement this. I tried something as below,
var datasetIdList = List.empty[String]
val datasetidsFeeder = datasetIdList.map(datasetId => Map("datasetId" -> datasetId)).iterator
def createData() = {
repeat(20){
feed("").exec(http("create dataset").post("/create/data").header("content-type", "application/json")
.body(StringBody("""{"name":"name"}"""))
.asJson.check(jsonPath("$.id").saveAs("userId"))))
.exec(session => { var usrid = session("userId").as[String].trim
datasetIdList:+= usrid session})
}}
def upload()= feed(datasetidsFeeder).exec(http("file upload").post("/compute-metaservice/datasets/${datasetId}/uploadFile")
.formUpload("File","./src/test/resources/data/File.csv")
.header("content-type","multipart/form-data")
.check(status is 200))
val scn = scenario("create data and upload").exec(createData()).exec(upload())
setUp(scn.inject(atOnceUsers(1))).protocols(httpConf)
}
I am seeing an exception that ListFeeder is empty when trying to run above script. Can someone please help
Updated Code:
class ParallelcallsSimulation extends Simulation{
var idNumbers = (1 to 50).iterator
val customFeeder = Iterator.continually(Map(
"name" -> ("test_gatling_"+ idNumbers.next())
))
val httpConf = http.baseUrl("http://localhost:8080")
.header("Authorization","Bearer 6a4aee03-9172-4e31-a784-39dea65e9063")
def createDatasetsAndUpload() = {
repeat(3) {
//create dataset
feed(customFeeder).exec(http("create data").post("/create/data").header("content-type", "application/json")
.body(StringBody("""{ "name": "${name}","description": "create data and upload file"}"""))
.asJson.check(jsonPath("$.id").saveAs("userId")))
.exec(session => {
val name = session("name").asOption[String]
println(name.getOrElse("COULD NOT FIND NAME"))
val userId = session("userId").as[String].trim
println("%%%%% User ID ====>"+userId)
val datasetIdList = session("datasetIdList").asOption[List[_]].getOrElse(Nil)
session.set("datasetIdList", userId :: datasetIdList)
})
}
}
// File Upload
def fileUpload() = foreach("${datasetIdList}","datasetId"){
exec(http("file upload").post("/uploadFile")
.formUpload("File","./src/test/resources/data/File.csv")
.header("content-type","multipart/form-data")
.check(status is 200))
}
def getDataSetId() = foreach("${datasetIdList}","datasetId"){
exec(http("get datasetId")
.get("/get/data/${datasetId}")
.header("content-type","application/json")
.asJson.check(jsonPath("$.dlp.dlp_job_status").optional
.saveAs("dlpJobStatus")).check(status is 200)
).exec(session => {
val datastId = session("datasetId").asOption[String]
println("request for datasetId >>>>>>>>"+datastId.getOrElse("datasetId not found"))
val jobStatus = session("dlpJobStatus").asOption[String]
println("JOB STATUS:::>>>>>>>>>>"+jobStatus.getOrElse("Dlp Job Status not Found"))
println("Time: >>>>>>"+System.currentTimeMillis())
session
}).pause(10)
}
val scn1 = scenario("create multiple datasets and upload").exec(createDatasetsAndUpload()).exec(fileUpload())
val scn2 = scenario("get datasetId").pause(100).exec(getDataSetId())
setUp(scn1.inject(atOnceUsers(1)),scn2.inject(atOnceUsers(1))).protocols(httpConf)
}
I see below error when I try to execute above script
[ERROR] i.g.c.s.LoopBlock$ - Condition evaluation crashed with message 'No attribute named 'datasetIdList' is defined', exiting loop
var datasetIdList = List.empty[String] defines a mutable variable pointing to a immutable list.
val datasetidsFeeder = datasetIdList.map(datasetId => Map("datasetId" -> datasetId)).iterator uses the immutable list. Further changes to datasetIdList is irrelevant to datasetidsFeeder.
Mutating a global variable with your virtual user is usually not a good idea.
You can save the value into the user's session instead.
In the exec block, you can write:
val userId = session("userId").as[String].trim
val datasetIdList = session("datasetIdList").asOption[List[_]].getOrElse(Nil)
session.set("datasetIdList", userId :: datasetIdList)
Then you can use foreach to iterate them all without using a feeder at all.
foreach("${datasetIdList}", "datasetId") {
exec(http("file upload")
...
}
You should put more work in your question.
Your code is not syntax-highlighted, and is formatted poorly.
You said "I am seeing an exception that ListFeeder is empty" but the words "ListFeeder" are not seen anywhere.
You should post the error message so that it's easier to see what went wrong.
In the documentation linked, there is a Warning. Quoted below:
Session instances are immutable!
Why is that so? Because Sessions are messages that are dealt with in a multi-threaded concurrent way, so immutability is the best way to deal with state without relying on synchronization and blocking.
A very common pitfall is to forget that set and setAll actually return new instances.
This is why the code in the updated question doesn't update the list.
session => {
...
session.set("datasetIdList", userId :: datasetIdList)
println("%%%% List =====>>>" + datasetIdList.toString())
session
}
The updated session is simply discarded. And the original session is returned in the anonymous function.
I am trying to write NHibernate queryover to select all records which has been deleted between two dates. I am using IsBetween().And(). But how do i write if i dont want to include both the fromdate and todate?
Here is my query:
public IEnumerable<DeletedRecord> Search(
DateTime deletedFrom,
DateTime deletedTo
)
{
DeletedRecord delAlias = null;
var query = Session.QueryOver(() => delAlias);
query.Where(() => delAlias.DeletedDate.IsBetween(deletedFrom).And(deletedTo));
return query.Future<DeletedRecord>();
}
Can anyone help me how to achieve so that i can bring all records after the deletedFrom date and before the deletedTo date?
Thanks
Just construct your date in 2 steps:
var query = Session.QueryOver(() => delAlias);
if(youNeedFromDate) //first step
query = query.Where(() => delAlias.DeletedDate >= deletedFrom);
if(youNeedToDate) //second step
query = query.Where(() => delAlias.DeletedDate <= deletedTo);
youNeedFromDate and youNeedToDate are bool variables that you can pass to your function or it could be different condition upon your logic.
Such cases happen quite often . defining extension method helps a lot. see below
public static IQueryOver<T, T> WhereIf<T>(this IQueryOver<T, T> query,bool condition, Expression<Func<T, bool>> expression) where T : class
{
if (condition)
{
query = query.And(expression);
}
return query;
}
Using the above you can chain you conditions and only it will only include the where condition if the first arg evaluates to true.
var query= Session.QueryOver<DeletedRecord>()
.WhereIf(filterByFrom,d=>d.DeletedDate>=fromDate)
.WhereIf(filterByTo,d=>d.DeletedDate<=toDate);
Our code has a SqlExpression, which at its bare minimum is something like:
var q = db.From<Users>();
q.Where(u => u.Age == 25);
totalRecords = db.Scalar<int>(q.ToCountStatement());
q.ToCountStatement() generates the following query:
SELECT COUNT(*) FROM "Users" WHERE ("Age" = #0)
However, db.Scalar() throws an exception: Must declare the scalar variable "#0". This has started occurring in recent versions (tested in 4.0.54). The same code was working fine until v4.0.50. I've checked the release notes, but couldn't find a related change.
Even passing a parameter throws the same exception:
totalRecords = db.Scalar<int>(q.ToCountStatement(), 25);
Is it a bug, or my oversight?
Secondly, is it possible to get q.ToCountStatement() to generate a more optimized query with COUNT(Age) or COUNT([PrimaryKey]) instead of COUNT(*)?
Now that OrmLite defaults to parameterized queries you also need to provide the queries db parameters when executing a query (if you've specified any params), e.g:
var q = db.From<Users>().Where(u => u.Age == 25);
var count = db.Scalar<int>(q.ToCountStatement(), q.Params);
You can also use OrmLite's explicit Count() API's, e.g:
db.Count<User>(x => x.Age == 25);
Or with a typed SqlExpression:
var q = db.From<User>().Where(x => x.Age == 25);
db.Count(q);
Otherwise another way to specify db params is to use an anonymous object, e.g:
db.Scalar<int>("SELECT COUNT(*) FROM Users WHERE Age=#age", new { age = 25});
I have read this (http://www.albahari.com/nutshell/predicatebuilder.aspx)
Here is my code :
var predicateOuter = PredicateBuilder.True<T_Users>();
predicateOuter.And(d => d.code== 357);
var count=tService.GetCount(predicateOuter.Expand());
my service in code first:
public int GetCountSearch(Expression<Func<T, bool>> exp)
{
return _entities.Count(exp);
}
all record in T_Users: 6548
all record where code==357 : 26
But it always returns all records. but why ?
You need to use the results of Add:
// Assign result here to predicateOuter -
predicateOuter = predicateOuter.And(d => d.code== 357);
// This should now function properly
var count = tService.GetCount(predicateOuter.Expand());
Add doesn't modify the predicate, but rather returns a new one with the additional criteria.