How do I get elapsed query time in the REST interface with ArangoDB? (an additional json field with the elapsed time)
Thanks.
Its possible to get profile information for the different execution phases of AQL queries via setting the profile option to true.
It can be done in arangosh like this:
q = "FOR doc IN _users RETURN doc";
s = db._createStatement({ query: q, options: { profile: true } });
res = s.execute().getExtra();
The resulting json of the getExtra() will look like that:
{
"stats" : {
"writesExecuted" : 0,
"writesIgnored" : 0,
"scannedFull" : 1,
"scannedIndex" : 0,
"filtered" : 0
},
"profile" : {
"initializing" : 0.0000040531158447265625,
"parsing" : 0.00003600120544433594,
"optimizing ast" : 0.0000040531158447265625,
"instantiating plan" : 0.000010967254638671875,
"optimizing plan" : 0.000023126602172851562,
"executing" : 0.00004601478576660156
},
"warnings" : [ ]
}
For shure https://docs.arangodb.com/Aql/Invoke.html should and will mention this.
Related
I am using firebase real-time database. I don't want to get all child nodes for a particular parent node, I am concerned this with a particular node, not the sibling nodes. Fetching all the sibling nodes increases my billing in firebase as extra XXX MB of data is fetched. I am using NodeJs admin library for fetching this.
Adding a sample JSON
{
"phone" : {
"shsjsj" : {
"battery" : {
"isCharging" : true,
"level" : 0.25999999046325684,
"updatedAt" : "2018-05-15 12:45:29"
},
"details" : {
"deviceCodeName" : "sailfish",
"deviceManufacturer" : "Google",
"deviceName" : "Google Pixel",
},
"downloadFiles" : {
"7bfb21ff683f8652ea390cd3a380ef53" : {
"uploadedAt" : 1526141772270,
}
},
"token" : "cgcGiH9Orbs:APA91bHDT3mI5L3N62hqUT2LojqsC0IhwntirCd6x0zz1CmVBz6CqkrbC",
"uploadServer" : {
"createdAt" : 1526221336542,
}
},
"hshssjjs" : {
"battery" : {
"isCharging" : true,
"level" : 0.25999999046325684,
"updatedAt" : "2018-05-15 12:45:29"
},
"details" : {
"deviceCodeName" : "sailfish",
"deviceManufacturer" : "Google",
"deviceName" : "Google Pixel",
},
"downloadFiles" : {
"7bfb21ff683f8652ea390cd3a380ef53" : {
"uploadedAt" : 1526141772270,
}
},
"token" : "cgcGiH9Orbs:APA91bH_oC18U56xct4dRuyw9qhI5L3N62hqUT2LojqsC0IhwntirCd6x0zz1CmVBz6CqkrbC",
"uploadServer" : {
"createdAt" : 1526221336542,
}
}
}
}
In the above sample JSON file, i want to fetch all phone->$deviceId->token. Currently, I am fetching the whole phone object, then I iterate over all the phone ID's to fetch the token. This spikes my database download usage and increases billing. I am only concerned with the token of all the devices. Siblings of the token is unnecessary.
All queries to Realtime Database fetch everything under the location requested. There is no way to limit to certain children under that location. If you want only certain children at a location, but not everything under that location, you'll have to query for each one of them separately. Or, you can restructure or duplicate your data to support the specific queries you want to perform - duplication is common for nosql type databases.
I have a collection data which has around 300k entries and its document looks like
{
"_id" : ObjectId("5xxx85"),
"user_id" : "1",
"name" : "test",
"user_private" : "0"
}
now i want to update all the documents in this collection and new document will look like
{
"_id" : ObjectId("5xxx85"),
"rid" : "1",
"user_name" : "test",
"is_private" : "private",
"is_moderator" : "true",
"amount" : "11111"
}
i.e i need to add new fields, update field names and check if user_private = 0 then put is_private as private or else put is_private as not_private.
I am a bit new so I am not able to get how can i do this efficiently as entries are around 300k.
Please suggest some ways, pseudo code will be really helpful
To update a document a filter criteria. Check pseudo code below and follow link to read more.
You'll need to have an existing value for user_private
db.messages.updateMany([
{ "user_private" : 0 }, // filter.
{ $set: {
"user_private" : "private",
"is_moderator" : "true"
}
}, // update.
{
upsert: true
}
]);
upserts - Creates a new document if no documents match the filter or Updates documents that match the filter based on the filter and update parameters provided.
I'm working on a project that uses flexible schemas. I've setup a local mongodb server and am using mongoose inside node.
Having an interesting scaling problem and was wondering if these response times were normal. If a query returns 50 documents, I takes 5-10 seconds for mongo to respond. In the same collection, a query that returns 2 documents is milliseconds.
It's not a slow connection because it's local, was wondering if anyone had an idea as to what was causing this.
I'm using OS X and mongo 3.0.1
Edit: The documents are nearly empty at the moment, with just one or two properties.
Edit: The total number of documents doesn't really matter, just the returned size. If there are 51 documents, 50 like {_id: "...", _schema:"bar"} and 1 {_id:"...", _schema: "foobar" } then collection.find({_schema:"bar"}) takes several seconds and collection.find({_schema:"foobar"}) takes no time.
Explain output:
"queryPlanner" : {
"plannerVersion" : 1,
"namespace" : "mean-dev.documentmodels",
"indexFilterSet" : false,
"parsedQuery" : {
"$and" : [ ]
},
"winningPlan" : {
"stage" : "COLLSCAN",
"filter" : {
"$and" : [ ]
},
"direction" : "forward"
},
"rejectedPlans" : [ ]
},
"serverInfo" : {
"host" : "Sams-MBP.local",
"port" : 27017,
"version" : "3.0.1",
"gitVersion" : "nogitversion"
},
"ok" : 1
No, it should not take that much time.
The issue is probably in the operations in your query (projections, sorting, geosearch, grouping, etc). The best way to solve that is by creating an index to speed up such query.
To create an index on _schema field execute that command in mongodb:
db.collection.ensureIndex({"_schema":1});
I am using MongoDB and the 10Gen node.js driver.
I have a collection in MongoDB that has docs similar to this:
{
_id: ObjectId( "500310affdc47af710000001" ),
name:'something',
tags:['apple','fruit','red'],
created: Date( 1342378221351 )
}
What I would like to get is look at all the documents and get a distinct count of all tags across all documents in the collection. I tried the group function and got this:
> db.photos.group(
... {cond:{}
... ,key:{tags:true}
... ,initial:{count:0}
... ,reduce:function(doc,out){out.count++}
... ,finalize:function(out){}});
[
{
"tags" : null,
"count" : 35
},
{
"tags" : [
"#strawberry",
"#friutpicture"
],
"count" : 1
}
]
Which is not right. I can get the distinct without the counts like this:
> db.photos.distinct('tags')
[
"#friutpicture",
"#strawberry",
"#factory",
"#wallpaper",
"#bummber",
"#test",
"#fire",
"#watermark"
]
but I need the counts.
You can use the following in the new Aggregation Framework (2.1+):
> db.photos..aggregate([{$unwind:"$tags"},
{$group:{_id:"$tags"}},
{$group:{_id:"DistictCount",count:{$sum:1}}}])
Your result will be:
{
"result" : [
{
"_id" : "DistictCount",
"count" : 8
}
],
"ok" : 1
}
The group function won't do what you need because you need to "unroll" the tag array before you can group on it and that means you need a map function that emits each tag in a document, as well as a reduce. You can use map/reduce if you are stuck on 2.0 or earlier and can't use aggregation framework. Here is an example that may help.
My site using mongodb for the chat application. Mongodb queries are getting timed out so i checked the db.currentOp(). Below is the currentOp() and Mongodb details,
637 active operations
750 inactive operations
Other details about mongodb:
Mongo db is running with sharding
I have two databases
a)First database having, two table only
b)Second database having , 5 tables
My questions are, why the current.Op() count got increased suddenly and what are the causes we have to taken care if currentOp() count is increased. Please help me on this and apologies for my bad English.
Below are the sample output of my currentOp()
MongoDB shell version: 1.8.2
> db.currentOp()
{
"inprog" : [
{
"opid" : "msdata1:234234234",
"active" : true,
"lockType" : "read",
"waitingForLock" : false,
"secs_running" : 43534,
"op" : "getmore",
"ns" : "local.oplog.rs",
"query" : {
},
"client_s" : "70.52.078.123:12345",
"desc" : "conn"
},
{
"opid" : "msdata1:2342323423",
"active" : true,
"lockType" : "read",
"waitingForLock" : false,
"secs_running" : 231231,
"op" : "query",
"ns" : "ichat.chatmemberlist",
"query" : {
"count" : "chatmemberlist",
"query" : {
"Mid" : "23423",
"bmid" : "23423"
}
},
"client_s" : "70.52.078.123:12345",
"desc" : "conn"
},
{
"opid" : "msdata1:2342323423",
"active" : false,
"lockType" : "write",
"waitingForLock" : true,
"op" : "update",
"ns" : "?ichat.useravail",
"query" : {
"Mid" : "23423"
},
"client_s" : "70.512.078234.423:12345",
"desc" : "conn"
},
...
...
...
From the limited amount of info, I can see that your queries are just running a really long time: "secs_running" : 231231, means 231 seconds. It's likely that you don't have enough resources available for the type of queries that you are running. That could be that you don't have enough memory, or perhaps too much queries that are acquiring a lock. If you're not on MongoDB 2.0.x yet, then you might want to upgrade to that too as it has vastly improved locking: http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html
I would advice to check the mongodb.log file to see which queries are being slow, then use explain to figure out whether you've indexes on the columns and then either add indexes, or see how you can re-design your schema if that might look like a better solution.