My site using mongodb for the chat application. Mongodb queries are getting timed out so i checked the db.currentOp(). Below is the currentOp() and Mongodb details,
637 active operations
750 inactive operations
Other details about mongodb:
Mongo db is running with sharding
I have two databases
a)First database having, two table only
b)Second database having , 5 tables
My questions are, why the current.Op() count got increased suddenly and what are the causes we have to taken care if currentOp() count is increased. Please help me on this and apologies for my bad English.
Below are the sample output of my currentOp()
MongoDB shell version: 1.8.2
> db.currentOp()
{
"inprog" : [
{
"opid" : "msdata1:234234234",
"active" : true,
"lockType" : "read",
"waitingForLock" : false,
"secs_running" : 43534,
"op" : "getmore",
"ns" : "local.oplog.rs",
"query" : {
},
"client_s" : "70.52.078.123:12345",
"desc" : "conn"
},
{
"opid" : "msdata1:2342323423",
"active" : true,
"lockType" : "read",
"waitingForLock" : false,
"secs_running" : 231231,
"op" : "query",
"ns" : "ichat.chatmemberlist",
"query" : {
"count" : "chatmemberlist",
"query" : {
"Mid" : "23423",
"bmid" : "23423"
}
},
"client_s" : "70.52.078.123:12345",
"desc" : "conn"
},
{
"opid" : "msdata1:2342323423",
"active" : false,
"lockType" : "write",
"waitingForLock" : true,
"op" : "update",
"ns" : "?ichat.useravail",
"query" : {
"Mid" : "23423"
},
"client_s" : "70.512.078234.423:12345",
"desc" : "conn"
},
...
...
...
From the limited amount of info, I can see that your queries are just running a really long time: "secs_running" : 231231, means 231 seconds. It's likely that you don't have enough resources available for the type of queries that you are running. That could be that you don't have enough memory, or perhaps too much queries that are acquiring a lock. If you're not on MongoDB 2.0.x yet, then you might want to upgrade to that too as it has vastly improved locking: http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html
I would advice to check the mongodb.log file to see which queries are being slow, then use explain to figure out whether you've indexes on the columns and then either add indexes, or see how you can re-design your schema if that might look like a better solution.
Related
Maybe I'm missing something, but according to the documentation and all the posts online, setting
cursor.maxTimeMS(1000).toArray(...)
should time out after 1000ms, and MongoDB should kill the operation after timeout.
But none of this is happening.
First, there is no timeout. It keeps going.
Second, I check db.currentOp() and the operation is still there, eating up all the memory. This later adds up and crashes the database with OOM.
Anyway running db.currentOp() after several minutes of no response prints:
{
"inprog" : [
{
"host" : "db2:27017",
"desc" : "conn20",
"connectionId" : 20,
"client" : "127.0.0.1:59214",
"clientMetadata" : {
"driver" : {
"name" : "nodejs",
"version" : "3.1.4"
},
"os" : {
"type" : "Linux",
"name" : "linux",
"architecture" : "x64",
"version" : "4.15.0-30-generic"
},
"platform" : "Node.js v8.10.0, LE, mongodb-core: 3.1.3"
},
"active" : true,
"currentOpTime" : "2018-09-14T00:10:29.903+0000",
"opid" : 11056,
"lsid" : {
"id" : UUID("78a2d853-30bf-4d6d-a208-0a150d9bf8be"),
"uid" : BinData(0,"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
},
"secs_running" : NumberLong(649),
"microsecs_running" : NumberLong(649968360),
"op" : "command",
As you can see, this has been running for 649 seconds, even though I explicitly specified 1000ms.
What is going on here? I've been pulling my hair out for two days and can't figure this out.
I had the same issue and had to update the mongodb driver from 3.1.1 to 3.3.5 and it worked like a charm!
I am using firebase real-time database. I don't want to get all child nodes for a particular parent node, I am concerned this with a particular node, not the sibling nodes. Fetching all the sibling nodes increases my billing in firebase as extra XXX MB of data is fetched. I am using NodeJs admin library for fetching this.
Adding a sample JSON
{
"phone" : {
"shsjsj" : {
"battery" : {
"isCharging" : true,
"level" : 0.25999999046325684,
"updatedAt" : "2018-05-15 12:45:29"
},
"details" : {
"deviceCodeName" : "sailfish",
"deviceManufacturer" : "Google",
"deviceName" : "Google Pixel",
},
"downloadFiles" : {
"7bfb21ff683f8652ea390cd3a380ef53" : {
"uploadedAt" : 1526141772270,
}
},
"token" : "cgcGiH9Orbs:APA91bHDT3mI5L3N62hqUT2LojqsC0IhwntirCd6x0zz1CmVBz6CqkrbC",
"uploadServer" : {
"createdAt" : 1526221336542,
}
},
"hshssjjs" : {
"battery" : {
"isCharging" : true,
"level" : 0.25999999046325684,
"updatedAt" : "2018-05-15 12:45:29"
},
"details" : {
"deviceCodeName" : "sailfish",
"deviceManufacturer" : "Google",
"deviceName" : "Google Pixel",
},
"downloadFiles" : {
"7bfb21ff683f8652ea390cd3a380ef53" : {
"uploadedAt" : 1526141772270,
}
},
"token" : "cgcGiH9Orbs:APA91bH_oC18U56xct4dRuyw9qhI5L3N62hqUT2LojqsC0IhwntirCd6x0zz1CmVBz6CqkrbC",
"uploadServer" : {
"createdAt" : 1526221336542,
}
}
}
}
In the above sample JSON file, i want to fetch all phone->$deviceId->token. Currently, I am fetching the whole phone object, then I iterate over all the phone ID's to fetch the token. This spikes my database download usage and increases billing. I am only concerned with the token of all the devices. Siblings of the token is unnecessary.
All queries to Realtime Database fetch everything under the location requested. There is no way to limit to certain children under that location. If you want only certain children at a location, but not everything under that location, you'll have to query for each one of them separately. Or, you can restructure or duplicate your data to support the specific queries you want to perform - duplication is common for nosql type databases.
Hello all I got stuck somewhere, I am working on mongodb with node.js where my collection data deleted automatically after 1 year on certain date and I want to stop that permanently how can I do that ? I have checked the available material on google but didn't got much success please help me friends ...
I have checked the index in one of my collection and it is showing data like this . Can you please tell me its is having TTL index or not
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "firstfive.teachers"
},
{
"v" : 1,
"key" : {
"_fts" : "text",
"_ftsx" : 1
},
"name" : "firstname_lastname_text",
"weights" : {
"firstName" : 1,
"lastName" : 1
},
"default_language" : "english",
"language_override" : "language",
"ns" : "firstfive.teachers",
"textIndexVersion" : 2
}
]
most likely you have TTL (time to limit) index defined on collection you're working with (https://docs.mongodb.com/v3.2/core/index-ttl/)
yu can check it by running db.your_collection.getIndexes() (it will be one with expireAfterSeconds) in mongo shell.
as any other index it can be removed - but do it carefully, apparently someone did it deliberately
I'm working on a project that uses flexible schemas. I've setup a local mongodb server and am using mongoose inside node.
Having an interesting scaling problem and was wondering if these response times were normal. If a query returns 50 documents, I takes 5-10 seconds for mongo to respond. In the same collection, a query that returns 2 documents is milliseconds.
It's not a slow connection because it's local, was wondering if anyone had an idea as to what was causing this.
I'm using OS X and mongo 3.0.1
Edit: The documents are nearly empty at the moment, with just one or two properties.
Edit: The total number of documents doesn't really matter, just the returned size. If there are 51 documents, 50 like {_id: "...", _schema:"bar"} and 1 {_id:"...", _schema: "foobar" } then collection.find({_schema:"bar"}) takes several seconds and collection.find({_schema:"foobar"}) takes no time.
Explain output:
"queryPlanner" : {
"plannerVersion" : 1,
"namespace" : "mean-dev.documentmodels",
"indexFilterSet" : false,
"parsedQuery" : {
"$and" : [ ]
},
"winningPlan" : {
"stage" : "COLLSCAN",
"filter" : {
"$and" : [ ]
},
"direction" : "forward"
},
"rejectedPlans" : [ ]
},
"serverInfo" : {
"host" : "Sams-MBP.local",
"port" : 27017,
"version" : "3.0.1",
"gitVersion" : "nogitversion"
},
"ok" : 1
No, it should not take that much time.
The issue is probably in the operations in your query (projections, sorting, geosearch, grouping, etc). The best way to solve that is by creating an index to speed up such query.
To create an index on _schema field execute that command in mongodb:
db.collection.ensureIndex({"_schema":1});
4:32 AM (2 minutes ago)
i followed the replication (master-slave) setup guide, and got data replicated fine, but the applier is stopped with the following error. i searched manual and googled without finding much other than the error message itself. really appreciate if you could provide some clues as how to troubleshoot this. thanks!
arangosh [_system]> require("org/arangodb/replication").applier.state();
{
"state" : {
"running" : false,
"lastAppliedContinuousTick" : null,
"lastProcessedContinuousTick" : null,
"lastAvailableContinuousTick" : null,
"progress" : {
"time" : "2014-04-15T20:20:13Z",
"message" : "applier stopped",
"failedConnects" : 0
},
"totalRequests" : 5,
"totalFailedConnects" : 0,
"totalEvents" : 0,
"lastError" : {
"time" : "2014-04-15T20:20:13Z",
"errorMessage" : "no start tick",
"errorNum" : 1413
},
"time" : "2014-04-15T20:37:50Z"
},
"server" : {
"version" : "2.0.4",
"serverId" : "83323931320193"
},
"endpoint" : "tcp://master:8529",
"database" : "_system"
}
This might happen if you start the applier for the very first time without specifying a tick. Use
require("org/arangodb/replication").applier.start(1)
instead.