Mongodb replica set attaching "/" with Spring Data Framework - linux

My resplica set configuration is something like this:
rs.glu:SECONDARY> rs.status()
{
"set" : "rs.glu",
"date" : ISODate("2014-04-03T04:34:35Z"),
"myState" : 2,
"syncingTo" : "10.13.96.13:27017",
"members" : [
{
"_id" : 2,
"name" : "10.13.98.14:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 51453,
"optime" : Timestamp(1396448528, 1),
"optimeDate" : ISODate("2014-04-02T14:22:08Z"),
"lastHeartbeat" : ISODate("2014-04-03T04:34:33Z"),
"lastHeartbeatRecv" : ISODate("2014-04-03T04:34:33Z"),
"pingMs" : 1
},
{
"_id" : 3,
"name" : "10.13.98.13:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 51453,
"lastHeartbeat" : ISODate("2014-04-03T04:34:33Z"),
"lastHeartbeatRecv" : ISODate("2014-04-03T04:34:33Z"),
"pingMs" : 1
},
{
"_id" : 4,
"name" : "10.13.96.13:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 51453,
"optime" : Timestamp(1396448528, 1),
"optimeDate" : ISODate("2014-04-02T14:22:08Z"),
"lastHeartbeat" : ISODate("2014-04-03T04:34:34Z"),
"lastHeartbeatRecv" : ISODate("2014-04-03T04:34:34Z"),
"pingMs" : 0,
"syncingTo" : "10.13.98.14:27017"
},
{
"_id" : 5,
"name" : "10.13.96.12:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 51533,
"optime" : Timestamp(1396448528, 1),
"optimeDate" : ISODate("2014-04-02T14:22:08Z"),
"self" : true
}
],
"ok" : 1
}
Now my connector code is this:
List<ServerAddress> listOfServers = new ArrayList<ServerAddress>();
listOfServers.add(new ServerAddress(properties.getProperty("mongo.host1"), Integer.valueOf(properties.getProperty("mongo.port1"))));
listOfServers.add(new ServerAddress(properties.getProperty("mongo.host2"), Integer.valueOf(properties.getProperty("mongo.port2"))));
listOfServers.add(new ServerAddress(properties.getProperty("mongo.host3"), Integer.valueOf(properties.getProperty("mongo.port3"))));
MongoClient mongoClient = new MongoClient(listOfServers);
mongoClient.setWriteConcern(WriteConcern.ACKNOWLEDGED);
MongoDbFactory gameDbFactory = new SimpleMongoDbFactory(mongoClient, properties.getProperty("adminportal.db"));
MongoTemplate gameMongoTemplate = new MongoTemplate(gameDbFactory);
Here is my properties file:
mongo.host1=10.13.96.12
mongo.port1=27017
mongo.host2=10.13.96.13
mongo.port2=27017
mongo.host3=10.13.98.14
mongo.port3=27017
But when I try to connect my code I get this exception,
WARNING: Server seen down: /10.13.96.12:27017 - java.io.IOException - message: Read timed out
Apr 03, 2014 12:36:26 AM com.mongodb.ConnectionStatus$UpdatableNode update
WARNING: Server seen down: /10.13.96.13:27017 - java.io.IOException - message: Read timed out
Apr 03, 2014 12:36:46 AM com.mongodb.ConnectionStatus$UpdatableNode update
WARNING: Server seen down: /10.13.98.14:27017 - java.io.IOException - message: Read timed out
Apr 03, 2014 12:38:14 AM com.mongodb.ConnectionStatus$UpdatableNode update
why this is happening, why there is a slash before the ip?
there is no where I attached the "/" in my code?
Am I thinking correct that this is the cause of the issue?
Thanks
Ankur

This seems an issue with the logger configuration and not an error.
Please refer to https://groups.google.com/forum/#!topic/mongodb-user/eAV7erPoZGU and Configure logging for the MongoDB Java driver for logger configuration for mongoDB.

Related

How is bounce rate calculated in Google Analytics?

Pulling some of the metrics from Google Analytics (including bounce rate). Saved data given below.
Which metrics are used to calculate bounce rate?
How can I calculate bounce rate using other metrics values?
{
"_id" : ObjectId("5ecd09c83f80224b219b6827"),
"source" : "(direct)",
"medium" : "(none)",
"pagePath" : "/",
"channelGrouping" : "Direct",
"deviceCategory" : "desktop",
"date" : "20180326",
"users" : 6,
"sessions" : 6,
"bounces" : 3,
"avgSessionDuration" : 95.3333333333333,
"pageviews" : 6,
"newUsers" : 5,
"sessionDuration" : 572,
"pageviewsPerSession" : 1,
"bounceRate" : 50,
"goal" : 0,
"accId" : "92025510",
"agencyId" : ObjectId("5e3136e4c2a1b60c89ae07cc"),
"accountMongoId" : ObjectId("5e4ee454cdc4db6a02696405"),
"dataForDate" : "2018-04-01",
"dataForDateTime" : ISODate("2018-03-26T00:00:00.000Z")
}
/* 2 */
{
"_id" : ObjectId("5ecd09c83f80224b219b682c"),
"source" : "(direct)",
"medium" : "(none)",
"pagePath" : "/",
"channelGrouping" : "Direct",
"deviceCategory" : "desktop",
"date" : "20180401",
"users" : 1,
"sessions" : 1,
"bounces" : 1,
"avgSessionDuration" : 0,
"pageviews" : 1,
"newUsers" : 1,
"sessionDuration" : 0,
"pageviewsPerSession" : 1,
"bounceRate" : 100,
"goal" : 0,
"accId" : "92025510",
"agencyId" : ObjectId("5e3136e4c2a1b60c89ae07cc"),
"accountMongoId" : ObjectId("5e4ee454cdc4db6a02696405"),
"dataForDate" : "2018-04-01",
"dataForDateTime" : ISODate("2018-04-01T00:00:00.000Z")
}
bounce rate
The percentage of single-page session (i.e., session in which the person left the property from the first page).
This is an internal calculation done by google its not one that you can calculate yourself i suggest using ga:bouncerate

Structured Spark streaming metrics retrieval

I have an application with structured Spark streaming and I would like to get some metrics like scheduling delay, latency, etc. Usually, such metrics can be found in Spark UI Streaming tab, however, such functionality does not exist for structured streaming as far as I know.
So how can I get these metrics values?
For now, I have tried to use query progress but not all of the required metrics can be found in the results:
QueryProgress {
"timestamp" : "2019-11-19T20:14:07.011Z",
"batchId" : 1,
"numInputRows" : 8,
"inputRowsPerSecond" : 0.8429038036034138,
"processedRowsPerSecond" : 1.1210762331838564,
"durationMs" : {
"addBatch" : 6902,
"getBatch" : 1,
"getEndOffset" : 0,
"queryPlanning" : 81,
"setOffsetRange" : 20,
"triggerExecution" : 7136,
"walCommit" : 41
},
"stateOperators" : [ {
"numRowsTotal" : 2,
"numRowsUpdated" : 2,
"memoryUsedBytes" : 75415,
"customMetrics" : {
"loadedMapCacheHitCount" : 400,
"loadedMapCacheMissCount" : 0,
"stateOnCurrentVersionSizeBytes" : 17815
}
} ],
"sources" : [ {
"description" : "KafkaV2[Subscribe[tweets]]",
"startOffset" : {
"tweets" : {
"0" : 579
}
},
"endOffset" : {
"tweets" : {
"0" : 587
}
},
"numInputRows" : 8,
"inputRowsPerSecond" : 0.8429038036034138,
"processedRowsPerSecond" : 1.1210762331838564
} ]

How to get sum of a particular field of a collection in MongoDB collection using PyMongo?

My MongoDB contains the following data
{
"_id" : ObjectId("5c1b742eb1829b69963029e8"),
"duration" : 12,
"cost" : 450,
"tax" : 81,
"tags" : [],
"participants" : [
ObjectId("5c1b6a8f348ddb15e4a8aac7"),
ObjectId("5c1b742eb1829b69963029e7")
],
"initiatorId" : ObjectId("5c1b6a8f348ddb15e4a8aac7"),
"context" : "coach",
"accountId" : ObjectId("5bdfe7b01cbf9460c9bb5d68"),
"status" : "over",
"webhook" : "http://d4bdc1ef.ngrok.io/api/v1/webhook_callback",
"hostId" : "5be002109a708109f862a03e",
"createdAt" : ISODate("2018-12-20T10:51:26.143Z"),
"updatedAt" : ISODate("2018-12-20T10:51:44.962Z"),
"__v" : 0,
"endedAt" : ISODate("2018-12-20T10:51:44.612Z"),
"startedAt" : ISODate("2018-12-20T10:51:32.992Z"),
"type" : "voip"
}
{
"_id" : ObjectId("5c1b7451b1829b69963029ea"),
"duration" : 1,
"cost" : 150,
"tax" : 27,
"tags" : [],
"participants" : [
ObjectId("5c1b6a8f348ddb15e4a8aac7"),
ObjectId("5c1b7451b1829b69963029e9")
],
"initiatorId" : ObjectId("5c1b6a8f348ddb15e4a8aac7"),
"context" : "coach",
"accountId" : ObjectId("5bdfe7b01cbf9460c9bb5d68"),
"status" : "over",
"webhook" : "http://d4bdc1ef.ngrok.io/api/v1/webhook_callback",
"hostId" : "5be002109a708109f862a03e",
"createdAt" : ISODate("2018-12-20T10:52:01.560Z"),
"updatedAt" : ISODate("2018-12-20T10:52:08.018Z"),
"__v" : 0,
"endedAt" : ISODate("2018-12-20T10:52:07.667Z"),
"startedAt" : ISODate("2018-12-20T10:52:06.762Z"),
"type" : "voip"
}
I want to get the total duration (sum of duration field) for a particular accountID where status is equals to "over" for a particular date range. Anyway to accomplish this using PyMongo? I am unable to form the query
Well I was doing some pretty basic mistakes while converting the query to PyMongo aggregation function. All I would say is be careful with the query structure format and especially the keys are to be encapsulated within quotes(""). To solve this all I have to do was
from bson.objectid import ObjectId
pipe = [
{"$match": {"accountId": ObjectId(accountId),
"status": "over",
"startedAt": {"$gte": startDate,
"$lte": EndDate
}
}},
{"$project": {"readableDate":
{"$dateToString":
{"format": "%Y-%m-%d", "date": "$startedAt"}},
"accountId": str("$accountId"),
"duration": "$duration"
}},
{"$group": {"_id": {"date": "$readableDate",
"accountId": str("$accountId")}, "totalCallDuration": {"$sum": "$duration"}}}]
for doc in db.VoiceCall.aggregate(pipe):
print(doc)
Just a reminder : the startDate and EndDate are in Python datetime format.

Mongo Secondary Sync Stuck in Recovery State

I have one master node and two secondary node. Due to some issues with one secondary node, it got stopped few days back. Now I am trying to resync the secondary node but after some time it goes into recovery state and stuck there.
For sync I deleted the data directory and restarted mongod service.
DB Size - 1.1 TB
sms3:PRIMARY> rs.status()
{
"set" : "sms3",
"date" : ISODate("2015-09-01T08:33:40Z"),
"myState" : 1,
"members" : [
{
"_id" : 9,
"name" : "abc:27117",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 9415375,
"optime" : Timestamp(1441096420, 7),
"optimeDate" : ISODate("2015-09-01T08:33:40Z"),
"self" : true
},
{
"_id" : 10,
"name" : "def:27117",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 9411728,
"optime" : Timestamp(1441096418, 159),
"optimeDate" : ISODate("2015-09-01T08:33:38Z"),
"lastHeartbeat" : ISODate("2015-09-01T08:33:38Z"),
"lastHeartbeatRecv" : ISODate("2015-09-01T08:33:39Z"),
"pingMs" : 0,
"syncingTo" : "db330.oak1.omniture.com:27117"
},
{
"_id" : 11,
"name" : "ghi:27117",
"health" : 1,
"state" : 3,
"stateStr" : "RECOVERING",
"uptime" : 53615,
"optime" : Timestamp(1441042830, 300),
"optimeDate" : ISODate("2015-08-31T17:40:30Z"),
"lastHeartbeat" : ISODate("2015-09-01T08:33:39Z"),
"lastHeartbeatRecv" : ISODate("2015-09-01T08:33:39Z"),
"pingMs" : 0,
"syncingTo" : "db330.oak1.omniture.com:27117"
}
],
"ok" : 1
}
sms3:PRIMARY> rs.config()
{
"_id" : "sms3",
"version" : 87615,
"members" : [
{
"_id" : 9,
"host" : "abc:27117"
},
{
"_id" : 10,
"host" : "def:27117",
"priority" : 0.5
},
{
"_id" : 11,
"host" : "ghi:27117",
"priority" : 0.5
}
]
}
As I understand your capped collection is gone out of sync. So you will need to clear everything from secondary and then restart the sync.
I found the solution. I synced the data directory on secondary node which was down from another secondary node which was up and restarted mongo.

MongoClient.connect - Error: Could not locate any valid servers in initial seed list

i am trying to connect the mongodb using mongodb.MongoClient.connect() with simple url connection string replicaset. When i start the server its throwing the error as
Error: Could not locate any valid servers in initial seed list
this is my code where i am passing three mongodb server as follows
var MongoClient = mongodb.MongoClient;
MongoClient.connect('mongodb://192.168.0.16,192.168.0.23,192.168.0.17/test', function(err, db) {
if(err){
console.error("Error! Exiting... Must start MongoDB first");
console.log("The error is :::::::::::::::", err);
process.exit(1);
}else{
console.log("Connection successful");
}
});
I have done replica set also.
I have three servers one act as a primary and other act as secondary.Using rs.status(), i can able to see that all server working fine.But still i receiving the same error.
mongodb version = 2.2.3
mongdb lib version = 1.3.18
{
"set" : "rs01",
"date" : ISODate("2015-01-09T07:35:15Z"),
"myState" : 1,
"members" : [
{
"_id" : 0,
"name" : "192.168.0.23:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 2079,
"optime" : Timestamp(1420787077000, 1),
"optimeDate" : ISODate("2015-01-09T07:04:37Z"),
"lastHeartbeat" : ISODate("2015-01-09T07:35:13Z"),
"pingMs" : 0
},
{
"_id" : 1,
"name" : "192.168.0.16:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 2088,
"optime" : Timestamp(1420787077000, 1),
"optimeDate" : ISODate("2015-01-09T07:04:37Z"),
"self" : true
},
{
"_id" : 2,
"name" : "192.168.0.17:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1838,
"optime" : Timestamp(1420787077000, 1),
"optimeDate" : ISODate("2015-01-09T07:04:37Z"),
"lastHeartbeat" : ISODate("2015-01-09T07:35:14Z"),
"pingMs" : 0
}
],
"ok" : 1
}
But i don't know what could be a issue.This issue was occurring in my production setup also.
Well, looks straight forward. Are you sure you are running the mongod servers? If so, are they running on the default 27017 port (since you did not specify the port number, that would be the default).
I would simplify your connection string further and just use 1 server url -- for the sake of debugging. I would also explicitly specify a port number to spell it all out.
Is one of these servers a primary? Can you connect to it from Mongo shell? That would be the first test.

Resources