Elastic search undefined? - node.js

I used mongo-connector on my linux server
pip install mongo-connector
python mongo_connector.py -m localhost:27217 -t http://localhost:9200
Inside my local node.js server I am using mongoDB which is hosted on my linux server
var Movie = require('../models/Movie');
var mongoosastic = require("mongoosastic");
var elasticsearch = require('elasticsearch');
movie = new Movie({title: 'Warrior'});
movie.save(function() {
console.log(arguments)
Movie.search({query:"Warrior"}, function(err, results) {
console.log(results);
});
});
Schema
var mongoose = require('mongoose');
var elasticsearch = require('elasticsearch');
var mongoosastic = require("mongoosastic");
var movieSchema = new mongoose.Schema({
language: String,
year: String,
title: { type:String, es_indexed:true },
director: String
});
movieSchema.plugin(mongoosastic);
module.exports = mongoose.model('Movie', movieSchema);
I keep getting undefined. I think the data is indexed, but I can't search for it.
Testing
curl -XPUT 'http://localhost:9200/blog/user/dilbert' -d '{ "name" : "Dilbert Brown" }'
curl -XGET 'http://localhost:9200/blog/user/dilbert?pretty=true'
{
"_index" : "blog",
"_type" : "user",
"_id" : "dilbert",
"_version" : 1,
"found" : true,
"_source":{ "name" : "Dilbert Brown" }
}
The problem could be that node.js is hosted on my local machine and the mongoDB/elasticsearch is on my linux server.

By pointing at the fact that your node.js and ES are on two different hosts, you've probably put the finger on the issue.
Looking, at your command line :
python mongo_connector.py -m localhost:27217 -t http://localhost:9200
-m points to your Mongo DB on the localhost port 27217 (unless that port is the entry to an SSH tunnel to your remote linux server)
-t points to your replication endpoint (i.e. Elasticsearch on port 9200 of your local machine)
According to your statement that Elasticsearch and MongoDB are hosted on your remote linux server, you should replace localhost with the host name or IP address of your Linux server and make sure the ports 27217 and 9200 are open on that host.
Then you should also make sure that your Node.js and mongoosastic points to your linux server, because by default it uses localhost:9200, i.e. your local machine. So use this instead and point to your linux server:
movieSchema.plugin(mongoosastic, {
hosts: [
'your_linux_host:9200'
]
});
And also that your mongoose connection points to your linux server as well.
mongoose.connect('mongodb://your_linux_host/your_database');

Related

Cannot connect to DeepStream Node.js server if using any custom plugins

So, if I use my DeepStream server as the following
const {Deepstream} = require('#deepstream/server')
const server = new Deepstream({
server.start()
it's working just fine I can connect to it from my frontend app like the following
const {DeepstreamClient} = require('#deepstream/client')
const client = new DeepstreamClient('192.168.88.238:6020')
client.login()
but If I add MongoDB storage instance or RethinkDB
NPM - RethinkDB
const {Deepstream} = require('#deepstream/server')
const server = new Deepstream({
storage: {
name: 'rethinkdb',
options: {
host: 'localhost',
port: 28015
}
}
})
// start the server
server.start()
I get the following error message when trying to reach my ds server.
(I've also tried to connect via WSS:// instead of WS://)
So hi everybody who has the same problem as me...
I figured it out!
So first of all what the npm packages documentation says from the usage of the Mongo DB driver is completely out of data
so what they say how should u use the npm package :
var Deepstream = require( 'deepstream.io' ),
MongoDBStorageConnector = require( 'deepstream.io-storage-mongodb' ),
server = new Deepstream();
server.set( 'storage', new MongoDBStorageConnector( {
connectionString: 'mongodb://test:test#paulo.mongohq.com:10087/munchkin-dev',
splitChar: '/'
}));
server.start();
INSTEAD OF ALL THIS!
You ain't really need the 'deep stream.io-storage-MongoDB' because it's an old module (based on: ), and u don't really need to use this way...
The correct usage of the MongoDB connector :
const {Deepstream} = require('#deepstream/server');
const server = new Deepstream({
storage: {
name: "mongodb",
options: {
connectionString: MONGO_CONNECTION_STRING ,
splitChar: '/'
}
}
});
server.start();
or you can also create a config. yaml file from all this :
storage:
name: mongodb
options:
connectionString: 'MONGO_CONNECTION_STRING'
# optional database name, defaults to `deepstream`
database: 'DATABASE_NAME'
# optional table name for records without a splitChar
# defaults to deepstream_docs
defaultCollection: 'COLLECTION_NAME'
# optional character that's used as part of the
# record names to split it into a table and an id part
splitChar: '/'
and pass it to the deep stream constructor as below:
const {Deepstream} = require('#deepstream/server');
const server = new Deepstream('config.yaml');
server.start();

NodeJS connection to MongoDB (on AWS C9) returns undefined db object

I'm learning how to code on Amazon's Cloud 9, and I'm trying to connect to a MongoDB database from NodeJS, but when I call the connect() function the db returned in the callback is undefined. However, I can see that the data exists when I run show dbs in the mongo shell, and the mongo server is running without issue, and the connect() function itself doesn't throw any errors.
index.js: Here is my index.js file containing the MongDB connection (this file executes when I run npm start in the terminal):
var MongoClient = require('mongodb').MongoClient;
// Connect to the db
MongoClient.connect("mongodb://localhost:27017/testdb", function (err, db) {
if (err){ console.log(err)}
else{
console.log("Connected!")
console.log(db.name)
}
db.close()
})
mongo shell confirms the database exists (sample is the name of the collection):
use testdb
switched to db testdb
db.sample.find()
{ "_id" : ObjectId("5aed5fc7a44ab7d8a4efce2f"), "name" : "Luckyfield" }
mongo server is running as it says "waiting for connections on port 27017", and whenever I run the index.js file this server records the connection opening and closing in the terminal:
connection accepted from 127.0.0.1:43820 #16 (1 connection now open)
2018-05-05T11:07:16.128+0000 I NETWORK [conn16] received client metadata from 127.0.0.1:43820 conn16: { driver: { name: "nodejs", version: "3.0.7" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.9.91-40.57.amzn1.x86_64" }, platform: "Node.js v6.14.1, LE, mongodb-core: 3.0.7" }
2018-05-05T11:07:16.138+0000 I NETWORK [conn16] end connection 127.0.0.1:43820 (0 connections now open)
When I replace db.name to just db, the console shows a MongoClient object full of data, although I don't know what it is. I also tried inserting or querying documents but again, db seems to be undefined.
In summary, when I run npm start in a separate terminal, the index.js file executes and prints undefined in the console, even though the connection is properly established and the data actually exists. Thank you for any help!

double colon in host identifier when connecting to mongodb

I have 2 nodejs application on Linux CentOs7 server. The first is running on the main domain and the second on a subdomain. Both have to connect to the same MongoDb replicaset but on different databases. They have different username and password in the connection string.
The application on the main domain is connecting without problems, but the subdomain gets the error: double colon in host identifier.
This is the config file for the MongoDb on the subdomain:
module.exports = {
'secret': 'mysecret',
'database': 'mongodb://myUID:myPass#127.0.0.1:27017,127.0.0.1:27018,127.0.0.1:27019/mySubDomainApp,replset: { rs_name: "rs0" }',
'hashidsecret': 'theSecret',
'cryptrsecret': 'thecryptosecret'
};
I found the solution in the config file:
module.exports = {
'secret': 'jK5skCC5spUWqrs7p',
'database': 'mongodb://fIujhYes:24KWWisPjfB52#127.0.0.1:27017/challenger?replicaSet=rs0',
'hashidsecret': 'MCZ4584jHMQsfC',
'cryptrsecret': 'wYrdS8KV51Rsvd',
'presetRoles': ['systemadmin', 'admin']
};
Changing ,replset: { rs_name: "rs0" } to ?replicaSet=rs0 did the trick.
But very strange that the first config file works on the main domain on the same server.

Mongoose not connecting on Ubuntu Ubuntu 14.04

I've got a node app built on Hapi using MongoDB and mongoose. Locally, I can use the app without issue. I can connect to the db, add data, find it, etc.
I've created an Ubuntu 14.04 x64 droplet on Digital Ocean.
I can ssh into my droplet and verify that my db is there with the correct name. I'm using dokku-alt to deploy and I have linked the db name to the app using dokku's mongodb:link appName mydb
I was having issues once I deployed the app where it would hang and eventually timeout. After a lot of debugging and commenting out code I found that any time I try to hit mongo like this the app will hang:
var User = request.server.plugins.db.User;
User
.findOne({ id: request.auth.credentials.profile.raw.id })
.exec(function(err, user){
// do something
});
Without this, the app loads fine, albeit without data. So my thought is that mongoose is never properly connecting.
I'm using grunt-shell-spawn to run a script which checks if mongo is already running, if not it starts it up. I'm not 100% certain that this is needed on the droplet, but I was having issues locally where mongo was already running... script:
/startMongoIfNotRunning.sh
if pgrep mongod; then
echo running;
else
mongod --quiet --dbpath db/;
fi
exit 0;
/Gruntfile.js
shell: {
make_dir: {
command: 'mkdir -p db'
},
mongodb: {
command: './startMongoIfNotRunning.sh',
options: {
stdin: false,
}
}
},
And here's how I'm defining the database location:
/index.js
server.register([
{ register: require('./app/db'), options: { url: process.env.MONGODB_URL || 'mongodb://localhost:27017/mydb' } },
....
/app/db/index.js
var mongoose = require('mongoose');
var _ = require('lodash-node');
var models = require('require-all')(__dirname + '/models');
exports.register = function(plugin, options, next) {
mongoose.connect(options.url, function() {
next();
});
var db = mongoose.connection;
plugin.expose('connection', db);
_.forIn(models, function(value, key) {
plugin.expose(key, value);
});
};
exports.register.attributes = {
name: 'db'
};
My app is looking for db files in db/. Could it be that dokku's mongodb:link appName mydb linked it to the wrong location? Perhaps process.env.MONGODB_URL is not being set correctly? I really don't know where to go from here.
It turns out the solution to my problem was adding an entry to the hosts file of my droplet to point to the mongo db url:
127.0.0.1 mongodb.myurl.com
For some reason, linking the db to my app with Dokku didn't add this bit. I would have thought that it was automatic. The app container's host file did get a mongodb entry when i linked the db to the app.

Replica Set not working as expected

I have configured like below and my MongoDB don't need username or password:
mongo: {
module: 'sails-mongo',
url: "mongodb://127.0.0.1:27017/mydb",
replSet: {
servers: [
{
host: "127.0.0.1",
port : 27018
},
{
host: "127.0.0.1",
port : 27019
}
],
options: {connectWithNoPrimary:true, rs_name:"rs0"}
}
}
It's working fine, meaning I do not get a connection error and I am able to do querying. But when I brought down 127.0.0.1:27017, 127.0.0.1:27018 becomes PRIMARY as if I did a rs.status(). After this, I am no longer able to do any query and keep getting the following:
Error: no open connections
I am sure that I setup replica-set in my local machine correctly as I used MongoDB native driver to test the above mentioned scenario (bring down PRIMARY and SECONDARY take over as PRIMARY) and there is no problem.
var url = 'mongodb://127.0.0.1:27017,127.0.0.1:27018,127.0.0.1:27019/mydb?w=0&wtimeoutMS=5000&replicaSet=sg1&readPreference=secondary';
mongodb.MongoClient.connect(url, function(err, result) {
if(err || result === undefined || result === null) {
throw err;
} else {
db = result;
}
});
ok I found the answer. This message emitted because of session.js. I commented everything in the file and now it is working. The reason I guess is in session.js, it only pointing to a single host, which is the original PRIMARY. when you bring down this mongodb PRIMARY, session.js no longer can connect so it threw exception. I also tried the mongodb URL string in sessions.js by putting in also the hosts ip in the replica set (mongodb://127.0.0.1:27017,127.0.0.1:27018,127.0.0.1:27019/mydb) but failed to "sails lift". When put only single host then it is fine.
now if I need to store sessions info, I need to start another mongodb instance then session.js point to this new instant.

Resources