I'm running the following error when attempting to run knex seed:run against my remote postgres database (not localhost): Knex:Error Pool2 - Error: connect ECONNREFUSED 127.0.0.1:5432.
I am able to run knex migrate:latest successfully and can see that the tables are created on my postgres server, but when I try to seed I get that error. I've run the same migrations/seed file against my local configuration and it has worked without a problem, but when I attempt to seed my heroku postgres instance, it throws this error (I'm not running my local pg service when I'm seeding the new db, which is likely why it's throwing an error).
Any thoughts on why it is attempting to connect to localhost instead of the specified db? Sample of my file provided below:
var User = require("./models/User");
var Project = require("./models/Project");
exports.seed = function(knex, Promise) {
console.log(knex.client.config.connection); //This returns the correct db info.
return knex('user').del()
.then(function() {
return knex('project').del()
}).then(function() {
return new User({id: 1, firstName: "James", lastName: "Lee", phone: "123-456-2000", email: "test#test.com"}).save(null, {method: "insert"});
}).then(function() {
return new Project({id: 1, name: "Test"}).save(null, {method: "insert"});
})
};
This seems to have occurred due to how I was setting up the migrations / seeds. The configurations were actually pulling from two different places, one which had the correct SSL settings in place, the other without (seed file). Adding the correct settings in both places seemed to resolve the issue.
Related
I am unable to query my Postgres DB in Render. I researched this issue online and found that it was a common problem but none of the solutions I found worked.
Here is my server-side code (I am using NodeJS, Express, Typescript and Postgres)
import postgres, { RowList, Row } from 'postgres'
import appconfig from '../app.config'
type Query = (sql: string) => Promise<RowList<Row[]>>
const query: Query = async (sql) => {
try {
const q = postgres({
host: appconfig.database.host,
port: appconfig.database.port,
database: appconfig.database.schema,
username: appconfig.database.username,
password: appconfig.database.password,
})
const res = await q`${sql}`
return res
} catch (err) {
throw err
}
}
export default query
I receive the following error every time and have not had a successful attempt. It's worth noting I have no issues connecting from PGAdmin on the same PC with the same credentials
Error: read ECONNRESET
at TCP.onStreamRead (node:internal/stream_base_commons:217:20)
at cachedError (C:\Users\xxxxx\repos\one-watch\node_modules\postgres\cjs\src\query.js:171:23)
at new Query (C:\Users\xxxxx\repos\one-watch\node_modules\postgres\cjs\src\query.js:36:24)
at sql (C:\Users\xxxxx\repos\one-watch\node_modules\postgres\cjs\src\index.js:111:11)
at C:\Users\xxxxx\repos\one-watch\src\database\query.ts:15:24
I have never used postgres before, all of my database experience has been in mysql up to this point, although I don't expect this is a postgres problem but potentially just a nuance of Render's implementation of their Postgres service. Any help would be greatly appreciated, thanks!
The only articles I've found like this one are related but they are able to get at least some sort of successful connection at least once. In my case they are all failing.
Adding
?sslmode=no-verify
in the end of the url worked for me.
Working on a simple nodejs express app using Mongodb. I am getting a typeerror cannot read from null when I try to work with the connection object that is returned. The connection object isn't null so I am not clear on what the error message is actually trying to point out.
the line of code that is generating the error is:
const silos = conn.db('toolkit').collection('silos')
I have a debug console.log right before that of:
console.log("Connection: ",conn)
I am checking for an error on the connect callback and the error is null. Prior to this issue I had an issue with the connection and username/authentication so I know the error checking works as before it was triggering on bad logins.
The error is:
TypeError: Cannot read property 'db' of null
at mongoClient.connect (/var/projects/drake/Routes/Silos.js:22:28)
at err (/var/projects/drake/node_modules/mongodb/lib/utils.js:415:14)
at executeCallback (/var/projects/drake/node_modules/mongodb/lib/utils.js:404:25)
at executeOperation (/var/projects/drake/node_modules/mongodb/lib/utils.js:422:7)
at MongoClient.connect (/var/projects/drake/node_modules/mongodb/lib/mongo_client.js:168:10)
at getSilos (/var/projects/drake/Routes/Silos.js:18:17)
at Layer.handle [as handle_request] (/var/projects/drake/node_modules/express/lib/router/layer.js:95:5)
at next (/var/projects/drake/node_modules/express/lib/router/route.js:137:13)
at Route.dispatch (/var/projects/drake/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (/var/projects/drake/node_modules/express/lib/router/layer.js:95:5)
the console.log line generators this message:
Connection: MongoClient {
Here is the code that I use for the Mongo connection. This is in a file and the MongoClient object is exported that is used in other files to connect to the database.
const url = `mongodb://${process.env.MONGO_HOST}:${process.env.MONGO_PORT}`
const opts = {
useNewUrlParser: true,
authMechanism: process.env.MONGO_MECH,
authSource: process.env.MONGO_SRC,
auth: {
user: process.env.MONGO_USER,
password: process.env.MONGO_PWD
}
}
const mongoClient = require('mongodb').MongoClient;
const objectID = require('mongodb').ObjectID
const mongo = new mongoClient(url,opts)
module.exports.mongoClient = mongo
Here is the where I pull in that code and call the connect.
Importing the code:
const { mongoClient } = require('../mongo')
Using the imported code:
mongoClient.connect( (err, conn) => {
if (err) err(res,err, "Database Connection Error", 500)
console.log("Connection: ",conn)
const silos = conn.db('toolkit').collection('silos')
this last line is the one that gives the error.
Results of the console.log
Connection: MongoClient {
One common cause of this is when you try to interact with the mongodb server before a connection is actually made. I might be wrong but I suspect you might be calling your db operations in the wrong order. To avoid this, make sure you have actually successfully connected to the mongodb server before running queries on the collection. If you are using mongoose, then
mongoose.connect('/a/valid/mongo/uri')
will make
mongoose.connection
a valid object that can be interacted with.
Could you please update your question with your connection handling code?
No idea what this was about.
I appreciate the feedback that I got to this.
I spent a weekend rewriting the code from scratch and it worked this time. I did the same thing I believe. There must of been a typo that I was overlooking.
For what it's worth I want to share my experiences with this problem. I was using Mongoose to make the connection but it would crash on either a "replica set" or an "auth error". Eventually I decided to move to MongoClient, just like you did. This is because "Mongoose will not throw any errors by default if you use a model without connecting.".
Moving to Mongo the connection error also returned 'null'. I searched and searched and tried several things, including copying your code and adding the 'auth' option, etc. I eventually switched back to Mongoose (though that shouldn't matter for what solved it for me) and did this:
Create a new user with ReadWrite abilities (didn't help immediately but is definitely needed, since it gave an error when deploying to Heroku without the correct 'new' user.). I now have two users in my MongoDB account.
Add the following to my (Mongoose) Schema:
///Already had this:
let testSchema = new Schema {
foo: String,
bar: Number
}
///Added:
,{
bufferCommands: false,
autoCreate: false
});
Since the DB was still empty I added the following code after creating the model.
///Already had this:
let Test = mongoose.model('Test', testSchema);
///Added:
Test.createCollection();
I believe it then initiated the collection 'for real'. In the mongo shell I know that a collection is not visible until there is an item in there. Now that the app.js didn't crash upon starting I managed to add a document to the database using a form in my HTML and it worked. I now have a working DB connected to my app. Hope this could help anyone also landing on this page.
I'm trying out the Roxy deployer. The Roxy app was created using the default app-type. I setup a new ML 9 database, and I ran "ml local bootstrap" using the default ports (8040 and 8041)
Then I setup a node application. I tried the following (sample code from https://docs.marklogic.com/jsdoc/index.html)
var marklogic = require('marklogic');
var conn = {
host: '192.168.33.10',
port: 8040,
user: 'admin',
password: 'admin',
authType: 'DIGEST'
}
var db = marklogic.createDatabaseClient(conn);
db.createCollection(
'/books',
{author: 'Beryl Markham'},
{author: 'WG Sebald'}
)
.result(function(response) {
console.log(JSON.stringify(response, null, 2));
}, function (error) {
console.log(JSON.stringify(error, null, 2));
});
Running the script gave me an error like:
$ node test.js
{
"message": "write document list: cannot process response with 500 status",
"statusCode": 500,
"body": "<error:error xsi:schemaLocation=\"http://marklogic.com/xdmp/error error.xsd\" xmlns:error=\"http://marklogic.com/xdmp/error\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">\n <error:code>XDMP-IMPMODNS</error:code>\n <error:name>err:XQST0059</error:name>\n <error:xquery-version>1.0-ml</error:xquery-version>\n <error:message>Import module namespace mismatch</error:message>\n <error:format-string>XDMP-IMPMODNS: (err:XQST0059) Import module namespace http://marklogic.com/rest-api/endpoints/config does not match target namespace http://marklogic.com/rest-api/endpoints/config_DELETE_IF_UNUSED of imported module /MarkLogic/rest-api/endpoints/config.xqy</error:format-string>\n <error:retryable>false</error:retryable>\n <error:expr/>\n <error:data>\n <error:datum>http://marklogic.com/rest-api/endpoints/config</error:datum>\n <error:datum>http://marklogic.com/rest-api/endpoints/config_DELETE_IF_UNUSED</error:datum>\n <error:datum>/MarkLogic/rest-api/endpoints/config.xqy</error:datum>\n </error:data>\n <error:stack>\n <error:frame>\n <error:uri>/roxy/lib/rewriter-lib.xqy</error:uri>\n <error:line>5</error:line>\n <error:column>0</error:column>\n <error:xquery-version>1.0-ml</error:xquery-version>\n </error:frame>\n </error:stack>\n</error:error>\n"
}
If I change the port to 8000 (the default appserver that inserts into Documents), the node function executes correctly as expected. I'm not sure if I need to configure anything else with the Roxy-created appserver so that it works with the node.js application.
I'm not sure where the "DELETE_IF_UNUSED" part in the error message is coming from either. There doesn't seem to be any such text in the configuration files generated by Roxy.
Edit: When accessing 192.168.33.10:8040 via the browser, I get a an xml with a similar error:
<error:error xsi:schemaLocation="http://marklogic.com/xdmp/error error.xsd" xmlns:error="http://marklogic.com/xdmp/error" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<error:code>XDMP-IMPMODNS</error:code>
<error:name>err:XQST0059</error:name>
<error:xquery-version>1.0-ml</error:xquery-version>
<error:message>Import module namespace mismatch</error:message>
<error:format-string>XDMP-IMPMODNS: (err:XQST0059) Import module namespace http://marklogic.com/rest-api/endpoints/config does not match target namespace http://marklogic.com/rest-api/endpoints/config_DELETE_IF_UNUSED of imported module /MarkLogic/rest-api/endpoints/config.xqy</error:format-string>
<error:retryable>false</error:retryable>
<error:expr/>
<error:data>
<error:datum>http://marklogic.com/rest-api/endpoints/config</error:datum>
<error:datum>http://marklogic.com/rest-api/endpoints/config_DELETE_IF_UNUSED</error:datum>
<error:datum>/MarkLogic/rest-api/endpoints/config.xqy</error:datum>
</error:data>
<error:stack>
<error:frame>
<error:uri>/roxy/lib/rewriter-lib.xqy</error:uri>
<error:line>5</error:line>
<error:column>0</error:column>
<error:xquery-version>1.0-ml</error:xquery-version>
</error:frame>
</error:stack>
</error:error>
If it matters, MarkLogic version is 9.0-3.1. It's a fresh install too.
Any advice?
Based on the comments, it looks like the problem is that the Node.js Client API expects to talk to a REST API endpoint, but the default Roxy configuration is an MVC application. If you haven't already done anything major with your Roxy app, I'd remove it and create one with --app-type=rest.
$ ml new my-app --app-type=rest --server-version=9
$ ml local bootstrap
$ ml local deploy modules
Then try your Node app.
I've got a node app built on Hapi using MongoDB and mongoose. Locally, I can use the app without issue. I can connect to the db, add data, find it, etc.
I've created an Ubuntu 14.04 x64 droplet on Digital Ocean.
I can ssh into my droplet and verify that my db is there with the correct name. I'm using dokku-alt to deploy and I have linked the db name to the app using dokku's mongodb:link appName mydb
I was having issues once I deployed the app where it would hang and eventually timeout. After a lot of debugging and commenting out code I found that any time I try to hit mongo like this the app will hang:
var User = request.server.plugins.db.User;
User
.findOne({ id: request.auth.credentials.profile.raw.id })
.exec(function(err, user){
// do something
});
Without this, the app loads fine, albeit without data. So my thought is that mongoose is never properly connecting.
I'm using grunt-shell-spawn to run a script which checks if mongo is already running, if not it starts it up. I'm not 100% certain that this is needed on the droplet, but I was having issues locally where mongo was already running... script:
/startMongoIfNotRunning.sh
if pgrep mongod; then
echo running;
else
mongod --quiet --dbpath db/;
fi
exit 0;
/Gruntfile.js
shell: {
make_dir: {
command: 'mkdir -p db'
},
mongodb: {
command: './startMongoIfNotRunning.sh',
options: {
stdin: false,
}
}
},
And here's how I'm defining the database location:
/index.js
server.register([
{ register: require('./app/db'), options: { url: process.env.MONGODB_URL || 'mongodb://localhost:27017/mydb' } },
....
/app/db/index.js
var mongoose = require('mongoose');
var _ = require('lodash-node');
var models = require('require-all')(__dirname + '/models');
exports.register = function(plugin, options, next) {
mongoose.connect(options.url, function() {
next();
});
var db = mongoose.connection;
plugin.expose('connection', db);
_.forIn(models, function(value, key) {
plugin.expose(key, value);
});
};
exports.register.attributes = {
name: 'db'
};
My app is looking for db files in db/. Could it be that dokku's mongodb:link appName mydb linked it to the wrong location? Perhaps process.env.MONGODB_URL is not being set correctly? I really don't know where to go from here.
It turns out the solution to my problem was adding an entry to the hosts file of my droplet to point to the mongo db url:
127.0.0.1 mongodb.myurl.com
For some reason, linking the db to my app with Dokku didn't add this bit. I would have thought that it was automatic. The app container's host file did get a mongodb entry when i linked the db to the app.
I have configured like below and my MongoDB don't need username or password:
mongo: {
module: 'sails-mongo',
url: "mongodb://127.0.0.1:27017/mydb",
replSet: {
servers: [
{
host: "127.0.0.1",
port : 27018
},
{
host: "127.0.0.1",
port : 27019
}
],
options: {connectWithNoPrimary:true, rs_name:"rs0"}
}
}
It's working fine, meaning I do not get a connection error and I am able to do querying. But when I brought down 127.0.0.1:27017, 127.0.0.1:27018 becomes PRIMARY as if I did a rs.status(). After this, I am no longer able to do any query and keep getting the following:
Error: no open connections
I am sure that I setup replica-set in my local machine correctly as I used MongoDB native driver to test the above mentioned scenario (bring down PRIMARY and SECONDARY take over as PRIMARY) and there is no problem.
var url = 'mongodb://127.0.0.1:27017,127.0.0.1:27018,127.0.0.1:27019/mydb?w=0&wtimeoutMS=5000&replicaSet=sg1&readPreference=secondary';
mongodb.MongoClient.connect(url, function(err, result) {
if(err || result === undefined || result === null) {
throw err;
} else {
db = result;
}
});
ok I found the answer. This message emitted because of session.js. I commented everything in the file and now it is working. The reason I guess is in session.js, it only pointing to a single host, which is the original PRIMARY. when you bring down this mongodb PRIMARY, session.js no longer can connect so it threw exception. I also tried the mongodb URL string in sessions.js by putting in also the hosts ip in the replica set (mongodb://127.0.0.1:27017,127.0.0.1:27018,127.0.0.1:27019/mydb) but failed to "sails lift". When put only single host then it is fine.
now if I need to store sessions info, I need to start another mongodb instance then session.js point to this new instant.