I'm trying to implement a call back on NodeJS EC2 server that's interacting with AWS RDS Postgresql. I'm not quite sure how it's done. There seems to be a EventEmitter method within AWS-SDK's RDS module. It's designed for all RDS instance types like MySQL, Aurora, etc. Not specifically for postgres. All I'm trying to do is to get some kind of callback after an INSERT or DELETE query.
It is not specific if your postgres is RDS or standalone on EC2.
You will need
var pg = require('pg');
var dbe={"result":null};
function Q(sqlQuery,callback) {
/* async, vulnerable, simple */
var conString = "postgres://"+dbUser+":"+dbPass+"#"+dbHost+":"+dbPort+"/"+dbName+"?ssl=true";
pg.connect(conString, function(err, client, done) {
if(err) {
return console.error('error fetching client from pool', err);
}
client.query(sqlQuery, function(err, result) {
done();//call `done()` to release the client back to the pool
if(err) {
return console.error('error running query', err);
}
dbe.result = result;
//console.log(JSON.parse(result.setEncoding('utf8');));
callback();
});
});
}
And calling
var res = Q('select now()', function(a) {console.log(dbe.result)});
or similar - I don't have a playground to test atm
Related
I'm using the snowflake node driver to connect to a DB. When running the connector from a local server I have no issues. However, when I try the same function running in lambda I can't seem to connect. There are no errors, exceptions, or timeouts... just nothing. Here is the code I'm using per their documentation.
var snowflake = require("snowflake-sdk");
var connection = snowflake.createConnection({
account: "****",
username: "******",
password: "******",
});
connect(connection);
const response = {
statusCode: 200,
body: JSON.stringify("Hello from Lambda!"),
};
return response;
function connect(connection) {
console.log("in connection");
let connection_ID;
try {
connection.connect(function (err, conn) {
if (err) {
console.error("Unable to connect: " + err);
} else {
console.log("Successfully connected to Snowflake");
// Optional: store the connection ID.
connection_ID = conn.getId();
}
console.log(connection_ID);
});
} catch (err) {
console.log(err);
}
}
For clarity, my lambda has no issues connecting to other API's, and is not running behind a VPC.
Any help would be greatly appreciated.
If you have not selected any VPC for your lambda function, it will use the default VPC of the region.
Can you select a VPC, which has access to the snowflake public endpoints and check.
If still an issue, please post the Cloud watch logs, it should give a clue.
You can also check on snowflake History page, if you get any Client-side connection request from the lambda or not.
What is the recommended way to establish database connection and close it properly in Node.js using tedious?
For each and every request we are creating new connection and processing the request then closing them in callback.
app.get('/getData/:id', function(req, res){
var id = req.params.id;
var sqlGet = "exec MyStoreProcedure #Id='" + id + "'";
var connection = new Connection(config);
var request = new Request(sqlGet, function(err, result){
connection.close();
if(err)
console.log(err);
else
res.send(result);
});
connection.on('connect', function(err) {
if (err)
{
console.log(err)
}else{
console.log("Connected");
connection.execSql(request);
}
});
});
Is there any other recommended approach to handle this scenario?
UPDATE (Oct 19, 2020):
It appears that tedious-connection-pool is no longer supported/outdated. I've migrated my code to mssql: https://www.npmjs.com/package/mssql
Previous Answer:
You should check out tedious-connection-pool: https://github.com/tediousjs/tedious-connection-pool.
This makes it easy to manage and reuse connections rather than open/close connections continuously.
As part of using connnection pooling, you should extract it out into a separate file so it can be reused across your application.
We have this unique integration which we are working out from AWS Lambda Function -> Oracle 11g RAC(On Prem).
We have chosen AWS Lambda with a runtime of node v8 and hence tried by default to use node-oracledb as the driver. There were many challenges to establish the connections when a fork of node-oracledb --> oracledb-for-lambda was able to make this work between a function within AWS to a simple oracle DB within AWS.
However the code broke with following error when tried out with original environment where it connects to an on-premise Oracle 11g RAC cluster. Following is the error:
ORA-21561: OID generation failed
VPC[{AWS Node Lambda}] -> Direct Connect -> On prem n/w -> Oracle RAC Cluster
Additional Notes:
Added HOSTALIASES file for name resolution
var oracledb = require('oracledb-for-lambda');
var os = require('os');
var fs = require('fs');
'use strict';
str_host = os.hostname() + ' localhost\n';
fs.writeFileSync(process.env.HOSTALIASES,str_host , function(err){
if(err) throw err;
});
var connAttr = {
user: "user",
password: "pass",
connectString: "connection string"
};
oracledb.getConnection(connAttr, function (err, connection) {
if (err) {
log.error("Error Log>>>>>: " + err.message);
return;
}
log.info('Connection was successful!' + connection);
connection.close(
function (err) {
if (err) {
log.error('Error while closing connection'+err.message);
return;
}
});
});
Make sure the connection string which you have give is in below format
//server-ip:port/database_name
First of all, this is one of my first projects in Node.js so I'm very new to it.
I have a project I want to make that is a SOAP (I know, SOAP... backwards compatibility, huh?) interface that connects to an Oracle database.
So I have a WSDL describing what these functions look like (validation for addresses and stuff) and I have a connection to the database.
Now when using the SOAP npm module, you need to create a server and listen using a service that allows you to respond to requests. I have a separate file that contains my SOAP service but this service should do queries on the database to get its results.
How would I go about sort of 'injecting' my database service into my SOAP service so that whenever a SOAP call is done, it orchestrates this to the correct method in my database service?
This is what my code looks like:
databaseconnection.js
var oracledb = require('oracledb');
var dbConfig = require('../../config/development');
var setup = exports.setup = (callback) => {
oracledb.createPool (
{
user : dbConfig.user,
password : dbConfig.password,
connectString : dbConfig.connectString
},
function(err, pool)
{
if (err) { console.error(err.message); return; }
pool.getConnection (
function(err, connection)
{
if (err) {
console.error(err.message);
return callback(null);
}
return callback(connection);
}
);
}
);
};
databaseservice.js
var DatabaseService = function (connection) {
this.database = connection;
};
function doSomething(callback) {
if (!this.database) { console.log('Database not available.'); return; }
this.database.execute('SELECT * FROM HELP', function(err, result) {
callback(result);
});
};
module.exports = {
DatabaseService: DatabaseService,
doSomething: doSomething
};
soapservice.js
var myService = {
CVService: {
CVServicePort: {
countryvalidation: function (args, cb, soapHeader) {
console.log('Validating Country');
cb({
name: args
});
}
}
}
};
server.js
app.use(bodyParser.raw({type: function(){return true;}, limit: '5mb'}));
app.listen(8001, function(){
databaseconnection.setup((callback) => {
var temp = databaseservice.DatabaseService(callback);
soapservice.Init(temp);
var server = soap.listen(app, '/soapapi/*', soapservice.myService, xml);
databaseservice.doSomething((result) => {
console.log(result.rows.length, ' results.');
});
});
console.log('Server started');
});
How would I go about adding the databaseservice.doSomething() to the countryvalidation soap method instead of 'name: args'?
Also: I feel like the structure of my code is very, very messy. I tried finding some good examples on how to structure the code online but as for services and database connections + combining them, I didn't find much. Any comments on this structure are very welcome. I'm here to learn, after all.
Thank you
Dieter
The first thing I see that looks a little off is the databaseconnection.js. It should be creating the pool, but that's it. Generally speaking, a connection should be obtained from the pool when a request comes in and release when you're done using it to service that request.
Have a look at this post: https://jsao.io/2015/02/real-time-data-with-node-js-socket-io-and-oracle-database/ There are some sample apps you could have a look at that might help. Between the two demos, the "employees-cqn-demo" app is better organized.
Keep in mind that the post is a little dated now, we've made enhancements to the driver that make it easier to use now. It's on my list to do a post on how to build a RESTful API with Node.js and Oracle Database but I haven't had a chance to do it yet.
The basic idea of the following code is I read messages off an ActiveMQ Artemis installation and insert them into a MongoDB instance.
It works well for up to a hundred or so messages per second but crashes if I throw a few thousand at it. My first guess would be the constant opening and closing of database connections. Should I also think about using an in-memory store and doing bulk database inserts?
The code is all running in node using the mqtt and mongodb npm packages. The code below, the database and the queue are all running in docker containers if it makes any difference.
var mqtt = require('mqtt'),
client = mqtt.connect('mqtt://mq:1883', {
username: "*************",
password: "*************"
}),
MongoClient = require('mongodb').MongoClient,
ObjectId = require('mongodb').ObjectID,
assert = require('assert'),
url = 'mongodb://db:27017/uo-readings';
client.on('connect', function () {
client.subscribe('readings');
});
client.on('error', function(error){
console.log(error)
});
client.on('message', function (topic, message) {
console.log(message.toString());
MongoClient.connect(url, function(err, db) {
assert.equal(null, err);
console.log("Connected correctly to server.");
db.collection('readings').insertOne(JSON.parse(message.toString()), function(err, result) {
assert.equal(err, null);
console.log("Inserted a document into the readings collection.");
});
client.end(function(){
console.log("Closing Connection.");
db.close();
});
});
});
See #Jonathan Muller's comment above