I am using MongoDB hosted with mLab and Node.js with express and TypeScript. My problem is that I can make exactly one request to the database when I run my server, and any subsequent request throws "Topology was destroyed." Here's what my code looks like.
export function getTeamName(id: string, callbackSuccess: (name) => void, callbackError?: (error) => void) {
initDb(() => {
db.collection('teams', (err: Error, teams) => {
if (err) { callbackError(err); db.close(); return; }
else {
teams.findOne({ '_id': id }, { 'name': 1 }, (error, name) => {
if (error) { callbackError(error); db.close(); return; }
else { callbackSuccess(name); db.close(); }
});
}
})
}, (err) => {
callbackError(err);
})
}
And the initDb() method:
import { Server, Db } from 'mongodb'; //using mongodb typings
var server = new Server("*******.mlab.com", *****, { auto_reconnect: false });
var db = new Db('serverName', server, { w: 1 });
function initDb(callbackSuccess: (data) => void, callbackError?: (err) => void) {
db.open((err, db) => {
if (err) {
callbackError(err);
}
else {
db.authenticate("username", "password", (error, data) => {
if (error) {
callbackError(err);
}
else {
callbackSuccess(data);
}
});
}
});
}
Thank you for your help.
Make sure your network does not have a firewall... That was my issue.
Related
I have the following API, the API is inserting into a table based on user selection from the client. User can select different material belonging to same experiment. In my payload, I have materials as array, experiment as string. I tried several ways to resolve my error. Following was the last try:
app.post("/insertMaterials", (req, res) => {
for (let mat of req.body["material"]) {
try {
oracledb.getConnection(
{
user: "some_user",
password: "some_pw",
connectString: "someConnStr",
},
function (err, connection) {
if (err) {
console.error("1" + err);
return;
}
connection.execute(
"INSERT INTO MATERIALS (ID, MAT_NAME, EXPR) VALUES((SELECT max(ID) + 1 FROM MATERIALS), :1, :2)",
[mat, req.body["experiment"]],
(err, result) => {
if (err) {
console.error("log " + err);
}
connection.commit();
connection.close();
}
);
}
);
} catch (error) {
console.log(error);
}
}
return res.status(200).json({
title: "SUCCESS: Materials Inserted",
});
});
I always get:
triggerUncaughtException(err, true / fromPromise /);
^
[Error: DPI-1002: invalid dpiConn handle] { errorNum: 0, offset: 0 }
Before I had a separate function of the block inside the for loop and I also tried with execeuteMany. Still same error. After trying lot other ways and reading in internet, I couldn't solve the issue. Except for finally catching uncaughtException and logging the error:
process.on('uncaughtException', (error, next) => {
let date = new Date()
errorLogStream.write(`Date: ${date}. Err: ${error.stack} \n`)
return
})
By catching this exception, my program does not break anymore and data is always inserted. But it would be great to know how and when this is raised and how this can be resolved or where if I am doing a mistake.
UPDATE
Payload example: {'material': ['F99999.7', 'J84845.4'], 'experiment': 'NA32R'}
Function:
async function addMatToExpr(exp, mat) {
let connection;
try {
connection = await oracledb.getConnection(
{
user: "some_user",
password: "some_pw",
connectString: "someConnStr",
});
result = await connection.execute("INSERT INTO MATERIALS (ID,
MAT_NAME, EXPR) VALUES((SELECT max(ID) + 1 FROM MATERIALS), :1, :2)",
[exp, mat], { autoCommit: true })
} catch (error) {
return res.status(404).json({
title: error,
});
} finally {
if (connection) {
try {
await connection.close()
} catch(error) {
console.log(error)
}
}
}
}
API:
app.post("/insertMaterials", (req, res) => {
for (let mat of req.body["materials"]) {
addMatToExpr(req.body["experiment"], mat)
}
});
Added the async/await function and the api that calls the function.
You need to 'await' the Oracle function calls so each completes before continuing. Currently the connection is being closed before the statement is executed.
See all the node-oracledb documentation and examples.
E.g.
async function run() {
let connection;
try {
connection = await oracledb.getConnection(dbConfig);
result = await connection.execute(sql, binds, options);
console.dir(result, { depth: null });
} catch (err) {
console.error(err);
} finally {
if (connection) {
try {
await connection.close();
} catch (err) {
console.error(err);
}
}
}
}
I am trying to call a function inside my post API. I have multiple queries and want to wait for the function to complete in order to execute the next queries. I am having an issue here. The function isn't completing the execution and the rest of the API gets executed as expected. How can I wait for the function to complete its execution? I have searched but couldn't find something convenient and related to my code.
Here's the code:
Node.js
function verifyEmail(mailToUpper)
{
var emailResult;
db2.open(mydbConnection, (err, conn) => {
if(!err)
{
console.log("Connected Successfully");
}
else
{
console.log("Error occurred while connecting to the database " + err.message);
}
conn.query(checkEmailQuery, [mailToUpper], (err, results) => {
if(!err)
{
if(results.length > 0)
{
// res.write("Email already exists");
emailResult = 0;
}
else
{
emailResult = 1;
}
}
conn.close((err) => {
if(!err)
{
console.log("Connection closed with the database");
}
else
{
console.log("Error occurred while trying to close the connection with the database " +
err.message);
}
})
})
})
return emailResult;
}
router.post('/api/postData', (req, res) => {
//some stuff
var waitingForResult;
setTimeout(() => {
waitingForResult = verifyEmail(mailToUpper);
}, 2000)
console.log(waitingForResult); //throwing an error of undefined
if(waitingForResult === 1) //not executing this
{
//my other queries
}
else //always executes this
{
res.send("Email already exists");
}
});
function verifyEmail(mailToUpper) {
return new Promise((resolve, reject) => {
db2.open(mydbConnection, (err, conn) => {
if (!err) {
console.log("Connected Successfully");
} else {
console.log("Error occurred while connecting to the database " + err.message);
}
conn.query(checkEmailQuery, [mailToUpper], (err, results) => {
if (!err) {
if (results.length > 0) {
// res.write("Email already exists");
resolve(0);
} else {
resolve(1);
}
}
conn.close((err) => {
if (!err) {
console.log("Connection closed with the database");
} else {
console.log("Error occurred while trying to close the connection with the database " +
err.message);
}
})
})
})
})
}
router.post('/api/postData', async (req, res) => {
const waitingForResult = await verifyEmail( mailToUpper );
if( waitingForResult === 1 ){
//my other queries
} else {
res.send("Email already exists");
}
});
I'm new to working with Active Directory in general. I'm trying to update the thumbnailPhoto attribute using ldapjs npm package I have my code setup to be able to update attributes in general and it works well.
I get my user like so:
const customeParser = function(entry, raw, callback){ if (raw.hasOwnProperty("thumbnailPhoto")){ entry.thumbnailPhoto = raw.thumbnailPhoto; } callback(entry) }
find(filter, cb) {
const client = ldap.createClient(this.ldapOptions)
client.on('error', err => {
console.error(err.message)
})
//Serach for users
client.bind(
this.options.dn,
this.options.password,
(err) => {
if (err) {
console.error((new Date).toUTCString(), 'Initial Bind Error', err)
client.unbind(() => {
client.destroy()
cb(err)
})
} else {
client.search(
'DC=foo,DC=local', {
scope: 'sub',
attributes: [
'distinguishedName',
'name',
'sn',
'givenName',
'mail',
'sAMAccountName',
'telephoneNumber',
'thumbnailPhoto',
// 'photoURL',
// 'profileImage',
'extensionAttribute1',
'msExchExtensionCustomAttribute1'
],
entryParser: customeParser,
filter: `${filter}`
},
(err, ee) => {
if (err) {
console.log((new Date).toUTCString(), 'SEARCH RESULT', err)
client.unbind(() => {
client.destroy()
cb(err)
})
}
ee.on('searchEntry', (entry) => {
ee.on('end', () => {
client.unbind(() => {
client.destroy()
cb(null, entry.object)
})
})
});
});
}
});
}
I call my modifyUser method from my LDAP class like so:
let changes = {
thumbnailPhoto: 'http://<ip>:<port>/img/photo.jpg'
}
ad.modifyUser(user.dn, changes, function (err, mod) {
if (err) {
console.log('ERROR: ' + JSON.stringify(err));
return;
}
if (!mod) {
console.log('Search: ' + mod + ' not found.');
} else {
console.log('MOD: ', JSON.stringify(mod));
}
})
modifyUser:
modifyUser(user, attributes, cb) {
const client = ldap.createClient(this.ldapOptions)
client.on('error', err => {
console.error(err.message)
})
//Serach for users
client.bind(
this.options.dn,
this.options.password,
(err) => {
if (err) {
console.error((new Date).toUTCString(), 'Initial Bind Error', err)
client.unbind(() => {
client.destroy()
cb(err)
})
} else {
var change = new ldap.Change({
operation: 'replace',
modification: attributes
});
console.log('CHANGE: ', attributes)
client.modify(user, change, function(err) {
if(err) console.error('ERROR: ', err);
},
(err, ee) => {
if (err) {
console.log((new Date).toUTCString(), 'SEARCH RESULT', err)
client.unbind(() => {
client.destroy()
cb(err)
})
}
ee.on('searchEntry', (entry) => {
ee.on('end', () => {
client.unbind(() => {
client.destroy()
cb(null, entry.object)
})
})
});
});
}
});
}
When I update the photo using my LDAP Tool it converts it to what i believe to be an octet or some sort of hex value. How do i mimic that in nodejs?
Am I on the right track? All other attributes get updated instantly using the code above, but the image fails to update...
Also is there a way to update multiple attributes at once? I get the error that only one attribute can be modified at a time.
Thanks in advance for your time!
I am trying to replace a string in url . Here is image of it
in this image I want to replace lssplalpha with lssplprod which are in pics array. For that I created an api . Here is a code
apiRoutes.get('/SchoolListing_lssplalpha_Replace',function(req, res) { schoolListModel.find({},function(err,check){
if(err){
return console.log(err);
}
else{
for(var i=0;i<check.length;){
var f=0;
for(var j=0;j<check[i].pics.length;j++){
f++;
var newTitle = check[i].pics[j].replace("lssplalpha","lsslprod");
check[i].pics[j] = newTitle;
console.log("after change",check[i].pics[j]);
check[i].save(function (err) {
if(err) {
console.error('ERROR!');
}
});
}
if(j==check[i].pics.length&&j==f){
i++;
}
}
console.log("i value",i);
console.log("check length",check.length);
if(i==check.length){
return res.json({status:true,message:"Updated Schools"}); }
}
});
});
I am getting success response . When I go and check database nothing changed in db. To know the reason I write log of it. When I see logs it was replacing correctly. But I didn't understand why those are not reflecting in database? Here is an image of log in console
Please help me to come out of this
The issue here is you are running a for loop (synchronous) where you are calling the model.save() operation which is asynchronous and the loop keeps iterating but the results of the async calls come later. The process of saving a database item in an array takes some time and Node.js knows this, so it starts the update and then just moves on trying to update the next item in the array. Once the write operation is complete a callback function is run, but by that point the loop has completed and there is no way to know which items finish in what order.
You could use the Bulk Write API to update your models. This allows you to sends multiple write operations to the MongoDB server in one command. This is faster than sending multiple independent operations (like) if you use create()) because with bulkWrite() there is only one round trip to MongoDB.
The following examples show how you can use the bulkWrite.
Using async/await:
apiRoutes.get('/SchoolListing_lssplalpha_Replace', async (req, res) => {
try {
let ops = [];
const docs = await schoolListModel.find({}).lean().exec();
docs.forEach(doc => {
const pics = doc.pics.map(pic => pic.replace("lssplalpha", "lsslprod"));
ops.push({
"updateOne": {
"filter": { "_id": doc._id },
"update": {
"$set": { pics }
}
}
});
});
const result = await schoolListModel.bulkWrite(ops);
console.log('Bulk update complete.', result);
res.status(200).json({
status: true,
message: "Updated Schools"
});
} catch (err) {
res.status(400).send({
status: false,
message: err
});
}
});
Using Promise API:
const bulkUpdate = (Model, query) => (
new Promise((resolve, reject) => {
let ops = [];
Model.find(query).lean().exec((err, docs) => {
if (err) return reject(err);
docs.forEach(doc => {
const pics = doc.pics.map(pic => (
pic.replace("lssplalpha", "lsslprod")
));
ops.push({
"updateOne": {
"filter": { "_id": doc._id },
"update": {
"$set": { pics }
}
}
});
if (ops.length === 500) {
Model.bulkWrite(ops).then((err, result) => {
if (err) return reject(err);
ops = [];
resolve(result);
});
}
});
if (ops.length > 0) {
Model.bulkWrite(ops).then((err, result) => {
if (err) return reject(err);
resolve(result);
});
}
});
})
);
apiRoutes.get('/SchoolListing_lssplalpha_Replace', (req, res) => {
bulkUpdate(schoolListModel, {}).then(result => {
console.log('Bulk update complete.', result);
res.status(200).json({
status: true,
message: "Updated Schools"
});
}).catch(err => {
res.status(400).send({
status: false,
message: err
});
});
});
You are running asynchronous call model.save() in for loop(synchronous)
To make your for loop synchronous you can use for...of loop which works asynchronous, also you will not need to add multiple checks like you have done in your code.
Try following code, it will work
apiRoutes.get('/SchoolListing_lssplalpha_Replace', function (req, res) {
schoolListModel.find({},function (err, check) {
if (err) {
return console.log(err);
}
else {
for (let checkObj of check) {
let newPicArr=[];
for (let pic of checkObj.pics) {
pic = pic.replace("lssplalpha", "lsslprod");
newPicArr.push(pic);
}
checkObj.pics=newPicArr;
checkObj.save(function (err) {
if (err) {
console.error('ERROR!');
}
});
}
return res.json({ status: true, message: "Updated Schools" });
}
});
});
I'm having a problem in My code using nodejs and mongoose. where I have a function that is supposed to get books from mongodb ( book.findOne) then update them using second query (updateMany).
the problem is that UpdateMany query executes before getting the books ..so my array stays empty and nothing is updated.
I know that node.js is asynchronous but how could i solve this ?
function UpdateBulk(completeValidBooks){
var existingRowsInDb = [];
completeValidBooks.forEach(function(currentBook) {
book.findOne({'ISIN': currentBook.ISIN },
function(error, result) {
existingRowsInDb.push(result);
});
});
book.updateMany(existingRowsInDb, (err,docs) => {
err ? console.log(err) : console.log(`updated ${docs.length}`);
});
}
Using promise you can do something like this:
function UpdateBulk(completeValidBooks) {
GetExistingRows(completeValidBooks).then((existingRowsInDb) => {
book.updateMany(existingRowsInDb, (err, docs) => {
err ? console.log(err) : console.log(`updated ${docs.length}`);
});
})
}
function GetExistingRows(completeValidBooks) {
return new Promise((resolve, reject) => {
var existingRowsInDb = [];
completeValidBooks.forEach((currentBook) => {
book.findOne({
'ISIN': currentBook.ISIN
}, (error, result) => {
existingRowsInDb.push(result);
});
});
resolve(existingRowsInDb);
})
}
Using callback you can do like this:
function UpdateBulk(completeValidBooks) {
GetExistingRows(completeValidBooks, (existingRowsInDb) => {
book.updateMany(existingRowsInDb, (err, docs) => {
err ? console.log(err) : console.log(`updated ${docs.length}`);
});
})
}
function GetExistingRows(completeValidBooks, callback) {
var existingRowsInDb = [];
completeValidBooks.forEach((currentBook) => {
book.findOne({
'ISIN': currentBook.ISIN
}, (error, result) => {
existingRowsInDb.push(result);
});
});
callback(existingRowsInDb);
}