I like to split out my code where possible - and having SQL in a route makes me cringe.
Node JS is new to me and I'm learning it at the moment, but I'm having a problem with the following (the page just hangs and doesn't serve anything).
It makes sense to me with my current level of knowledge - but I imagine there will be a better way of doing this.
Thank you for taking the time for reading this, and your help is greatly appreciated.
Route
var express = require('express');
var router = express.Router();
var db = require('../lib/db.js');
var model = require('../models/contacts.js');
/* GET contacts. */
router.get('/', function(req, res) {
// Get data from model - RETURN THE ROWSET HERE
model.get_names(db, function(rowset) {
res.render('contacts', {
title: 'Contacts | Rhubarb',
nav_active: "contacts",
}, function(err, output) {
res.status(200).send(rowset);
}); // res.render
}); // model.get_names
}); // router
module.exports = router;
Model
module.exports.get_names = function(db) {
var sql = " SELECT attr_name " +
" , attr_value " +
" FROM contacts a " +
" , contact_attributes b " +
" WHERE a.contact_id = b.contact_id " +
" AND (attr_name = 'forename' OR attr_name = 'surname')";
db.all(sql, function(err, rowset) {
if (err) throw err;
return rowset;
}); // db.all
} // get_names
Sorted it by adding an actual call back to the function... sigh...
module.exports.get_names = function(db, callback) {
var sql = " SELECT attr_name " +
" , attr_value " +
" FROM contacts a " +
" , contact_attributes b " +
" WHERE a.contact_id = b.contact_id " +
" AND (attr_name = 'forename' OR attr_name = 'surname')";
return db.all(sql, function(err, rowset) {
if (err) throw err;
return callback(rowset);
});
}
Related
I'm trying to run a NodeJS code that reads some data fields from an array, use them to do a database query to check if the data is duplicate before inserting them into the corresponding table.
My NodeJS code will be called from a PHP script so I need to know when it ends this is why I need to add process.exit(0) somewhere. The problem I have is that if I add it, the script is terminated and my promise never gets the time to send back the result.
Here is my code:
var bar = new Promise((resolve, reject) => {
result.forEach((row, index, array) => {
var escaped = _.map(row, mysql.escape);
var checkQuery = "SELECT COUNT(*) as found FROM data WHERE field1 = " + escaped[0] + " AND field2 = " + escaped[1] + " AND field3 = " + escaped[2] + " AND field4 = " + escaped[3] + " AND field5 = " + escaped[4] + " AND field6 = " + escaped[5] + " AND field7 = " + escaped[6] + ";";
conn.query(checkQuery, function (err, res) {
if (err) {
console.log("Error checking row for duplicate");
console.log(checkQuery);
process.exit(1);
} else {
if (res[0].found == 0) {
var query = " (";
var escaped = _.map(row, mysql.escape);
var csv = escaped.join(',');
query += csv;
query += ")";
query += row !== _.last(result) ? ',' : ';';
console.log(query);//This will change to inserting the data to the table
}else{
console.log("Duplicate found!");
}
}
});
if (index === array.length -1) resolve();
});
});
bar.then(() => {
console.log('All done!');
process.exit(0);
});
If I remove process.exit(0); I see "All done" first then console.log(query) result.
If I add it, the script is terminated and I see "All done" only.
Is there a better approach to do this task please?
Thanks.
Here is a way to wait for a promise before the application exits.
class Waiter {
private timeout: any
constructor() {
this.waitLoop()
}
private waitLoop():void {
this.timeout = setTimeout(() => { this.waitLoop() }, 100 * 1000)
}
okToQuit():void {
clearTimeout(this.timeout)
}
}
// Your app.
const appPromise:Promise<any> = ...
const w = new Waiter()
appPromise.finally(() => {
console.log("Quitting")
w.okToQuit()
})
Running multiple asynchronous operations in a loop and tracking when everything is done is just way, way, way easier if you use promises for all the individual asynchronous operation rather than trying to track asynchronous operations that use plain callbacks.
You don't say exactly what your database is, but if it's mysql, then there is a mysql2/promise driver that natively supports promises and that would be my recommendation to switch to that. Then you can directly use a promise returned from .query(). But, without the info about your specific database driver, I've shown how to manually promisify .query().
Then, the looping code can use a for loop and await to sequence the database calls so it's easy to know when they are all complete.
const { promisify } = require('util');
async function someFunc() {
// other code here
// promisify conn.query (or use promise interface directly from the database)
conn.queryP = promisify(conn.query);
try {
for (const row of result) {
const escaped = _.map(row, mysql.escape);
const checkQuery = "SELECT COUNT(*) as found FROM data WHERE field1 = " + escaped[0] + " AND field2 = " +
escaped[1] + " AND field3 = " + escaped[2] + " AND field4 = " + escaped[3] + " AND field5 = " +
escaped[4] + " AND field6 = " + escaped[5] + " AND field7 = " + escaped[6] + ";";
let res = await con.queryP(checkQuery);
if (res[0].found == 0) {
const csv = _.map(row, mysql.escape).join(',');
const terminator = row !== _.last(result) ? ',' : ';';
const query = " (" + csv + ")" + terminator;
console.log(query); //This will change to inserting the data to the table
} else {
console.log("Duplicate found!");
}
}
} catch (e) {
console.log("Error checking row for duplicate: ", checkQuery);
console.log(e);
process.exit(1);
}
console.log('All done!');
process.exit(0);
}
The code appears to be trying to build a query inside the loop where each iteration of the loop will add-on to the next (that's what _.last(result) ? ',' : ';'; look like anyway). If that's the case, then the query variable has to be moved outside the loop so it can build from one iteration of the loop to the next. But, you don't show what you're really trying to do with that query so you're on your own for that part.
you decide how many promises will go out before hand and then count them as they resolve, then exit
in this example the same principle is applied but it has callback functions instead of promises. For promises you would call a count function from the .then() or .finally(), and the count function will decide whether it is time to exit
mongoose example from a javascript server:
let g_DB = null;
//init mongoose
const mongoose = require("mongoose");
const connectionParams = {
useNewUrlParser: true,
useUnifiedTopology: true,
};
const connStr1 = "mongodb+srv://XX:XX#clusterXX.XX.mongodb.net/XX?
retryWrites=true&w=majority";
mongoose.set("strictQuery", false);
mongoose.connect(connStr1, connectionParams)
.then(handleConnection())
.catch((err) => console.log("Error:", err));
//end script
//handleConnection - start on successful response from mongoose connection
function handleConnection(msg) {
console.log("mongoose has connected to Mongo Atlas successfully");
g_DB = mongoose.connection;
g_DB.once("open", function () {
console.log(
"mongoose has connected to Mongo Atlas Cluster using database XX"
);
doTest();
});
}
//---------------------------------------------------
function doTest() {
console.log("test-05: create 500 books");
//---- MODEL ----
const _schema = new mongoose.Schema({
name: String,
price: Number,
quantity: Number,
});
//g_DB is a mongoose connection set earlier in the script
const _model = g_DB.model("book_schema", _schema, "bookstore");
let loopcount = 500;
let waitcount = loopcount;
for (let i = 0; i < loopcount; i++) {
_m = new _model({
name: `WHY MAKE 500 BOOKS ${new Date().toISOString()}`,
price: 200,
quantity: 2000,
});
_m.save((e, x) => {
if (e) return console.error(e);
console.log(x, `waitcount: ${--waitcount}`);
if (!waitcount) doExit();
});
}
}
//--
function doExit() {
console.log("exit from server");
process.exit();
}
Use Reject/Resolve to manage promise in Nodejs
When your task fulfils your request send result with resolve(); and if its failing use reject();
In your case you are not managing promise properly that's why it's running asynchronously, better to use following way with the proper returns.
var bar = new Promise((resolve, reject) => {
return result.forEach((row, index, array) => {
var escaped = _.map(row, mysql.escape);
var checkQuery = "SELECT COUNT(*) as found FROM data WHERE field1 = " + escaped[0] + " AND field2 = " + escaped[1] + " AND field3 = " + escaped[2] + " AND field4 = " + escaped[3] + " AND field5 = " + escaped[4] + " AND field6 = " + escaped[5] + " AND field7 = " + escaped[6] + ";";
return conn.query(checkQuery, function (err, res) {
if (err) {
console.log("Error checking row for duplicate");
console.log(checkQuery);
return reject(err);
} else {
if (res[0].found == 0) {
var query = " (";
var escaped = _.map(row, mysql.escape);
var csv = escaped.join(',');
query += csv;
query += ")";
query += row !== _.last(result) ? ',' : ';';
console.log(query);//This will change to inserting the data to the table
return resolve(query)
} else {
console.log("Duplicate found!");
return reject('Duplicate Found');
}
}
});
});
});
bar.then((data) => {
console.log('All done!');
});
In above code I am returning query + resolve/reject so it makes better to run in more synchronised way.
return conn.query(checkQuery, function (err, res) {
Plus, while processing this promise I am handling with .then((data) so I can handle that resolve values here.
bar.then((data) => {
console.log('All done!');
});
Note: If you are rejecting any promise it won't be available in above .then block you'll find this reject in catch block so code will be changed in following way.
bar.then((data) => {
console.log('All done!');
}).catch(err=>{
console.log(err);
});
You can try the following:
(async () => {
await new Promise((resolve, reject) => {
result.forEach((row, index, array) => {
var escaped = _.map(row, mysql.escape);
var checkQuery = "SELECT COUNT(*) as found FROM data WHERE field1 =
" + escaped[0] + " AND field2 = " + escaped[1] + " AND field3 = " +
escaped[2] + " AND field4 = " + escaped[3] + " AND field5 = " + escaped[4] + " AND field6 = " + escaped[5] + " AND field7 = " + escaped[6] + ";";
conn.query(checkQuery, function (err, res) {
if (err) {
console.log("Error checking row for duplicate");
console.log(checkQuery);
process.exit(1);
} else {
if (res[0].found == 0) {
var query = " (";
var escaped = _.map(row, mysql.escape);
var csv = escaped.join(',');
query += csv;
query += ")";
query += row !== _.last(result) ? ',' : ';';
console.log(query);//This will change to inserting the data to the table
}else{
console.log("Duplicate found!");
}
}
});
if (index === array.length -1) resolve();
});
});
console.log('All done!');
})();
You don't even need to call the process.exit(0) because the code will always terminate when the job is done :)
I understand that this question has been asked to oblivion, but I havent been able to solve my particular issue with it.
When I started my project, I had cors error too, but after adding the following they dissapeared, until today I decided to refractore the code of an app, and it suddenly stopped working, fun fact is that the old code was working ok.:
const cors = require('cors');
app.use(cors());
app.options('*', cors());
I have tried too
app.use(cors({
origin: 'http://localhost:3000'
}));
But the point is, that every single api call works, except one. And even better, it works randomly, sometimes it gives a cors error and sometimes it doesnt. Thats why im lost.
I have downloaded a chrome extension Moesif origin and cors changer and that seems to be the only way to make it work. But its not a valid solution.
Why is this happening?
My api call is this one. As u can observe a simple call with some database manipulation and thats it, it should return a res.status of 100, but im unable to see it when using the chrome extension, and otherwise it will be ccors blocked
//generate calendar
app.post('/generateCalendar', async (req, res) => {
const userId = req.body.userId
const yearsToLive = req.body.yearsToLive
const registerDate = req.body.registerDate
function filterDate(date) {
var stringDate = moment(date).format('YYYY-MM-DD').toString();
console.log("dentro del filter date")
var result = stringDate.match(/(?:(?!T).)*/)
return result[0];
}
function getWeeksToLive(death_date, birth_date) {
//returns the weeks to live between death and birth date, rounded to upper week
var weeks_to_live = moment(death_date).diff(moment(birth_date), 'days') / 7;
console.log("semanas a vivir: " + Math.ceil(weeks_to_live))
return Math.ceil(weeks_to_live);
}
//sets the deathDate and weeksToLive in the database
db.query("SELECT * from users where id =" + userId).then(data => {
var birth_date = data[0].birth_date
var deathDate = ""
//sets the death_date and the weeks to live
deathDate = moment(filterDate(birth_date)).add(yearsToLive, 'years')
db.query("UPDATE users SET death_date = '" + moment(deathDate).format('YYYY-MM-DD').toString() + "' , weeks_to_live = '" + getWeeksToLive(deathDate, birth_date) + "' WHERE id = '" + userId + "';").then(data => {
console.log(data)
/*******/
//Sets the yearsToLive and registerDate in the database
db.query("UPDATE users SET years_to_live = '" + yearsToLive + "' , register_date = '" + registerDate + "' WHERE id = '" + userId + "';").then(data => {
console.log(data)
db.query("INSERT INTO calendar (user_id) values ('" + userId + "');").then(data => {
console.log(data)
// res.send(data)
/*******/
//Sets all the field for the calendar
db.query("INSERT INTO calendar_field (text, rating, calendar_id, week_number) select '', 0, c.id, g.wn from calendar c join users u on u.id = c.user_id cross join generate_series(1, u.weeks_to_live) as g(wn);").then(data => {
console.log(data);
console.log("series generated")
/******/
//the lifeExpectanceSet restriction is removed and access to dashboard is granted
db.query("UPDATE user_permissions SET life_expectancy = 'false' , dashboard = 'true' WHERE user_id = '" + userId + "';").then(data => {
console.log("everything generated")
res.sendStatus(100)
console.log(data)
}).catch(err => console.log(err))
}).catch(err => console.log(err))
})
}).catch(err => {
console.log(err)
res.send(err)
})
}).catch(err => console.log(err))
})
})
This is the old code when it used to work, I havent really done anything apart from refactoring some stuff
//generate calendar
app.post('/generateCalendar', async (req, res) => {
const userId = req.body.userId
const yearsToLive = req.body.yearsToLive
const registerDate = req.body.registerDate
function filterDate(date) {
var stringDate = moment(date).format('YYYY-MM-DD').toString();
console.log("dentro del filter date")
var result = stringDate.match(/(?:(?!T).)*/)
return result[0];
}
function getWeeksToLive(death_date, birth_date) {
//returns the weeks to live between death and birth date, rounded to upper week
var weeks_to_live = moment(death_date).diff(moment(birth_date), 'days') / 7;
console.log("semanas a vivir: " + Math.ceil(weeks_to_live))
return Math.ceil(weeks_to_live);
}
//sets the deathDate and weeksToLive in the database
db.query("SELECT * from users where id =" + userId).then(data => {
var birth_date = data[0].birth_date
var deathDate = ""
//sets the death_date and the weeks to live
deathDate = moment(filterDate(birth_date)).add(yearsToLive, 'years')
db.query("UPDATE users SET death_date = '" + moment(deathDate).format('YYYY-MM-DD').toString() + "' , weeks_to_live = '" + getWeeksToLive(deathDate, birth_date) + "' WHERE id = '" + userId + "';").then(data => {
console.log(data)
}).catch(err => console.log(err))
})
//Sets the yearsToLive and registerDate in the database
db.query("UPDATE users SET years_to_live = '" + yearsToLive + "' , register_date = '" + registerDate + "' WHERE id = '" + userId + "';").then(data => {
console.log(data)
db.query("INSERT INTO calendar (user_id) values ('" + userId + "');").then(data => {
console.log(data)
res.send(data)
})
}).catch(err => {
console.log(err)
res.send(err)
})
//Sets all the field for the calendar
db.query("INSERT INTO calendar_field (text, rating, calendar_id, week_number) select '', 0, c.id, g.wn from calendar c join users u on u.id = c.user_id cross join generate_series(1, u.weeks_to_live) as g(wn);").then(data => {
console.log(data);
}).catch(err => console.log(err))
})
Ok so the problem is that res.sendStatus() is not working for some reason, I switched to res.send("string") and it does the trick
I have an issue with not able to get the affected rows result from the following
During the debug I notice it always crashes at conn.querySync(query.sqlUpdate, params);
Console.log is not showing anything as well.
What did I do wrong here?
CODE
//imports
const format = require('string-format');
const query = require('../db/query');
const message = require('../common/message');
const constant = require('../common/constant');
var ibmdb = require("ibm_db");
require('dotenv').config();
// access the environment variables for this environment
const database = "DATABASE=" + process.env.DATABASE + ";";
const hostname = "HOSTNAME=" + process.env.HOSTNAME + ";";
const uid = "UID=" + process.env.UID + ";";
const pwd = "PWD=" + process.env.PWD + ";";
const dbport = "PORT=" + process.env.DBPORT + ";";
const protocol = "PROTOCOL=" + process.env.PROTOCOL;
const connString = database+hostname+uid+pwd+dbport+protocol;
function updateContact(params) {
ibmdb.open(connString, function(err, conn){
//blocks until the query is completed and all data has been acquired
var rows = conn.querySync(query.sqlUpdate, params);
console.log(rows);
});
}
module.exports.updateContact = updateContact;
I finally understand what the problem is.
The problem lies in me using the querySync function. This function not return affected row counts.
https://github.com/ibmdb/node-ibm_db/blob/master/APIDocumentation.md#querySyncApi
The proper way is to use prepare followed by executeNonQuery.
https://github.com/ibmdb/node-ibm_db/blob/master/APIDocumentation.md#executeNonQueryApi
So from the API, i modify my codes.
...
conn.prepare(query.SQL_UPDATE, function (error, stmt) {
if (err) {
console.log(err);
return conn.closeSync();
}
stmt.executeNonQuery(params, function (err, result) {
if( err ) {
console.log(err);
}
else {
console.log("Affected rows = " + result);
}
//Close the connection
conn.close();
});
});
...
===================This is my code below=========================
function spotifyThisSong() {
var spotify = new Spotify({
id: 'myid',
secret: 'mysecret'
});
var songName = process.argv[3];
if(!songName){
songName = "What's my age again";
}
var params = songName;
spotify.search({ type: 'track', query: params }, function(err, data) {
if ( err ) {
console.log('Error occurred: ' + err);
return; //from spotify npm docs
}
else{
console.log(data);
};
});
}
===================END OF CODE=========================
keeps giving me undefine. I need to extract the song name, year, album and its url. Thanks in advance.
//So this is what I found. This actually involves a few things.
//FIRST, capture user-input,
//SECOND, npm install spotify, and use the template as suggested in the DOCS.
//THIRD, parse through the JSON correctly as done below.
//HOPE this helps someone. (dont forget your keys from spotify)
var songName = process.argv[3]; //capture userInput and query it below
params = songName;
spotify.search({ type: 'track', query: params }, function(err, data) {
if ( err ) {
console.log('Error occurred: ' + err);
return;
}
else{
output = space + "================= DATA HERE ==================" +
space + "Song Name: " + "'" +songName.toUpperCase()+ "'" +
space + "Album Name: " + data.tracks.items[0].album.name +
space + "Artist Name: " + data.tracks.items[0].album.artists[0].name +
space + "URL: " + data.tracks.items[0].album.external_urls.spotify + "\n\n\n";
console.log(output);
};
});
This code doesn't work and I couldn't find out why?
It is always pushing obj right serialized JSON string but it always return with wrong key. In obj id is regularly increasing but key isn't.
var c = redis.createClient(),
obj = {id:0, name:"dudu"},
key="person:";
c.select(0);
c.multi()
.incr("idx:person", function(err, _idx) {
console.log("incr -> #idx: " + _idx);
key += obj.id = _idx;
console.log("After Inc obj: " + JSON.stringify(obj));
})
.set(key, JSON.stringify(obj), function(err, _setResp) {
console.log("set -> #_setResp: " + _setResp);
console.log(JSON.stringify(ihale));
})
.get(key, function(er, _obj) {
console.log("get -> " + key);
if (er) {
res.json(er);
} else {
console.log("Found: " + JSON.stringify(_obj));
res.json(_obj);
}
})
.exec(function(err, replies) {
console.log("MULTI got " + replies.length + " replies");
replies.forEach(function(reply, index) {
console.log("Reply " + index + ": " + reply.toString());
});
});
c.quit();
This worked:
c.INCR("idx:person", function(a,b) {
obj.id = b;
console.dir(obj);
key = "pa:" + b;
c.set(key, JSON.stringify(obj), function(err, _setResp) {
console.log("set -> #_setResp: " + _setResp);
console.log(JSON.stringify(obj));
c.get(key, function(er, _obj) {
console.log("get -> " + key);
if (er) {
res.json(er);
} else {
console.log("Found: " + JSON.stringify(_obj));
res.json(_obj);
}
});
});
});
The way to do this is simple :)
Event driven node is executing every part inside of previous one:
c.INCR("idx:person", function(a,b) {
obj.id = b;
key = "pa:" + b;
c.set(key, JSON.stringify(obj), function(err, _setResp) {
c.get(key, function(er, _obj) {
if (er) {
res.json(er);
} else {
res.json(_obj);
}
});
});
});
In transaction mode command are grouped and passed to Redis . The Exec command execute code you passed . When you pass the key value to a the set command , there is no incremental key value on right of it.
For this kind of use, and if you still want have merged commands in one , script it In Lua:
local keyid = redis.call('INCR', 'idx:person')
local result = redis.call('SET', 'person:'..keyid,ARGV[1])
return 'person:'..keyid
for using it in a redis eval command :
eval "local keyid = redis.call('INCR', 'idx:person'); local result = redis.call('SET', 'person:'..keyid,ARGV[1]);return 'person:'..keyid" 0 "yourJSONObject"
this should work:
client.eval([ "local keyid = redis.call('INCR', 'idx:person'); local result = redis.call('SET', 'person:'..keyid,ARGV[1]);return result", 0,JSON.stringify(obj) ], function (err, res) {
console.log(res); // give the personID
});
You Can also use a Hash instead of a simple key in your example for separate id, name , and json object . Return the hash from the lua script will be like return it from a hset.