Trying to use the wait.for module with a mysql query ( can do it with callbacks but would be nice to be able to do it with 'wait.for' )
I know that the sql connection query is non-standard so I'm not sure how to convert it.
var getUserQuery = "SELECT * FROM x WHERE id= '5'";
connection.query(getUserQuery, function(err, rows, fields){
....
});
How would I go about waiting to get the rows of this before my code proceeds on ?
link to module 'wait.for'
The part I am not understanding is at the bottom of that page - (Notes on usage on non-standard callbacks. e.g.: connection.query from mysql ).
https://github.com/luciotato/waitfor#notes-on-usage-on-non-standard-callbacks-eg-connectionquery-from-mysql
Notes on usage on non-standard callbacks. e.g.: connection.query from mysql
wait.for expects standardized callbacks. A standardized callback always returns (err,data) in that order.
A solution for the sql.query method and other non-standard callbacks is to create a wrapper function standardizing the callback, e.g.:
connection.prototype.q = function(sql, params, stdCallback){
this.query(sql,params, function(err,rows,columns){
return stdCallback(err,{rows:rows,columns:columns});
});
}
usage:
try {
var getUserQuery = "SELECT * FROM x WHERE id= ?";
var result = wait.forMethod(connection, "q", getUserQuery, ['5']);
console.log(result.rows);
console.log(result.columns);
}
catch(err) {
console.log(err);
}
Related
I am trying to get the value of variable w_name outside query even I have defined w_name before query but cannot getting it after query. Please check the code and help:
connection.query('SELECT * FROM workstations WHERE id=?',[results[0].w_id],function(err,res,field){
w_name = res[0].name;
});
console.log(w_name);
console.log showing undefined but if I am putting console.log inside the query after w_name it is showing proper result. What's wrong in it?
connection.query(...params) is asynchronous function, which requires some time to read and get resulting data for query passed to it.
Defining console.log(w_name) prints undefined because result is not yet assigned to w_name variable. Whereas defining it inside callback function assigns value to w_name.
Check this article for understanding asynchronous javascript.
connection.query('SELECT * FROM workstations WHERE id=?',
[results[0].w_id],function(err,res,field){
w_name = res[0].name;
console.log(w_name);
});
NodeJs is asynchronous, It uses callback for the results.
The best solution as far as i know about your problem is given below:
async function myQuery(){
return new Promise((resolve,reject)=>{
connection.query('SELECT * FROM workstations WHERE id=?',[results[0].w_id],function(err,res,field){
resolve(res[0].name)
});
})
}
Now you can use then to get the value or use async await to get the value like this.
//this is where you would like to call that function.
async function index(){
let w_name = await myQuery()
console.log(w_name)
}
here you can use Promise() and resolve() the query of the result.
const w_name = await new Promise(function(resolve, reject) {
connection.query('SELECT * FROM workstations WHERE id=?', [results[0].w_id], function(err, res, field) {
if (err) throw err
resolve(res[0].name);
})
});
console.log(w_name);
NodeJS 6.9.3
What I previously had went like this:
An outer function called "get_user()":
return database_queries.get_user(user_name)
.then(function(results_from_database) {
and that function then ran a database call, using Knex, and returned:
var dbquery = Multiline.stripIndent(function () {/*
SELECT
u.id as profile_id,
'user' as type_of_profile
FROM
user_profile u
WHERE name REGEXP "[[:<:]]||user_name||[[:>:]]"
*/});
dbquery = dbquery.replaceAll('||user_name||', user_name);
return DB.knex.raw(dbquery).then(function(result1) {
for(var index_of_results = 0; index_of_results < result1[0].length; index_of_results++) {
var document1 = result1[0][index_of_results];
array_of_maps_with_profile_type_and_profile_id[document1["type_of_profile"]].push(document1["profile_id"]);
}
When I did this, the database query ran, and got data, but this happened asynchronously, without the results ever being returned to the outer function. In other words, the outer function had completed long before the database queries had run.
So I tried to wrap the inner function in a Promise:
function get_user(user_name) {
return new Promise(function(resolve, reject) {
resolve ()
.then(function() {
var dbquery = Multiline.stripIndent(function () {/*
SELECT
u.id as profile_id,
'user' as type_of_profile
FROM
user_profile u
WHERE name REGEXP "[[:<:]]||user_name||[[:>:]]"
*/});
dbquery = dbquery.replaceAll('||user_name||', user_name);
return DB.knex.raw(dbquery).then(function(result1) {
for(var index_of_results = 0; index_of_results < result1[0].length; index_of_results++) {
var document1 = result1[0][index_of_results];
array_of_maps_with_profile_type_and_profile_id[document1["type_of_profile"]].push(document1["profile_id"]);
}
Now the database calls don't seem to ever be called. When they run, they appear in the logs, but now there are no database queries appearing in the logs. It would seem this inner function now returns a Promise, but the "resolve()" part of the Promise is never called.
What am I doing wrong here?
Here is a bit simpler way to write essentially the same query:
function get_user(user_name) {
const regex = `[[:<:]]${user_name}[[:>:]]`;
return DB.knex('user_profile')
.where(DB.knex.raw(`?? REGEXP ?`, ['name', regex]))
.then(res => {
// do what ever you like with returned rows here
});
}
You might want to simplify that a bit.
Take a look at these example projects.
https://github.com/joicenunes/helloapp
https://github.com/joicenunes/exercicio02
https://github.com/joicenunes/exercicio03
https://github.com/joicenunes/exercicio-04
(there are more, but you can find the rest)
Also avoid "replaceAll" and use binding variables.
Finally, since you are using node 6.x, you can use a few es6 goodies (arrow functions, multi-line strings, classes, etc), make the language work for you.
I'm running nodejs function in Amazon Lambda. It is supposed to do an insert to mysql DB after a HTTP get. Everything seems to be fine -- looking at the cloudwatch logs the query is parsed correctly and if I copy paste the query to mysql console it does exactly what it is supposed to.
Essentially:
var mysql = require('mysql')
var connection = createConnection({ connection details });
connection.connect();
var query = connection.query('Insert into AAA select * \
from BBB where BBB.a = ?;', [parameter],
function(err, result) {}
);
connection.end();
The problems is that the Lambda version simply does nothing. Query is visible and correct and the function returns cleanly but it never actually inserts anything. I have the same problem with update query as well but all the mysql selects work and return stuff so the problem is not that. The insert also works when I run it on my machine -- when I push it to lambda the problem appears.
I tried to add a separate commit statement but couldn't get it working either. I'm clearly missing something but can't figure out what. Do I need to have a transaction block for updates?
EDIT: Per Mark B's request. I think I tried to be smarter than I am by showing only part of the code. The whole logic was:
exports.handler = function(event, context, callback){
if ( event.A == -1 ){
exports.updateDB(event, function(res) {
context.succeed(res)
}
}
};
exports.updateDB = function(event, callback) {
var mysql = require('mysql')
var connection = createConnection({ connection details });
connection.connect();
var query = connection.query( 'update products set A=? where product_id = ?;',
[parameters],
function(err,result){ });
var query = connection.query( 'insert into other_table select * from products where product_id = ?;',
[parameters],
function(err,result){ });
connection.commit(function(err) {
if(err) {
connection.rollback(function() {
throw(err);
});
}
connection.end();
});
callback({"ok":"ok"})
};
Per advice given here I made the following changes. I took the last callback away, and did put callbacks inside both connection.queries:
var query = connection.query( 'insert into other_table select * from products where product_id = ?;',
[parameters],
function(err,result){
callback({"ok":"ok"})
});
And it seems to work. I'm guessing now that the commit -part does nothing but it doesn't seem to break it either. It probably is obvious at this point that I'm not much of a developer and even less so familiar with node.js so I truly appreciate the help I got!
Please note that the query function is an asynchronous function, meaning that it will be no result available until the callback function is triggered. In your sample code, the connection is closed immediately after it was triggered, long before the callback is executed. Try changing the code so that connection is closed by in the callback function, e.g.
var query = connection.query('Insert into AAA select * \
from BBB where BBB.a = ?;', [parameter],
function(err, result) {
// now it is ok to close the connection
connection.end();
if (err) {
// error handling
}
else {
// do something with the result
}
}
);
By the way, since you are working with Lambda, the same thing applies to the callback(), context.succeed() and context.fail() function handlers. In other words, it is likely that you would like to call them where I wrote the comments about error and result handling above.
I have two problems implementing a RESTful service using Node.js / node-postgres lib / PostgreDB and both are due to the async nature of JS.
A) I need to pass an extra argument to a callback in client.query(query, callback) call
I am inside a callback of a query and going through an array of recently fetched rows from a DB and want to launch a subsequent query for each of them:
var query = client.query('SELECT * FROM event', queryAllEventsHandler);
function queryAllEventsHandler(err, result){
allEvents = result.rows;
/* allEvents is an JSON array with the following format
[ {"id_event":1, "name":"name of the event"},
{"id_event":1, "name":"name of the event"}
]
*/
for(var i = 0; i<allEvents.length; i++){
client.query('SELECT * FROM days where id_event = $1',[allEvents[i].id_event], function( err, result){
//I want to have a reference to variable i
}
}
In the above example I want to do something like:
client.query('SELECT * FROM days where id_event = $1',[allEvents[i].id_event], function( AN_EXTRA_ARG, err, result)
Where the AN_EXTRA_ARG is an extra argument or a closure in the callback function... How can I achieve this? Should I create an closure with the of i and pass it as a callback's arg? How ? :|
B) "Synchronizing" queries
I need to launch various queries and create a custom JSON from all of them. Since every query and it's callback are asynchronous (waiting for no one) I was looking for a way to "tame" it and among other stuff I found a solution that occured to me in the first place, but seemed a bit "bad/lousy":
Keeping the query count is really the way to go as #jslatts suggests in Synchronous database queries with Node.js?
Hope I
With regards to question A, you could create a function to handle both your queries and only return when the last query is executed and return both results to the callback.
for(var i = 0; i<allEvents.length; i++){
query(client, allEvents[i], function(result1, result2) {
//do something
});
}
function query(client, event, callback) {
client.query('SELECT * FROM days where id_event = $1',[event.id_event], function( err1, result1){
client.query('SELECT * FROM days where id_event = $1',[event.id_event], function( err2, result2){
callback(result1, result2);
});
});
}
I don't like answering my on question, but this might be of interest to someone.... Regarding the A part of my question. You can assign a custom object to this in your function.
As you know a keyword this corresponds to the Window (top) object when inside a function (unless it's a method function). Using the bind function you can change the reference of this to your own object...
So what I did was, I created a named function queryCallback
function queryCallback(err, result){
//this == Window (default)
}
changed the anonymous callback function to the named one queryCallback:
client.query('SELECT * ... where id_event = $1',[allEvents[i].id_event], queryCallback.bind( {"position":i}, err, result));
Now, note queryCallback.bind( {"position":i}, err, result));
What bind(my_custom_this, [other args]) does is it binds a custom object (in my case {"position":i}) to this inside the function upon which the bind was called...
Now we have this scenario:
function queryCallback(err, result){
//this == {"position":i}
}
Bind explained: http://fitzgeraldnick.com/weblog/26/
A) I personally like lodash (or underscore if you prefer) partial() for this. It takes a function and a number of arguments and returns a function with the provided arguments applied and the remaining arguments still open. It's very much like the functional concept of currying.
B) For combining multiple asynchronous results I highly recommend async. The syntax will take a little getting used to, but makes thing like this very easy. Quick sample:
async.parallel([
one: function(callback){
db.fetch(options, callback);
},
two: function(callback){
db.fetch(options, callback);
}
],
function(err, results){
// this callback will get called when either parallel call gives an error
// or when both have called the callback
if (err) {
// handle error
return;
}
// get the results from results.one and results.two
});
== Added in edit ==
Actually lodash also provides a nicer (imho) albeit slightly more expensive (due to function calls) solution for your problem A):
_(allEvents).each(function(event, index, array) {
client.query('SELECT * FROM days where id_event = $1',[event.id_event], function( err, result) {
// Just use 'index' here, which doesn't change during the each
}
});
For B), your options include async or via a Promise library (such as Q, when.js, Bluebird, etc...)
I [new to node.js and programming in general] have two mysql query results (member info and list of workshops that members can attend) and need to send them to res.render() to be presented in .jade template (Member edit page).
To do this I'm using node-mysql and mysql-queue modules. Problem is I don't know how to pass callback function to render the response before queue.execute() finishes so I made workaround and put first two queries in the queue (mysql-queue feature), executed the queue, and afterwards added third "dummy query" which has callback function that renders the template.
My question is can I use this workaround and what would be the proper way to this using this modules?
exports.memberEdit = function (req, res) {
var q = connection.createQueue();
var membersResults,
htmlDateSigned,
htmlBirthDate,
servicesResults;
q.query("SELECT * FROM members WHERE id= ?;", req.id, function (err, results) {
console.log("Članovi: " + results[0]);
membersResults = results[0];
htmlDateSigned = dater.convertDate(results[0].dateSigned);
htmlBirthDate = dater.convertDate(results[0].birthDate);
});
q.query("SELECT * FROM services", function (err, results) {
console.log("Services: " + results);
servicesResults = results;
});
q.execute();
// dummy query that processes response after all queries and callback execute
// before execute() statement
q.query("SELECT 1", function (err,result) {
res.render('memberEdit', { title: 'Edit member',
query:membersResults,
dateSigned:htmlDateSigned,
birthDate:htmlBirthDate,
services:servicesResults });
})
};
I think an alternative could be to use a transaction to wrap your queries with:
var trans = connection.startTransaction();
trans.query(...);
trans.query(...);
trans.commit(function(err, info) {
// here, the queries are done
res.render(...);
});
commit() will call execute() and it provides a callback which will be called when all query callbacks are done.
This is still a bit of a workaround though, it would make more sense if execute() would provide the option of passing a callback (but it doesn't). Alternatively, you could use a module which provides a Promise implementation, but that's still a workaround.