I'm having alot of trouble getting couchdb's response handlers to do anything useful with node.js and now.js. My aim is to call back to a client or group with certain information that's been found and also sent from a client.
everyone.now.login = function(){
var username = this.now.lusername;
var password = this.now.lpassword;
var test;
db.get(this.now.lusername, function (err, doc, test, username) {
if (err) {
console.log(+ username + " doesn't exist!");
} else {
console.log('Found user!');
console.log(doc);
test = 1;
}
});
console.log(test);
}
I'm using test as an example here. I declare it outside db.get, assign a value inside to feed back but when console.log is run outside it doesn't have a value.
Am I simply not getting it or is something wrong?
(I do release I can use doc.(whatever value) but there is a particular variable i wanted to pass on back to the client outside of this call)
The test variable will not have a value in the console.log because the CouchDB response has not arrived yet. Your login function executes the following steps.
Set some variables (username, password, and test is undefined)
Begin a CouchDB fetch (for this.now.lusername) with a function assigned to run when it is complete
Run console.log(test) which is still undefined
The CouchDB response completes, and your function from step 2 runs
What I do for things like this is use function calls.
everyone.now.login = function(){
var username = this.now.lusername;
var password = this.now.lpassword;
db.get(this.now.lusername, function (err, doc, test, username) {
if (err) {
console.log(+ username + " doesn't exist!");
} else {
console.log('Found user!');
console.log(doc);
login_complete(doc);
}
});
function login_complete(doc) {
console.log('The login finished! Doc is ' + doc)
}
}
This way the code still looks correct (the "story" of the code mostly flows from top to bottom), but it also executes correctly (the login_complete() function runs only after the login is actually complete).
Related
I am new to node.js.
I am trying to create function, where a randomly generated String is queried to check if it exists or not. If it already exists, the String is randomly generated till it is unique.
let validID = false;
console.log(temp); //temp is the randomly generated String.
while(!validID){
Website.findOne({shortcut: temp},function(err,docs){
if(docs==null){
validID = true;
console.log("The shortcut for the url is" + temp);
}else{
console.log("FOUND");
temp = generateId();
}
});
}
When run, the code is stuck in an infinite while loop.
I tried to see whether the code works with a String value ( not a variable ) passed in as the query inside findOne(). It worked. I am assuming that the fact that temp is a variable is causing the problem. Can variables be passed in as a value in a query? If so, what is the correct way?
Website.findOne operates asynchronously, i.e. the callback-function you passed to it, will be run once the results from the mongodb are fetched. However, node will not be able to actually process this callback, since your callstack never gets emptied due to your while-loop. If you're interested, you can find out more about this here.
One way to solve this is to wrap your Mongo-DB call in a promise, wait for it to resolve, then return if the ID is unique and continue by calling it recursively otherwise (note that this can be highly simplified by using async/await but for understanding how this works using promised are beneficial imo):
function findIdPromise(temp) {
return new Promise((resolve, reject) => {
Website.findOne({
shortcut: temp
}, function (err, docs) {
if (err) {
return reject(err);
}
resolve(docs);
});
});
}
function getNextIsUniqueIdPromise(shortcut) {
return findIdPromise()
.then(docs => {
if (docs == null) {
return shortcut;
}
return getNextIsUniqueIdPromise(generateId());
});
}
// call it initially with
getNextIsUniqueIdPromise(firstShortcutToCheck)
.then(shortcut => {
console.log("The shortcut for the url is" + shortcut):
})
.catch(err => {
console.log("an error occured", err):
});
This is some of my code that I have in my index.js. Its waiting for the person to visit url.com/proxy and then it loads up my proxy page, which is really just a form which sends back an email and a code. From my MongoDB database, I grab the users order using the code, which contains some information I need (like product and the message they're trying to get). For some reason, it seems like its responding before it gets this information and then holds onto it for the next time the form is submitted.
The newline in my res.send(product + '\n' + message) isnt working either, but thats not a big deal right now.
But.. for example, the first time I fill out the form ill get a blank response. The second time, I'll get the response to whatever I filled in for the first form, and then the third time ill get the second response. I'm fairly new to Web Development, and feel like I'm doing something obviously wrong but can't seem to figure it out. Any help would be appreciated, thank you.
app.get('/proxy', function(req,res){
res.sendFile(__dirname+ "/views/proxy.html");
});
var message = "";
var product = "";
app.post('/getMessage', function(req,res)
{
returnMsg(req.body.user.code, req.body.user.email);
//res.setHeader('Content-Type', 'text/plain');
res.send(product + "\n" + message);
});
function returnMsg(code, email){
MongoClient.connect(url, function(err, db){
var cursor = db.collection('Orders').find( { "order_id" : Number(code) })
cursor.each(function(err, doc){
assert.equal(err, null);
if (doc!= null)
{
message = doc["message"];
product = doc["product"];
}
else {
console.log("wtf");
// error code here
}
});
console.log(email + " + " + message);
var document = {
"Email" : email,
"Message" : message
}
db.collection("Users").insertOne(document);
db.close();
});
}
You need to do lots of reading about your asynchronous programming works in node.js. There are significant design problems with this code:
You are using module level variables instead of request-level variables.
You are not correctly handling asynchronous responses.
All of this makes a server that simply does not work correctly. You've found one of the problems already. Your async response finishes AFTER you send your response so you end up sending the previously saved response not the current one. In addition, if multiple users are using your server, their responses will tromp on each other.
The core design principle here is first that you need to learn how to program with asynchronous operations. Any function that uses an asynchronous respons and wants to return that value back to the caller needs to accept a callback and deliver the async value via the callback or return a promise and return the value via a resolved promise. The caller then needs to use that callback or promise to fetch the async value when it is available and only send the response then.
In addition, all data associated with a request needs to stay "inside" the request handle or the request object - not in any module level or global variables. That keeps the request from one user from interfering with the requests from another user.
To understand how to return a value from a function with an asynchronous operation in it, see How do I return the response from an asynchronous call?.
What ends up happening in your code is this sequence of events:
Incoming request for /getMessage
You call returnMsg()
returnMsg initiates a connection to the database and then returns
Your request handler calls res.send() with whatever was previously in the message and product variables.
Then, sometime later, the database connect finishes and you call db.collection().find() and then iterate the cursor.
6/ Some time later, the cursor iteration has the first result which you put into your message and product variables (where those values sit until the next request comes in).
In working out how your code should actually work, there are some things about your logic that are unclear. You are assigning message and product inside of cursor.each(). Since cursor.each() is a loop that can run many iterations, which value of message and product do you actually want to use in the res.send()?
Assuming you want the last message and product value from your cursor.each() loop, you could do this:
app.post('/getMessage', function(req, res) {
returnMsg(req.body.user.code, req.body.user.email, function(err, message, product) {
if (err) {
// send some meaningful error response
res.status(500).end();
} else {
res.send(product + "\n" + message);
}
});
});
function returnMsg(code, email, callback) {
let callbackCalled = false;
MongoClient.connect(url, function(err, db) {
if (err) {
return callback(err);
}
var cursor = db.collection('Orders').find({
"order_id": Number(code)
});
var message = "";
var product = "";
cursor.each(function(err, doc) {
if (err) {
if (!callbackCalled) {
callback(err);
callbackCalled = true;
}
} else {
if (doc != null) {
message = doc["message"];
product = doc["product"];
} else {
console.log("wtf");
// error code here
}
}
});
if (message) {
console.log(email + " + " + message);
var document = {
"Email": email,
"Message": message
}
db.collection("Users").insertOne(document);
}
db.close();
if (!callbackCalled) {
callback(null, message, product);
}
});
}
Personally, I would use promises and use the promise interface in your database rather than callbacks.
This code is still just conceptual because it has other issues you need to deal with such as:
Proper error handling is still largely unfinished.
You aren't actually waiting for things like the insert.One() to finish before proceeding.
I met an annoyance when trying to use simple function to query data for my web app. The idea is to use one function to list the contents of one table; the other function to use the user-selected record_id in this table to query the detailed contents data in another table.
When running, the app ran the two functions without any error while no data got. Checked the console and found the second function query results is null (I console.log the input for the second function, found they got and using the correct query keys). Since I am sure database has the data for query.
I tried:
use psql command line to query data using the same query keys, I have the results without problem;
I run a node command line, and try to run the two functions by providing the query keys, it also gave me the correct results.
So the functions should work. Now my question is why put them in the app and let them run by themselves, they did not get the query results?
I am using pg = require("pg"); and const pool = new pg.Pool(config) for database connection;
Your sharing of your experience will be very appreciated.
(UPDATE)The functions are like below:
function listItemDB(callback){
pool.connect(function(err, client, done) {
if(err) {
return console.error('error fetching client from pool', err);
}
//use the client for executing the query
client.query(`SELECT * FROM basicdb.items`,
function(err, result) {
//call `done(err)` to release the client back to the pool (or destroy it if there is an error)
done(err);
if(err) {
return console.error('error running query', err);
}
// console.log(result.rows);
callback(result.rows);
});
});
}
The above function is only trying to get "item1" and "dataset1" for future use and pass them to below function args. It does its job perfectly.
function getFileName(itemName,datasetName, callback) {
let fileName;
console.log(itemName,datasetName);
pool.connect(function(err, client, done) {
if(err) {
return console.error('error fetching client from pool', err);
}
client.query("SELECT * "+
"FROM basicdb.dataset "+
"INNER JOIN basicdb.items "+
"ON basicdb.dataset.item_id = basicdb.items.item_id "+
"WHERE (basicdb.items.item_name = ($1)) "+
"AND (basicdb.dataset.datasetname = ($2))",[itemName,datasetName],
function (err, result){
done();
if(err) {
return console.error('error running query', err);
}
let records = result.rows;
fileName = records[records.length-1].filename;
callback(fileName);
});
});
}
This above function is trying to get the filename so the main app can use it. The code to call the above function in my main app.js is like below:
db.getFileName("item1","dataset1",function(fileName) {
//do something with the fileName....}
("db" is the module name which include the functions.)
I finally found the problem, which is a low-level mistake and has nothing to do with the database and the queries.
The item names got from the dropdown list in the app, which was feed to the function args, has one " "(space) attached to the end of the name(i dont know why?), which always "!=" the record in the database:-(, so always no query result in the app. But for the function test, I hardcode the item name which is correct "==" the record in the database. Since it is " ", even when I console.log(itemName), I did not find the space at the end.
It turns out to be 'A mistake of space'.
So I've been trying to set a global variable from inside a request, but seem to be getting nothing. The code I'm using
A username for testing is test2 after username=
var forSearching = "test2";
var name = "";
console.log(forSearching);
request("http://mercsystem.esy.es/get.php?username=" + forSearching, function(err, res, body)
{
if (err) return console.error(err);
var main = JSON.parse(body);
if (main.success == "false")
{
message.reply("Sorry, invalid user!")
}
else
{
name = main.Username
}
});
If you insert a console.log(name) right after you set the value, you will see that the value is set just fine.
The issue is likely one of timing. request() is an asynchronous operation. That means calling request() starts the asynchronous operation, then the rest of your code continues to run to completion and then, some time LATER, the callback gets called with the final asynchronous results.
Though you don't show where you are trying to use the name variable, you are probably checking the value of this global variable before the callback has been called and thus before the value has been set.
In node.js, what you are doing is not how you use asynchronous results. It will never work reliably (or at all). Instead, the only place to use your asynchronous result is in the callback itself or in some function you call from the callback and pass the result to.
var forSearching = "test2";
console.log("begin");
request("http://mercsystem.esy.es/get.php?username=" + forSearching, function (err, res, body) {
console.log("in request() callback");
if (err) return console.error(err);
var main = JSON.parse(body);
if (main.success == "false") {
message.reply("Sorry, invalid user!")
} else {
var name = main.Username
console.log(name); // value shows fine here
// use the name variable here or call some function and pass
// the name variable to it
}
});
console.log("after request() call");
// You cannot use the name value here (even if it was in a global) because
// the async callback has not yet been called
If you ran this code with the console.log() statement I've added, you would see this sequence of events:
begin
after request() call
in request() callback
From this sequence, you can see that code after your request() call runs BEFORE the async callback runs. Thus, you cannot use your name variable there, even if it is in a global.
router.post("/application_action", function(req,res){
var Employee = req.body.Employee;
var conn = new jsforce.Connection({
oauth2 : salesforce_credential.oauth2
});
var username = salesforce_credential.username;
var password = salesforce_credential.password;
conn.login(username, password, function(err, userInfo, next) {
if (err) { return console.error(err); res.json(false);}
// I want this conn.query to execute first and then conn.sobject
conn.query("SELECT id FROM SFDC_Employee__c WHERE Auth0_Id__c = '" + req.user.id + "'" , function(err, result) {
if (err) { return console.error(err); }
Employee["Id"] = result.records[0].Id;
});
//I want this to execute after the execution of above query i.e. conn.query
conn.sobject("SFDC_Emp__c").update(Employee, function(err, ret) {
if (err || !ret.success) { return console.error(err, ret);}
console.log('Updated Successfully : ' + ret.id);
});
});
I have provided my code above. I need to modify Employee in the conn.query and use it in conn.sobject. I need to make sure that my first query executes before 2nd because I am getting value from 1st and using in the 2nd. Please do let me know if you know how to accomplish this.
New Answer Based on Edit to Question
To execute one query based on the results of the other, you put the second query inside the completion callback of the first like this:
router.post("/application_action", function (req, res) {
var Employee = req.body.Employee;
var conn = new jsforce.Connection({
oauth2: salesforce_credential.oauth2
});
var username = salesforce_credential.username;
var password = salesforce_credential.password;
conn.login(username, password, function (err, userInfo, next) {
if (err) {
return console.error(err);
res.json(false);
}
// I want this conn.query to execute first and then conn.sobject
conn.query("SELECT id FROM SFDC_Employee__c WHERE Auth0_Id__c = '" + req.user.id + "'", function (err, result) {
if (err) {
return console.error(err);
}
Employee["Id"] = result.records[0].Id;
//I want this to execute after the execution of above query i.e. conn.query
conn.sobject("SFDC_Emp__c").update(Employee, function (err, ret) {
if (err || !ret.success) {
return console.error(err, ret);
}
console.log('Updated Successfully : ' + ret.id);
});
});
});
});
The only place that the first query results are valid is inside that callback because otherwise, you have no way of knowing when those asynchronous results are actually available and valid.
Please note that your error handling is unfinished since you don't finish the response in any of the error conditions and even in the success case, you have not yet actually sent a response to finish the request.
Original Answer
First off, your code shows a route handler, not middleware. So, if you really intend to ask about middleware, you will have to show your actual middleware. Middleware that does not end the request needs to declare next as an argument and then call it when it is done with it's processing. That's how processing continues after the middleware.
Secondly, your console.log() statements are all going to show undefined because they execute BEFORE the conn.query() callback that contains the code that sets those variables.
conn.query() is an asynchronous operation. It calls its callback sometime IN THE FUTURE. Meanwhile, your console.log() statements execute immediately.
You can see the results of the console.log() by putting the statements inside the conn.query() callback, but that is probably only part of your problem. If you explain what you're really trying to accomplish, then we could probably help with a complete solution. Right now, you're just asking questions about flawed code, but not explaining the higher level problem you're trying to solve so you're making it hard for us to give you the best answer to your actual problem.
FYI:
app.locals - properties scoped to your app, available to all request handlers.
res.locals - properties scoped to a specific request, available only to middleware or request handlers involved in processing this specific request/response.
req.locals - I can't find any documentation on this in Express or HTTP module. There is discussion of this as basically serving the same purpose as res.locals, though it is not documented.
Other relevants answers:
req.locals vs. res.locals vs. res.data vs. req.data vs. app.locals in Express middleware
Express.js: app.locals vs req.locals vs req.session
You miss the basics of the asynchronous flow in javascript. All the callbacks are set to the end of event loop, so the callback of the conn.query will be executed after console.logs from the outside. Here is a good article where the the basic concepts of asynchronous programming in JavaScript are explained.