I m actually developping a little application with mongodb and nodejs to create my REST Api.
I face a problem when I need to access an object reference :
I have a roadmap collection which reference a user object
When I want to get all the roadmaps, I have to loop on my roadmap array to lazy load my users, by the ref ID stored in the roadmap collection
I have a callback problem, the user is not loaded when I need it
I have found a solution using Wait.for library : https://github.com/luciotato/waitfor , but I dont know how it works. I tried everything, but no way to make it work
all: (req,res)->
#em.collection(#collection).find().toArray((err, result)=>
roadmaps = []
for r in result
r.user = #getUser(r.user.oid)
roadmaps.push r
res.send(roadmaps))
getUser: (oid)->
#em.collection('user').findOne {_id: new #objectId(oid)}, (err, res)=>
if !err
return res
return undefined
Does anybody have an idea of how to make it works properly ? Where should I put the wait.lauchFiber ? where should I put the wait.for ?
Thanks for all
I'm not familiar with CoffeeScript, please correct me and I'll edit this answer.
all: (req,res)->
var result = wait.forMethod(#em.collection(#collection).find(), "toArray")
roadmaps = []
for r in result
r.user = #getUser(r.user.oid)
roadmaps.push r
res.send(roadmaps)
getUser: (oid)->
try
return wait.forMethod(#em.collection('user'),"findOne",{_id:new #objectId(oid)})
catch(err)
return undefined
As you can see, for "getUser", if the method is that simple, you better use your version, with the callback.
"where to put the launchFiber()?"
you put the launchFiber when a request arrives. see https://github.com/luciotato/waitfor#proper-use
Related
My question is if is it possible to achieve the following implementation in NodeJS without using any callback or asnyc/await or promises or synchrounous libraries in the caller file?
Caller File:
const isUnique = require("is-unique");
if(!isUnique({email:req.user.email})){
//call is-unique to check if unique, if not unique return 400
return res.status(400).json({"error":"Profile exist"})
}
Callee File:
// is-unique.js
const isUnique = (field) => {
//call an async function countRecordsInDatabase, check the result and return a boolean
const count = countRecordsInDatabaseAsync(field);
return (count <= 0);
}
module.exports = isUnique;
Thanks in advance and hope I have made the question short and sweet. :D
You want to call an async function, in a sync function and not have the sync function be async. It seems like that is what you are asking.
There are libraries that suggest they can do this, however it doesn't sound like a great idea.
What are you trying to achieve?
I'll keep it real with you. If most people aren't doing things this way, which they aren't it's prolly not a good idea to do it differently.
I'm trying to get my head around callbacks in Node.JS and it's pretty foreign compared to what I'm used to.
I have the following example from another post on here:
var someOutput;
var myCallback = function(data) {
console.log('got data: '+data);
someOutput = data;
};
var usingItNow = function(callback) {
callback('get it?');
};
//now do something with someOutput outside of the functions
I have added the someOutput variable - I want to get the text 'get it?' (obviously I don't) into there so that I can concatenate it on to a string, but it's not working in any combination I've tried.
I must be missing something fundamental here!
Thank you for the help.
You should call the function:
usingItNow(myCallback);
But anyway it is not a good practice to use callback to set variable and consume later. It will work for you now, but later if callback will be async, it will fail.
Consider using Promise and use someOutput in the then function.
A good Promise package is Bluebird
If you want to do it like a pro, consider using Promise.coroutine
http://bluebirdjs.com/docs/api/promise.coroutine.html
Or async + await if you are using an updated version of Node.js
Look on http://node.green/#ES2017-features-async-functions to check what avilable on your node version
As commented by emil:
"call usingItNow(myCallback);"
doh!
I'm new to promises and Q, and I'm trying to convert a route that uses query in node-mysql. Here are excerpts from my code:
var Q = require('q');
// connection defined elsewhere
router.get('/', function(req, res) {
var query = Q.denodeify(connection.query);
var promise = query("-- query ommitted --", [req.user.id]);
promise.then(console.log, console.error);
});
I'm trying to convert this from an existing set up that doesn't use promises, so I know the connection is set up properly and the query is valid. Whenever I try to request this route, I get the same message in stderr:
[TypeError: Cannot read property 'connectionConfig' of undefined]
I don't know where it's coming from so I'm not sure how to find the full stacktrace. I also tried an alternate version of this code where I had my own function instead of console.log and console.error, but this function was never called and the same error appeared.
Updated answer:
You're probably losing the lexical scope to connection when you denodeify the query method.
Looking at the Q documentation it says this can be an issue:
If you are working with methods, instead of simple functions, you can easily run in to the usual problems where passing a method to another function—like Q.nfcall—"un-binds" the method from its owner.
To fix this try using Q.nbind instead:
var query = Q.nbind(connection.query, connection);
var promise = query("-- query ommitted --", [req.user.id]);
promise.then(console.log, console.error);
Original answer:
Looking at the node-mysql source, the only place connectionConfig is accessed is in Pool.js. So my guess would be that you've got an issue with how the pool is being configured.
I think this is a very simple question? I am a beginner trying to learn mongo with node.
Once I have saved something to a collection, how can I pull it out in simple var format?
db.highschools.save({
hsid :10,
name :"Johnson High School",
location:"San Diego, CA"
});
I simply want to store a var as 'Johnson High School'.
My failed attempts that have returned undefined are as follows...
var hsName = db.highschools.find({hsid:10}).name;
var hsName = db.highschools.find({hsid:10}).name.str;
Pretty sure I'm missing the big picture here, would someone please be kind enough to help me figure this out?
Use findOne instead:
var hsName = db.highschools.findOne({hsid:10}).name;
Also, note that this is a Mongo script, not a NodeJS script.
You'll need to make it async when you write the logic in NodeJS.
db.collection('students', function(err, collection) {
collection.findOne({hsid:10}, function(err, student) {
if (err) { throw err; }
console.log(student.name);
});
});
If you're confident that there should be only one result, then you can use the shortcut method findOne which simply calls find internally with a limit of one. If you were to use find, it returns an array of matches.
I'm trying to write a scraper using 'request' and 'cheerio'. I have an array of 100 urls. I'm looping over the array and using 'request' on each url and then doing cheerio.load(body). If I increase i above 3 (i.e. change it to i < 3 for testing) the scraper breaks because var productNumber is undefined and I can't call split on undefined variable. I think that the for loop is moving on before the webpage responds and has time to load the body with cheerio, and this question: nodeJS - Using a callback function with Cheerio would seem to agree.
My problem is that I don't understand how I can make sure the webpage has 'loaded' or been parsed in each iteration of the loop so that I don't get any undefined variables. According to the other answer I don't need a callback, but then how do I do it?
for (var i = 0; i < productLinks.length; i++) {
productUrl = productLinks[i];
request(productUrl, function(err, resp, body) {
if (err)
throw err;
$ = cheerio.load(body);
var imageUrl = $("#bigImage").attr('src'),
productNumber = $("#product").attr('class').split(/\s+/)[3].split("_")[1]
console.log(productNumber);
});
};
Example of output:
1461536
1499543
TypeError: Cannot call method 'split' of undefined
Since you're not creating a new $ variable for each iteration, it's being overwritten when a request is completed. This can lead to undefined behaviour, where one iteration of the loop is using $ just as it's being overwritten by another iteration.
So try creating a new variable:
var $ = cheerio.load(body);
^^^ this is the important part
Also, you are correct in assuming that the loop continues before the request is completed (in your situation, it isn't cheerio.load that is asynchronous, but request is). That's how asynchronous I/O works.
To coordinate asynchronous operations you can use, for instance, the async module; in this case, async.eachSeries might be useful.
You are scraping some external site(s). You can't be sure the HTML all fits exactly the same structure, so you need to be defensive on how you traverse it.
var product = $('#product');
if (!product) return console.log('Cannot find a product element');
var productClass = product.attr('class');
if (!productClass) return console.log('Product element does not have a class defined');
var productNumber = productClass.split(/\s+/)[3].split("_")[1];
console.log(productNumber);
This'll help you debug where things are going wrong, and perhaps indicate that you can't scrape your dataset as easily as you'd hoped.