I am new to nodejs. Using bluebird promises to get the response of an array of HTTP API calls, and storing derived results in an ElasticSearch.
Everything is working fine, except I am unable to access the variables within the 'then' function. Below is my code:
Promise.map(bucket_paths, function(path) {
this.path = path;
return getJson.getStoreJson(things,path.path);
}, {concurrency:1}).then(function(bucketStats){
bucketStats.map(function(bucketStat) {
var bucket_stats_json = {};
bucket_stats_json.timestamp = new Date();
bucket_stats_json.name = path.name ==> NOT WORKING
});
});
How can I access the path.name variable within the 'then' ? Error says 'path' is undefined.
The best way to do this is to package the data you need from one part of the promise chain into the resolved value that is sent onto the next part of the chain. In your case with Promise.map(), you're sending an array of data onto the .then() handler so the cleanest way to pass each path down to the next stage is to make it part of each array entry that Promise.map() is resolving. It appears you can just add it to the bucketStat data structure with an extra .then() as show below. When you get the data that corresponds to a path, you then add the path into that data structure so later on when you're walking through all the results, you have the .path property for each object.
You don't show any actual result here so I don't know what you're ultimately trying to end up with, but hopefully you can get the general idea from this.
Also, I switched to Promise.mapSeries() since that's a shortcut when you want concurrency set to 1.
Promise.mapSeries(bucket_paths, function(path) {
return getJson.getStoreJson(things,path.path).then(bucketStat => {
// add the path into this item's data so we can get to it later
bucketStat.path = path;
return bucketStat;
});
}).then(function(bucketStats){
return bucketStats.map(function(bucketStat) {
var bucket_stats_json = {};
bucket_stats_json.timestamp = new Date();
bucket_stats_json.name = bucketStat.path.name;
return bucket_status_json;
});
});
Related
I am trying to create intent in which when the user sends a parameter, then it should be checked in the database that is already there or not. If it already exists user gets a response A and if not it is added to the database, & the user gets response B. I am using Axios to make the API calls. But the code is not working.
I am getting a following errors in the console.
TypeError: Cannot set property 'response' of undefined
at axios.get.then.response (/srv/index.js:33:18)
at <anonymous>
at process._tickDomainCallback (internal/process/next_tick.js:229:7)"
Show matching entries
Hide matching entries
Add field to summary line
I am not a professional, I am doing all this coding by learning from online tutorials and youtube videos Please help.
function idHandler(agent) {
const idcard = agent.parameters.idcard;
const idvalue = ' ';
const idname= ' ';
axios.get('API' + idcard)
.then(response => {
this.response = response.data;
idvalue = this.response[0].IDcard;
idname = this.response[0].Name;
});
if (idcard === idvalue) {
const ans = 'Name of ' + idname + ' is already present in the List';
agent.add(ans);
} else {
const data = [{
idcard: idcard
}];
axios.post('API', data);
}
}
You have a few issues with both how to code for Dialogflow, and how to express some of the syntax correctly. Some of that is because it looks like you're mixing code that expects Promises with code that doesn't.
The axios library is meant to work with Promises - the get and post methods both return Promises. So you can either work with them by using an await method, or by using the .then() method that the returned Promise has. You may want to look up details of both.
Additionally, Dialogflow requires that you either return the Promise, or that your handler be an async function (which will return a Promise for you).
If you're using the .then() approach, however, then everything you do that relies on the call to axios must be done inside the .then() block and you must return that Promise. If you use await, then your function must be declared as an async function. (You can probably mix the two - but don't.)
So that part of your code might look something like:
return axios.get( url )
.then( response => {
// Do EVERYTHING in here
});
And inside the then() code block is where you would extract the values you want, call agent.add() with the message, and possibly make the API call to add data.
While this didn't get flagged as an error, you are trying to assign a value to a const after it is initially set. This is an error. Those should probably be declared using let instead.
Your error looks like it is saying that this is undefined. Which seems... odd. But also not really an issue since you probably don't need to be using this for most of what you're trying to do. If you scope the variables with let, then they should be available for the life of the function.
In this simplified example, associative array A cannot be populated in a Node.js Firestore query---it's as if there is a scoping issue:
var A = {};
A["name"] = "nissa";
firestore.collection("magic: the gathering")
.get()
.then(function(query) {
query.forEach(function(document) {
A[document.id] = document.id;
console.log(A);
});
})
.catch(function(error) {
});
console.log(A);
Console output:
{ name: 'nissa' } < last console.log()
{ name: 'nissa', formats: 'formats' } < first console.log() (in forEach loop)
{ name: 'nissa', formats: 'formats', releases: 'releases' } < second console.log() (in forEach loop)
Grateful for any assistance, please request for further detail if needed.
Data is loaded from Firestore asynchronously, and while that is happening, your main code continues to run.
It's easiest to see what that means by placing a few logging statements:
console.log("Starting to load data");
firestore.collection("magic: the gathering")
.get()
.then(function(query) {
console.log("Got data");
});
console.log("After starting to load data");
When you run this code, it prints:
Starting to load data
After starting to load data
Got data
This is probably not the order that you expected the logging to be in. But it is actually working as intended, and explains the output you see. By the time your last console.log(A); runs, the data hasn't been loaded yet, so A is empty.
The solution is simple, but typically takes some time to get used to: all code that needs the data from the database must be inside the callback, or be called from there.
So something like this:
var A = {};
A["name"] = "nissa";
firestore.collection("magic: the gathering")
.get()
.then(function(query) {
query.forEach(function(document) {
A[document.id] = document.id;
});
console.log(A);
})
Also see:
Array of JSON object is not empty but cannot iterate with foreach, show zero length
NodeJS, Firestore get field
Unable to add Google markers inside a loop, a more complex problem, calling multiple asynchronous API
scope issue in javascript between two Functions, which also shows using the more modern async and await keywords, instead of then()
How to get data from firestore DB in outside of onSnapshot, which uses an onSnapshot listener instead of get()
I've got a problem with redis and nodejs. I have to loop through a list of phone numbers, and check if this number is present in my redis database. Here is my code :
function getContactList(contacts, callback) {
var contactList = {};
for(var i = 0; i < contacts.length; i++) {
var phoneNumber = contacts[i];
if(utils.isValidNumber(phoneNumber)) {
db.client().get(phoneNumber).then(function(reply) {
console.log("before");
contactList[phoneNumber] = reply;
});
}
}
console.log("after");
callback(contactList);
};
The "after" console log appears before the "before" console log, and the callback always return an empty contactList. This is because requests to redis are asynchronous if I understood well. But the thing is I don't know how to make it works.
How can I do ?
You have two main issues.
Your phoneNumber variable will not be what you want it to be. That can be fixed by changing to a .forEach() or .map() iteration of your array because that will create a local function scope for the current variable.
You have create a way to know when all the async operations are done. There are lots of duplicate questions/answers that show how to do that. You probably want to use Promise.all().
I'd suggest this solution that leverages the promises you already have:
function getContactList(contacts) {
var contactList = {};
return Promise.all(contacts.filter(utils.isValidNumber).map(function(phoneNumber) {
return db.client().get(phoneNumber).then(function(reply) {
// build custom object
constactList[phoneNumber] = reply;
});
})).then(function() {
// make contactList be the resolve value
return contactList;
});
}
getContactList.then(function(contactList) {
// use the contactList here
}, funtion(err) {
// process errors here
});
Here's how this works:
Call contacts.filter(utils.isValidNumber) to filter the array to only valid numbers.
Call .map() to iterate through that filtered array
return db.client().get(phoneNumber) from the .map() callback to create an array of promises.
After getting the data for the phone number, add that data to your custom contactList object (this is essentially a side effect of the .map() loop.
Use Promise.all() on the returned array of promises to know when they are all done.
Make the contactList object we built up be the resolve value of the returned promise.
Then, to call it just use the returned promise with .then() to get the final result. No need to add a callback argument when you already have a promise that you can just return.
The simplest solution may be to use MGET with a list of phone numbers and put the callback in the 'then' section.
You could also put the promises in an array and use Promise.all().
At some point you might want your function to return a promise rather than with callback, just to stay consistent.
Consider refactoring your NodeJS code to use Promises.
Bluebird is an excellent choice: http://bluebirdjs.com/docs/working-with-callbacks.html
you put async code into a for loop (sync operations). So, each iteration of the for loop is not waiting for the db.client(...) function to end.
Take a look at this stackoverflow answer, it explains how to make async loops :
Here
I need to use bluebird in my code and I have no idea how to use it. My code contains nested loops. When the user logs in, my code will run. It will begin to look for any files under the user, and if there are files then, it will loop through to get the name of the files, since the name is stored in a dictionary. Once it got the name, it will store the name in an array. Once all the names are stored, it will be passed along in res.render().
Here is my code:
router.post('/login', function(req, res){
var username = req.body.username;
var password = req.body.password;
Parse.User.logIn(username, password, {
success: function(user){
var Files = Parse.Object.extend("File");
var object = [];
var query = new Parse.Query(Files);
query.equalTo("user", Parse.User.current());
var temp;
query.find({
success:function(results){
for(var i=0; i< results.length; i++){
var file = results[i].toJSON();
for(var k in file){
if (k ==="javaFile"){
for(var t in file[k]){
if (t === "name"){
temp = file[k][t];
var getname = temp.split("-").pop();
object[i] = getname;
}
}
}
}
}
}
});
console.log(object);
res.render('filename', {title: 'File Name', FIles: object});
console.log(object);
},
error: function(user, error) {
console.log("Invalid username/password");
res.render('logins');
}
})
});
EDIT:The code doesn't work, because on the first and second console.log(object), I get an empty array. I am suppose to get one item in that array, because I have one file saved
JavaScript code is all parsed from top to bottom, but it doesn't necessarily execute in that order with asynchronous code. The problem is that you have the log statements inside of the success callback of your login function, but it's NOT inside of the query's success callback.
You have a few options:
Move the console.log statements inside of the inner success callback so that while they may be parsed at load time, they do not execute until both callbacks have been invoked.
Promisify functions that traditionally rely on and invoke callback functions, and hang then handlers off of the returned value to chain the promises together.
The first option is not using promises at all, but relying solely on callbacks. To flatten your code you will want to promisify the functions and then chain them.
I'm not familiar with the syntax you're using there with the success and error callbacks, nor am I familiar with Parse. Typically you would do something like:
query.find(someArgsHere, function(success, err) {
});
But then you would have to nest another callback inside of that, and another callback inside of that. To "flatten" the pyramid, we make the function return a promise instead, and then we can chain the promises. Assuming that Parse.User.logIn is a callback-style function (as is Parse.Query.find), you might do something like:
var Promise = require('bluebird');
var login = Promise.promisify(Parse.User.logIn);
var find = Promise.promisify(Parse.Query.find);
var outerOutput = [];
return login(yourArgsHere)
.then(function(user) {
return find(user.someValue);
})
.then(function(results) {
var innerOutput = [];
// do something with innerOutput or outerOutput and render it
});
This should look familiar to synchronous code that you might be used to, except instead of saving the returned value into a variable and then passing that variable to your next function call, you use "then" handlers to chain the promises together. You could either create the entire output variable inside of the second then handler, or you can declare the variable output prior to even starting this promise chain, and then it will be in scope for all of those functions. I have shown you both options above, but obviously you don't need to define both of those variables and assign them values. Just pick the option that suits your needs.
You can also use Bluebird's promisifyAll() function to wrap an entire library with equivalent promise-returning functions. They will all have the same name of the functions in the library suffixed with Async. So assuming the Parse library contains callback-style functions named someFunctionName() and someOtherFunc() you could do this:
var Parse = Promise.promisifyAll(require("Parse"));
var promiseyFunction = function() {
return Parse.someFunctionNameAsync()
.then(function(result) {
return Parse.someOtherFuncAsync(result.someProperty);
})
.then(function(otherFuncResult) {
var something;
// do stuff to assign a value to something
return something;
});
}
I have a few pointers. ... Btw tho, are you trying to use Parse's Promises?
You can get rid of those inner nested loops and a few other changes:
Use some syntax like this to be more elegant:
/// You could use a map function like this to get the files into an array of just thier names
var fileNames = matchedFiles.map(function _getJavaFile(item) {
return item && item.javaFile && item.javaFile.name // NOT NULL
&& item.javaFile.name.split('-')[0]; // RETURN first part of name
});
// Example to filter/retrieve only valid file objs (with dashes in name)
var matchedFiles = results.filter(function _hasJavaFile(item) {
return item && item.javaFile && item.javaFile.name // NOT NULL
&& item.javaFile.name.indexOf('-') > -1; // and has a dash
});
And here is an example on using Parse's native promises (add code above to line 4/5 below, note the 'then()' function, that's effectively now your 'callback' handler):
var GameScore = Parse.Object.extend("GameScore");
var query = new Parse.Query(GameScore);
query.select("score", "playerName");
query.find().then(function(results) {
// each of results will only have the selected fields available.
});
I'm trying to write a scraper using 'request' and 'cheerio'. I have an array of 100 urls. I'm looping over the array and using 'request' on each url and then doing cheerio.load(body). If I increase i above 3 (i.e. change it to i < 3 for testing) the scraper breaks because var productNumber is undefined and I can't call split on undefined variable. I think that the for loop is moving on before the webpage responds and has time to load the body with cheerio, and this question: nodeJS - Using a callback function with Cheerio would seem to agree.
My problem is that I don't understand how I can make sure the webpage has 'loaded' or been parsed in each iteration of the loop so that I don't get any undefined variables. According to the other answer I don't need a callback, but then how do I do it?
for (var i = 0; i < productLinks.length; i++) {
productUrl = productLinks[i];
request(productUrl, function(err, resp, body) {
if (err)
throw err;
$ = cheerio.load(body);
var imageUrl = $("#bigImage").attr('src'),
productNumber = $("#product").attr('class').split(/\s+/)[3].split("_")[1]
console.log(productNumber);
});
};
Example of output:
1461536
1499543
TypeError: Cannot call method 'split' of undefined
Since you're not creating a new $ variable for each iteration, it's being overwritten when a request is completed. This can lead to undefined behaviour, where one iteration of the loop is using $ just as it's being overwritten by another iteration.
So try creating a new variable:
var $ = cheerio.load(body);
^^^ this is the important part
Also, you are correct in assuming that the loop continues before the request is completed (in your situation, it isn't cheerio.load that is asynchronous, but request is). That's how asynchronous I/O works.
To coordinate asynchronous operations you can use, for instance, the async module; in this case, async.eachSeries might be useful.
You are scraping some external site(s). You can't be sure the HTML all fits exactly the same structure, so you need to be defensive on how you traverse it.
var product = $('#product');
if (!product) return console.log('Cannot find a product element');
var productClass = product.attr('class');
if (!productClass) return console.log('Product element does not have a class defined');
var productNumber = productClass.split(/\s+/)[3].split("_")[1];
console.log(productNumber);
This'll help you debug where things are going wrong, and perhaps indicate that you can't scrape your dataset as easily as you'd hoped.