In my javascript class, i have a function that return a list of things like so,
//ProjectClass.js
getProjectList: () =>
#Project.find (err, projects) =>
console.log(projects)
return projects
However, whenever I try to send a server response from nodejs
//App.js
project = new projectSchema.Project()
res.send(project.getProjectList())
I get the following as a response
{
"options": {
"populate": {}
},
"_conditions": {},
"_updateArg": {},
"op": "find"
}
Ironically, if I pass the res object to my getProjectList and send server response from my getProjectList function then everything works just fine.
Thanks for the help!
#Project.find is an asynchronous function, calling your anonymous callback once the find results are available. getProjectList returns the result of the last synchronous statement executed in that function which is the return value of #Project.find. That's not the projects array your callback returns but the query object you see in your response.
Related
My goal is to create an abstracted POST function for an express running on Node similar to Django's inbuilt REST methods. As a part of my abstracted POST function I'm checking the database (mongo) for valid foreign keys, duplicates, etc..., which is where we are in the process here. The function below is the one that actually makes the first calls to mongo to check that the incoming foreign keys actually exist in the tables/collections that they should exist in.
In short, the inbuilt response functionality inside the native .map function seems to be causing an early return from the called/subsidiary functions, and yet still continuing on inside the called/subsidiary functions after the early return happens.
Here is the code:
const db = require(`../../models`)
const findInDB_by_id = async params => {
console.log(`querying db`)
const records = await Promise.all(Object.keys(params).map(function(table){
return db[table].find({
_id: {
$in: params[table]._ids
}
})
}))
console.log('done querying the db')
// do stuff with records
return records
}
// call await findIndDB_by_id and do other stuff
// eventually return the response object
And here are the server logs
querying db
POST <route> <status code> 49.810 ms - 9 //<- and this is the appropriate response
done querying the db
... other stuff
When I the function is modified so that the map function doesn't return anything, (a) it doesn't query the database, and (b) doesn't return the express response object early. So by modifying this:
const records = await Promise.all(Object.keys(params).map(function(table){
return db[table].find({ // going to delete the `return` command here
_id: {
$in: params[table]._ids
}
})
}))
to this
const records = await Promise.all(Object.keys(params).map(function(table){
db[table].find({ // not returning this out of map
_id: {
$in: params[table]._ids
}
})
}))
the server logs change to:
querying db
done querying the db
... other stuff
POST <route> <status code> 49.810 ms - 9 // <-appropriate reponse
But then I'm not actually building my query, so the query response is empty. I'm experiencing this behavior with the anonymous map function being an arrow function of this format .map(table => (...)) as well.
Any ideas what's going on, why, or suggestions?
Your first version of your code is working as expected and the logs are as expected.
All async function return a promise. So findInDB_by_id() will always return a promise. In fact, they return a promise as soon as the first await inside the function is encountered as the calling code continues to run. The calling code itself needs to use await or .then() to get the resolved value from that promise.
Then, some time later, when you do a return someValue in that function, then someValue becomes the resolved value of that promise and that promise will resolve, allowing the calling code to be notified via await or .then() that the final result is now ready.
await only suspends execution of the async function that it is in. It does not cause the caller to be blocked at all. The caller still has to to deal with a promise returned from the async function.
The second version of your code just runs the db queries open loop with no control and no ability to collect results or process errors. This is often referred to as "fire and forget". The rest of your code continues to execute without regard for anything happening in those db queries. It's highly unlikely that the second version of code is correct. While there are a very few situations where "fire and forget" is appropriate, I will always want to at least log errors in such a situation, but since it appears you want results, this cannot be correct
You should try something like as when it encounter first async function it start executing rest of the code
let promiseArr = []
Object.keys(params).map(function(table){
promiseArr.push(db[table].find({
_id: {
$in: params[table]._ids
}
}))
})
let [records] = await Promise.all(promiseArr)
and if you still want to use map approach
await Promise.all(Object.keys(params).map(async function(table){
return await db[table].find({
_id: {
$in: params[table]._ids
}
})
})
)
I'm new to Node and the async programming model. I'm having problems dealing with a simple requirement that seems pretty basic in synchronous environments: paging through an API response until the response is empty.
More specifically, the API, on a successful call, will return data and a status of 200 or 206 (partial content). If I see the 206 response, I need to keep making calls to the API (also sending a page query param that I increment each time) until I see the 200 response.
In a synchronous language, the task will be a piece of cake:
// pseudocode
data = []
page = 1
do {
response = api.call(page)
data.append(response.data)
page++
} while (response != 200)
return data
Now, in Node, for a single api call, code like this will work:
// fire when '/' has a GET request
app.get('/', (req, res) => {
axios.get('https://api.com/v1/cats')
.then(response => {
// now what??
});
});
});
See the //now what?? comment? That's the point where I'm wondering how to proceed. I came across this somewhat-relevant post but am not able to convert this to a format that will work for me in Node and Axios.
Is it enough to just wrap the axios code in a separate function? I don't think so, because if I do this:
function getData(pageNum) {
axios.get('https://api.com/v1/cats')
.then(response => {
// now what??
});
});
}
I can't rely on a return value because as soon axios.get() gets executed, the function will be over. I can call getData() again after I get the first response, but then, suppose I want to return all the data from these multiple calls as the HTTP response from my Express server . . . how do I do that?
I hope I will not get downvoted for laziness or something. I've really looked around but not found anything relevant.
First, a counter-question: Is the data set so big that you need to worry about using up all the memory? Because if so then it will take more work to structure your code in a way that streams the data all the way through. (In fact I'm not even sure whether express allows streaming... you are using express aren't you?)
From the axios documentation, it looks like response is a readable stream which provides the response body. So reading it is also an asynchronous task. So you should write a function that does that. See the "Stream" page of the nodejs docs for more details. Or I could be persuaded to help with that too, time permitting. But for now, I'll assume you have a function readResponse, which takes an axios response object as an argument and returns a promise, and the promise resolves to an object such as { statusCode: 206, result: ['thing1', 'thing2'] }. I'll also assume that your goal is to get all the result arrays and concatenate them together to get e.g. ['thing1', 'thing2', 'thing3', 'thing4', 'thing5', 'thing6'].
You could write a self-calling version of your getData function. This will retrieve all data from a given page onwards (not just the page itself):
function getData(pageNum) {
axios.get('https://api.com/v1/cats' + (pageNum ? '?page=' + pageNum) : '')
.then(readResponse)
.then(function(parsedResponse) {
if(parsedResponse.statusCode == 200) {
return parsedResponse.result;
} else if(parsedResponse.statusCode == 206) {
return getData(pageNum + 1).then(function(laterData) {
return parsedResponse.result.concat(laterData);
});
} else {
// error handling here, throw an exception or return a failing promise.
}
});
});
}
Then, to get all data, just call this function with pageNum = 0:
// fire when '/' has a GET request
app.get('/', (req, res) => {
getData(0)
.then(function(results) {
// results is now the array you want.
var response = JSON.stringify(results); // or whatever you're doing to serialise your data
res.send(response);
});
});
I have some intents that need to trigger the fulfillment webhook and don't care about the response. The webhook takes longer than the timeout to respond so I'd like the intent to simply respond with "Thanks for chatting" and then close the conversation while actually triggering the webhook.
Feels easy but I'm missing something. Also I'm new to the dialogflow stuff.
I can do this in any language, but here's an example in Javascript:
fdk.handle(function (input) {
// Some code here that takes 20 seconds.
return {'fulfillmentText': 'i can respond but I will never make it here.'}
});
EDIT 1 - Trying async
When I use an async function, the POST request never happens. So in the following code:
fdk.handle(function (input) {
callFlow(input);
return { 'fulfillmentText': 'here is the response from the webhook!!' }
});
async function callFlow(input) {
console.log("input is --> " + input)
var url = "some_url"
console.log("Requesting " + url)
request(url, { json: true, headers: {'Access-Control-Allow-Origin' : '*'} }, (err, res, body) => {
if (err) { return console.log(err); }
console.log("body is...")
console.log(body)
});
}
I see in the logs the two console.log outputs but nothing from the request. And the request doesn't seem to happen either because I don't see it at my endpoint.
SOLUTION
Thanks Prisoner for the tip. Seems like I needed to return the fulfillment JSON back through the callFlow() and handle() functions. Now Google Home doesn't timeout and both the HTTP call and response are generated.
const fdk = require('#fnproject/fdk');
const request = require('request');
fdk.handle(function (input) {
return callFlow(input);
});
async function callFlow(input) {
var searchPhrase = input || "cats"
var url = "some url"
return new Promise((resolve, reject) => {
request.post(url, {
headers: { 'content-type': 'application/x-www-form-urlencoded' },
body: searchPhrase
},
function (err, resp, body) {
if (err) { return console.log(err) }
r = { 'fulfillmentText': `OK I've triggered the flow function with search term ${searchPhrase}` }
resolve(r)
}
);
});
}
You cannot trigger the fulfillment asynchronously. In a conversational model, it is expected that the fulfillment will perform some logic that determines the response.
You can, however, perform an asynchronous operation in the fulfillment that does not complete before you return the result.
If you are using a sufficiently modern version of node (version 8 and up), you can do this by declaring a function as an async function, but not calling it with the await keyword. (If you did call it with await, it would wait for the asynchronous operation to complete before continuing.)
So something like this should work, given your example:
async function doSomethingLong(){
// This takes 20 seconds
}
fdk.handle(function (input) {
doSomethingLong();
return {'fulfillmentText': 'This might respond before doSomethingLong finishes.'}
});
Update 1 based on your code example.
It seems odd that you report that the call to request doesn't appear to be done at all, but there are some odd things about it that may be causing it.
First, request itself isn't an async function. It is using a callback model and async functions don't just automatically wait for those callbacks to be called. So your callFlow() function calls console.log() a couple of times, calls request() and returns before the callbacks are called back.
You probably should replace request with something like the request-promise-native package and await the Promise that you get from the call. This makes callFlow() truly asynchronous (and you can log when it finishes the call).
Second, I'd point out that the code you showed doesn't do a POST operation. It does a GET by default. If you, or the API you're calling, expect a POST, that may be the source of the error. However, I would have expected the err parameter to be populated, and your code does look like it checks for, and logs, this.
The one unknown in the whole setup, for me, is that I don't know how fdk handles async functions, and my cursory reading of the documentation hasn't educated me. I've done this with other frameworks, and this isn't a problem, but I don't know if the fdk handler times out or does other things to kill a call once it sends a reply.
I want to be access array outside the function or outside the loop in nodejs. I written following code.
var result = [];
function setid (swfid){
crud.getswift(swfid).then(function (response) {
console.log("response",response);
result = response;
// res.send(response);
}).catch(function (err) {
return ("error:" + err);
});
console.log("result",result);
}
console.log("result",result);
But its returning null. your suggestions please
You wrote a new statement in the function call and therefore you scoped it. This is one of the things wrong there. Apart from that, as the first person commenting to this answer mentioned, you have an async call here. Therefore, you need to return a promise from setid and wait for the response to get the result.
You're mixing your Aysnc logic with Sync. You won't get the response outside the .then function scope because there's no response available at the time you're trying to get the results.
Try using a callback in the promise - You'd need to invoke the function in the promise callback and send the response as function param, then play with the data.
> Promise / API call etc
.then(() => gotDataCallBack(data));
gotDataCallBack(data){
// handle your data and logic here.
// this will make sure you have the data available before you move ahead with
your application/manipulation logic.
}
I'm making a DNS Lookup API using Node.js and Express.js framework such that when it sends a POST request, it should return the addresses of different record types.
app.post('/', (req, res) => {
// Request format
// const l = {
// lookup: 'twitter.com',
// recordTypes: ['A', 'TXT']
// };
// Using destructor to fetch properties
const { lookup, recordTypes } = req.body;
console.log(lookup, recordTypes);
// For each record type
recordTypes.forEach(function(type) {
// setTimeout to get something async
setTimeout(function() {
dns.resolve(lookup.toLowerCase(), type, (err, addresses) => {
console.log(type);
if (err) {
return console.log(`\nType(${type}):\n`, err);
}
result = result + JSON.stringify({ type: `${type}`, response: { addresses } });
console.log(result);
});
}, 2000);
});
res.send(result);
});
It logs the correct stuff in the console but when it comes to the response, it returns an empty string. I used setTimeout to mimic the asynchronous nature of the request but it just does not work.
Please assume that I have declared stuff like result etc. because it is working. Also, please don't to redirect me to the Node.js documentation because I have already read that stuff and that's not the problem here. The problem is that I need to get every record type in an array and send that back as a response.
Here's what I have tried:
Tried to push response for each record type in the result array,
Tried to use a for of loop instead of forEach
Please help!
The way I'm reading your code is that for each item in the array you correctly use callbacks to do each individual bit of processing.
However, remember that forEach itself is not asynchronous. Thus you are setting up a bunch of tasks that will complete sometime, then returning undefined... then your results start to trickle in.
There's a couple ways to correctly. As you are using callbacks here I will use that style. You want to get a callback when all items in an array have been completely processed. The async module does this very well, providing a lot of high quality methods that act on arrays and such and give you a way to have a callback when they are all over.
Your function will look something like:
let res = []
async.each( recordTypes,
( type, done ) => {
dns.resolve(lookup.toLowerCase(), type, (err, addresses) => {
result = result + JSON.stringify({ type: `${type}`, response: { addresses } });
done(err)
} )
},
(allOverError) => {
res.send(result);
}
)
Notice there are two function parameters here: the first one is called for every item in the list, and the last is called when every item in the list has been completely processed.
There are other ways too, promises or the async/await keywords (confusing because of the name of the async module), but callbacks are good.