return response data from async call - node.js

I created this function to get list all my drives from GDrive.
async getAllDrives(token) {
let nextPageToken = ""
let resultArray = []
const config= {
headers: {
Authorization: `Bearer ${token}`
}
};
const bodyParams = {
pageSize: 2,
fields: 'nextPageToken, drives(id, name)',
q:`hidden=false`,
};
do {
axios.get(
`https://www.googleapis.com/drive/v3/drives`,
config,
bodyParams,
).then(result => {
nextPageToken = result.data.nextPageToken;
resultArray.push(result.data.drives);
resultArray = resultArray.flat();
console.log("result", resultArray);
}).catch(error => {
console.log(error);
//res.send(error);
});
}while(nextPageToken);
resultArray = resultArray.flat();
resultArray.map(drive => {
drive.isSharedDrive = true;
return drive;
});
return JSON.stringify(resultArray);
}
When I look in console.log
then(result => {
nextPageToken = result.data.nextPageToken;
resultArray.push(result.data.drives);
resultArray = resultArray.flat();
console.log("result", resultArray);
})
I have the expected result,
result [
{
kind: 'drive#drive',
id: '**',
name: ' ★ 🌩'
},
]
but return JSON.stringify(resultArray); is empty.
I found a similar question here, How do I return the response from an asynchronous call? but the answer is not satisfying.

You used the async call slightly incorrectly. You calling axios.get without await keyword, but with .then chaining. Since you don't wait for result to return, you getting empty array first, returning you nothing. And only then your callback function inside .then is getting called. To simplify, you doing this in your example:
function getAllDrives() {
// Local variable where you want your result
let result = [];
// You calling the axios.get method, but don't wait for result
axios.get().then(result => {})
// Empty result is getting returned immediately
return result;
}
And when response is returned from the remote server, function inside .then trying to save result to local variable. But function is already completed, so you don't get anything.
What you actually should do is call axios.get with await keyword:
// You should always cover your asynchronous code with a try/catch block
try {
// Instead of `then` callback use `await` keyword. Promise returned from
// this method will contain result. If error occurs, it will be thrown,
// and you can catch it inside `catch`.
const result = await axios.get(
`https://www.googleapis.com/drive/v3/drives`,
config,
bodyParams
);
// Here is your code as you wrote it inside `then` callback
nextPageToken = result.data.nextPageToken;
resultArray.push(result.data.drives);
resultArray = resultArray.flat();
console.log("result", resultArray);
} catch (error) {
// And here is your error handling code as you wrote it inside `catch`
console.log(error);
}
This way your method will not complete until your request is not executed.
You can read more about async/await functions here.

I believe your goal is as follows.
You want to retrieve the drive list using axios.
Your access token can be used for retrieving the drive list using Drive API.
Modification points:
In order to use nextPageToken in the request, in this case, it is required to run the script with a synchronous process. So, async/await is used. This has already been mentioned in the existing answers.
When I saw your script, I thought that the query parameter might be required to be included in the 2nd argument of axios.get().
In order to use nextPageToken, it is required to include the property of pageToken. In your script, pageToken is not used. By this, the infinite loop occurs because nextPageToken is continued to be returned.
When these points are reflected in your script, how about the following modification?
Modified script:
let resultArray = [];
const config = {
headers: {
Authorization: `Bearer ${token}`,
},
params: {
pageSize: 2,
fields: "nextPageToken, drives(id, name)",
q: `hidden=false`,
pageToken: "",
},
};
do {
const { data } = await axios
.get(`https://www.googleapis.com/drive/v3/drives`, config)
.catch((error) => {
if (error.response) {
console.log(error.response.status);
console.log(error.response.data);
}
});
if (data.drives.length > 0) {
resultArray = [...resultArray, ...data.drives];
}
nextPageToken = data.nextPageToken;
config.params.pageToken = nextPageToken;
} while (nextPageToken);
resultArray.map((drive) => {
drive.isSharedDrive = true;
return drive;
});
return JSON.stringify(resultArray);
Testing:
When this script is run, the following result is obtained.
[
{"id":"###","name":"###","isSharedDrive":true},
{"id":"###","name":"###","isSharedDrive":true},
,
,
,
]
Note:
From the official document of "Drives: list",
pageSize: Maximum number of shared drives to return per page. Acceptable values are 1 to 100, inclusive. (Default: 10)
So, when pageSize is 100, the number of loops can be reduced. If you want to test the loop using nextPageToken, please reduce the value.
References:
axios
Drives: list

I recommend you study a little more about async/await.
It makes no sense for you to use async and put a .then().catch(), the purpose of async to get these encapsulated syntaxes.
async getAllDrives(token) {
try {
const getDrives = await this.request(token)
console.log(getDrives)
const results = this.resultArray(getDrives)
return results
} catch (e) {
console.log(e)
}
}
I didn't quite understand your while or your objective, adapt it to your code or remove it
async request(token) {
let nextPageToken = 1 // ????????
const config = {
headers: {
Authorization: `Bearer ${token}`
}
};
const bodyParams = {
pageSize: 2,
fields: 'nextPageToken, drives(id, name)',
q: `hidden=false`,
};
let getDrives = [];
// loop for each request and create a request array
for (let x = 0; x < fields.nextPageToken; x++) {
const request = axios.get(
`https://www.googleapis.com/drive/v3/drives`,
config,
bodyParams
);
getDrives.push(request)
}
const drives = await Promise.all(getDrives)
return drives
}
async resultArray(drivers) {
// result treatment here
}
The return of promise all will be an array of the driver's responses
Note: The response in request.data
const request = await axios.get()
const resposta = request.data
Read about
https://developer.mozilla.org/pt-BR/docs/Web/JavaScript/Reference/Global_Objects/Promise/all

Related

node using await in for loop which calls an async function

I'm having a little trouble getting back a result from an async function I am calling in a for loop, I should be getting back a string but it is returning a promise instead
I am following this syntax but I haven't been able to get it to work properly and I'm hoping for some insight into why this fails https://eslint.org/docs/rules/no-await-in-loop
here's the function I am trying to work with, in this case decodeTheObject is async and returns a promise but if I use await decodeTheObject eslint will give me an error saying I can't use await inside a for loop, unfortunately the workaround above still results in a promise being returned instead of a resolved value
async function decode (request, encodedObj) {
const decodedArr = [];
try{
for (const id of encodedObj) {
decodedArr.push({
data: decodeTheObject(request, id), // this is an async function
partial: true,
});
}
await Promise.all(decodedArr);
return decodedArr;
}catch (err) {
throwError(
req,
{
errorDetails: [
{
issue: 'INVALID_ENCODING',
description: 'Invalid encoded obj',
},
],
},
);
}
};
// ----- calling function -----
const decodedObj = await decode(request, encodedItem);
const payload = {
newprop: decodedObj
};
Promise.all() has to work directly on an array of promises. You are passing it an array of objects which won't know how to reach into the objects to get the promises so thus, it won't accomplish anything useful. There are a couple ways to approach this.
This will await each async call in sequence, get the value and push the value into the array:
async function decode (request, encodedObj) {
const decodedArr = [];
try{
for (const id of encodedObj) {
let data = await decodeTheObject(request, id);
decodedArr.push({data, partial: true});
}
return decodedArr;
} catch(e) {
...
}
}
Or, you could run them in parallel with Promise.all() like this by creating an array of promises and using Promise.all() on that array:
async function decode (request, encodedObj) {
return Promise.all(encodedObj.map(id => {
return decodeTheObject(request, id).then(data => {
return {data, partial: true};
});
})).catch(err => {
...
});
}

NodeJS: make less requests by API

I am trying to process response data and do not make next request before current data didn't processed. I tried use async/await and generators.
Generator:
private *readData() {
const currentUrl = this.getUrl();
const requestSettings = this.getRequestSettings();
yield axios.get( currentUrl, requestSettings).then( (response: any) => {
console.log('Make request');
return this.getData(response);
});
}
*readItem() {
let responseData: any;
if (!responseData) {
const response = this.readData();
responseData = response.next();
}
console.log('res data:', responseData['value']);
yield responseData['value'].then((res: any) => {
return res;
});
}
and then in the main code I do next:
for (let i=0;i<10;i++) {
item = transport.readItem().next();
console.log("R:", item);
}
Another idea was using async/await
async readItems() {
const headers = this.settings.headers;
const response = await axios.get( url, {
headers: headers
});
return response.data;
}
But in this case I get a promise in the response, if I just try to call this method 10 times.
I read 10 items, but I still make 10 requests to the server. Is it possible make one request, processed 10 items, and then make the second request? Maybe I have to use another pattern or whatever.
Async/await is right approach, just put await in front of readItem() and the Promise you get will be awaited that will give you desired. If your loop is in top level use readItem().then(). The latest NodeJS version allows await in top level.
for (let i=0;i<10;i++) {
item = await transport.readItem();
console.log("R:", item);
}
I found next solution
(async () => {
for (let i = 0; i < 10; i++) {
item = await transport.readItem();
console.log("R:", item);
}
})();
because I ran this part of code inside script without any functions/methods

NodeJS Controller function returns before data from API call and array.filter completes

I'm working on a Slackbot that compares a repo branch file with the same file in Master, which requires making two API calls to bitbucket's API. The first grabs all the most recent branches in our workspace, which includes a URL that I can then use to call the API for the difference between the two files:
Snippet from NodeJS Controller File
let diffs = await axios.get('API-Call-To-BB-For-Recent-Branches', {
headers: {
'Authorization': `${AUTH}`
}
})
let filteredByUser = diffs.data.values.filter(element => {
if (element.target.author.raw.includes(username)) {
if (element.target.repository.full_name.includes("nameOfMasterBranch")) {
axios.get(element.target.links.diff.href, {
headers: {
'Authorization': `${AUTH}`
}
}).then(response => {
let clippedBranch = {
branch: element.name,
author: element.target.author.user.display_name,
diff: response.data
}
// Console Logging here shows the data I'm looking for
console.log(clippedBranch)
return clippedBranch
}).catch(error => {
console.log(error)
})
}
}
})
// Console logging Here returns an empty array.
console.log(filteredByUser)
// Console logging the returned value on the main server file returns a Promise<Pending>
return filteredByUser
What I've Tried
I've tried using a Promise.resolve and Promise.All to fix the issue.
Making the second API call inside of a for Of statement and a forEach statement
I've tried nesting the array processing and second API call inside of a .then on the first API call to BitBucket.
Whats preventing the data from being resolved in time to be returned?
Thanks in advance for your time!
Your problem is in the .filter loop. You are calling axios.get but not doing anything with the Promise it returns. The filter function should return true if the object is to be kept and false if not. But it returns nothing at all (which is equivalent to false. You might think it returns clippedBranch but it doesn't.
I suspect you want filteredByUser to be an array of clippedBranch objects, but it will end up being an empty array.
The following code will get you further:
let diffs = await axios.get("API-Call-To-BB-For-Recent-Branches", {
headers: {
Authorization: `${AUTH}`
}
});
const filteredByUser = [];
// use forEach instead of filter
diffs.data.values.forEach(async element => { // <--- use async to allow await
if (element.target.author.raw.includes(username)) {
if (element.target.repository.full_name.includes("nameOfMasterBranch")) {
const clippedBranch = await axios.get( // <--- wait for the result!
element.target.links.diff.href, {
headers: {
Authorization: `${AUTH}`
}
})
.then(response => {
let clippedBranch = {
branch: element.name,
author: element.target.author.user.display_name,
diff: response.data
};
// Console Logging here shows the data I'm looking for
console.log(clippedBranch);
return clippedBranch;
})
.catch(error => {
console.log(error);
});
if (clippedBranch) {
filteredByUser.push(clippedBranch); // <-- add to array if exists
}
}
}
});
Your original code didn't do anything with the value that was finally resolved by the axios.get(href). In fact those calls would complete long after your function ended since you didn't wait on them at all.
I used forEach instead of filter because we want to process each entry but not simply keep or discard it (which is what filter is for). We want to build a new array, so we just push the things we want onto it if found while looping through each entry.
You might want to chain calls to both .filter and .map separately. Not only will this "flatten" your code but it should read a bit easier as well. Use filter first to remove elements that you don't need. Then map through the resulting list and return a list of promises which can be passed to Promise.all
Here is what I believe you're attempting to do:
async function getBranchesByUsername(username) {
try {
const diffs = await axios.get("API-Call-To-BB-For-Recent-Branches", {
headers: {
Authorization: `${AUTH}`,
},
});
const requests = diffs.data.values
.filter(element => {
return element.target.author.raw.includes(username) &&
element.target.repository.full_name.includes("nameOfMasterBranch")
})
.map(element => {
return axios
.get(element.target.links.diff.href, {
headers: {
Authorization: `${AUTH}`,
},
})
.then((response) => {
const clippedBranch = {
branch: element.name,
author: element.target.author.user.display_name,
diff: response.data,
};
console.log(clippedBranch);
return clippedBranch;
});
});
return await Promise.all(requests)
} catch (error) {
console.log(error)
throw error
}
}
I have not been able to test this but the concept still applies

request-promise loop, how to include request data sent with response?

I am trying to use request-promise in a loop and then send a response back to the client with all of the responses. The below code works, however I want to also include the request data with each response so that the request ID can be correlated with the result. Is there a built in way to do this:
promiseLoop: function (req, res) {
var ps = [];
for (var i = 0; i < 3; i++) {
// var read_match_details = {
// uri: 'https://postman-echo.com/get?foo1=bar1&foo2=bar2',
// json: true // Automatically parses the JSON string in the response
// };
var session = this.sessionInit(req, res);
if (this.isValidRequest(session)) {
var assertion = session.assertions[i];
const options = {
method: 'POST',
uri: mConfig.serviceURL,
body: assertion,
headers: {
'User-Agent': 'aggregator-service'
},
json: true
}
logger.trace(options);
ps.push(httpClient(options));
}
}
Promise.all(ps)
.then((results) => {
console.log(results); // Result of all resolve as an array
res.status(200);
res.send(results);
res.end();
}).catch(err => console.log(err)); // First rejected promise
}
Assuming httpClient() is request-promise that you refer to and the assertion value is what you're trying to pass through with this result, you could change this:
ps.push(httpClient(options));
to this:
ps.push(httpClient(options).then(result => {
return {id: assertion, result};
}));
Then, your promise would resolve to that object which contains both the result and the id and you could access each in the final array of results.
Your code doesn't show what the current result is. If it's already an object, you could also just add the id property to that object if you'd rather. This is up to you exactly how you put that final result together.
ps.push(httpClient(options).then(result => {
// add id into final result
result.id = assertion;
return result;
}));
Anyway, the general idea is that before putting the promise in the array, you use a .then() handler to slightly modify the returned result, adding in whatever data you want to add and then returning that new modified result so it becomes the resolved value of the promise chain.
To make sure you process all responses, even if some have an error, you can use the newer [Promise.allSettled()][1] instead of Promise.all() and then look through which responses succeeded or failed in processing the results. Or, you can catch any errors, turn them into resolved promises, but give them a sential value (often null) that you can see in processing the final results:
ps.push(httpClient(options).then(result => {
// add id into final result
result.id = assertion;
return result;
}).catch(err => {
console.log(err);
// got an error, but don't want Promise.all() to stop
// so turn the rejected promise into a resolved promise
// that resolves to an object with an error in it
// Processing code can look for an `.err` property.
return {err: err};
}));
Then, later in your processing code:
Promise.all(ps)
.then((results) => {
console.log(results); // Result of all resolve as an array
// filter out error responses
let successResults = results.filter(item => !item.err);
res.send(successResults );
}).catch(err => console.log(err)); // First rejected promise
Promise.allSettled will not stop at error. It make sure you process all responses, even if some have an error.
const request = require('request-promise');
const urls = ["http://", "http://"];
const promises = urls.map(url => request(url));
Promise.allSettled(promises)
.then((data) => {
// data = [promise1,promise2]
})
.catch((err) => {
console.log(JSON.stringify(err, null, 4));
});

Unable to get async/await working with Axios and array.map()

I'm trying to build a function that maps over a given array and then runs an axios call on each item. Once that is done, I want to return the mapped array for use in another function.
Here is the code:
require('dotenv').config();
const axios = require('axios');
const consola = require('consola');
function getLeagues(listItems) {
const today = new Date();
const season = today.getMonth() >= 6 ? today.getFullYear() : today.getFullYear() - 1; // this gets the game ready for the new season every July 1st
// we will end up directly returning listItems.map once the issue is solved
const selectedLeagues = listItems.map(async (item) => {
const countryName = item.country;
const leagueName = item.league;
try {
const response = await axios({
method: 'GET',
url: `https://api-football-v1.p.rapidapi.com/v2/leagues/country/${countryName}/${season}`,
headers: {
'content-type': 'application/octet-stream',
'x-rapidapi-host': 'api-football-v1.p.rapidapi.com',
'x-rapidapi-key': process.env.FOOTY_API_KEY,
},
});
const leagueData = response.data.api.leagues
.filter((league) => league.name === leagueName)
.map((data) => {
return {
leagueId: data.league_id,
name: data.name,
seasonStart: data.season_start,
seasonEnd: data.season_end,
country: data.country,
};
})
.pop(); // we use pop() because filter() and map() return arrays and we don't want an array of 1 object, just that object
consola.ready({
// this displays all of the data as needed
// this also runs after the below consola block
message: `leagueData: ${JSON.stringify(leagueData, null, 2)}`,
badge: true,
});
return leagueData;
} catch (error) {
throw new Error(error);
}
});
consola.ready({
// this displays an array with an empty object, not an array with above leagueData object
// this also runs before the above consola block
message: `selectedLeagues: ${JSON.stringify(selectedLeagues, null, 2)}`,
badge: true,
});
return selectedLeagues;
}
module.exports = getLeagues;
I'm not sure why the selectedLeagues array is being returned before the leagueData object is even ready. I thought async/await held everything. Instead, in my console, I am getting:
selectedLeagues: [
{}
]
leagueData: {
"leagueId": 753,
"name": "Liga 3",
"seasonStart": "2019-07-19",
"seasonEnd": "2020-05-16",
"country": "Germany"
}
What am I doing wrong?
You have to wrap your listItems.map in a promise all function because map on its own isn't compatible with async.
// Now magically you can add async to your map function...
Promise.all(listItems.map(async item => {
// Then put your try catch here so that it only wraps around
// the results of the function you're awaiting...
let response
try {
response = await axios()
} catch (err) {
return err;
}
// Anything else you want to do with the response...
return response
})).then(results => {
// All the resolved promises returned from the map function.
console.log(results)
})
When you use the await keyword inside an async function the rest of the code will just wait for the result of the awaited function, the try catch part is to catch any error you might get that's out of your control which is why you only try catch around the awaited function.
If you wrap too much of your code inside a try catch you won't be able to diagnose and handle the error properly.
You could put a try catch around the whole of your code if you wanted but the problem would be that the whole code would error out whenever you have any kind of small problem.
You can also do this with a for of loop which might look a bit cleaner...
for await (let item of listItems) {
// try catch await axios etc...
}
You can use async together with a Promise,
const arr = [1, 2, 3];
const asyncRes = await Promise.all(arr.map(async (i) => {
await sleep(10);
return i + 1;
}));
console.log(asyncRes);
// 2,3,4
What's happening in your .map() is async, not what's outside of it. The .map() kicks off but it doesn't block the consola.ready({ at the bottom.

Resources