Node - Receiving ECONNRESET error when doing an http request with axios - node.js

Question

I have a function in Node that needs to fetch all of the Users from Azure AD (I'm utilizing Microsoft Graph and Axios).
This function queries Microsoft Graph, the response includes an array of users. The max number of objects in this array is capped at 999. If there are more users, this response will also include an #odata.nextLink. My plan was to query, and if there is an #odata.nextLink in the response, query again until there is no #odata.nextLink. Not entirely sure if there is another way to do this better.
async function fetchUsers(tokenEndpoint, client_id, client_secret, query) {
try {
let allUsers = [];
let bearer = `Bearer ${await fetchToken(
tokenEndpoint,
client_id,
client_secret
)}`;
let res = await fetchData(bearer, `${msAPI}${query}`);
if (res.status === 200) {
allUsers.push(...res.data.value);
console.log(allUsers.length);
let nextURL = res.data["#odata.nextLink"];
while (nextURL != undefined) {
bearer = `Bearer ${await fetchToken(
tokenEndpoint,
client_id,
client_secret
)}`;
res = await fetchData(bearer, nextURL);
allUsers.push(...res.data.value);
nextURL = res.data["#odata.nextLink"];
console.log(allUsers.length);
}
}
return allUsers;
} catch (err) {
handleError(err);
}
}
async function fetchToken(tokenEndpoint, client_id, client_secret) {
try {
const res = await axios.post(
tokenEndpoint,
qs.stringify({
client_id,
client_secret,
grant_type: "client_credentials",
scope
})
);
return res.data.access_token;
} catch (err) {
handleError(err);
}
}
async function fetchData(bearer, url) {
try {
const res = await axios.get(url, {
headers: {
Authorization: bearer
}
});
return res;
} catch (err) {
handleError(err);
}
}
I have this statement in another async function
const allUsers = await fetchUsers(tokenEndpoint,clientId,clientSecret,query);
Sometimes this runs perfectly and I get all my Azure AD users and sometimes I receive an ECONNRESET error in the middle of the function being run and it won't finish. This error seemed to happen a few times on one network, then I tested on a different network and didn't get the error once. Swapped back to the original network and got the errors again. Not sure if this is a network issue or the way I am doing the Axios requests in my loop.
Regardless, how would I go about resolving this ECONNRESET issue?

Answer 1

An ECONNRESET occurs when a TCP abruptly closed. This could be caused by a failure in your local network, the remote service, or anywhere in between the two.
The most likely cause here is requesting $top=999. That is an excessively large (and therefore long) page size. You should be setting this to a smaller value or, ideally, leaving it to the default. This ensures the page sizes are reasonable, requests complete quickly, and helps avoid potential getting throttled by AAD.
Your "follow each #odata.nextLink plan is the correct pattern to use. This allows you to pull a page of data and process it prior to making subsequent requests for additional pages. In between requests you'll want to store the current #odata.nextLink so if anything goes wrong, you can pick up where you left off.