Nodejs loop through response data from request - node.js

The below data is returned from the request() library.
var data = [{
service: "trains",
...
}, {
service: "buses",
...
}]
I have the following code on a NodeJS application.
request(options)
.then(data => {
data.forEach(record => {
if (record.service !== 'buses') {
return next('You do not have permission to access buses.')
}
return next()
})
})
.catch(err => next(err))
So when this code runs on Node it always evaluates to false? This is odd to me because running this without the request and just using the raw response directly seems to work ok? I guess this is an issue with async behavior?

You're looping through all the elements of the array, so if the first element does not match 'buses', it will result in 'You do not have permission to acces buses'.
I think what you're looking for is a method to check if an element exists in an array. In your case something like Array.filter:
.then(data => {
if (data.filter(el => el.service === 'buses').length === 0) {
return next('You do not have permission to access buses.');
}
return next();
})

Related

NodeJS Controller function returns before data from API call and array.filter completes

I'm working on a Slackbot that compares a repo branch file with the same file in Master, which requires making two API calls to bitbucket's API. The first grabs all the most recent branches in our workspace, which includes a URL that I can then use to call the API for the difference between the two files:
Snippet from NodeJS Controller File
let diffs = await axios.get('API-Call-To-BB-For-Recent-Branches', {
headers: {
'Authorization': `${AUTH}`
}
})
let filteredByUser = diffs.data.values.filter(element => {
if (element.target.author.raw.includes(username)) {
if (element.target.repository.full_name.includes("nameOfMasterBranch")) {
axios.get(element.target.links.diff.href, {
headers: {
'Authorization': `${AUTH}`
}
}).then(response => {
let clippedBranch = {
branch: element.name,
author: element.target.author.user.display_name,
diff: response.data
}
// Console Logging here shows the data I'm looking for
console.log(clippedBranch)
return clippedBranch
}).catch(error => {
console.log(error)
})
}
}
})
// Console logging Here returns an empty array.
console.log(filteredByUser)
// Console logging the returned value on the main server file returns a Promise<Pending>
return filteredByUser
What I've Tried
I've tried using a Promise.resolve and Promise.All to fix the issue.
Making the second API call inside of a for Of statement and a forEach statement
I've tried nesting the array processing and second API call inside of a .then on the first API call to BitBucket.
Whats preventing the data from being resolved in time to be returned?
Thanks in advance for your time!
Your problem is in the .filter loop. You are calling axios.get but not doing anything with the Promise it returns. The filter function should return true if the object is to be kept and false if not. But it returns nothing at all (which is equivalent to false. You might think it returns clippedBranch but it doesn't.
I suspect you want filteredByUser to be an array of clippedBranch objects, but it will end up being an empty array.
The following code will get you further:
let diffs = await axios.get("API-Call-To-BB-For-Recent-Branches", {
headers: {
Authorization: `${AUTH}`
}
});
const filteredByUser = [];
// use forEach instead of filter
diffs.data.values.forEach(async element => { // <--- use async to allow await
if (element.target.author.raw.includes(username)) {
if (element.target.repository.full_name.includes("nameOfMasterBranch")) {
const clippedBranch = await axios.get( // <--- wait for the result!
element.target.links.diff.href, {
headers: {
Authorization: `${AUTH}`
}
})
.then(response => {
let clippedBranch = {
branch: element.name,
author: element.target.author.user.display_name,
diff: response.data
};
// Console Logging here shows the data I'm looking for
console.log(clippedBranch);
return clippedBranch;
})
.catch(error => {
console.log(error);
});
if (clippedBranch) {
filteredByUser.push(clippedBranch); // <-- add to array if exists
}
}
}
});
Your original code didn't do anything with the value that was finally resolved by the axios.get(href). In fact those calls would complete long after your function ended since you didn't wait on them at all.
I used forEach instead of filter because we want to process each entry but not simply keep or discard it (which is what filter is for). We want to build a new array, so we just push the things we want onto it if found while looping through each entry.
You might want to chain calls to both .filter and .map separately. Not only will this "flatten" your code but it should read a bit easier as well. Use filter first to remove elements that you don't need. Then map through the resulting list and return a list of promises which can be passed to Promise.all
Here is what I believe you're attempting to do:
async function getBranchesByUsername(username) {
try {
const diffs = await axios.get("API-Call-To-BB-For-Recent-Branches", {
headers: {
Authorization: `${AUTH}`,
},
});
const requests = diffs.data.values
.filter(element => {
return element.target.author.raw.includes(username) &&
element.target.repository.full_name.includes("nameOfMasterBranch")
})
.map(element => {
return axios
.get(element.target.links.diff.href, {
headers: {
Authorization: `${AUTH}`,
},
})
.then((response) => {
const clippedBranch = {
branch: element.name,
author: element.target.author.user.display_name,
diff: response.data,
};
console.log(clippedBranch);
return clippedBranch;
});
});
return await Promise.all(requests)
} catch (error) {
console.log(error)
throw error
}
}
I have not been able to test this but the concept still applies

Express returns empty array

I currently have the following code
router.get('/uri', (request,response) => {
let final = [];
TP.find({userID: request.userID})
.then(tests =>{
tests.forEach(test => {
A.findById(test.assignmentID)
.then(assignment => {
final.push({
testID: test._id,
name: assignment.title,
successRate: `${test.passedTests}/${test.totalTests}`
})
})
.catch(error => {
console.log(error)
})
})
return response.send(final);
})
.catch(err => {
console.log(err);
return response.sendStatus(500);
})
})
The code is supposed to query 2 MongoDB databases and construct an array of objects with specific information which will be sent to the client.
However, I always get an empty array when I call that endpoint.
I have tried making the functions async and make them wait for results of the nested functions but without success - still an empty array.
Any suggestions are appreciated!
forEach doesn't care about promises inside it. Either use for..of loop or change it to promise.all. The above code can be simplified as
router.get('/uri', async (request,response) => {
const tests = await TP.find({userID: request.userID});
const final = await Promise.all(tests.map(async test => {
const assignment = await A.findById(test.assignmentID);
return {
testID: test._id,
name: assignment.title,
successRate: `${test.passedTests}/${test.totalTests}`
};
}));
return response.send(final);
});
Hope this helps.

How to returned poll data after each nodejs api call to reactjs component

I need to poll the data until the response.Status === 'UpdatesComplete'.
I have written this node js API function which basically polls the data -
const getResults = async (location) => {
try {
const response = await poll(location);
if (response.Status && response.Status === 'UpdatesComplete') {
return response;
}
return await getResults(location);
} catch (err) {
throw err;
}
};
app.get('/url', async (req, res) => {
try {
const results = await getResults(req.query);
res.json(formatData(results));
} catch (err) {
res.status(500).send(err);
console.error(err);
}
});
I am calling this API from ReactJS class component inside ComponentDidMount lifecycle method -
componentDidMount = () => {
axios.get('url', {
params: params
})
.then(response => {
console.log(response.data, 'data');
// setting the data on state
this.setState({ filteredList: response.data });
})
.catch(err => {
this.setState({ error: true });
});
};
This is working fine. But since the API is returning data till all the data has been fetched(after doing polling), it's taking a very long time on view to load the data. I am basically looking for returning the polling data to view as soon as the data fetched and simultaneously polling the API until all the data has been fetched. In this case, data will keep on updating after each polling on the view which will improve the user experience.
Thanks in advance.
You are finding the lazy loading or infinite scroll solution on the server-side. There is no simple way to do this.
The basic idea of the solution is to paginate your result with polling. ie.
call url?size=2&offset=0 from the client-side. Then on the server-side just poll first 2 results and return. next time call url?size=2&offset=2 and so-on.

Express API returning an unwanted "data" section in my GET all requests

I'm currently building a rest API and I'm having an unexpected output when I make a /GET request.
When i make a get request to the API, it returns
{
data: {
[{myExpectedObjects},{myExpectedObjects}]
}
}
however, I'm expecting my get request to return just the array of objects. Below is the code im using to accomplish the rest calls
Create controller
const create = (req, res) => {
let dataModel = generateModel(genericDataFromReq);
dataModel = new dataModel({
genericData,
specificData,
});
dataModel.save().then((data) => {
res.status(201).send(data);
}, (e) => {
res.status(500).send(e);
});
}
};
get all controller
const list = (req, res) => {
const dataModel = generateModel(dataToGet);
dataModel.find().then((data) => {
if (data.length === 0) {
res.status(404).send('failed');
} else {
res.status(200).send({ data });
}
}, (e) => {
res.status(500).send(e);
});
};
generate data model
function generateModel(dbCollectionName) {
try {
return generateDataModel(dbCollectionName);
} catch (e) {
return mongoosee.model(`${dbCollectionName}`);
}
}
I know the code is a bit unconventional but I've set up a generic rest API to take in different types of requests and I found this solution to be the best way of doing this.
Any ideas on why my get all request is tacking on a "data" section before my array of objects (which is what I'm actually interest in)?
I believe the issue is in this line:
else {
res.status(200).send({ data });
}
When you put curly braces around a variable, it creates a key-value pair where the key is the name of the variable and the value is the value of the variable. So get rid of the curly braces and it should work as you expect. See the parts that mention this ES2015 notation here:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Object_initializer

Promises and condition from async call

I'm implementing my service using Node.js with AWS DynamoDB (aws-sdk).
It's unclear for me how to implement the following scenario with promises:
I get a request to modify an entity with specified attributes
I'm trying to find the entity in the DB (async call find)
If the entity not found then create one with initial state (async call createInitialStateObject)
Modify the entity (which was in the DB before or just created on step 3) according to specific rules (async call applyModifications)
This is my first attempt:
function scenario(params) {
find(params).then((data) => {
let objectExists = checkExistense(data);
if (!objectExists) {
createInitialStateObject(params).then((data) => {
console.log("Object created");
// OK
}).catch((err) => {
console.error("Object not created");
// exit and return error
});
}
applyModifications(params).then((data) => {
// OK, return data
}).catch((err) => {
// exit and return error
});
}).catch((err) => {
// exit and return error
});
}
But the flaw here is that creation could happen before the modification, it's not bound to happen one after another.
The other attempt works, but looks a bit weird. I create an empty promise to call in case the object already exists:
function scenario(params) {
find(params).then((data) => {
let conditionalPromise = new Promise((resolve) => {
resolve(null);
});
let objectExists = checkExistense(data);
if (!objectExists) {
conditionalPromise = createInitialStateObject(params);
}
conditionalPromise.then((data) => {
applyModifications(params).then((data) => {
// OK, return data
}).catch((err) => {
// exit and return error
});
}).catch((err) => {
// exit and return error
});
}).catch((err) => {
// exit and return error
});
}
How it should be implemented in a right way?
Creating 'empty' or sync. Promises isn't unusual. There is even a short way of doing that: Promise.resolve(value) creates and resolves a Promise immediately.
Besides that you should make use of proper chaining and stop nesting things so much. Once you are in a chain, you don't even need to resolve an empty promise as a return value of a non thenable object is interpreted as exactly this.
function scenario(params) {
return find(params)
.then(data => {
let objectExists = checkExistense(data);
if (!objectExists) {
return createInitialStateObject(params);
}
// if we return nothing (or null in your case) this will be the same as return Promise.resolve()
return null;
})
.then(data => applyModifications(params))
.then(data => console.log(data))
.catch(err => console.log(err));
// exit and return error
}

Resources