So i'm getting some data from an external API for my web-app without any issues. However, these requests takes a bit too long so i'd like to cache this data once retrieved with the help of REDIS. I've managed to successfully do these steps separately but i need to be able to call the external API if there is no existing data in the redis-client.
I tried the /players GET request in Postman and it gets stuck on 'Sending request..'
app.get('/players', async (req, res) => {
const players = await getOrSetCache('players', async () => {
return shl.players();
})
res.json(players);
})
Function to return a Promise. If there is no data, a callback is used to make the request to the external api and store that data in the redis-client
function getOrSetCache(key, callback) {
return new Promise((resolve, reject) => {
redisClient.get(key, async(error, data) => {
if (error) return reject(error)
if (data != null) return resolve(JSON.parse(data))
const freshData = await callback()
redisClient.setex(key, 3600, JSON.stringify(freshData))
resolve(freshData)
})
})
}
Callback code:
players() {
return this.connection.get('/teams').then(teams => {
const teamCodes = teams.map(team => team.team_code);
Promise.all(teamCodes.map(teamCode => this.playersOnTeam(teamCode)))
.then(teamData => {
const teamInfo = [].concat(...teamData);
const playerInfo = [].concat(...teamInfo.map(team => team.players.map(player => ({
player_id: player.player_id,
first_name: player.first_name,
last_name: player.last_name,
default_jersey: player.default_jersey,
position: player.position,
}))));
return playerInfo;
}
);
});
}
Grateful for any tip of what the issue might be or if someone can spot errors in this implementation!
Related
I want to add data to my MongoDB collection. I'm getting this data via a local Flask API. I'm GETting the data on my React Frontend and it's displaying fine. I'm not sure why I can't do the same thing on my express nodejs backend. I want to get that same data and use it to build the entity that I'm going to store.
This is how I'm attempting to get the data
app.get('/', async (req, res) => {
let initialData = {};
axios.get('http://localhost:3000/details').then((res) => {
initialData = res.data;
});
const recruit = new RecruitModel({ email:initialData.email,
mobile_number:initialData.mobile_number,
name:initialData.name});
try {
await recruit.save()
res.send("inserted data")
} catch (error) {
console.log(error)
}
})
I'm pretty sure something wrong there and nowhere else. Because if I pass static information instead it's correctly stored, no issues.
You are saving to the database's Recruit Collection before the promise is resolved. Since data to save in the Recruit Collection is dependent upon the result from the API which will initially return the promise, therefore, use promise resolving functions to wait for its result.
Solution#1 (using .then function):
app.get('/', async (req, res) => {
let initialData = {};
try {
axios.get('http://localhost:3000/details').then((response) => {
initialData = response.data;
const recruit = new RecruitModel({
email: initialData.email,
mobile_number: initialData.mobile_number,
name: initialData.name,
});
recruit.save().then((response) => res.send('inserted data'));
});
} catch (error) {
console.log(error);
}
});
Solution#2 (using async await keywords):
app.get('/', async (req, res) => {
try {
const response = await axios.get('http://localhost:3000/details');
const recruit = new RecruitModel({
email: response.data.email,
mobile_number: response.data.mobile_number,
name: response.data.name,
});
await recruit.save();
res.send('inserted data');
} catch (error) {
console.log(error);
}
});
Either solution will work in your case.
I'm currently working on an API call that works exactly the way it was intended within Postman, but when the same call is made on the front-end browser, it does not return the same data. Stack is MERN.
The very interesting part in all of this is that Axios request returns a different array of data each and every time. To help explain this, for testing purposes the database has only 4 messages to return when making the call. In Postman, all 4 messages are returned each and every time. But with React-Redux, the Axios call sometimes returns 2 messages, somtimes returns 3 messages, but never returns the full 4 messages as expected.
Edit:
Looks like this is a useEffect issue, I created a button that calls the same API request and all 4 messages are returned. Anyone else ever had this issue?
Here is the API call:
router.get('/all', auth, async (req, res) => {
try {
const lastMessagesArr = [];
const contacts = await Message.aggregate([
{ $match: { user: ObjectId(req.user.id) } },
{ $group: { _id: '$number' } },
]);
// console.log('contacts', contacts);
// console.log('req.user.id', req.user.id);
const getAllLast = async () => {
for (i = 0; i < contacts.length; i++) {
// let shortenedMessage;
let lastMessage = await Message.find({
user: ObjectId(req.user.id),
number: contacts[i]._id,
}).sort({ createdAt: -1 });
lastMessagesArr.push(lastMessage[0]);
// console.log('lastMessage', lastMessage);
}
lastMessagesArr.sort().reverse();
res.json(lastMessagesArr);
// console.log('lastMessagesArr', lastMessagesArr);
};
await getAllLast();
} catch (err) {
console.error(err.message);
res.status(500).json({ msg: 'Server Error' });
}
});
Here is my axios call React-Redux:
export const getAllMessages = () => async (dispatch) => {
try {
const res = await axios.get('/api/messages/all');
console.log('res.data all messages', res.data);
dispatch({
type: SET_ALL_MESSAGES,
payload: res.data,
});
} catch (err) {
const errors = err.response.data.errors;
if (errors) {
errors.forEach((error) => dispatch(setAlert(error.msg, 'danger')));
}
dispatch({
type: MESSAGE_ERROR,
payload: { msg: err.response.statusText, status: err.response.status },
});
}
};
And this is being called on the front-end through a useEffect hook when the component loads:
const AllConvos = ({ getAllMessages, message }) => {
useEffect(() => {
getAllMessages();
}, []);
return (
....
)
Is this a browser issue? (Using Chrome as the browser) Is this an Axios issue? React-Redux Issue? Any help is much appreciated
export const getAllMessages = () => async (dispatch) => {
The above code is a thunked action (a function that returns a function).
The way it works is that instead of an object being dispatched to the store, a function is sent instead. The redux-thunk middleware checks if it is a function and executes it with dispatch.
const dispatch = useDispatch(); // From react-redux
useEffect(() => {
dispatch(getAllMessages());
}, [dispatch]);
The thunked action needs to be dispatched so that it reaches the redux-thunk middleware. Otherwise nothing is there to pick it up.
Figured this out- it was a async/await issue within the useEffect hook. I moved the useEffect hook from the AllConvos component to component that functions as the page of the site and called the useEffect function there. The useEffect function now looks like this:
useEffect(async () => {
await getAllMessages();
}, []);
If I take away the async/await, the prior behavior experienced before is replicated.
I'm working on building an inventory management application using PERN stack. I have a modal where I need to make 2 GET requests and when I console.log in front end both requests are getting Status 200 response. However in my express server, first get request is working fine but the second request is not receiving anything.
My frontend code
const openModal = async () => {
setDetailModalOpen(true)
try {
await Promise.all([
(async () => {
const serial_number = props.bacsSerial
const response = await fetch(`http://localhost:5000/bacslist/demoinventory/${serial_number}`)
const parseResponse = await response.json()
console.log(response)
setInputs({
bacsUnit: parseResponse.bacs_unit,
serialNumber: parseResponse.serial_number,
partNumber: parseResponse.part_number,
bacsLocation: parseResponse.bacs_location,
description: parseResponse.bacs_description
})
setBacsId(parseResponse.id)
setBacsData(parseResponse)
})(),
(async () => {
const response2 = await fetch(`http://localhost:5000/bacslist/demoinventory/${bacsId}`)
console.log(response2)
})()
])
} catch (err) {
console.error(err.message)
}
}
My backend code
router.get("/demoinventory/:serial_number", async (req, res) => {
console.log('This one is working')
try {
const {serial_number} = req.params
const getDemoBacs = await pool.query(
"SELECT * FROM demo_inventory WHERE serial_number = $1", [serial_number]
)
res.json(getDemoBacs.rows[0])
} catch (err) {
console.error(err.message)
}
})
router.get("/demoinventory/:bacsId", async (req, res) => {
console.log(req.params)
console.log('This one is not working')
try {
const getHistoryData = await pool.query(
"SELECT * FROM demo_inventory_history"
)
console.log(getHistoryData)
res.json(getHistoryData)
} catch (err) {
console.error(err.message)
}
})
Sorry, Kinda new to this stuff so this isn't exactly an answer but I'm not allowed to leave a comment. I can't see your state variables with the code you posted, but are you sure that BacsId is being set to state before it is used in the second call, or is the parameter in the second call being sent empty, thus not using the right URL? Just a thought.
I am trying to get the array after it is filled in a function but it is empty when I print it out.
Code:
var usersList = [];
app.post('/submitLogin',function(req,res) {
getUsersGroups();
console.log(usersList);
});
function getUsersGroups() {
const users = new Promise((resolve, reject) => {
dbConnection
.getUsers()
.then(data => {
resolve(data)
})
});
const groups = new Promise((resolve, reject) => {
dbConnection
.getGroups()
.then(data => {
resolve(data)
})
});
Promise.all([users, groups])
.then(data => {
usersList = data[0];
groupsList = data[1];
console.log(usersList)
});
}
However, the console.log in the getUsersGroups(), prints out the filled array but it is empty in the app.post('/submitLogin...')
Why is this happening assuming that getUsersGroups() runs before I try to print out the array?
You are not observing the async execution of your db call.
The console.log(usersList); happens BEFORE the db call, so it is empty.
You must adapt your code like this and post the code AFTER the db execution:
app.post('/submitLogin', function (req, res) {
getUsersGroups().then((data) => {
console.log(data)
res.send(data.usersList)
})
})
function getUsersGroups () {
const users = new Promise((resolve, reject) => {
dbConnection
.getUsers()
.then(data => {
resolve(data)
})
})
const groups = new Promise((resolve, reject) => {
dbConnection
.getGroups()
.then(data => {
resolve(data)
})
})
return Promise.all([users, groups])
.then(data => {
console.log(data[0])
return {
usersList: data[0],
groupsList: data[1]
}
})
}
I strongly suggest to don't change a global variable like usersList (that I removed from my example)
because if you receive two request contemporaneously, the second one could overwrite the data of the first one and
cause many side effect.
app.post('/submitLogin', async function(req,res) {
await getUsersGroups();
console.log(usersList);
});
try this
I'm writing an API using Node.js Express framework, which makes requests to some other external APIs. I need to have logs of each request in database (I use MongoDB via mongoose).
The problem is that when I'm trying to push log to array of sub documents by mongoose Update method, sometimes it saves in database and sometimes not, in both cases without any error.
Here's some parts of executing code:
// externalApiCtrl module
const request = require('request');
const mongoose = require('mongoose');
const httpContext = require('express-http-context');
const PolicyLog = mongoose.model('PolicyLog');
const updatePolicyLog = (id, log) => {
return new Promise((resolve, reject) => {
PolicyLog.update({ "policyId": mongoose.Types.ObjectId(id) },
{ $push: { logs: log } }
).then(() => {
resolve();
}).catch(err => {
reject(err);
});
});
};
exports.createPolicy = (policy) => {
return new Promise((resolve, reject) => {
// prepare request body and do some other stuff here
let options = {}; // request options (url, method, headers & body)
request(options, (error, response, body) => {
if (error)
reject(error);
let policyLocalId = httpContext.get("policyLocalId");
// here comes the trouble
updatePolicyLog(policyLocalId, {
method: "reqName",
request: "reqBody",
response: body
}).then(() => {
resolve();
}).catch(err => {
return reject(err)
});
});
});
};
// Main controller module
exports.create = (req, res) => {
let externalApiCtrl = require('./controllers/external-api.controller.js');
externalApiCtrl.createPolicy(req.policy)
.then(result => {
return res.json(result);
}).catch(err => {
return res.status(501).json(err);
});
};
So when updatePolicyLog function is called, looks like sometimes it resolves before mongoose Update method (PolicyLog.update()) is resolved.
Is there any way of solving such issue?
Finally I found out the solution.
The problem was not with mongoose, but with express-http-context module. Sometimes during multiple parallel requests to API, the context was lost and the value of policyLocalId variable was undefined, so mongoose did not update the logs.
It happened on Node boron lts (v.6.16.0), updating it to carbon lts (v8.15.0) fixed the problem.