How to: GET Data from 2 APIs, compare, POST bool - node.js

I'm working on a project that requires me to:
GET IDs from API1, push the IDs into an array, then map over those IDs, using them for a second GET request, where IDs are used as params for API2 GET request, populates an array with IDs or N for "Not existing" -- this array is then called in:
A POST request. This post maps over the returned array from the GET request. IF the item is not "N", it POSTS to API1 with checked: true. IF the item is "N", it emails us telling us API2 is missing this project.
I want this system to automatically do a GET and POST every 2 hours, so I'm using setInterval (not sure this is the best idea). EDIT: Cron job would be a better solution.
I'm working with NodeJS, Express, Request-Promise, Async / Await.
Here is some of my pseudo code so far:
// Dependencies
const express = require('express');
const axios = require('axios');
const mailgun = require('mailgun-js')({ apiKey, domain });
// Static
const app = express();
app.get('/', (req, res, next) => {
// Replace setInterval with Cron job in deployment
// Get All Ids
const orders = await getGCloud();
// Check if IDs exist in other API
const validations = await getProjectManagementSystem(orders);
// If they exist, POST update to check, else, mailer
validations.map(id => {
if (id !== 'n') {
postGCloud(id);
} else {
mailer(id);
}
});
}
// Method gets all IDs
const getGCloud = async () => {
try {
let orders = [];
const response = await axios.get('gCloudURL');
for (let key in response) {
orders.push(response.key);
}
return orders;
} catch (error) {
console.log('Error: ', error);
}
}
// Method does a GET requst for each ID
const getProjectManagementSystem = async orders => {
try {
let idArr = [];
orders.map(id => {
let response = await axios.get(`projectManagementSystemURL/${id}`);
response === '404' ? idArr.push('n') : idArr.push(response)
})
return idArr;
} catch (error) {
console.log('Error: ', error);
}
}
const postGCloud = id => {
axios.post('/gcloudURL', {
id,
checked: true
})
.then(res => console.log(res))
.catch(err => console.log(err))
}
const mailer = id => {
const data = {
from: 'TESTER <test#test.com>',
to: 'customerSuppoer#test.com',
subject: `Missing Order: ${id}`,
text: `Our Project Management System is missing ${id}. Please contact client.`
}
mailgun.messages().send(data, (err, body) => {
if (err) {
console.log('Error: ', err)
} else {
console.log('Body: ', body);
}
});
}
app.listen(6000, () => console.log('LISTENING ON 6000'));
The TL;DR: Need to do a GET request to API 1, then another GET request to API 2 following it (using IDs from API 1 as params), then send data from second GET to a POST request that then either updates API 1's data or emails Customer support. This is an automatic system that runs every two hours.
Main Questions:
1. Is it okay to have a setInterval in a get req?
2. Can I have a GET request automatically call a POST request?
3. If so, how can I pass GET request data onto a POST request?

To make it work for both of your calls one post and one get you have to do an Ajax call to get post processed information in another method.
I hope this works.

Related

server crashes after failed axios get request in Node.js

I created a function to find data by ID from api if there is one and create new one if there isn't. But server crashes if the first axios.get() fails with 400 code when data corresponding to the provided ID doesn't exist. Originally had try and catch block around the axios get and post and thought they might be causing the crash but removing them yields same result.
async function findOrCreateDocument(id) {
if (id == null) return;
let response = await axios.get(api + `/${id}`);
if (response.status >= 200 && response.status < 300) return response.data;
response = await axios.post(api, { id: id, data: defaultValue });
document = response.data;
return document;
}
This function is called inside of a socket.io server.
io.on("connection", (socket) => {
socket.on("join-document", async (documentId) => {
const document = await findOrCreateDocument(documentId);
socket.join(documentId);
socket.emit("load-document", document.data);
socket.on("send-text-change", (delta) => socket.broadcast.to(documentId).emit("recieve-text-change", delta));
...
}
}

axios.put is not working when sending query object

I was trying to send put request using axios to update information about a book in mongodb, the code is working fine when using postman but it is not working using axios.put used inside react component when submitting, while axios.delete is working fine. I think the problem is that sending the query object in that way is not the right way but I am not able to find the solution.
This is the function that handleSubmit,
the 'id' is the id of the book
'updates' is a state object that contains all the changes that should happen in the book data.
And the second function handleChange is the function that setupdates according to changes in the inputs
const handleSubmit = async (e) => {
e.preventDefault();
try {
const res = await axios.put("http://localhost:8000/book/edit/" + id, {
params: {
updates,
},
});
setNewBook(res.data)
} catch (err) {
console.log(err);
}
};
const handleChange = (e) => {
e.preventDefault();
const value = e.target.value;
setUpdates({
...updates,
[e.target.name]: value,
});
};
maybe you misunderstood the axios put method. as per the axios docs provided (https://github.com/axios/axios#axiosputurl-data-config) the correct structure is axios.put(url[, data[, config]])
The first parameter is URL
The second parameter is data (body request).
3rd parameter is config (you leave params in this parameter)
It should be like this
const res = await axios.put("http://localhost:8000/book/edit/" + id, null, {
params: {
updates,
},
});
I hope this can help you

Node js repeating a get request until there is a change in response

I will start off by saying I am a complete newbie when it comes to node js. I have the code below which currently sends a get request to the URL. It parses a specific value of the response and stores it as the search variable. It then uses the instagram api to change the bio on my instagram account to that search variable. However I would like the get request to continue until it detects a change. Ex. When the program is first run it fires off a get request. The first response value we get we will call 1. However after the first response I want it to continue to do get requests say every 5 seconds. The moment the response value changes from 1 to anything else I want that new value to be sent to the instagram bio. Can anyone help?
const { IgApiClient } = require("instagram-private-api")
const ig = new IgApiClient()
const https = require('https')
const USERNAME = "MYUSERNAME"
const PASSWORD = "MYPASS"
ig.state.generateDevice(USERNAME)
const main = async () => {
let url = "https://11z.co/_w/14011/selection";
https.get(url,(res) => {
let body = "";
res.on("data", (chunk) => {
body += chunk;
});
res.on("end", async () => {
try {
search = await JSON.parse(body).value;
} catch (error) {
console.error(error.message);
};
});
}).on("error", (error) => {
console.error(error.message);
});
await ig.simulate.preLoginFlow()
await ig.account.login(USERNAME, PASSWORD)
// log out of Instagram when done
process.nextTick(async () => await ig.simulate.postLoginFlow())
// fill in whatever you want your new Instagram bio to be
await ig.account.setBiography(search)
}
main()
// code is written in main() so that I can use async/await
to be good citizen to the target endpoint:
have a look on exponential-backoff package - https://www.npmjs.com/package/exponential-backoff
A utility that allows retrying a function with an exponential delay between attempts.

Why does Async firebase fetching is not working? (NODE JS)

Building a NodeJS REST API.
Trying to send load data from FireBase collection, then sending it to the user (as API response).
Looks like the problem is that it's not waits for the firebase fetch to resolve, but send back a response without the collection data. (tried to use ASYNC-AWAIT but its not working)
exports.getChatMessages = async (req, res, next) => {
const chatId = req.params.chatId
const getChatData = () => {
db
.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc')
.onSnapshot((snapshot) => {
snapshot.docs.forEach(msg => {
console.log(msg.data().messageContent)
return {
authorID: msg.data().authorID,
messageContent: msg.data().messageContent,
timeStamp: msg.data().timeStamp,
}
})
})
}
try {
const chatData = await getChatData()
console.log(chatData)
res.status(200).json({
message: 'Chat Has Found',
chatData: chatData
})
} catch (err) {
if (!err.statusCode) {
err.statusCode(500)
}
next(err)
}
}
As you can see, I've used 2 console.logs to realize what the problem, Terminal logs looks like:
[] (from console.logs(chatData))
All messages (from console.log(msg.data().messageContent))
Is there any way to block the code unti the firebase data realy fetched?
If I correctly understand, you want to send back an array of all the documents present in the messages subcollection. The following should do the trick.
exports.getChatMessages = async (req, res, next) => {
const chatId = req.params.chatId;
const collectionRef = db
.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc');
try {
const chatsQuerySnapshot = await collectionRef.get();
const chatData = [];
chatsQuerySnapshot.forEach((msg) => {
console.log(msg.data().messageContent);
chatData.push({
authorID: msg.data().authorID,
messageContent: msg.data().messageContent,
timeStamp: msg.data().timeStamp,
});
});
console.log(chatData);
res.status(200).json({
message: 'Chat Has Found',
chatData: chatData,
});
} catch (err) {
if (!err.statusCode) {
err.statusCode(500);
}
next(err);
}
};
The asynchronous get() method returns a QuerySnapshot on which you can call forEach() for enumerating all of the documents in the QuerySnapshot.
You can only await a Promise. Currently, getChatData() does not return a Promise, so awaiting it is pointless. You are trying to await a fixed value, so it resolves immediately and jumps to the next line. console.log(chatData) happens. Then, later, your (snapshot) => callback happens, but too late.
const getChatData = () => new Promise(resolve => { // Return a Promise, so it can be awaited
db.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc')
.onSnapshot(resolve) // Equivalent to .onSnapshot((snapshot) => resolve(snapshot))
})
const snapshot = await getChatData();
console.log(snapshot)
// Put your transform logic out of the function that calls the DB. A function should only do one thing if possible : call or transform, not both.
const chatData = snapshot.map(msg => ({
authorID: msg.data().authorID,
messageContent: msg.data().messageContent,
timeStamp: msg.data().timeStamp,
}));
res.status(200).json({
message: 'Chat Has Found',
chatData
})
Right now, getChatData is this (short version):
const getChatData = () => {
db
.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc')
.onSnapshot((snapshot) => {}) // some things inside
}
What that means is that the getChatData function calls some db query, and then returns void (nothing). I bet you'd want to return the db call (hopefully it's a Promise), so that your await does some work for you. Something along the lines of:
const getChatData = async () =>
db
.collection('chats')
// ...
Which is the same as const getChatData = async() => { return db... }
Update: Now that I've reviewed the docs once again, I see that you use onSnapshot, which is meant for updates and can fire multiple times. The first call actually makes a request, but then continues to listen on those updates. Since that seems like a regular request-response, and you want it to happen only once - use .get() docs instead of .onSnapshot(). Otherwise those listeners would stay there and cause troubles. .get() returns a Promise, so the sample fix that I've mentioned above would work perfectly and you don't need to change other pieces of the code.

Node.js backend return response before all the API calls within the endpoints are made

I have a GET endpoint, which basically makes some API calls to the Spoonacular API. Essentially, I make two API calls within the endpoint.
The first API call gets the list of recipe ID's for the specific ingredients
The second API calls gets the metadata for each of the recipe ID's.
After the first API call I store all the Id's in an array (recipeArray), and I want to make the second api call for each ID in my array (function recipeTest does this).
When I try to do this and then return my response to the front end, it always returns a response before completing all the API calls in the second step.
Here, is my code. The first API calls works just fine, but the second API call (recipeTest function), is where it messed up. Before that function finishes making all the API calls to the Spoonacular API, my endpoint returns an empty Array (res.send(toSend)). So, I was just wondering if there is any way around this?
Thank you so much in advance, I really appreciate it!
module.exports = (app) => {
app.get('/api/search', async (req, res) => {
console.log("endpoint working");
let ingredientList = "apples,+eggs,+bacon"; // needs to be given from the front end
let ingredientSearchUrl = `https://api.spoonacular.com/recipes/findByIngredients?ingredients=${ingredientList}&number=1&ignorePantry=true&apiKey=${keys.spoonacularKey}`;
try {
const ingredientSearchResult = await axios({
method: 'get',
url: ingredientSearchUrl
});
var recipeArray = ingredientSearchResult.data.map(info => {
return info.id;
});
} catch (err) {
console.log("error in finding recipe ID ", err);
}
let toSend = [];
try {
const check = await recipeTest(recipeArray, toSend);
} catch (err) {
console.log("error in finding recipe information ", err);
}
res.send(toSend);
});
}
const recipeTest = async (recipeArray, toSend) => {
return Promise.all(
_.forEach(recipeArray, async (recipeId) => {
let recipeInfoUrl = `https://api.spoonacular.com/recipes/${recipeId}/information?includeNutrition=false&apiKey=${keys.spoonacularKey}`;
let recipeInfo = {};
const recipeData = await axios({
method: 'get',
url: recipeInfoUrl
});
// console.log("recipeInfo search working", recipeData.data);
recipeInfo['id'] = recipeData.data.id;
recipeInfo['title'] = recipeData.data.title;
recipeInfo['time'] = recipeData.data.readyInMinutes;
recipeInfo['recipeUrl'] = recipeData.data.sourceUrl;
recipeInfo['imageUrl'] = recipeData.data.image;
// console.log('recipe info dict', recipeInfo);
toSend.push(recipeInfo);
console.log('toSend inside', toSend);
})
);
}
_.forEach return collection itself and not all your async handlers.
Use recipeArray.map to get an array of async functions to let Promise.all do its work:
Promise.all(
recipeArray.map(x => async (recipeId) => {

Resources