I'm doing a web application that after posting some tweets it return informations like number of retweets etc.
The problem I'm having is that I'm unable to store the data returned by the twitter api.
The twitter function to load the data from the API is:
exports.loadTwit = function(){
return function(req,res,next){
let responseDataObject = [];
req.twitter.forEach(function(entry) {
T.get('statuses/show/'+entry.id,loaded);
});
async function loaded(error, data, response){
if (error) {
console.log(error);
}else{
dati = await data;
responseDataObject.push(dati);
console.log(responseDataObject); // shows the data
}
}
console.log(responseDataObject); // shows []
res.render('db',{dati: responseDataObject});
}
}
I want to store the data in the responseDataObject. I've put the async/await because I think it's asyncronous so I have to wait for the data.
The problem, showed in the comments, is that inside the loaded function I can see the content of responseDataObject, outside it gives me an empty array.
I don't know how to solve it, any help is appreciated.
Thanks
Related
I am working on building a blog API for a practice project, but am using the data from an external API. (There is no authorization required, I am using the JSON data at permission of the developer)
The idea is that the user can enter multiple topic parameters into my API. Then, I make individual requests to the external API for the requested info.
For each topic query, I would like to:
Get the appropriate data from the external API based on the params entered (using a GET request to the URL)
Add the response data to my own array that will be displayed at the end.
Check if each object already exists in the array (to avoid duplicates).
res.send the array.
My main problem I think has to do with understanding the scope and also promises in Axios. I have tried to read up on the concept of promise based requests but I can't seem to understand how to apply this to my code.
I know my code is an overall mess, but if anybody could explain how I can extract the data from the Axios function, I think it could help me get the ball rolling again.
Sorry if this is a super low-level or obvious question - I am self-taught and am still very much a newbie!~ (my code is a pretty big mess right now haha)
Here is a screenshot of the bit of code I need to fix:
router.get('/:tagQuery', function(req, res){
const tagString = req.params.tagQuery;
const tagArray = tagString.split(',');
router.get('/:tag', function(req, res){
const tagString = req.params.tag;
const tagArray = queryString.split(',');
const displayPosts = tagArray.map(function(topic){
const baseUrl = "https://info.io/api/blog/posts";
return axios
.get(baseUrl, {
params: {
tag: tag
}
})
.then(function(response) {
const responseData = response.data.posts;
if (tag === (tagArray[0])){
const responseData = response.data.posts;
displayPosts.push(responseData);
} else {
responseData.forEach(function(post){
// I will write function to check if post already exists in responseData array. Else, add to array
}); // End if/then
})
.catch(function(err) {
console.log(err.message);
}); // End Axios
}); // End Map Function
res.send(displayPosts);
});
Node.js is a single thread non-blocking, and according to your code you will respond with the result before you fetching the data.
you are using .map which will fetch n queries.
use Promise.all to fetch all the requests || Promise.allsettled.
after that inside the .then of Promise.all || promise.allsettled, map your result.
after that respond with the mapped data to the user
router.get('/:tag', function (req, res) {
const tagString = req.params.tag;
const tagArray = queryString.split(',');
const baseUrl = "https://info.io/api/blog/posts";
const topicsPromises=tagArray.map((tobic)=>{
return axios
.get(baseUrl, {
params: {
tag: tag
}
})
});
Promise.all(topicsPromises).then(topicsArr=>{
//all the data have been fetched successfully
// loop through the array and handle your business logic for each topic
//send the required data to the user using res.send()
}).catch(err=>{
// error while fetching the data
});
});
your code will be something like this.
note: read first in promise.all and how it is working.
My code is
main.post('/userlist', function(req, res, next) {
// where did you get this?
// var db = req.db;
var query = connection.query("SELECT name,zaman,giriscikis FROM giriscikis where date_format(zaman,'%Y-%m-%d') between ? and ?", [req.body.bas, req.body.bitis], function(err, result) {
if (err) throw err;
console.log(result);
res.send(httpResponse(result));
return res.redirect('/zamansorgu');
//console.log(req.body.bas)
//console.log(req.body.bitis)
});
});
I want to fecth data from database and redirect to same page in the code(zamansorgu.html)
But I get an error
Cannot set headers after they are sent to the client
How can I solve this problem
thank you for your helps
You are attempting to send back JSON data and redirect to a different page. That's not possible. Each endpoint request can have one response, not more. You can either send back the data, or redirect. That's because redirecting really does send back data too (the html of the new target page).
Think about it from the caller's point of view. If it did allow this how would it work? If someone uses this link from a browser should the browser show the JSON data you returned, or should it take the user to the new page?
The error is saying "hey, I already sent back data. I can't redirect now because we are already down the path of returning some JSON".
If you want to use the data to format the output that can be done, or if you want to redirect to a new location and pass the data in the url, that's also possible. Consider code like this:
main.post('/userlist', function(req, res, next) {
// var db = req.db;
var query = connection.query("SELECT name,zaman,giriscikis FROM giriscikis where date_format(zaman,'%Y-%m-%d') between ? and ?", [req.body.bas, req.body.bitis], function(err, result) {
if (err) return next(err);
if (result.urlparam) {
// this builds a new url using the query value
const nextUrl = `/zamansorgu?param=${result.urlparam}`;
return res.redirect(nextUrl);
}
else {
// this builds html here
const html = `<html><body><h1>${result.title}</h1></body></html>`;
return res.send(html);
}
});
});
I also ran into this, in my case it was quite a deceptive little bug. A node-inspector session helped me pinpoint the problem quickly however. The problem in my case was pretty bone-headed, the res.end call in the sample below is the offending line.
res.writeHead(200, {"Content-Type": "application/json"});
res.end(JSON.stringify(someObject));
someObject did not exist after a refactor and that was causing a ReferenceError to get thrown. There is a try-catch in Router.prototype._dispatch that is catching the ReferenceError and passing it into next
res.status(301).redirect(`/zamansorgu?name=${"John"}&email=${"john#email.com"}`)
So, this is something I explored but it will be dependent on the structure of your application. You could always pull the data out using query params and hydrate your application.
I am trying to learn Angular/node & express. I am currently attempting to write an endpoint for my backed that
Needs to use axios to run a get request to an external API that will return a list of IDs (numbers)
For each of those IDs supplied run a request that gets the details based off that id.
But my question is in regards to number 2. I have a list of IDs as input, and I'd like to run a request to an external API (using axios) based on each of these IDs. It may be helpful to note - The request to an external API based on the ID returns an object with details of that ID so my overall goal with this endpoint of my API is to return an array of objects, where each object contains the details of the IDs.
There have been a couple questions similar to mine...
Pushing responses of axios request into array (This one is very similiar to my question)
Axios random number of requests
However, they are using React.js, and I am having difficulties adapting their solution to node/express.
I am trying to model my approach based off of the code snippet provided in the top asnwer of the first question. But, my solution is returning an empty object as a response.
My question: What can I do differently to make multiple axios GET requests to an external API where each request is created dynamically
app.route('/test').get((req, res) => {
axios
.get('https://www.thecocktaildb.com/api/json/v1/1/filter.php?i=vodka')//This will not be hardcoded, but grabbed as a parameter from the endpoint
.then(function(response) {
//Purpose of this initial .then() clause is to make a call to the cocktaildb API to get the IDs of all the cocktails with a given ingredient eg: vodka.
var data = response.data;//response.data contains the JSON object containing the contents received by the cocktaildb API request.
var cocktailIds = [];
//collect all cocktail ids so we can later make more requests to obtain the details associated with that ID.
data.drinks.forEach(drink => {
cocktailIds.push(drink['idDrink']);
});
//this is passed on to the next then clause. It is a list of cocktail ids.
return cocktailIds;
})
.then((drinks) => {
//the promises variable contains a list of all the requests we will have to make in order to get the details of all cocktail ids. I have tested that they are valid requests.
const promises = drinks.map(id => {
//console.log(getCocktailDetailsUrl + id);
return axios.get(getCocktailDetailsUrl + id)
.then(({data}) => {
return data;
})
})
//I was hoping Promise.All to execute all of the requests in the promise and response to be stored in the cocktailDetails variable
const cocktailDetails = Promise.all(promises)
.then(values => {
return values;
})
.catch(error => {
console.log("There was an error when sending requests for details of all cocktails");
console.log(error);
})
//Sending response only formatted this way for testing purposes
if(cocktailDetails) {
//this block is executed, and an empty object is returned as response
console.log("cocktails was sent as response");
res.send(cocktailDetails);
} else {
console.log("cocktails was not sent as response");
res.send("cocktailDetails was not poppulated at the time of sending response");
}
})
.catch(function (error) {
res.send("There was an iswsue with your request to the cocktaildb API.");
console.log('The following is the error from the request to the cocktaildb API: ' + error);
})
});
As I mentioned before, my response contains an empty object. I know I must use promise.all somehow, but I am not sure how to implement it properly.
Your issue is that you aren't waiting for all the promises to resolve before sending the response. Additional critique, your code can be dramatically simplified to the following:
app.route('/test').get((req, res) => {
axios.get('https://www.thecocktaildb.com/api/json/v1/1/filter.php?i=vodka')//This will not be hardcoded, but grabbed as a parameter from the endpoint
.then(function(response) {
const promises = response.data.drinks.map(drink =>axios.get(getCocktailDetailsUrl + drink.idDrink).then(({data})=>data));
//I was hoping Promise.All to execute all of the requests in the promise and response to be stored in the cocktailDetails variable
return Promise.all(promises)
.then(values => {
res.send(values);// shouldn't you be using res.json() here?
});//if an error is thrown for any reason here, it will be caught by your outer catch
})
.catch(function (error) {
res.send("There was an iswsue with your request to the cocktaildb API.");
console.log('The following is the error from the request to the cocktaildb API: ' + error);
});
});
You should wait for the promise before trying to send a response.
Add await before Promise.all and you don't need the then block
const cocktailDetails = await Promise.all(promises)
I am using Meteor in one of my project where i need to simply show the data on meteor template say inventory.html. I have a method at server side which hit a query and get the data from mongoDb . I call the server side method from client side and getting all the data at client side as well but as the data coming to client side takes some time on the mean time the template is rendered without any values so how would i show the values in the template or there is any technique so that i can show my values or data.
Currently i am using a service to setting and getting data i.e testService.
var testservice = require('./TestService');
Template.allInventory.rendered = function() {
Template.allInventory.helpers({
productDetails: function() {
return testservice.getData();
}
})
}
channels.js where i am setting the data coming from DB
var testservice = require('./TestService');
Template.channels.rendered = function(){
Meteor.call('getDetialsFromDB',function(err, res){
if(res){
console.log("res is...getAllProductDetail.."+res);
testservice.setData(res);
}
if(err){
console.log("Error while calling getAllProductDetail");
}
})
if i call the above method without using the service than its render the template without any data because data coming from backend taking some time like:
Template.allInventory.rendered = function() {
Template.allInventory.helpers({
productDetails: function() {
var data;
Meteor.call('getDetialsFromDB',function(err, res){
if(res){
console.log("res is...getAllProductDetail.."+res);
data = res;
}
if(err){
console.log("Error while calling getAllProductDetail");
}
})
return res;
}
})
so i just want to render my data coming from one of Meteor .method of server side calling from client side or please give any example
Any help would be appriciated!
Thanks
You can use SSR Package for server side rendering.
Template.someTemplate.onCreated(function(){
this.someVar = new ReactiveVar(null)
let self = this
Meteor.call('someMethod',function(err,res){
if(!err){
self.someVar.set(res)
}
})
})
We captured the result of meteor method into reactive variable so as to execute helper function until the method retrieves the result
now make helpers to get the result of the meteor method
Template.someTemplate.helpers({
getSomeVar:function(){
if(Template.instance().someVar.get())
return Template.instance().someVar.get()
}
})
I am using node.js for server side development and backbone.js for client side development. i want to fetch data from multiple table(more than 3) by sending only one request to node.js. but i cant merge all that result with each other beacuse of asynchronous execution of node.js. i have done this but it sending a lots of get request to node js for getting data from all the tables and because of these performance of my site is become slower. please help me if anyone having any idea.
I would create a method which aggregates the results from each of the requests and sends the response back. Basically each of your three async db calls would pass their data to the same method. That method would check to see if it had all of the data it needed to complete the request, and if it did, send the response.
Here is a pseudo code example:
function handleRequest(req, res) {
var results = {};
db.getUsers(function(data) {
aggregate('users', data);
});
db.getPosts(function(data) {
aggregate('posts', data);
});
db.getComments(function(data) {
aggregate('comments', data);
});
function aggregate(name, data) {
results[name] = data;
if(results.users && results.posts && results.comments) {
res.send(results);
}
}
}
This is simplified greatly, you should also of course check for errors and timeouts to the db calls, but this will allow you to wait for all the async commands to complete before sending the data.