To learn node.js I'm creating a small app that get some rss feeds stored in mongoDB, process them and create a single feed (ordered by date) from these ones.
It parses a list of ~50 rss feeds, with ~1000 blog items, so it's quite long to parse the whole, so I put the following req.connection.setTimeout(60*1000); to get a long enough time out to fetch and parse all the feeds.
Everything runs quite fine, but the request is called twice. (I checked with wireshark, I don't think it's about favicon here).
I really don't get it.
You can test yourself here : http://mighty-springs-9162.herokuapp.com/feed/mde/20 (it should create a rss feed with the last 20 articles about "mde").
The code is here: https://github.com/xseignard/rss-unify
And if we focus on the interesting bits :
I have a route defined like this : app.get('/feed/:name/:size?', topics.getFeed);
And the topics.getFeed is like this :
function getFeed(req, res) {
// 1 minute timeout to get enough time for the request to be processed
req.connection.setTimeout(60*1000);
var name = req.params.name;
var callback = function(err, topic) {
// if the topic has been found
if (topic) {
// aggregate the corresponding feeds
rssAggregator.aggregate(topic, function(err, rssFeed) {
if (err) {
res.status(500).send({error: 'Error while creating feed'});
}
else {
res.send(rssFeed);
}
},
req);
}
else {
res.status(404).send({error: 'Topic not found'});
}};
// look for the topic in the db
findTopicByName(name, callback);
}
So nothing fancy, but still, this getFeed function is called twice.
What's wrong there? Any idea?
This annoyed me for a long time. It's most likely the Firebug extension which is sending a duplicate of each GET request in the background. Try turning off Firebug to make sure that's not the issue.
I faced the same issue while using Google Cloud Functions Framework (which uses express to handle requests) on my local machine. Each fetch request (in browser console and within web page) made resulted in two requests to the server. The issue was related to CORS (because I was using different ports), Chrome made a OPTIONS method call before the actual call. Since OPTIONS method was not necessary in my code, I used an if-statement to return an empty response.
if(req.method == "OPTIONS"){
res.set('Access-Control-Allow-Origin', '*');
res.set('Access-Control-Allow-Headers', 'Content-Type');
res.status(204).send('');
}
Spent nearly 3hrs banging my head. Thanks to user105279's answer for hinting this.
If you have favicon on your site, remove it and try again. If your problem resolved, refactor your favicon url
I'm doing more or less the same thing now, and noticed the same thing.
I'm testing my server by entering the api address in chrome like this:
http://127.0.0.1:1337/links/1
my Node.js server is then responding with a json object depending on the id.
I set up a console log in the get method and noticed that when I change the id in the address bar of chrome it sends a request (before hitting enter to actually send the request) and the server accepts another request after I actually hit enter. This happens with and without having the chrome dev console open.
IE 11 doesn't seem to work in the same way but I don't have Firefox installed right now.
Hope that helps someone even if this was a kind of old thread :)
/J
I am to fix with listen.setTimeout and axios.defaults.timeout = 36000000
Node js
var timeout = require('connect-timeout'); //express v4
//in cors putting options response code for 200 and pre flight to false
app.use(cors({ preflightContinue: false, optionsSuccessStatus: 200 }));
//to put this middleaware in final of middleawares
app.use(timeout(36000000)); //10min
app.use((req, res, next) => {
if (!req.timedout) next();
});
var listen = app.listen(3333, () => console.log('running'));
listen.setTimeout(36000000); //10min
React
import axios from 'axios';
axios.defaults.timeout = 36000000;//10min
After of 2 days trying
you might have to increase the timeout even more. I haven't seen the express source but it just sounds on timeout, it retries.
Ensure you give res.send(); The axios call expects a value from the server and hence sends back a call request after 120 seconds.
I had the same issue doing this with Express 4. I believe it has to do with how it resolves request params. The solution is to ensure your params are resolved by for example checking them in an if block:
app.get('/:conversation', (req, res) => {
let url = req.params.conversation;
//Only handle request when params have resolved
if (url) {
res.redirect(301, 'http://'+ url + '.com')
}
})
In my case, my Axios POST requests were received twice by Express, the first one without body, the second one with the correct payload. The same request sent from Postman only received once correctly. It turned out that Express was run on a different port so my requests were cross origin. This caused Chrome to sent a preflight OPTION method request to the same url (the POST url) and my app.all routing in Express processed that one too.
app.all('/api/:cmd', require('./api.js'));
Separating POST from OPTIONS solved the issue:
app.post('/api/:cmd', require('./api.js'));
app.options('/', (req, res) => res.send());
I met the same problem. Then I tried to add return, it didn't work. But it works when I add return res.redirect('/path');
I had the same problem. Then I opened the Chrome dev tools and found out that the favicon.ico was requested from my Express.js application. I needed to fix the way how I registered the middleware.
Screenshot of Chrome dev tools
I also had double requests. In my case it was the forwarding from http to https protocol. You can check if that's the case by looking comparing
req.headers['x-forwarded-proto']
It will either be 'http' or 'https'.
I could fix my issue simply by adjusting the order in which my middlewares trigger.
Related
I am using AXIOS in a web client in order to POST a file to the express backend as an upload. Since both the file's size and the client user's bandwidth is variable, it may take a certain amount of time for the POST request to finish. On the backend some logic applies and the request is promptly rejected.
The problem is that the client receives the response only after the request is finished, which can be several seconds.
I have already tested that it is not the backend's fault, as the behavior is the same when POSTing to any arbitrary post-enabled url in the web, regardless of the technology. Here is an (over)simplified example of the case.
Here's the post action. Notice the commended request to the arbitrary post-enabled url. It behaves exactly the same:
try{
console.log("posting....")
const res = await axios.post("http://localhost:4000/upload", formData)
// const res = await axios.post("https://github.com/logout", formData)
console.log("result:")
console.log(res)
}catch(err){
console.error(err)
}
And the demo express backend route:
app.post("/upload", (req, res) => {
console.log("Rejecting...")
res.status(403).send()
console.log("Rejected.")
return
})
For testing purposes I choose a 3.7Mb file, and throttle down my browsers bandwidth to the Fast 3G preset.
The backend immediately outputs:
Rejecting...
Rejected.
Whereas the request is pending for about 43 seconds before returning the 403 error:
Am I missing something obvious here? It is such a common functionality, it makes me doubt that this is the correct way to be handled. And if it really is, do we have any information on whether express's thread is active during that time, or is it just a client inconvenience?
Thanks in advance!
I believe you could just use res.status(403) rather than res.status(403).send() .
You could also try using res.status(403).end() and I am not sure why you should use a return statement in the router part.
It seems that first sending the response headers and then manually destroying the request does the trick:
app.post("/upload", (req, res) => {
console.log("Rejecting...")
res.status(403).send("Some message")
return req.destroy()
})
The AXIOS request stays pending until just the current chunk is uploaded, and then immediately results in the correct status and message. In the throttled down fast 3g example, pending time went down from 43s to 900ms.
Also, this solution emerged through trial and error, so it can possibly not be the best practice.
I would still be interested in an AXIOS oriented solution, if one exists.
I would like to handle both POST and GET requests as a single request, such that all of my routings and subsequent functions only need to process a single request, rather than duplicating everything once for GET and again for POST.
So I figure the simplest way of doing this is to convert a POST to a GET early on using middleware, is there any problem with this ?
if(req.method=='POST'){
req.method = 'GET';
req.query = req.body;
delete(req.body);
}
You can have the same handler function for the both requests i.e
app.get('/', handlerFunction);
app.post('/', handlerFunction);
You can have express respond to all POST requests as 302 redirects to the same URL (these are always GET requests).
Here's some sample code:
// Redirect all post requests
app.post('^*$', function(req, res) {
// Now just issue the same request again, this time as a GET
res.redirect(302, req.url);
});
});
Side note: this will work but I wouldn't recommend this as a long term solution. If you decide you do need to handle POST requests differently from GET requests and the maintainability will become a pain. In the long run, you're better off having a clear definition for how to handle POST and GET requests rather than treating them the same.
Recommended approach is to have the same handler function for both in this case. for eg.
app.get('/path', handler);
app.post('/path', handler);
I am turning around in stackoverflow without finding an answer to my question. I have used expressJS fur several days in order to make an access webpage that returns first an interstitial and then a webpage depending on several informations I can get from the requester IP and so on.
My first idea for the interstitial was to use this piece of code:
var interstitial = function(req, res, next) {
res.render('interstitial');
next();
}
router.get('/', interstitial, nextPage);
setting a timeout on the next nextPage callback function of router.get().
However it looks that I could not do that. I had an error "Error: Can't set headers after they are sent.". I suppose this is due to the fact that res.render already give a response to the request and in the philosophy of express, the next function is passing the req, res args for another reply to another function that possibly could do it. Am I right?
In that case, is there a way to give several answer, with timeout to one request? (a res.render, and after that in the next callback a rest.send...).
Or is this mandatory to force client to ask a request to give back another response? (using js on the client side for instance, or timers on client side, or maybe discussing with client script using socket.io).
Thanks
Not sure I fully understand, but you should be placing all your deterministic logic within the function of the handler you're using for your endpoint.
Kinda like so:
router.get('/', function(req, res){
var origin = request.origin;
if (origin == '11.22.33.44'){
res.send('Interstitial Page.');
}else{
res.send('Home Page');
}
});
You would replace the simple text responses with your actual pages, but the general idea is that once that endpoint is handled you can't next() it to secondary handler.
I'm struggling to pass response time data at the application-level to every route in my Express app. I've Googled and Googled but everything comes back suggesting one throw a new Date() into API calls, which is gross, or the Express response-time package (including many results from here on SO). While I'm sure it's great, I don't understand what purpose it serves other than adding a header seeing as how I can get response times in my browser's dev tools.
What I want to do is access all response time data from the server into the view, on every route.
Express already provides some request data, but not all of it. I just can't seem to access all responses. My terminal looks like this when I load up a basic page, even though I have 6 images on the page besides the one CSS file.
GET / 200 64.439 ms - 1663
GET /styles/index.css 200 36.582 ms - 2035
If I use the response-time package, I can't seem to access the 'X-Response-time' header. req.headers only seems to return a subset of all headers, similar to the Express traffic output mentioned above.
Maybe I'm just dense, but even the docs of the response-time package mention how to configure it with Express, but I still don't understand what it's supposed to be adding or how I would access it outside of my console.
Create a new middleware that records the response time of a request and makes this available to your own function fn. The fn argument will be invoked as fn(req, res, time), where time is a number in milliseconds.
Couldn't you just do this:
var responseTime = require('response-time')
app.use(responseTime(function(req, res, time) {
res.header('X-Response-Time', time);
}));
Now every route below this will have a response time header on it.
The response-time package will add the X-Response-Time header at the same time as you send the request:
app.use(responseTime());
app.get('/', function(req, res) {
console.log(res.get('X-Response-Time')); // undefined
res.send('Hello');
console.log(res.get('X-Response-Time')); // 3.720ms
});
It does this by listening out for when headers are about to be written (e.g. when a request is about to be sent), appending the response time just before (using the on-headers package on npm).
It sounds like what you want to do is record the response time, then set it into res.locals so you can render it in your view.
I've played around for a while trying to use the on-headers package to set the response time inside res.locals, but I think it must be too late by then to write to the response... Maybe this is the right path? Sorry I don't have more time right now to play around :(
I need to monitor my application's uptime via New Relic. I'm using the Restify framework which is largely based on Express.
New Relic wants to make HEAD requests to my application, but I'm not sure how to set up a HEAD route correctly to satisfy New Relic. Currently, my Restify app returns a 405 error for "Method Not Allowed", which causes New Relic to have fits and send me non-stop emails about how my application is down, and I can't find any documentation from New Relic that shows how to set up a simple ping URL to satisfy them.
Is there anything I need to do other than this:
server.head('/ping', function(error, req, res) {
res.send("hello");
});
EDIT:
The parameters are mislabeled so the res.send() is actually trying to call next().send() which would be undefined. Removing the error parameter and shifting everything over fixed the code as discovered by the OP.
As per the restify documentation, you need to call return next() in your callback function:
http://mcavage.me/node-restify/#Routing
server.head('/ping', function (req, res) {
res.send('hello');
});
If you would like to respond immediately and not continue down the chain, you can pass false as a parameter in your call to next()