I successfully wrote app in node.js working with http.get. Problem is, that if page doesn't exist, it make an error that terminates the app. Is any way how to declare timeout. After timeout it should stop waiting for response and let app continue (if written synchronously)
Thanks for all advices...
I'm responding thinking that you are trying to retrieve info via http.get.
You can see the validate using the response status code.
http.get(url, function(res) {
if (res.statusCode !== 200) {
Your custom handler
} else {
var body = '';
res.on('data', function(chunk) {
body += chunk;
});
res.on('end', function() {
console.log(body);
});
}
}).on('error', function(e) {
console.log("error: ", e);
});
A fully implemented example can be found her https://gist.github.com/pasupulaphani/9630789
You can set timeout and emit an error/abort if the request takes too long but this has to be done in application logic.
Found a better/sophisticated solution to handle timeouts in other post :
https://stackoverflow.com/a/12815321/2073176
Related
I am performing some http calls in an AWS lambda. The number of calls are ~400 per minute.
The calls are performed as in the following snippet
var req = http.get("https://www.google.com", res => {
let body = '';
res.on('data', chunk => {
body += chunk;
});
res.on('end', chunk => {
if (body.includes('html')) {
console.log('Got back healthy response');
} else {
console.log('Got an unexpected response');
}
})
});
req.on('error', e => {
console.log('Got an error response');
})
That is a simple https request. When the Lambda is invoked, it performs ~40 requests in a single execution.
My issue is that at the beginning, everything looks good and all the requests are performed correctly. After a while (that can be after ~30 min) calls start to degrade and I get back "socket hang up ECONNRESET" error.
I then tried using the request module and change the code with the following
const request = require('request');
request("https://www.google.com", function (error, response, body) {
if (!error && response.statusCode == 200 && body.includes('html')) {
console.log('Got back healthy response' );
} else {
console.log('Got an unexpected response');
console.log('Error: ' + error);
console.log('Response: ' + response);
console.log('Body: ' + body);
}
});
In this case, with the same number of requests within the same lambda, same setting I never get the ECONNRESET error.
I'm ok using the request module but I'm curious to know why this was happening with the default http implementation.
Is this caused by socket allocation that the request module handles in a more appropriate way?
I know similar questions have been asked already but I didn't find a good answer for my case.
This isn't really an answer but I cannot write comments.
The main difference I can see is the encoding. The default encoding in request is utf8, whereas in the http module it's buffer. Adding res.setEncoding('utf8'); might be helpful. It might not be faster. In the line body += chunk you just implicitly convert a Buffer to a string so it should be the same.
If adding the setEncoding won't change anything then you might want to report an issue to the nodejs team because it's the same as the example in http_http_get_url_options_callback. They should fix it or change the example.
I have a consistent problem with using the request module where, regardless of the URL I provide, I am getting a socket hang up error, generally a connection reset. What makes this more strange is that this code works on another developers machine without problem.
var request = require("request");
request("http://google.com", function(error, response, body) {
console.log(error);
console.log(response);
if (!error && response.statusCode === 200) {
}
});
This is a simplified version of the code but it illustrates the point. If I do something like this though
var http = require('http');
var options = {
host: 'www.google.com',
port: 80
};
http.get(options, function (resp) {
var bdy = "";
resp.on('data', function (chunk) {
bdy = bdy + chunk;
});
resp.on('end', function () {
//var r = JSON.parse(bdy);
console.log(bdy);
});
}).on("error", function (e) {
console.log("Got error: " + e.message);
});
I get a response back from Google as I would expect. What is strange is that both the request module and another module I am using (weather-js) exhibiting the same behavior: all requests result in some sort of socket error. Additionally, when I run Fiddler I can see the http.get request go out, but I never see an entry when the code from the request or weather-js module runs.
I am running Node 10.5.0 on Windows 10.
I have the following backend http endpoint implemented with NodeJS and expressJS
app.get("/api/stocks/lookup/:qry", function(req, res) {
getJson(lookupSearch(req.params.qry), function(json) {
var quotes = [];
und.forEach(json, function(d) {
getJson(quoteSearch(d.Symbol), function(j) {
quotes.push(j);
if (json.length == quotes.length) {
res.send(quotes);
}
});
});
});
});
var getJson = function(search, cb) {
http.request(search, function(response) {
var raw = '';
response.on('data', function(d) {
raw += d;
});
response.on('end', function() {
cb(JSON.parse(raw));
});
response.on('error', function(err) {
console.error(err);
});
}).end();
};
What I do not understand is why this only works sometimes and why other times I get this error message from curl: curl: (52) Empty reply from server My solution that checks the lengths of json and quotes seems correct to me and so I think I must have misused some library. However, as I am new to asynchronous code (the http request in getJson) I am not 100% sure this is correct.
I asked a question with similar code here: syntax and methods for placing callbacks in nodejs Note that the 'homebrewed' solution with incrementing and decrementing a count only works sometimes as well. I do not want to use JQuery inside node either.
Why does the above http request only work sometimes?
I try to put many callbacks in a callback, but the program will shut down after the return of several success requests and post "socket hang up" error. I try to collect data from response and output them at once, can someone tell me which part goes wrong...
By the way, I hide the details of request method, I promise the request method works on http call.
http.request(options1,function(data){
var condition=data.length;
var result=[];
data.foreach(item,function(data){
http.request(options2, function(data){
if(data) result.push(data);
condition--;
if(condition<=0) console.log(result);
}
});
});
for my http.request method
var request=function(options,callback){
http.request(options,function(res){
var body;
res.on('data',function(chunk){
body+=chunk;
});
res.on('end',function(){
callback(JSON.parse(body));
});
request.end();
};
That's not the correct usage of http.request().
The http.request() callback is passed an IncomingMessage object, not buffered response data.
EDIT: Your custom request() should look something like this, although you should handle errors too. Note the proper placement of request.end() and initializing var body = '':
function request(options, callback) {
http.request(options,function(res) {
var body = '';
res.setEncoding('utf8');
res.on('data',function(chunk) {
body += chunk;
}).on('end',function() {
callback(JSON.parse(body));
});
}).end();
}
You're missing .end() for your requests so that node.js knows you are ready to actually send the HTTP request: http.request(..., ....).end();. This is the cause of your particular error... the server hangs up the connection because it got tired of waiting for your request after the TCP connection was opened.
I'm trying to read a PDF from a URL and display it to a user's browser (via the passed in 'response' object). I've tried to use the code below and it works sometimes, but generally fails:
function writePdfToBrowser(url, response) {
http.get(url, function(res) {
logger.verbose('about to start download...');
var chunks = [];
res.on('data', function(chunk) {
chunks.push(chunk);
});
res.on("end", function() {
logger.verbose('downloaded');
var buffer = new Buffer.concat(chunks);
//write downloaded pdf to the original response
response.write(buffer);
//response.send(buffer);
response.end();
});
}).on("error", function() {
logger.error("error!");
});
}
In the new page where I attempted to load the pdf it would just say "Failed to load pdf".
I'm new to Node, so not sure where the problem lies, any ideas? Anyone have any working code to do the same thing?
Thank you for any help!
Mark
Use piping:
function pipe(url, res) {
var request = http.get(url, function(response) {
res.writeHead(response.statusCode, response.headers)
response.pipe(res);
});
request.on('error', function(error){
res.statusCode = 500;
res.end(error.message);
});
}
... and please provide next time more information about what and how it fails, some logs, inspect response im browser before. And so on..