I am using node.js v. 0.4.8. When I create a HTTPS server, I never get the response.on('end', ...) event (but I do for a HTTP one). I read the issue reports on node's github page - https://github.com/joyent/node/issues/728, and apparently this is an issue that regressed into 0.4.8. response.on('close', ...) seems to have the same functionality, but I do not know enough about HTTP/HTTPS to judge. Can I use it as replacement for response.on('end', ...), and is this likely to cause any problems in future?
You can see a code sample below.
Thanks in advance!
var request = "JSON_request";
var https = require('https');
var options = {
host:"someHost.com",
path:"somePath",
method: "POST",
headers: {
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(request)
}
};
var req = https.request(options, function(res){
var response = "";
res.setEncoding('utf8');
res.on('data', function(chunk){
console.log("INFO: "+chunk);
response += chunk;
});
// This never happens
res.on('end', function(){
console.log("End received!");
});
// But this does
res.on('close', function(){
console.log("Close received!");
});
});
req.on('error', function(error){
console.log("Error: "+error);
});
req.write(request);
req.end();
I believe this issue has been fixed as of commit de09168, and there hasn't been any action on this question for months BUT here is the answer:
On the comments for issue 728 Node.js creator Ryan Dahl says:
This isn't a bug. Responses are not guaranteed to have an 'end' event. Use 'close'.
Though later he says:
i take back what i said before. we should probably make this work.
Either way it seems close and end are pretty much interoperable in this use case. The relevent code in the node.js core tls.js:681 renforces that interpretation:
process.nextTick(function() {
self.encrypted.emit('end');
self.cleartext.emit('end');
self.encrypted.emit('close');
self.cleartext.emit('close');
});
Since node.js 0.8.12 finish event is emitted on HTTP responses' end function. I wrote all the details in a similar question.
Related
I am performing some http calls in an AWS lambda. The number of calls are ~400 per minute.
The calls are performed as in the following snippet
var req = http.get("https://www.google.com", res => {
let body = '';
res.on('data', chunk => {
body += chunk;
});
res.on('end', chunk => {
if (body.includes('html')) {
console.log('Got back healthy response');
} else {
console.log('Got an unexpected response');
}
})
});
req.on('error', e => {
console.log('Got an error response');
})
That is a simple https request. When the Lambda is invoked, it performs ~40 requests in a single execution.
My issue is that at the beginning, everything looks good and all the requests are performed correctly. After a while (that can be after ~30 min) calls start to degrade and I get back "socket hang up ECONNRESET" error.
I then tried using the request module and change the code with the following
const request = require('request');
request("https://www.google.com", function (error, response, body) {
if (!error && response.statusCode == 200 && body.includes('html')) {
console.log('Got back healthy response' );
} else {
console.log('Got an unexpected response');
console.log('Error: ' + error);
console.log('Response: ' + response);
console.log('Body: ' + body);
}
});
In this case, with the same number of requests within the same lambda, same setting I never get the ECONNRESET error.
I'm ok using the request module but I'm curious to know why this was happening with the default http implementation.
Is this caused by socket allocation that the request module handles in a more appropriate way?
I know similar questions have been asked already but I didn't find a good answer for my case.
This isn't really an answer but I cannot write comments.
The main difference I can see is the encoding. The default encoding in request is utf8, whereas in the http module it's buffer. Adding res.setEncoding('utf8'); might be helpful. It might not be faster. In the line body += chunk you just implicitly convert a Buffer to a string so it should be the same.
If adding the setEncoding won't change anything then you might want to report an issue to the nodejs team because it's the same as the example in http_http_get_url_options_callback. They should fix it or change the example.
I'm aware that there are several questions related to mine, but I didn't find any of them useful:
this one doesn't apply to my case, I'm actually getting the answer, it's the contents that I can't get.
on this one, on the other hand, the problem is a wrong handling of an asynchronous call, which is not my case
there, well, I really didn't fully understand this question
And so on...
Then, I think this is a legitimate question. I'm actually performing some encryption in my server (express routing in node) through a post request:
app.post('/encrypt', encrypt);
Encrypt is doing:
function encrypt(req,res) {
if(req.body.key && req.body.message) {
var encryptedMessage = Encrypter.encrypt(req.body.key,req.body.message);
return res.status(200).json({ message: encryptedMessage });
}
res.status(409).json({ message: 'the message could not be encrypted, no key found' });
}
}
So, I tested this via console.log, and it's working. When the server receives the request, the encrypted message is being generated.
At the same time, I'm testing my thing with mocha and I'm doing it like so:
describe('# Here is where the fun starts ', function () {
/**
* Start and stop the server
*/
before(function () {
server.listen(port);
});
after(function () {
server.close();
});
it('Requesting an encrypted message', function(done) {
var postData = querystring.stringify({
key : key,
message : message
});
var options = {
hostname: hostname,
port: port,
path: '/encrypt',
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'Content-Length': postData.length
}
};
var req = http.request(options, function(res) {
res.statusCode.should.equal(200);
var encryptedMessage = res.message;
encryptedMessage.should.not.equal(message);
done();
});
req.on('error', function(e) {
//I'm aware should.fail doesn't work like this
should.fail('problem with request: ' + e.message);
});
req.write(postData);
req.end();
});
});
So, whenever I execute the tests, it fails with Uncaught TypeError: Cannot read property 'should' of undefined because res.message does not exist.
None of the res.on (data, end, events is working, so I suppose the data should be available from there. First I had this:
var req = http.request(options, function(res) {
res.statusCode.should.equal(200);
var encryptedMessage;
res.on('data', function (chunk) {
console.log('BODY: ' + chunk);
encryptedMessage = chunk.message;
});
encryptedMessage.should.not.equal(message);
done();
});
But res.on was never accessed (the console.log didn't show anything). I'm therefore a bit stuck here. I'm surely doing some basic stuff wrong, but I don't have a clue, and the many questions I found doesn't seem to apply to my case.
Weird enough, if I launch a test server and then I curl it
curl --data "key=secret&message=veryimportantstuffiabsolutellyneedtoprotect" localhost:2409/encrypt
Curl justs waits ad aeternam.
Actually I was doing it properly at the beginning, and the problem was indeed the same than in the second question I mentionned I was actually "clearing" my context with done() before the post data arrived. The solution is:
var req = http.request(options, function(res) {
res.statusCode.should.equal(200);
res.on('data', function(data) {
encryptedMessage = JSON.parse(data).message;
encryptedMessage.should.not.equal(message);
done();
});
});
In such a way that done() is only called when the data has been threated. Otherwise, mocha will not wait for the answer.
I try to put many callbacks in a callback, but the program will shut down after the return of several success requests and post "socket hang up" error. I try to collect data from response and output them at once, can someone tell me which part goes wrong...
By the way, I hide the details of request method, I promise the request method works on http call.
http.request(options1,function(data){
var condition=data.length;
var result=[];
data.foreach(item,function(data){
http.request(options2, function(data){
if(data) result.push(data);
condition--;
if(condition<=0) console.log(result);
}
});
});
for my http.request method
var request=function(options,callback){
http.request(options,function(res){
var body;
res.on('data',function(chunk){
body+=chunk;
});
res.on('end',function(){
callback(JSON.parse(body));
});
request.end();
};
That's not the correct usage of http.request().
The http.request() callback is passed an IncomingMessage object, not buffered response data.
EDIT: Your custom request() should look something like this, although you should handle errors too. Note the proper placement of request.end() and initializing var body = '':
function request(options, callback) {
http.request(options,function(res) {
var body = '';
res.setEncoding('utf8');
res.on('data',function(chunk) {
body += chunk;
}).on('end',function() {
callback(JSON.parse(body));
});
}).end();
}
You're missing .end() for your requests so that node.js knows you are ready to actually send the HTTP request: http.request(..., ....).end();. This is the cause of your particular error... the server hangs up the connection because it got tired of waiting for your request after the TCP connection was opened.
I am trying to learn node.js.
I am trying to understand streams and piping.
Is it possible to pipe the response of http request to console.log?
I know how to do this by binding a handler to the data event but I am more interested in streaming it to the console.
http.get(url, function(response) {
response.pipe(console.log);
response.on('end', function() {
console.log('finished');
});
});
console.log is just a function that pipes the process stream to an output.
Note that the following is example code
console.log = function(d) {
process.stdout.write(d + '\n');
};
Piping to process.stdout does exactly the same thing.
http.get(url, function(response) {
response.pipe(process.stdout);
response.on('end', function() {
console.log('finished');
});
});
Note you can also do
process.stdout.write(response);
I successfully wrote app in node.js working with http.get. Problem is, that if page doesn't exist, it make an error that terminates the app. Is any way how to declare timeout. After timeout it should stop waiting for response and let app continue (if written synchronously)
Thanks for all advices...
I'm responding thinking that you are trying to retrieve info via http.get.
You can see the validate using the response status code.
http.get(url, function(res) {
if (res.statusCode !== 200) {
Your custom handler
} else {
var body = '';
res.on('data', function(chunk) {
body += chunk;
});
res.on('end', function() {
console.log(body);
});
}
}).on('error', function(e) {
console.log("error: ", e);
});
A fully implemented example can be found her https://gist.github.com/pasupulaphani/9630789
You can set timeout and emit an error/abort if the request takes too long but this has to be done in application logic.
Found a better/sophisticated solution to handle timeouts in other post :
https://stackoverflow.com/a/12815321/2073176