(I am using node-amqp and rabbitmq server.)
I am trying to guess why I have a close event if something goes wrong. For example, If I try to open a a connection to a queue (with bad parameters) I receive an error event. That it is perfect ok.
But, after any error I will receive also a close connection (in that case, maybe because close the failed socket to the queue). And after that, auto-reconnect and I receive the (initial) ready event.
The problem:
connection.on('ready', function() {
do_a_lot_of_things
}).on(error, function(error){
solve_the_problem
});
if something goes wrong, I receive the error, but then "ready" event and it will re do_a_lot_of_things. Is my approach wrong?
best regards
You can use connection.once('ready', function () { … }) (see the documentation), which will execute the handler only on the first event.
Related
In a Node server I have a series of streams piped together. Say, for example:
streamA.pipe(streamB).pipe(streamC)
Eventually streamA completes and at that point I want to switch the downstreams to another source:
streamD.pipe(streamB).pipe(streamC)
But when I try to do this I get the following error:
Error: write after end
How can I prevent streamA from closing my downstreams? Or how can I open my downstreams back up to switch them over to streamD?
Also, my use case requires that I wait until streamA end. I can't switch the streams prematurely.
Been googling the same thing, you can just add
streamA.pipe(streamB, {end: false}).pipe(streamC, {end: false})
In my case I found that if I unpipe() my downstream when I receive the 'end' event then the upstream won't have a chance to close my downstream and everything is Ok.
streamA.on('end', function() {
downstreams.forEach(downstream => {
streamA.unpipe(downstream);
});
}
If there is a more canonical solution then please feel free to post it.
I am trying to stream some audio to my server and then stream it to a service specified by the user, the user will be providing me with someHostName, which can sometimes not support that type of request.
My problem is that when it happens the clientRequest.on('end',..) is never fired, I think it's because it's being piped to someHostReq which gets messed up when someHostName is "wrong".
My question is:
Is there anyway that I can still have clientRequest.on('end',..) fired even when the stream clientRequest pipes to has something wrong with it?
If not: how do I detect that something wrong happened with someHostReq "immediately"? someHostReq.on('error') doesn't fire up except after some time.
code:
someHostName = 'somexample.com'
function checkIfPaused(request){//every 1 second check .isPaused
console.log(request.isPaused()+'>>>>');
setTimeout(function(){checkIfPaused(request)},1000);
}
router.post('/', function (clientRequest, clientResponse) {
clientRequest.on('data', function (chunk) {
console.log('pushing data');
});
clientRequest.on('end', function () {//when done streaming audio
console.log('im at the end');
}); //end clientRequest.on('end',)
options = {
hostname: someHostName, method: 'POST', headers: {'Transfer-Encoding': 'chunked'}
};
var someHostReq = http.request(options, function(res){
var data = ''
someHostReq.on('data',function(chunk){data+=chunk;});
someHostReq.on('end',function(){
console.log('someHostReq.end is called');
});
});
clientRequest.pipe(someHostReq);
checkIfPaused(clientRequest);
});
output:
in the case of a correct hostname:
pushing data
.
.
pushing data
false>>>
pushing data
.
.
pushing data
pushing data
false>>>
pushing data
.
.
pushing data
console.log('im at the end');
true>>>
//continues to be true, that's fine
in the case of a wrong host name:
pushing data
.
.
pushing data
false>>>>
pushing data
.
.
pushing data
pushing data
false>>>>
pushing data
.
.
pushing data
true>>>>
true>>>>
true>>>>
//it stays true and clientRequest.on('end') is never called
//even tho the client is still streaming data, no more "pushing data" appears
if you think my question is a duplicate:
it's not the same as this: node.js http.request event flow - where did my END event go? , the OP was just making a GET instead of a POST
it's not the same as this: My http.createserver in node.js doesn't work? , the stream was in paused mode because the none of the following happened:
You can switch to flowing mode by doing any of the following:
Adding a 'data' event handler to listen for data.
Calling the resume() method to explicitly open the flow.
Calling the pipe() method to send the data to a Writable.
source: https://nodejs.org/api/stream.html#stream_class_stream_readable
it's not the same as this: Node.js response from http request not calling 'end' event without including 'data' event , he just forgot to add the .on('data',..)
The behaviour in case of a wrong host name seems some problem with buffers, if the destination stream buffer is full (because someHost is not getting the sended chunks of data) the pipe will not continue to read the origin stream because pipe automatically manage the flow. As pipe is not reading the origin stream you never reach 'end' event.
Is there anyway that I can still have clientRequest.on('end',..) fired
even when the stream clientRequest pipes to has something wrong with
it?
The 'end' event will not fire unless the data is completely consumed. To get 'end' fired with a paused stream you need to call resume() (unpiping first from wrong hostname or you will fall in buffer stuck again) to set the steam into flowMode again or read() to the end.
But how to detect when I should do any of the above?
someHostReq.on('error') is the natural place but if it takes too long to fire up:
First try to set a low timeout request (less than someHostReq.on('error') takes to trigger, as seems too much time for you) request.setTimeout(timeout[, callback]) and check if it doesn't fail when correct hostname. If works, just use the callback or timeout event to detect when the server timeOut and use one of the techniques above to reach to the end.
If timeOut solution fails or doesn't fits your requirements you have to play with flags in clientRequest.on('data'), clientRequest.on('end') and/or clienteRequest.isPaused to guess when you are stuck by the buffer. When you think you are stuck just apply one of the techniques above to reach to the end of the stream. Luckily it takes less time to detect buffer stuck than wait for someHostReq.on('error') (maybe two request.isPaused() = true without reach 'data' event is enought to determine if you are stuck).
How do I detect that something wrong happened with someHostReq
"immediately"? someHostReq.on('error') doesn't fire up except after
some time.
Errors triggers when triggers. You can not "immediately" detect it. ¿Why not just send a prove beacon request to check support before piping streams? Some kind of:
"Cheking service specified by the user..." If OK -> Pipe user request stream to service OR FAIL -> Notify user about wrong service.
i use web sockets and the onclose method does not get triggert when the server is down.
i tried to close the ws connections manually with progress.exit but it seems this only gets triggert on a clean exit.
is their a way to catch a stopped nodejs app and execute code before the stop?
it should trigger on ctr+c and any other crash/exit.
basicly i want to tell the client when the connection is not alive anymore before he is trying to do something, since onclose does not handel every case, what can i do to check the connection?
the only solution i came up with is to ping the server from time to time.
since this is not possible i started sending pings as a workarround:
var pingAnswer = true;
pingInterval = setInterval(function(){
if(pingAnswer){
ws.send(JSON.stringify({type:'ping'})); //on the serverside i just send a ping back everytime i recive one.
pingAnswer = false;
}else{
clearInterval(pingInterval);
//reload page or something
}
},1000);
ws.onMessage(function(e){
m = JSON.parse(e.data);
switch(m.type){
....
case 'ping':
pingAnswer=true;
return;
}
}
);
You don't provide a snippet showing how you're defining your on('close',...) handler, but it's possible that you're defining the handler incorrectly.
At the time of this writing, the documentation for websocket.onclose led me to first implement the handler as ws_connection.onclose( () => {...});, but I've found the proper way to be ws_connection.on('close', () => {...} );
Maybe the style used in the ws documentation is some kind of idiom I'm unfamiliar with.
I've tested this with node 6.2.1 and ws 1.1.1. The on.('close',...) callback is triggered on either side (server/client) when the corresponding side is shutdown via Ctrl+c or crashes for whatever reason (for example, for testing, JSON.parse("fail"); or throw new Error('');).
Take this code, where f is a stream that has an event 'body', which calls the listeners with a m -- which is itself a stream emitting events:
f.on('message', function(m) {
m.on('body', function(stream, info) {
var b = '';
stream.on('data', function(d) {
b += d;
});
stream.on('end', function() {
if (/^header/i.test(info.which))
msg.header = Imap.parseHeader(b);
else
msg.body = b;
});
});
m.on('attributes', function(attrs) {
msg.attrs = attrs;
msg.contentType = partID[1];
});
});
f.on('end', function() {
if (hadErr)
return;
cb(undefined, msg);
});
The backend is emitting a 'message' event, passing it a m object. The code then listens to the events body and attributes. It's all straightforward except that my little brain is in a bit of a crisis (I am not used to dealing with streams). Especially: how is the backend emitting from the f and m objects, to guarantee that events are indeed called at the right time?
Specifically:
How would f have to be coded, in general terms, in order to make sure that mm doesn't emit till m.on('body', function(stream, info) { is called?
Does a listener need to be added with on() before the event is emitted in order for it to be caught?
If so, does that mean that f and m will emit events after the code here has registered?
If the backend is supposed to guarantee that b.emit('end') is called after m.emit('end'), how is that even supposed to happen really, still guaranteeing that on() is called before any one of the events are emitted?
OK I am 100% confused about this matter. I am obviously missing something basic and crucial, and I am not even able to ask the right questions because of this...! (Apologies)
Does a listener need to be added with on() before the event is emitted in order for it to be caught?
Yes.
If so, does that mean that f and m will emit events after the code here has registered?
No, events are not queued anywhere. If nothing is listening for them, they will be lost. I think that's what you're asking anyway... f and m don't seem to emit events in your code.
If the backend is supposed to guarantee that b.emit('end') is called after m.emit('end'), how is that even supposed to happen really, still guaranteeing that on() is called before any one of the events are emitted?
b is a string in your example? I'm not sure what you're asking here.
Think of it differently. When .on is called, a function is subscribed to a channel of messages. Those messages are already flowing before that function is subscribed, and will continue to flow if that function is unsubscribed. .on and .removeListener() just set the subscription status for a particular function.
Events can be emitted even if nothing is listening for them. Events can fire all the time and if nothing is listening, they just don't go anywhere. (An exception to this are the error events built into Node.js, which are turned into real exceptions if there isn't an error handler.)
How would f have to be coded, in general terms, in order to make sure that mm doesn't emit till m.on('body', function(stream, info) { is called?
I still don't follow specifically what you're asking, since none of the code you show emits anything. But, you wouldn't really want to do this. You need to be setting up your event handlers before opening a stream, or doing whatever you are doing that causes the events to be fired.
You might be getting confused on the ordering of event handling on new objects. In Node.js, there is a rule... Never emit directly from your constructor. Always use nextTick() or similar. This way, after instantiation, any code to attach itself to event handlers can do so before the events are emitted.
Also, if you are using streams, consider using the readable event so that the stream remains paused until you're ready to read from it. Pull vs. push.
I'm trying to read some file by opening read stream and send chunks of the file through ZMQ to another process to consume them. The stream is working like it should, however when I start the worker, it doesn't see the data that's been sent.
I tried sending data through socket every 500ms, not in a callback, and when I start the worker it collects all previous chunks of data:
sender = zmq.socket('push')
setInterval(() ->
console.log('sending work');
sender.send('some work')
, 500)
receiver = zmq.socket("pull")
receiver.on "message", (msg) ->
console.log('work is here: %s', msg.toString())
Outputs:
sending work
sending work
sending work
sending work
sending work
// here I start the worker
sending work
work is here: some work
work is here: some work
work is here: some work
work is here: some work
work is here: some work
work is here: some work
sending work
work is here: some work
sending work
work is here: some work
sending work
work is here: some work
So, when the worker starts, it begins with pulling all the previous data and then it pulls it every time sth new comes in. This does not apply when I do this:
readStream = fs.createReadStream("./data/pg2701.txt", {'bufferSize': 100 * 1024})
readStream.on "data", (data) ->
console.log('sending work');
sender.send('some work'); // I'd send 'data' if it worked..
In this scenario, the worker doesn't pull any data at all.
Are those kind of sockets supposed to create a queue or not? What am I missing here?
Yes, push socket is blocking until HWM is reached, and there's nobody to send to.
Maybe the sender hasn't bound yet, try something like this:
sender.bind('address', function(err) {
if (err) throw err;
console.log('sender bound!');
// the readStream code.
}
also a connect is missing from your code example, I bet it's there, but maybe you forgot it.