it's the first time I've come across SSE, and after a while, i finally got something working. The problem is, that wherever I look, there are always the same examples, of a interval or a loop right inside the original middleware, someting like this:
app.get('/events', async function(req, res) {
console.log('Got /events');
res.set({
'Cache-Control': 'no-cache',
'Content-Type': 'text/event-stream',
'Connection': 'keep-alive'
});
res.flushHeaders();
// Tell the client to retry every 10 seconds if connectivity is lost
res.write('retry: 10000\n\n');
let count = 0;
while (true) {
await new Promise(resolve => setTimeout(resolve, 1000));
console.log('Emit', ++count);
// Emit an SSE that contains the current 'count' as a string
res.write(`data: ${count}\n\n`);
}
});
Now this works, but it's not useful. I am building a simple ordering app. I need to send an sse every time someone orders, so you can see the orders in real time. To do this, I need to send the sse messages after an order has been checked and added to the db. I am not only adding the order in a separate middleware, but even in a separate file, and as a response to a different request. I can't just use res.write, as in the example, because it will affect the response to the /add-order request, and not to the /sse. Is there any way I could do this, possibly even without using sse at all? (I am open to using npm packages as well)
The general idea of SSE remains the same, in your particular case you could use something like the app EventEmitter, when an order is created emit the event, while on the other side subscribe to the event, and each time your receive an event send the data over SSE.
Related
I am new to node.js. I am creating a async operation in node with http server module. I want to print hello first and world after 5 seconds on a web page. I have referred this example form Introduction from node js by Ryan Dahl which is 8 years old.
I was wondering whether I have missed anything or something has changed in node.
I have tried with setTimeout and setInterval function but both dosent seem to work.
var http = require('http');
var server = http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' })
res.write('Hello \n');
setTimeout(() => {
res.end('World');
},5000)
})
server.listen(8000);
Expected result : Hello prints first followed by world after 5 seconds.
Actual result : Both Hello World prints together after 5 seconds.
Conceptually you are mixing up client and server.
The browser can only render your page when it receives the entire page. (not entirely true*, but you should think of it this way).
If you don't res.end (or res.send with express), the server will hold onto the message(see caveat below). The browser will never render no matter how many .writes you have, because it simply doesn't have the final webpage to render.
Anyway your async code is correct. If you want to see whether the asynchronity is working on the server, use console.logs instead. Whether the .write is actually sent is honestly dependent on many optimization factors you can't see.
*This is actually quite dependent on OS behavior.
https://nodejs.org/api/http.html#http_response_write_chunk_encoding_callback
Observe the API, it may or may not flush the chunk to OS. If its too small, it will usually not be flushed (for performance reasons). The OS then may or may not flush the chunk out (also for performance reasons).
If you want to see this behavior, write a much much much bigger string.
Literally this big:
var server = http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' })
res.write('HelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHello');
res.write('HelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHello');
res.write('HelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHello');
res.write('HelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHello');
res.write('HelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHello');
setTimeout(() => {
res.end('World');
},5000)
})
You're getting Hello and world printed on the screen after 5 seconds because the res.end would be the function fired last irrespective of where or how you write it. This is because the res.end function terminates the servers response and only one response can go to the client the way you have configured.
If you truly want to see the output you desire try using console.log instead of res.end and res.write.
I'm currently looking to set up an endpoint that accepts a request, and returns the response data in increments as they load.
The application of this is that given one upload of data, I would like to calculate a number of different metrics for that data. As each metric gets calculated asynchronously, I want to return this metric's value to the front-end to render.
For testing, my controller looks as follows, trying to use res.write
uploadData = (req, res) => {
res.write("test");
setTimeout(() => {
res.write("test 2");
res.end();
}, 3000);
}
However, I think the issue stems from my client-side which I'm writing in React-Redux, and calling that route through an Axios call. From my understanding, it's because the axios request closes once receiving the first response, and the connection doesn't stay open. Here is what my axios call looks like:
axios.post('/api', data)
.then((response) => {
console.log(response);
})
.catch((error) => {
console.log(error);
});
Is there an easy way to do this? I've also thought about streaming, however my concern with streaming is that I would like each connection to be direct and unique between clients that are open for short amount of time (i.e. only open when the metrics are being calculated).
I should also mention that the resource being uploaded is a db, and I would like to avoid parsing and opening a connection multiple times as a result of multiple endpoints.
Thanks in advance, and please let me know if I can provide any more context
One way to handle this while still using a traditional API would be to store the metrics in an object somewhere, either a database or redis for example, then just long poll the resource.
For a real world example, say you want to calculate the following metrics of foo, time completed, length of request, bar, foobar.
You could create an object in storage that looks like this:
{
id: 1,
lengthOfRequest: 123,
.....
}
then you would create an endpoint in your API that like so metrics/{id}
and would return the object. Just keep calling the route until everything completes.
There are some obvious drawbacks to this of course, but once you get enough information to know how long the metrics will take to complete on average you can tweak the time in between the calls to your API.
3 years ago I could do multiple res.send in express.js.
even write a setTimeout to show up a live output.
response.send('<script class="jsbin" src="http://code.jquery.com/jquery-1.7.1.min.js"></script>');
response.send('<html><body><input id="text_box" /><button>submit</button></body></html>');
var initJs = function() {
$('.button').click(function() {
$.post('/input', { input: $('#text_box').val() }, function() { alert('has send');});
});
}
response.send('<script>' + initJs + '</script>');
Now it will throw:
Error: Can't set headers after they are sent
I know nodejs and express have updated. Why can't do that now? Any other idea?
Found the solution but res.write is not in api reference http://expressjs.com/4x/api.html
Maybe you need: response.write
response.write("foo");
response.write("bar");
//...
response.end()
res.send implicitly calls res.write followed by res.end. If you call res.send multiple times, it will work the first time. However, since the first res.send call ends the response, you cannot add anything to the response.
response.send sends an entire HTTP response to the client, including headers and content, which is why you are unable to call it multiple times. In fact, it even ends the response, so there is no need to call response.end explicitly when using response.send.
It appears to me that you are attempting to use send like a buffer: writing to it with the intention to flush later. This is not how the method works, however; you need to build up your response in code and then make a single send call.
Unfortunately, I cannot speak to why or when this change was made, but I know that it has been like this at least since Express 3.
res.write immediately sends bytes to the client
I just wanted to make this point about res.write clearer.
It does not build up the reply and wait for res.end(). It just sends right away.
This means that the first time you call it, it will send the HTTP reply headers including the status in order to have a meaningful response. So if you want to set a status or custom header, you have to do it before that first call, much like with send().
Note that write() is not what you usually want to do in a simple web application. The browser getting the reply little by little increases the complexity of things, so you will only want to do it it if it is really needed.
Use res.locals to build the reply across middleware
This was my original use case, and res.locals fits well. I can just store data in an Array there, and then on the very last middleware join them up and do a final send to send everything at once, something like:
async (err, req, res, next) => {
res.locals.msg = ['Custom handler']
next(err)
},
async (err, req, res, next) => {
res.locals.msg.push('Custom handler 2')
res.status(500).send(res.locals.msg.join('\n'))
}
I'm writing a Node.js application using a global event emitter. In other words, my application is built entirely around events. I find this kind of architecture working extremely well for me, with the exception of one side case which I will describe here.
Note that I do not think knowledge of Node.js is required to answer this question. Therefore I will try to keep it abstract.
Imagine the following situation:
A global event emitter (called mediator) allows individual modules to listen for application-wide events.
A HTTP Server is created, accepting incoming requests.
For each incoming request, an event emitter is created to deal with events specific to this request
An example (purely to illustrate this question) of an incoming request:
mediator.on('http.request', request, response, emitter) {
//deal with the new request here, e.g.:
response.send("Hello World.");
});
So far, so good. One can now extend this application by identifying the requested URL and emitting appropriate events:
mediator.on('http.request', request, response, emitter) {
//identify the requested URL
if (request.url === '/') {
emitter.emit('root');
}
else {
emitter.emit('404');
}
});
Following this one can write a module that will deal with a root request.
mediator.on('http.request', function(request, response, emitter) {
//when root is requested
emitter.once('root', function() {
response.send('Welcome to the frontpage.');
});
});
Seems fine, right? Actually, it is potentially broken code. The reason is that the line emitter.emit('root') may be executed before the line emitter.once('root', ...). The result is that the listener never gets executed.
One could deal with this specific situation by delaying the emission of the root event to the end of the event loop:
mediator.on('http.request', request, response, emitter) {
//identify the requested URL
if (request.url === '/') {
process.nextTick(function() {
emitter.emit('root');
});
}
else {
process.nextTick(function() {
emitter.emit('404');
});
}
});
The reason this works is because the emission is now delayed until the current event loop has finished, and therefore all listeners have been registered.
However, there are many issues with this approach:
one of the advantages of such event based architecture is that emitting modules do not need to know who is listening to their events. Therefore it should not be necessary to decide whether the event emission needs to be delayed, because one cannot know what is going to listen for the event and if it needs it to be delayed or not.
it significantly clutters and complexifies code (compare the two examples)
it probably worsens performance
As a consequence, my question is: how does one avoid the need to delay event emission to the next tick of the event loop, such as in the described situation?
Update 19-01-2013
An example illustrating why this behavior is useful: to allow a http request to be handled in parallel.
mediator.on('http.request', function(req, res) {
req.onceall('json.parsed', 'validated', 'methodoverridden', 'authenticated', function() {
//the request has now been validated, parsed as JSON, the kind of HTTP method has been overridden when requested to and it has been authenticated
});
});
If each event like json.parsed would emit the original request, then the above is not possible because each event is related to another request and you cannot listen for a combination of actions executed in parallel for a specific request.
Having both a mediator which listens for events and an emitter which also listens and triggers events seems overly complicated. I'm sure there is a legit reason but my suggestion is to simplify. We use a global eventBus in our nodejs service that does something similar. For this situation, I would emit a new event.
bus.on('http:request', function(req, res) {
if (req.url === '/')
bus.emit('ns:root', req, res);
else
bus.emit('404');
});
// note the use of namespace here to target specific subsystem
bus.once('ns:root', function(req, res) {
res.send('Welcome to the frontpage.');
});
It sounds like you're starting to run into some of the disadvantages of the observer pattern (as mentioned in many books/articles that describe this pattern). My solution is not ideal – assuming an ideal one exists – but:
If you can make a simplifying assumption that the event is emitted only 1 time per emitter (i.e. emitter.emit('root'); is called only once for any emitter instance), then perhaps you can write something that works like jQuery's $.ready() event.
In that case, subscribing to emitter.once('root', function() { ... }) will check whether 'root' was emitted already, and if so, will invoke the handler anyway. And if 'root' was not emitted yet, it'll defer to the normal, existing functionality.
That's all I got.
I think this architecture is in trouble, as you're doing sequential work (I/O) that requires definite order of actions but still plan to build app on components that naturally allow non-deterministic order of execution.
What you can do
Include context selector in mediator.on function e.g. in this way
mediator.on('http.request > root', function( .. ) { } )
Or define it as submediator
var submediator = mediator.yield('http.request > root');
submediator.on(function( ... ) {
emitter.once('root', ... )
});
This would trigger the callback only if root was emitted from http.request handler.
Another trickier way is to make background ordering, but it's not feasible with your current one mediator rules them all interface. Implement code so, that each .emit call does not actually send the event, but puts the produced event in list. Each .once puts consume event record in the same list. When all mediator.on callbacks have been executed, walk through the list, sort it by dependency order (e.g. if list has first consume 'root' and then produce 'root' swap them). Then execute consume handlers in order. If you run out of events, stop executing.
Oi, this seems like a very broken architecture for a few reasons:
How do you pass around request and response? It looks like you've got global references to them.
If I answer your question, you will turn your server into a pure synchronous function and you'd lose the power of async node.js. (Requests would be queued effectively, and could only start executing once the last request is 100% finished.)
To fix this:
Pass request & response to the emit() call as parameters. Now you don't need to force everything to run synchronously anymore, because when the next component handles the event, it will have a reference to the right request & response objects.
Learn about other common solutions that don't need a global mediator. Look at the pattern that Connect was based on many Internet-years ago: http://howtonode.org/connect-it <- describes middleware/onion routing
I'm new to node.js and have this very simple code. I just want to say a Hi User every second to the users who have connected to the server.
Here's the code I have:
var http = require('http');
function newfunc(request, response) {
response.writeHead(200, {
"Content-Type": "text/plain",
"connection" : "keep-alive"
});
setInterval(function() {
response.write('Hi User\n');
response.end('');
}, 1000);
}
http.createServer(newfunc).listen(7070);
I see the Hi User message only once, and seems as if setInterval is writing it only once.
What am I doing wrong in this?
EDIT: I stand corrected in the comments... Just remove the response.end() call and you should see something like what you were expecting.
ORIGINAL RESPONSE: What you are trying to do cannot be done in this fashion... The response to an HTTP request is only ever sent once: when response.end() is called (the first time). Commenter points out that this is not really correct: what would be correct to say is that no further data can be sent in the response after end() is called.
If you want to show an HTML page whose contents change every second based on server-side actions, you will need to use something like WebSockets (e.g. the Node-compatible http://socket.io/ library) and some client-side Javascript, which is somewhat more complicated than the code you have above. In general, for non-trivial UI's that do more than just append to the response or do bi-directional communication, this type of approach is preferrable