Server Sent Events are a valuable tool to open a persistent connection to a web server, where the server has the ability to push new data to the client, when available.
Using this technology in Node.js is quite straightforward and can be implemented with the following code example:
#!/usr/bin/env node
'use strict';
const http = (options, listener) => require('http').createServer(listener).listen(options.port);
http({ port: 8080 }, (req, res) => {
switch (req.url) {
case '/server-sent-events': {
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Connection': 'keep-alive',
'Cache-Control': 'no-cache',
});
const sendDate = () => res.write(`data: ${new Date()}\n\n`);
sendDate();
const interval = setInterval(sendDate, 1000);
req.on('close', () => clearInterval(interval));
} break;
default: {
res.writeHead(200, {
'Content-Type': 'text/html; charset=utf-8',
});
res.end(`
<!DOCTYPE html>
<html>
<head>
<title>Server Send Events</title>
<meta charset="utf-8">
<script>
const sse = new EventSource('/server-sent-events');
sse.onerror = () => document.body.innerHTML = 'Connection Error';
sse.onmessage = ({ data }) => document.body.innerHTML = data;
</script>
</head>
<body></body>
</html>
`);
}
}
});
Unfortunately I am not able to achieve the same goal with Deno, as there is no simple write method on the request object, but I guess it has to be implemented somehow using the req.w buffer. Can you help me please finish off the following example code, so the Server Sent Events can be utilised with Deno as well?
#!/usr/bin/env deno run --allow-net
import { listenAndServe as http } from 'https://deno.land/std/http/server.ts';
http({ port: 8080 }, (req) => {
switch (req.url) {
case '/server-sent-events': {
// missing steps:
// * setup the server sent event headers
// * create the interval and send the date periodically
// * clear the interval when the connection gets closed
} break;
default: {
req.respond({
headers: new Headers({
'Content-Type': 'text/html; charset=utf-8',
}),
body: `
<!DOCTYPE html>
<html>
<head>
<title>Server Send Events</title>
<meta charset="utf-8">
<script>
const sse = new EventSource('/server-sent-events');
sse.onerror = () => document.body.innerHTML = 'Connection Error';
sse.onmessage = ({ data }) => document.body.innerHTML = data;
</script>
</head>
<body></body>
</html>
`,
});
}
}
});
Thank you very much for your support!
[Update 2021-11-04]:
I have made some progress doing some research across different sources (https://deno.land/std#0.76.0/http/server.ts, https://github.com/denoland/deno/issues/4817) and got a step closer to the solution. Using the updated example below at least the setup and usage of the Server Sent Events do work now. The remaining issue (besides cleaning up and refactoring of the code) remains the safe detection when the incoming request has been closed (see comments in the source code below):
#!/usr/bin/env deno run --allow-net
import { listenAndServe as http } from 'https://deno.land/std/http/server.ts';
http({ port: 8080 }, (req) => {
switch (req.url) {
case '/server-sent-events': {
// set up a quick´n´dirty write method without error checking
req.write = (data) => {
req.w.write(new TextEncoder().encode(data));
req.w.flush();
};
// setup the server sent event headers
let headers = '';
headers += 'HTTP/1.1 200 OK\r\n';
headers += 'Connection: keep-alive\r\n';
headers += 'Cache-Control: no-cache\r\n';
headers += 'Content-Type: text/event-stream\r\n';
headers += '\r\n';
req.write(headers);
// create the interval and send the date periodically
const sendDate = () => req.write(`data: ${new Date()}\n\n`);
sendDate();
const interval = setInterval(sendDate, 1000);
// final missing step:
// * clear the interval when the connection gets closed
// currently dropping the connection from the client will
// result in the error: Uncaught (in promise) BrokenPipe:
// Broken pipe (os error 32)
// this error also does not seem to be catchable in the
// req.write method above, so there needs to be another safe
// way to prevent this error from occurring.
} break;
default: {
req.respond({
headers: new Headers({
'Content-Type': 'text/html; charset=utf-8',
}),
body: `
<!DOCTYPE html>
<html>
<head>
<title>Server Send Events</title>
<meta charset="utf-8">
<script>
const sse = new EventSource('/server-sent-events');
sse.onerror = () => document.body.innerHTML = 'Connection Error';
sse.onmessage = ({ data }) => document.body.innerHTML = data;
</script>
</head>
<body></body>
</html>
`,
});
}
}
});
[Update 2021-04-16]
All issues have been resolved and are posted in my accepted answer below.
Deno's http library doesn't support SSE, but you can use Oak Framework, or implement it yourself.
import { Application, Router } from "https://deno.land/x/oak/mod.ts";
const app = new Application();
const router = new Router();
router.get('/', ctx => {
ctx.response.body = `
<!DOCTYPE html>
<html>
<head>
<title>Server Send Events</title>
<meta charset="utf-8">
<script>
const sse = new EventSource('/server-sent-events');
sse.onerror = () => document.body.innerHTML = 'Connection Error';
sse.onmessage = ({ data }) => document.body.innerHTML = data;
</script>
</head>
<body></body>
</html>
`;
})
router.get("/server-sent-events", (ctx) => {
const target = ctx.sendEvents();
const sendDate = () => target.dispatchMessage(`${new Date()}`);
sendDate();
const interval = setInterval(sendDate, 1000);
});
app.use(router.routes());
await app.listen({ port: 8080 });
In the end I have found an answer to my question and so the full answer with plenty of comments follows, so you get a working version of server sent events in Deno. The solution below also solves the os error 32, which gets caused by not catching the connection writer flash method:
#!/usr/bin/env deno run --allow-net
// imports
import { ServerRequest, listenAndServe as http } from 'https://deno.land/std/http/server.ts';
// commodity
const encoder = new TextEncoder();
const print = console.log;
// start the web-server
// this one allows the endpoint `/server-sent-events`, which hosts a clock that
// will be refreshed every second (the efficiency of the clock solution could of
// course be optimised, as every client gets its own clock interval, but this
// this does not matter as this example wants to show how to setup and clean a
// task for every connecting client)
// all other requests will be answered with a simple html page that subscribes
// to the sse-based clock
http({ port: 8080 }, async (req) => {
// ip address of the client (formatted as `ip:port`, so we cut the `:port` part
// of it)
const ip = req.headers.get('host').split(':').slice(0, -1).join(':');
// determine the endpoint to access
switch (req.url) {
// host the server sent event based clock
case '/server-sent-events': {
// logging
print(`+ Client ${ip} connected`);
// prepare the disconnect promise. we will use this one later on to await
// the clients disconnect, so we can properly clean up. so the promise will
// be resolved manually by us when we detect a disconnect from the client
// on an attempt to send new data to him (unfortunately there seems to be
// no other way to detect when the client actually closed the connection)
let resolver;
const disconnect = new Promise((resolve) => resolver = resolve);
// write helper
req.write = async (data) => {
// send the current data to the client
req.w.write(encoder.encode(data));
// to actually send the data we need to flush the writer first. we need
// to try/catch this part, as not handling errors on flush will lead to
// the `Broken pipe (os error 32)` error
try {
await req.w.flush();
} catch(err) {
// throw any errors but the broken pipe, which gets thrown when the
// client has already disconnected and we try to send him new data
// later on
if (err.name !== 'BrokenPipe') {
throw err;
}
// close the connection from our side as well
req.conn.close();
// resolve our `disconnect` promise, so we can clean up
resolver();
}
};
// date writer (interval method which pushes the current date to the client)
const sendDate = async () => await req.write(`data: ${new Date()}\n\n`);
// prepare and send the headers
let headers = '';
headers += `HTTP/1.1 200 OK\r\n`;
headers += `Connection: keep-alive\r\n`;
headers += `Cache-Control: no-cache\r\n`;
headers += `Content-Type: text/event-stream\r\n`;
headers += `\r\n`;
await req.write(headers);
// send the date now for the first time and then every second
sendDate();
const interval = setInterval(sendDate, 1000);
// await until the clients disconnects to clean up. so we will be "stuck"
// here until a disconnect gets detected as we use a promise based approach
// to detect the disconnect
await disconnect;
clearInterval(interval);
// logging
print(`- Client ${ip} disconnected`);
} break;
// all other requests host a simple html page which subscribes to the clock
default: {
print(`* Serve website to ${ip}`);
req.respond({
headers: new Headers({
'Content-Type': 'text/html; charset=utf-8',
}),
body: `
<!DOCTYPE html>
<html>
<head>
<title>Server Sent Events</title>
<meta charset="utf-8">
<script>
const sse = new EventSource('/server-sent-events');
sse.onerror = () => document.body.innerHTML = 'Connection Error';
sse.onmessage = ({ data }) => document.body.innerHTML = data;
</script>
</head>
<body></body>
</html>
`,
});
}
}
});
This example seems more idiomatic.
import { serve } from "https://deno.land/std#0.116.0/http/server.ts";
const msg = new TextEncoder().encode("data: hello\r\n");
serve(async (_) => {
let timerId: number | undefined;
const body = new ReadableStream({
start(controller) {
timerId = setInterval(() => {
controller.enqueue(msg);
}, 1000);
},
cancel() {
if (typeof timerId === "number") {
clearInterval(timerId);
}
},
});
return new Response(body, {
headers: {
"Content-Type": "text/event-stream",
},
});
});
console.log("Listening on http://localhost:8000");
Related
I have been reading about Deno for a few hours and finally got an http static server running.
I would like to know what else is needed to add https to it.
I understand Leaf to Root arrangement of certificates but not in Deno.
Working code:
import {
gray,
green,
cyan,
bold,
yellow,
red,
} from 'https://deno.land/std#0.58.0/fmt/colors.ts';
import { Application, HttpError, send, Status } from 'https://deno.land/x/oak/mod.ts';
const app = new Application();
// Middleware 1 - Error handler
app.use(async (context, next) => {
try {
await next();
} catch (e) {
if (e instanceof HttpError) {
context.response.status = e.status as any;
// expose
// Determines if details about the error should be automatically exposed in a response.
// This is automatically set to true for 4XX errors, as they represent errors in the request...
if (e.expose) {
context.response.body = `
<!DOCTYPE html>
<html>
<body>
<h1>${e.status} - ${Status[e.status]}</h1>
<!-- <h1>${e.status} - ${e.message}</h1> -->
</body>
</html>`;
} else {
context.response.body = `
<!DOCTYPE html>
<html>
<body>
<h1>${e.status} - ${Status[e.status]}</h1>
<!-- <h1>${e.status} - ${e.message}</h1> -->
</body>
</html>`;
}
} else if (e instanceof Error) {
context.response.status = 500;
// ...while 5XX errors are set to false as they are internal server errors and
// exposing details could leak important server security information.
context.response.body = `
<!DOCTYPE html>
<html>
<body>
<h1>500 - Internal Server Error</h1>
</body>
</html>`;
console.log('Unhandled Error:', red(bold(e.message)));
console.log(e.stack);
}
}
});
// Middleware 2 - Logger
app.use(async (context, next) => {
await next();
const rt = context.response.headers.get('X-Response-Time');
console.log(`${green(context.request.method)} ${cyan(context.request.url.pathname)} - ${bold(String(rt),)}`, );
});
// Middleware 3 - Response Time
app.use(async (context, next) => {
const start = Date.now();
await next();
const ms = Date.now() - start;
context.response.headers.set('X-Response-Time', `${ms}ms`);
});
// End point - Send static content
app.use(async (context) => {
await context.send({
root: `${Deno.cwd()}/var/www/example1.com/public`,
index: 'index.html',
});
});
// Welcome message
app.addEventListener('listen', ({ hostname, port }) => {
console.clear();
console.log(' ');
console.log(bold('Deno 1.1.1 - Static HTTP Server'));
console.log(gray(' #host: ') + yellow(`${hostname}`));
console.log(gray(' #port: ') + yellow(`${port}`));
console.log(gray(' Status: ') + green('online'));
console.log(gray(' #root: ') + cyan('/var/www/example1.com/public'));
console.log(gray(' #index: ') + cyan('index.html'));
console.log(' ');
});
await app.listen({ hostname: '127.0.0.1', port: 80 });
// run the server
// deno run --allow-net --allow-read httpServer.ts
Read the documentation on serveTLS and listenAndServeTLS.
If you have a valid certificate, you shouldn't have any issues. However, I had some difficulty overriding rejections of self-signed SSL certificates. Also, I ended up serving ssl on a port other than 443, due to permission errors I received while trying to run an HTTPS server as a local user.
Those are the issues I ran into and how I got around them. I hope it helps.
Following is my POST function in Node js. I want to call a funtion in my client side HTML to display an error message on the page.
router.post('/',(req,res)=>{
const data = JSON.stringify({
institute_name: req.body.institute_name,
email : req.body.email,
password : req.body.password
})
const options = {
host:'localhost',
port:'8888',
path:'/registerInstitute',
method:'POST',
headers: {
'Content-Type':'application/json'
}
}
const req1 = http.request(options, (res1)=>
{
const status = res1.statusCode
if (status == 201)
{
//admin created
res.redirect('/?account_created=true')
}
else if ( status == 409)
{
//CALL AJAX FUNCTION TO DISPLAY ERROR MSG
}
})
req1.write(data)
req1.end()
})
The simple answer is NO. But there are workarounds to do so and I've demonstrated the one that will suite your scenario
Sockets
You can use web sockets to trigger event to notify client to run specific function
On service side you can do something like this:
var io = require('socket.io').listen(80); // initiate socket.io server
if (
io.sockets.on('connection', function (socket) {
if ( error == 403 ) {
socket.emit('runErrorFunction', { error: "error data" });
}
});
You can do something like this on client side:
<script src="/socket.io/socket.io.js"></script>
<script>
var socket = io.connect('http://localhost'); // connect to server
socket.on('runErrorFunction', function (data) { // listen to runErrorFunction event raised by the server
// Here goes your error showing logic
});
</script>
Here is the link for more info Socket.io
In an Express-based app, I'm running an async function to fetch image source from a database and render them into a Nunjucks view engine
Nunjucks is rendering the following in the DOM
<img src="[object Promise]">
I have already enabled Nunjucks's async rendering with web: { async: true } and enabled the nunjucks async api with a callback like so
// controller.js (KeystoneJS app)
view.render('index', function (err, res) {
console.log('err at index render', err); // undefined
return res;
});
How can i get the resolved value of my async function?
As I understand it, Nunjucks doesn't support asynchronous render directly. You can use asynchronous filters to get it. Maybe I'm wrong.
Imho, use feature with Be Careful! mark is a not good idea.
// template.njk
Hello {{user_id | find | attr('name') }}!
// app.js
var nunjucks = require('nunjucks');
var env = nunjucks.configure();
// Async filter
env.addFilter('find', function(a, cb) {
setTimeout(function () {
cb(null, {
name: 'Smith'
});
}, 10);
}, true)
// Sync filter
env.addFilter('attr', function(obj, attr) {
return obj && attr && obj[attr];
});
env.render('template.njk',
{user_id: 1}, // pass sync vars
function(err, res) {
if (err)
return;
console.log(res);
return res
}
);
I don't know about nunjucks, but you can implement async functionality regardless of the view engine being used. To show the idea, I tried to reproduce your situation. I created an HTML file named index.html with an img tag without any src attribute:
<html>
<head>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<title>Reproduce</title>
</head>
<body>
<img id='bg'></img>
<script src='./so.js'></script>
</body>
</html>
I have a <script> tag in my HTML which links to my so.js file shown below by which an image is requested by sending a HTTP request to NodeJS/Express server:
getImage();
function getImage(){
// Get an image by its name as URL parameter
fetch('/bg/background.jpg',{
method: 'GET',
headers: {
'Content-Type': 'application/json'
}
}).then(result=>{
return result.blob()
}).then(result=>{
console.log('result -> ', result)
document.querySelector('#bg').src=URL.createObjectURL(result)
document.querySelector('#bg').style.width='200px'
}).catch(err=>{
console.log('err -> ', err)
})
}
Here is my NodeJS/ExpressJS code inside a file name server.js:
express=require('express')
bodyParser=require('body-parser')
path=require('path')
fetch=require('node-fetch')
server=express()
//Body-parser middleware
server.use(bodyParser.json())
server.use(bodyParser.urlencoded({extended:false}))
server.use(bodyParser.raw())
//Set static path
server.use(express.static(path.join(__dirname,'public')))
server.get('/',(req,res)=>{
res.render('index.html')
})
// Pick a file on hard disk
// and send it as "Blob" to the browser
server.get('/bg/:name',(req,res)=>{
var options = {
root: __dirname + '/public/',
dotfiles: 'deny',
headers: {
'x-timestamp': Date.now(),
'x-sent': true
}
};
var fileName = req.params.name;
res.sendFile(fileName, options, function (err) {
if (err) {
next(err);
} else {
console.log('Sent:', fileName);
}
});
})
server.listen('8000',()=>{
console.log('Server listening on port 8000...')
})
As can be seen, I'm doing my async communication between browser and server by implementing fetch API and without even touching the view engine.
I just wanted to provide an alternative idea to use fetch API and do the async HTTP communications with it regardless of any view rendering engine that is being used.
I'm trying to use SSE with node + express: I intercept requests using an express route, then I initiate a SSE session by directly writing headers:
res.writeHead(200, {
"content-type": "text/event-stream",
"cache-control": "no-cache"
});
I proceed with writing intermittent payloads using "res.write()"s.
This works well with Chrome's EventSource, up until the time when I call ".close()" to end the session. Then, the connection keeps hanging: Chrome doesn't reuse the connection to initiate additional EventSource requests (or any other requests), and node never triggers a "close" event on the IncomingMessage instance.
My question is: How do I handle "eventSource.close()" properly using node's http API?
It's worth noting that:
Since I don't set a "content-length", Node automatically assumes "chunked" transfer encoding (this shouldn't be a problem AFAIK). It also defaults to "connection: keep-alive".
The session terminates OK when I'm using Firefox.
When the browser closes the event source it does let the server side know. On the server side, the response socket object (res.socket) will generate an end event followed by a close event. You can listen to this event and respond appropriately.
E.g.
res.socket.on('end', e => {
console.log('event source closed');
sseResponses = sseResponses.filter(x => x != res);
res.end();
});
If your server is trying to write to a socket closed on the browser, it should not raise an error, but will return false from res.write.
If both your server side code and client side code are hanging after you close the event source, you may have bugs on both sides.
More complete prototype, with your writeHead code from above.
var app = new (require('express'));
var responses = [];
app.get("/", (req, res) => {
res.status(200).send(`
<html>
<script>
var eventsource = null;
function connect() {
if (!eventsource) {
eventsource = new EventSource("/sse");
eventsource.onmessage = function(e) {
var logArea = window.document.getElementById('log');
logArea.value += e.data;
};
}
}
function disconnect() {
if (eventsource) {
var myeventsource = eventsource;
eventsource = null;
myeventsource.close();
}
}
</script>
<div>
<span>
Connect
Disconnect
<span>
</div>
<textarea id="log" style="width: 500px; height: 500px"></textarea>
</html>`);
});
app.get("/sse", (req, res) => {
res.writeHead(200, {
"content-type": "text/event-stream",
"cache-control": "no-cache"
});
res.socket.on('end', e => {
responses = responses.filter(x => x != res);
res.end();
});
responses.push(res);
});
app.listen(8080);
setInterval(() => {
responses.forEach(res => {
res.write('data: .\n\n');
});
}, 100);
I am using the node.js agent for the new relic. I am using Node.js custom instrumentation to monitor socket.io, new relic web transaction appearing under root and no data is visible as I am not able to monitor socket.io. Below is the output under Transaction Root Path
Breakdown table
Category Segment % Time Avg calls (per txn) Avg time (ms)
WebTransaction Root path 100.0 1.0 2,150
Server Code
var http = require('http'),
fs = require('fs'),
nr = require('newrelic'),
index = fs.readFileSync(__dirname + '/sock_client.html');
var app = http.createServer(function(req, res) {
res.writeHead(200, {'Content-Type': 'text/html'});
res.end(index);
});
// Socket.io server listens to our app
var io = require('socket.io').listen(app);
// Send current time to all connected clients
function sendTime() {
io.emit('time', { time: new Date().toJSON() });
}
// Send current time every 10 secs
setInterval(sendTime, 10000);
// Emit welcome message on connection
io.on('connection', function(socket) {
// Use socket to communicate with this particular client only, sending it it's own id
socket.emit('welcome', { message: 'Welcome!', id: socket.id });
socket.on('rings', function(data){
nr.startWebTransaction('websocket/ping/test_v3', function transactionHandler() {
socket.emit('pong', { message: 'Welcome!' });
console.log(data);
nr.addCustomParameters({
"Discount Code": "Summer Super Sale",
"Item Code": 31456
});
});
});
socket.on('pings', function(data){
nr.startWebTransaction('websocket/ping/test_v5', function transactionHandler() {
let trans = nr.getTransaction();
someAsyncBeha("console.log", function(){
trans.end();
})
console.log(data);
});
});
});
function someAsyncBeha(data, cb){
setTimeout(function() {
console.log("Goodbye!");
console.log(data);
cb();
}, 5000);
};
Client Code
<!doctype html>
<html>
<head>
<link rel="shortcut icon" href="/favicon.png">
<script src='/socket.io/socket.io.js'></script>
<script>
var socket = io();
socket.on('welcome', function(data) {
addMessage(data.message);
// Respond with a message including this clients' id sent from the server
socket.emit('rings', {data: 'foo!', id: data.id});
});
socket.on('time', function(data) {
addMessage(data.time);
socket.emit('pings', {data: 'foo!', id: data.time});
});
socket.on('error', console.error.bind(console));
socket.on('message', console.log.bind(console));
socket.on('pong', console.log.bind(console));
function addMessage(message) {
var text = document.createTextNode(message),
el = document.createElement('li'),
messages = document.getElementById('messages');
el.appendChild(text);
messages.appendChild(el);
}
</script>
</head>
<body>
<ul id='messages'></ul>
</body>
</html>
The issue was in the new relic module version, there are two suggestions.
First, upgrade the Node.js agent to v2.2.0
This version includes a bug fix that should resolve the issue you're seeing.
Second, move nr = require('newrelic'); to the top of the requires.
This ensures the agent catches everything in your application before the code has a chance to spin up. Without this, the code in the other requires can launch before the agent has a chance to inject its methods, resulting in no tracking for those methods.