Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I am using Node.js to get data from a MQTT broker (with this) and store in a MongoDB database. Now I want to visualize this data in a web app (possibly) in real time.
The data comes from different Internet Of Things node, so every time a node send data, I would that a button in HTMl become green or blink.
I am not an expert backend programmer, so what could be the best way, in terms of complexity? There are some intuitive libraries?
API ? Socket.io ? Moongose ?
WebSocket is similar to MQTT but with no emphasised protocol on how a client or server should communicate with each other.
Since you have already implemented MQTT in NodeJS, I would recommend using the following pattern to establish communication.
IOT -> sends DATA via MQTT -> Server receives DATA -> Server sends DATA via WebSocket -> Client receives DATA.
I am using socket.io, a popular WebSocket library for the purpose of demonstration.
STEP 1: Installing socket.io
npm install socket.io
STEP 2: Server side implementation
var app = require('express')();
var http = require('http').Server(app);
var mqtt = require('mqtt'); // This is for demonstration only, use your existing mqtt library instead.
var client = mqtt.connect('mqtt://test.mosquitto.org');
var io = require('socket.io')(http);
io.on('connection', function(socket) { // Whenever a new client is connected, this event is triggered
console.log("[Live] A new client connected.", socket.id);
socket.on('disconnect', function(socket) {
console.log("[Live]", socket.id, "got disconnected.");
});
});
client.on('message', function (topic, message) {
io.emit('live', message.toString()); // io.emit(channel, message);
// Broadcasts message to all client instance's that are subscribed to 'live' channel.
});
http.listen(3000, function(){
console.log('listening on *:3000');
});
STEP 3: Client side implementation
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/1.7.2/socket.io.js"></script>
<script>
var socket = io('http://localhost:3000');
socket.on('live', function (data) { // This event is triggered when server publishes data to 'live' channel
console.log(data);
});
</script>
Simple, isn't it. Hope it helps.
Solution 1
You could let your front end to subscribe your MQTT broker. The potential problem is the data may not go into your DB but it shows up on the web page.
Solution 2
You could open a web socket or stream between your backend and frontend so that the backend will push data in real time.
Related
I have a web application that accepts an input from the client (browser) and calls an application at the server side with the input. The problem is that the application will run for a long time (about 20s). In order to not block the client (browser) and show the progress to the user, my server side POST function returns immediately and feeds the user
input to the application. Once the application computed the result, I send the result to the client using socket.io. Another twist to the scenario is that the application can not be repeatedly launched and closed (will cause some problem with CUDA) so the whole server side runs just one instance of the application. A sketch of my server-side code looks like the following. Embedded in the code are the three questions that I have searched through the Internet and tried various solutions but could not work out. Any help on them is really appreciated. If this is not the right place to ask, I would be thankful to any feedback about where to ask this question.
var express = require('express');
const{ spawn }= require('child_process');
var app = express();
var http = require('http').Server(app);
var io = require('socket.io')(http);
var darknet = spawn('./darknet', other arguments...) ;
io.on('connection', function(socket){
app.socket = socket;
console.log('A connection has been established');
})
darknet.stdout.on('data',(data) =>{
// This function is called once darknet finished computation and wrote the result to the stdout
// Question 1: now need to send the data back to the client but have no access to any socket here?
})
io.on('connection', function(socket){
//Question 2: how to store this socket so that I can use it in a solution to the Question 1? Can I use session? */
//Question 3: I observed that once the client side refreshed its page (for example, uploaded an image), the previous socket will be closed and a new socket will be established. How do I make sure that I send the output from the application to the right socket since the original one may be invalid when the output is ready to be sent?
})
app.post('/predict', function (req,res){
darknet.stdin.write(user_input+'\n');
res.render();
})
In my express server, I am using socket.io and Redis pubsub.
The server subscribes a Redis message channel, and forwards Redis message to a specific websocket client when someone publishes new message via Redis.
From what i read from socket.io and Redis documentation, I can send message to a specific client by using the varible socket and call socket.broadcast.to(mySocketID).emit.
But in the following code, what should I do if I wanna emit message to mySocketID inside the redis subscriber, which is out of the scope of io.on('connection')?
var redis = require('redis');
var server = require('http').createServer(app);
var io = require("socket.io")(server);
io.on('connection', function (socket) {
socket.on('whatever_event', function() {
socket.broadcast.to(mySocketID).emit('TEST', 'hello ' + mySocketID);
}
}
var subscriber = redis.createClient();
subscriber.on('message', function (channel, redisMessage) {
// I want to send redisMessage to the websocket client
// Can I access to "socket" in io.on('connection' ...) ?
});
I did something similar with WebSockets a while back (If I understand what you're trying to do). Effectively, I had to create a in-process event proxy layer between the Redis Pub/Sub (plain ol' EventEmitter) and the websocket. Without this layer you're going to have a lot of unneeded connection overhead, by using it you can limit the number of open connections to Redis.
So, the subscriber.on section sends a EventEmitter event (yourEventsObject.emit(/*...*/)). Then inside your socket.io connection callback you respond to these events with listeners (yourEventsObject.on(/*...*/)). When your socket.io connection closes, you need to make sure and clean up these EventEmitter listeners so you don't get errors with closed sockets.
Here is the full file on full source on and the article describing the problem/solution with websockets.
My goal is to build a chat application - similar to whatsapp
To my understanding, socket.io is a real-time communication library written in javascript and it is very simple to use
For example
// Serverside
io.on('connection', function(socket) {
socket.on('chat', function(msg) {
io.emit('chat', msg);
});
});
// ClientSide (Using jquery)
var socket = io();
$('form').submit(function(){
socket.emit('chat', $('#m').val());
$('#m').val('');
return false;
});
socket.on('chat', function(msg){
$('#messages').append($('<li>').text(msg));
});
1) do I always need to start an io.on('connection') to use the real-time feature or i could just start using socket.on object instead? for example i have a route
app.post('/postSomething', function(req, res) {
// Do i need to start an io.on or socket.on here?
});
because i want the real-time feature to be listen only on specific route.
2) Redis is a data structure library which handles the pub/sub, why do we need to use pub/sub mechanism?
I read alot of articles but couldn't grasp the concept. Article example http://ejosh.co/de/2015/01/node-js-socket-io-and-redis-intermediate-tutorial-server-side/
for example the code below
// Do i need redis for this, if so why? is it for caching purposes?
// Where does redis fit in this code?
var redis = require("redis");
var client = redis.createClient();
io.on('connection', function(socket) {
socket.on('chat', function(msg) {
io.emit('chat', msg);
});
});
3) Just wondering why I need nginx to scale node.js application? i found this stackoverflow answer:
Strategy to implement a scalable chat server
It says something about load balancing, read that online and couldn't grasp the concept as well.
So far I have only been dealing with node.js , mongoose simple CRUD application, but I'm willing work really hard if you guys could share some of your knowledge and share some useful resources so that I could deepen my knowledge about all of these technologies.
Cheers!
Q. Socket.on without IO.on
io.on("connection" ... )
Is called when you receive a new connection. Socket.on listens to all the emits at the client side. If you want your client to act as a server for some reason then (in short) yes io.on is required
Q. Redis pub/sub vs Socket.IO
Take a look at this SO question/anwer, quoting;
Redis pub/sub is great in case all clients have direct access to redis. If you have multiple node servers, one can push a message to the others.
But if you also have clients in the browser, you need something else to push data from a server to a client, and in this case, socket.io is great.
Now, if you use socket.io with the Redis store, socket.io will use Redis pub/sub under the hood to propagate messages between servers, and servers will propagate messages to clients.
So using socket.io rooms with socket.io configured with the Redis store is probably the simplest for you.
Redis can act like a message queue if it is a requirement. Redis is a datastore support many datatypes.
Q. Why Nginx with Node.js
Node.js can work standalone but nginx is faster to server static content.
Since nginx is a reverse proxy therefore servers are configured with nginx to handle all the static data (serving static files, doing redirects, handling SSL certificates and serving error pages.
) and every other request is sent to node.js
Check this Quora post as well: Should I host a node.js project without nginx?
Quoting:
Nginx can be used to remove some load from the Node.js processes, for example, serving static files, doing redirects, handling SSL certificates and serving error pages.
You can do everything without Nginx but it means You have to code it yourself, so why not use a fast and proven solution for this.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
Currently there is a plethora of websocket libraries for node.js, the most popular seem to be:
https://github.com/Worlize/WebSocket-Node
https://github.com/einaros/ws
https://github.com/LearnBoost/engine.io
https://github.com/learnboost/socket.io
https://github.com/sockjs
However I can't find any solid concrete comparisons between any of them...
Apparently Socket.io was awesome, but has become quite dated and has failing builds. Both ws and websocket-node claim they are the fastest. And engine.io seems new, but a lot heavier than the lighter aletarntives.
It would be amazing if we or someone could put together an answer that serves as a guide on which socket library to use and when, as well as a comparison between them.
Getting the ball rolling with this community wiki answer. Feel free to edit me with your improvements.
ws
WebSocket server and client for node.js. One of the fastest libraries if not the fastest one.
websocket-node
WebSocket server and client for node.js
websocket-driver-node WebSocket server and client protocol parser node.js - used in faye-websocket-node
faye-websocket-node WebSocket server and client for node.js - used in faye and sockjs
socket.io
WebSocket server and client for node.js + client for browsers + (v0 has newest to oldest fallbacks, v1 of Socket.io uses engine.io) + channels - used in stack.io. Client library tries to reconnect upon disconnection.
sockjs
WebSocket server and client for node.js and others + client for browsers + newest to oldest fallbacks
faye WebSocket server and client for node.js and others + client for browsers + fallbacks + support for other server-side languages
deepstream.io clusterable realtime server that handles WebSockets & TCP connections and provides data-sync, pub/sub and request/response
socketcluster WebSocket server cluster which makes use of all CPU cores on your machine. For example, if you were to use an xlarge Amazon EC2 instance with 32 cores, you would be able to handle almost 32 times the traffic on a single instance.
primus Provides a common API for most of the libraries above for easy switching + stability improvements for all of them.
When to use:
use the basic WebSocket servers when you want to use the native WebSocket implementations on the clientside, beware of the browser incompatibilities
use the fallback libraries when you care about browser fallbacks
use the full featured libraries when you care about channels
use primus when you have no idea about what to use, are not in the mood for rewriting your application when you need to switch frameworks because of changing project requirements or need additional connection stability.
Where to test:
Firecamp is a GUI testing environment for SocketIO, WS and all major real-time technology. Debug the real-time events while you're developing it.
Update: This answer is outdated as newer versions of libraries mentioned are released since then.
Socket.IO v0.9 is outdated and a bit buggy, and Engine.IO is the
interim successor. Socket.IO v1.0 (which will be released soon) will
use Engine.IO and be much better than v0.9. I'd recommend you to use
Engine.IO until Socket.IO v1.0 is released.
"ws" does not support fallback, so if the client browser does not
support websockets, it won't work, unlike Socket.IO and Engine.IO
which uses long-polling etc if websockets are not available. However,
"ws" seems like the fastest library at the moment.
See my article comparing Socket.IO, Engine.IO and Primus:
https://medium.com/p/b63bfca0539
npm ws was the answer for me. I found it less intrusive and more straight forward. With it was also trivial to mix websockets with rest services. Shared simple code on this post.
var WebSocketServer = require("ws").Server;
var http = require("http");
var express = require("express");
var port = process.env.PORT || 5000;
var app = express();
app.use(express.static(__dirname+ "/../"));
app.get('/someGetRequest', function(req, res, next) {
console.log('receiving get request');
});
app.post('/somePostRequest', function(req, res, next) {
console.log('receiving post request');
});
app.listen(80); //port 80 need to run as root
console.log("app listening on %d ", 80);
var server = http.createServer(app);
server.listen(port);
console.log("http server listening on %d", port);
var userId;
var wss = new WebSocketServer({server: server});
wss.on("connection", function (ws) {
console.info("websocket connection open");
var timestamp = new Date().getTime();
userId = timestamp;
ws.send(JSON.stringify({msgType:"onOpenConnection", msg:{connectionId:timestamp}}));
ws.on("message", function (data, flags) {
console.log("websocket received a message");
var clientMsg = data;
ws.send(JSON.stringify({msg:{connectionId:userId}}));
});
ws.on("close", function () {
console.log("websocket connection close");
});
});
console.log("websocket server created");
I'm using Node.js, Socket.io with Redisstore, Cluster from the Socket.io guys, and Redis.
I've have a pub/sub application that works well on just one Node.js node. But, when it comes under heavy load is maxes out just one core of the server since Node.js isn't written for multi-core machines.
As you can see below, I'm now using the Cluster module from Learnboost, the same people who make Socket.io.
But, when I fire up 4 worker processes, each browser client that comes in and subscribes gets 4 copies of each message that is published in Redis. If there are are three worker processes, there are three copies.
I'm guessing I need to move the redis pub/sub functionality to the cluster.js file somehow.
Cluster.js
var cluster = require('./node_modules/cluster');
cluster('./app')
.set('workers', 4)
.use(cluster.logger('logs'))
.use(cluster.stats())
.use(cluster.pidfiles('pids'))
.use(cluster.cli())
.use(cluster.repl(8888))
.listen(8000);
App.js
redis = require('redis'),
sys = require('sys');
var rc = redis.createClient();
var path = require('path')
, connect = require('connect')
, app = connect.createServer(connect.static(path.join(__dirname, '../')));
// require the new redis store
var sio = require('socket.io')
, RedisStore = sio.RedisStore
, io = sio.listen(app);
io.set('store', new RedisStore);io.sockets.on('connection', function(socket) {
sys.log('ShowControl -- Socket connected: ' + socket.id);
socket.on('channel', function(ch) {
socket.join(ch)
sys.log('ShowControl -- ' + socket.id + ' joined channel: ' + ch);
});
socket.on('disconnect', function() {
console.log('ShowControll -- Socket disconnected: ' + socket.id);
});
});
rc.psubscribe('showcontrol_*');
rc.on('pmessage', function(pat, ch, msg) {
io.sockets.in(ch).emit('show_event', msg);
sys.log('ShowControl -- Publish sent to channel: ' + ch);
});
// cluster compatiblity
if (!module.parent) {
app.listen(process.argv[2] || 8081);
console.log('Listening on ', app.address());
} else {
module.exports = app;
}
client.html
<script src="http://localhost:8000/socket.io/socket.io.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.0/jquery.min.js"></script>
<script>
var socket = io.connect('localhost:8000');
socket.emit('channel', 'showcontrol_106');
socket.on('show_event', function (msg) {
console.log(msg);
$("body").append('<br/>' + msg);
});
</script>
I've been battling with cluster and socket.io. Every time I use cluster function (I use the built in Nodejs cluster though) I get alot of performance problems and issues with socket.io.
While trying to research this, I've been digging around the bug reports and similar on the socket.io git and anyone using clusters or external load balancers to their servers seems to have problems with socket.io.
It seems to produce the problem "client not handshaken client should reconnect" which you will see if you increase the verbose logging. This appear alot whenever socket.io runs in a cluster so I think it reverts back to this. I.E the client gets connected to randomized instance in the socket.io cluster every time it does a new connection (it does several http/socket/flash connections when authorizing and more all the time later when polling for new data).
For now I've reverted back to only using 1 socket.io process at a time, this might be a bug but could also be a shortcoming of how socket.io is built.
Added: My way of solving this in the future will be to assign a unique port to each socket.io instance inside the cluster and then cache port selection on client side.
Turns out this isn't a problem with Node.js/Socket.io, I was just going about it the completely wrong way.
Not only was I publishing into the Redis server from outside the Node/Socket stack, I was still directly subscribed to the Redis channel. On both ends of the pub/sub situation I was bypassing the "Socket.io cluster with Redis Store on the back end" goodness.
So, I created a little app (with Node.js/Socket.io/Express) that took messages from my Rails app and 'announced' them into a Socket.io room using the socket.io-announce module. Now, by using Socket.io routing magic, each node worker would only get and send messages to browsers connected to them directly. In other words, no more duplicate messages since both the pub and sub happened within the Node.js/Socket.io stack.
After I get my code cleaned up I'll put an example up on a github somewhere.