Node + Redis + PHP getting RealTime Issue - node.js

I'm building a Pub/Sub Server in NodeJS + Redis + PHP, after dig in on the subject and starting building something I came across some confusion that I saw on the process.
Example Scenario:
5 users are connected to the socket through NodeJscorrectly.
One of them click a button to send an custom alert to all the connected users except him
Php receive the the message for the alert trough a call ajax (for purpose of example)
Php Publish back to Redis
Redis receive the event and trigger socket.io to broadcast the event
Set up view:
<input type="text" id="message">
<button id="sendMessage">Send alert</button>
Client side Js
var socket = io.connect('http://127.0.0.1:3000/');
socket.on('notification', function (data) {
alert(data.message);
console.log('message received');
});
// Ajax event
$(document).on('click','#sendMessage', function(){
$.ajax({
url: '/send/notification',
method: 'POST',
type: 'JSON',
data: { message: $('#message').val() },
success:function(data)
{
console.log('message sent');
}
})
});
Php Server Side for Communicate to redis
$message = $_POST['message'];
$redis = new Predis\Client();
$redis->publish('notification',$message);
NodeJs
At this point on my NodeJs server I will go to listen for the message event in redis for then broadcast the event to sockets but here is the first issue I met.
var http = require('http');
var app = http.createServer();
var io = require('socket.io')(app);
var redis = require('redis');
app.listen(3000,'127.0.0.1',function(){
console.log("listening:", app.address().port, app.address().address);
});
io.on('connection', function(client){
var redisClient = redis.createClient();
redisClient.subscribe('notification');
redisClient.on("message",function(channel,data){
// This line should broadcast to all the client except the sender
socket.broadcast.emit(channel,JSON.parse(data));
console.log(channel);
});
});
Issue
At this point where I console.log() the channel I can see on my terminal 5 logs "notification" Why 5?
When socket.io broadcast the event, it does that 5 times, sending in this case 5 alert() to the clients and 4 to the sender instead of 1 for all clients a 0 for the sender.
The times are depending on how many users are connected I really don't understand what I missed, because it shouldn't be the right behaviour.
I already tried to put the creation of the redisClient and the subscription of a channel outside of the io.on('connection') with no luck, same result.
Another Test
if I broadcast the event on the connection of the socket like so:
io.on('connection', function(client)
{
client.broadcast.emit('notification','hello');
});
It works correctly, so I think is redis problem. Any suggest will be really appreciated.

You are having 5 clients connected, so the "connection" event is fired 5 times.
Thus there are 5 listeners for redis which all then broadcast.
There are two ways to do it correctly:
a) 1 listener per connection, only send the message to the connection
io.on('connection', function(socket){
var redisClient = redis.createClient();
redisClient.subscribe('notification');
redisClient.on("message",function(channel,data){
// This line should broadcast to only the current connection
socket.emit(channel,JSON.parse(data));
console.log(channel);
});
});
b) 1 global listener, broadcast to all clients
var redisClient = redis.createClient();
redisClient.subscribe('notification');
redisClient.on("message",function(channel,data){
// This line should broadcast to all the client except the sender
io.sockets.emit(channel,JSON.parse(data));
console.log(channel);
});
I would prefer method b) as it only needs one redis connection.

I have done same in one project.Only difference is I publish notification from node.js but i did not get any issue.There is broadcast only open socket.

Related

Trigger `socket.io` event from server(`node.js`), not from client

I am trying to make a game server with node.js, socket.io.
The basic idea likes below.
Initialize socket.io instance when the server starts
Store instance in global scope, so controllers can access it
When API calls, we trigger some socket.io event in the controller or some other points
Here is the implementation I made ...
First, in server.js - entry point
let GlobalVars = require('./state/GlobalVars');
const apiRouters = require('./router');
...
app.use('/api', apiRouters);
app.get('/', (req, res) => {
res.sendFile(`${__dirname}/test/simpleClient.html`)
});
const httpServer = http.createServer(app);
let socketIOInstance = socketIO(httpServer);
socketIOInstance.on('connection', (socket) => {
console.log('SOCKET.IO A USER CONNECTED');
socket.on('create', (data) => {
console.log('SOCKET.IO create called', socket);
socket.join(data.room);
socketIOInstance.emit('message', 'New people joined');
});
socket.on('join', (data) => {
console.log('SOCKET.IO join called', data);
})
socket.emit('message', 'Hi');
});
GlobalVars.socketIO = socketIOInstance;
// Add to global, so the controllers can manage own actions like create, join ...
httpServer.listen(port, () => {
console.log(`Server Listening on the port ${port}`);
})
...
When I access from a client, I am able to see SOCKET.IO A USER CONNECTED and Hi in the browser console.
Second, In api controller.
let GlobalVars = require('../state/GlobalVars');
...
router.post('/create', (req, res) => {
console.log('GenerateGameSokect');
let game = new Game();
let gameId = game.gameId;
// console.log('Global vars ', GlobalVars.socketIO);
GlobalVars.socketIO.emit('create', {
room: gameId
});
res.json({
result : 'SUCCESS',
game : game
})
});
I imported GlobalVars which contains socketIO instance. So what I expected was, socket create event triggered from the statement GlobalVars.socketIO.emit('create', Object) but could not find message in the server logs.
I got no clue what I was missing.
The final form I pursue is something like...
When user call create API, I creates socket connection and room
API will called in HTTP protocol, but in the API, the server publishes some events. - pubsub like.
Thanks for reading my questions b. Here is full source code till now(bitbucket public)
================== EDIT ====================
I got understood (maybe...)
The user-flow I wanted was ...
The client call API
(In the server) Checking validation in API and if valid emit to socket.io
If event accepted send new status to all clients
However, creating socket.io connection in the server looks strange for me, the solution is up to the client.
New user-flow I will change
The client call a validation API
If return is valid, the client emit socket.io event. This time server only do validation, not emit socket.io
In socket event, send new status to all other users
================== EDIT #2 ====================
This is a kind of conclusion. It looks I just misunderstanding the concept of socket communication. Like answer and replies say, Socket and HTTP are totally different channels, there is no way to connect both. (At least, without open new connection from http server to socket)
If this is wrong, you could add reply, Thanks
Now I understand you. Or at least I think!
Let's put it this way: there are two (asymetric) sides on a socket, server and client. What I called, respectively, "global manager" and "socket" in my comment to your post.
const server = require('socket.io')(yourHttpServer);
// client is installed as well when `npm i socket.io`
const client = require('socket.io-client')('http://localhost:' + yourServerPort);
// `socket` is the server side of the socket
server.on('connection', (socket) => {
// this will be triggered by client sides emitting 'create'
socket.on('create', (data) => {
console.log('a client socket just fired a "create" event!');
});
});
// this will be triggered by server side emitting 'create'
client.on('create', (data) => {
server.emit('create', {content: 'this will result in an infinite loop of "create" events!'});
});
In your /create route, when you GlobalVars.socketIO.emit('create', ...), the server-side socket handler isn't triggered, however if you have clients connected through a browser (or, like I showed above, if you connect a client socket directly from the server) then these will trigger their 'create' listener, if any.
Hope this helps you get on the right tracks!

Nodejs request proxy stream(mjpeg) connection never ends

(unnecessary backstory)
I have a nodejs server with expressjs framework that's proxy streaming a webcam feed. The reason I need this is because the mjpg stream must come from this server due to complex CORS issues.
//proxy from webcam server to avoid CORS complaining
app.get('/stream1',function(req,res){
var url="http://camera.nton.lviv.ua/mjpg/video.mjpg"
request(url).pipe(res);
});
question :
The issue is simple. request(url).pipe(res) never closes, because the source is mjpeg which literally never ends. I need to find a way to force close this pipe when the client(browser; the destination) is no longer available - as in, closes the window.
The other answers did not work for me.
This line var pipe=request(url).pipe(res);
returns the pipe instead of the request object. So I needed to break the line up.
The request object is needed to abort. Calling the .end() didn't work either, but the .abort() did the trick. It took me hours to find the answer that worked for me, so I thought I would share.
app.get('/cam/frontdoor',function(req,res){
var request_options = {
auth: {
user: '',
pass: ''},
url: 'http:/xx.xx.xx.xx/mjpg/video.mjpg',
};
var req_pipe = request(request_options);
req_pipe.pipe(res);
req_pipe.on('error', function(e){
console.log(e)
});
//client quit normally
req.on('end', function(){
console.log('end');
req_pipe.abort();
});
//client quit unexpectedly
req.on('close', function(){
console.log('close');
req_pipe.abort()
})
})
Use socket.io to monitor the remote connection
// install it on your project
npm install socket.io
// require it on server side
var socket = require('socket.io');
// listen for sockets from your server
var mysocks = socket.listen(myexpressappvar);
// keep collection of sockets for use if needed
// its good practice
var connectedSockets = [];
// add event handelers on server side
mysocks.sockets.on("connection", function(socket){
// add socket to our collection
connectedSockets.push(socket);
// you will need to bind their stream id here.
exe.....
// listen for disconnected
socket.on("disconnect", function(){
// remove socket from collection
connections.splice(connections.indexOf(socket), 1);
// destory stream here
exe...
});
});
// last thing, is add socket.io to the client side.
<script src="/socket.io/socket.io.js"></script>
// then call connect from client side js file
var socket = io.connect();
I have found out a simpler way. Add a event listener for client connection closing, and force close the pipe when it happens.
app.get('/stream1',function(req,res){
var url="http://camera.nton.lviv.ua/mjpg/video.mjpg"
var pipe=request(url).pipe(res);
pipe.on('error', function(){
console.log('error handling is needed because pipe will break once pipe.end() is called')
}
//client quit normally
req.on('end', function(){
pipe.end();
}
//client quit unexpectedly
req.on('close', function(){
pipe.end();
}
});

Socket.io Callback Function Parameter Confused

I am not sure I understand the server side configuration of the socket.
var app = express();
var server = require('http').createServer(app);
var socketio = require('socket.io')(server, {
serveClient: config.env !== 'production',
path: '/socket.io-client'
});
Here, the code creates a socket server "attached" with the http server according to the api reference. What is attach?
socketio.on('connection', function (socket) {
socket.on('create', function(room) {
console.log('joining a room');
socket.join(room);
console.log('socket joined room: ', room);
});
socket.address = socket.handshake.address !== null ?
socket.handshake.address.address + ':' + socket.handshake.address.port :
process.env.DOMAIN;
socket.connectedAt = new Date();
// Call onDisconnect.
socket.on('disconnect', function () {
onDisconnect(socket);
console.info('[%s] DISCONNECTED', socket.address);
});
// Call onConnect.
onConnect(socket);
console.info('[%s] CONNECTED', socket.address);
});
};
Question: Here, the 'socket' variable is server's or client's ? if it is server's then why the socket.join(room) works ? (client's been added to a room) If it is client's, then why it has to listen to 'create' event.(client emits an event called create to change the room.)
to conclude, I confused by the three 'socket' in the following code.
socketio.on('connection', function (socket) {
socket.on('create', function(room) {
console.log('joining a room');
socket.join(room);
console.log('socket joined room: ', room);
});
});
Here, the code creates a socket server "attached" with the http server according to the api reference. What is attach?
webSocket connections (which socket.io is built on top of) all get initiated from the client by first making an HTTP connection. Thus, there must be an HTTP server that can be used for socket.io connections. That's why the initialization of socket.io needs an HTTP connection. Often, that web server is also acting as a normal web server too and thus can be used for both purposes. This simplifies cross-origin issues since all browsers all clients to connect to the same origin from which their web page was served. If, you don't already have another web server, socket.io can create it's own.
Question: Here, the 'socket' variable is server's or client's ?
It is the server-side object that represents the connection to a particular client. Socket.io has a socket object on the client that represents the client-side connection to the server and it has a socket object on the server (for each separate client connection) that represents the connection to a particular client.
So, if you want to send data to a particular client, you use the socket object that represents that client and you do:
socket.emit(msg, data);
if it is server's then why the socket.join(room) works ?
This works because this socket object represents the connection to a particular client. It is a server-side object, but it is specific to a particular client (e.g. there's a different one for each client connection).
to conclude, I confused by the three 'socket' in the following code.
socketio.on('connection', function (socket) {
socket.on('create', function(room) {
console.log('joining a room');
socket.join(room);
console.log('socket joined room: ', room);
});
});
socketio represents the overall socket.io module and is how you execute global commands. In this case, you are executing a global command to listen for any newly connection clients.
When you get a connection event, the argument to that event is the newly created socket object (a server-side object that represents a connection to a particular client).
The socket.on('create', ...) is a listener for the create message sent from the client to the server. So, this line of code says to listen for a create message sent from this particular (newly connected) client and when that message arrives, call a callback and pass it the data sent with the message (in this case a room name).
The socket.join(room); line uses the same socket as above and joins it to a specific room on the server.

NodeJS, Express, and eventful Socket.io

I want to build a NodeJS API so that when I hit an endpoint, the app will trigger an event that will cause its unique socket connection to emit a message to its listeners. I have built a solution before using Python/Django, Redis, and NodeJS/Socket.io with Django as the API and Redis as the 'event trigger', but I would like to consolidate the different technologies into NodeJS and Socket.io.
I tried moving the socket.emit() code into different modules and then app.use()'d those modules, but the code broke because it didn't have an instance of the socket.
I also know that you can broadcast to all socket connections inside on an endpoint, for example:
app.use('socket.io/help', function(req, res) {
console.log(req.query);
io.sockets.emit('help_message', 'You should help me.');
res.send('help msg sent');
});
But I am looking for a way that allows a client (that doesn't have a socket connection) to hit an endpoint and pass a query param that tells NodeJS which of its connected sockets to send a message to.
Is this possible? Or am I trying to fight the framework? e.g., is there a different way of doing this with different JS WebSocket frameworks/technologies?
I have been stuck on same situation but resolved easily
you have created socket on app.js
server = require('http').createServer(app),
io = require('socket.io').listen(server);
server.listen(port);
global.socketIO = io;
Now you can call this io instance to your any controller like
app.use('socket.io/help', function(req, res) {
console.log(req.query);
var io = global.socketIO;
// UID IS THE VARIABLE
io.sockets.emit('help_message'+UID, 'You should help me.');
res.send('help msg sent');
});
CLIENT SIDE
<script src="/socket.io/socket.io.js"></script>
<script>
window.socket.on("help_message_<%-UID%>", function(msg){
//// ACTION
});
You can join a room with the specific sockets you want to recieve the messages on.
see Rooms & Namespaces in the socket.io documentation
join a chan on your helpdesk conenctions:
io.on('connection', function(socket){
socket.join('helpdesk');
});
and broadcast to them:
app.use('socket.io/help', function(req, res) {
console.log(req.query);
var io = global.socketIO;
io.sockets.emit('adduser', req.body.uid);
io.to('helpdesk').emit('some event'):
res.send('help msg sent');
});

Node js net event : connect vs connection

When is the connection event of the net.Server fired vs the net.Socket connect event. Are they the same event? I have seen a code example where the handler for the net.createServer function (which handlers the connection event) also has the following code in it
var server = net.createServer(function (client) {
var id = client.remoteAddress + ':' + client.remotePort;
client.on('connect', function() {
channel.emit('join', id, client);
});
client.on('data', function(data) {
data = data.toString();
channel.emit('broadcast', id, data);
});
});
Is this incorrect? Is this listener not needed/never hit..i.e. the emit should be outside the listener.
Based on the code you post here, I'll assume you are reading Node.js in Action and it's the sample code that cause you the problem. If this is the case, you may reference to this similar question: NodeJS events on('connect') error.
Summary
In short, these 2 events are received by 2 different objects.
connect event is received by socket object and is emitted "when a socket connection is successfully established". On ther other hand, connection event is received by server object and is emitted "when a new connection is made".
Notice when a client hit node and create a socket, the callback for net.createServer(callback) is automatically called, thus you don't have to manually register another event handler on this client-server socket, which is client.on('connect', function() { }); in your code.
Concepts
As #3y3 mentioned, there are 2 ways to use net module in node.js, which are createConnection and createServer.
To better understand how it works, you may use this diagram as sample:
net.createServer
When you require net module and createServer, you basically create a server for clients (browser via http, terminal via telnet, etc.) to connect in.
Consider the following codes:
var net = require('net');
var events = require('events');
var channel = new events.EventEmitter();
var server = net.createServer(function (socket) {
var id = socket.remoteAddress + ': ' + client.remotePort;
console.log('Server connected', id);
channel.emit('join', id, socket);
socket.on('data', function(data) {
data = data.toString();
channel.emit('broadcast', id, data);
});
}).listen(8888);
In this case, the parameter in the callback of createServer is the "socket" between server and client. In the sample code, the author of Node.js in Action call it client, which might be a little confusing for you, but that's actually the same concept: it's a client related object which contains methods for your server to do something.
Notice that this callback is registered on the server and called at the same time the connection is built. This is a result of connection event which is emitted to the server when the client hit port 8888 on server in this case. Any functions you put inside this callback will be execute immediately. In our case, there are three things we did:
Console.log the client id on server
Emit a 'join' event to channel object
Register a event listener for 'data' event (i.e. called when there's data sending from client)
There's no need for client.on('connect', function() { // do something }), and since the callback for server connection event is the one we just mentioned. So what's the purpose of clinet.on('connect')?
net.createConnection
By utilizing this function, you create "a connection to the server" rather then create a server itself. In other words, it just like you open a terminal and use telnet to connect to the server, only that this happens in your server code base (i.e. in your node environment, as the diagram Client x shows)
Consider the following code: (based on #3y3's example again):
var Client = net.createConnection;
var client = Client({port: 8888, localAddress: '127.0.0.1', localPort: 51000});
client.on('connect', function() {
var id = this.localAddress + ': ' + this.localPort;
console.log('Client connected', id);
});
Here, we build a client in node and connect to the server through port 8888, which is what we defined previously. localPort and localAddress are defined arbitrary, just a mimic for a terminal connection in local environment.
We then register an event handler on this client, so when this client is connected to the server, it will receive a connect event and execute this event handler. If you run both snippets in the same file with node, you'll see both
Client connected 127.0.0.1: 51000
Server connected ::ffff:127.0.0.1: 51000
in the console.
Conclusion
If you want to do something when remote clients connect to the server, simply put lines you want to execute in the callback function of net.createServer(function(socket) { // your lines here }).
Use client.on('connect', function() {}); for other manually build clients in your node environment.
For further information about socket object and server object, you may refer to the official document here.
You can play with the following revised codes in your case:
var events = require('events');
var net = require('net');
var channel = new events.EventEmitter();
channel.clients = {};
channel.subscriptions = {};
channel.on('join', function (id, client) {
this.clients[id] = client;
this.subscriptions[id] = function (senderId, message) {
if (senderId !== id) {
this.clients[id].write(message);
}
};
this.on('broadcast', this.subscriptions[id]);
});
var server = net.createServer(function (client) {
var id = client.remoteAddress + ': ' + client.remotePort;
console.log('Server connected', id);
channel.emit('join', id, client);
client.on('data', function(data) {
data = data.toString();
channel.emit('broadcast', id, data);
});
});
server.listen(8888);
var Client = net.createConnection;
var client = Client({port: 8888, localAddress: '127.0.0.1', localPort: 51000});
client.on('connect', function() {
var id = this.localAddress + ': ' + this.localPort;
console.log('Client connected', id);
});
If your code is same as:
var Client = require('net').createConnection,
client = Client({port:4321});
client.on('connect', function(){
//channel.emit('join', id, client);
channel.emit('join', id, this); //avoid bad closure
});
It is valid code and when connect will emited, the server part emits connection, creates socket object and pass it to connection callback
UPDATE:
Your code is incorrect. You pass to createServer the callback for connection event it same as:
var server = net.createServer();
server.on('connection', callback);
In callback you have yet connected socket object. This is correct:
var server = net.createServer(function (client) {
var id = client.remoteAddress + ':' + client.remotePort;
channel.emit('join', id, client);
client.on('data', function(data) {
data = data.toString();
channel.emit('broadcast', id, data);
});
});
Please read the official Nodejs document for module 'net'.
The crux of your matter is, you have a server socket that you have named 'client' (a confusing name) and it is not a client socket. For a client socket you can subscribe to 'connect' event and not for a server socket.
net.createServer(callback) takes a callback which is called back when a new connection from a client occurs and the callback is passed the server socket. What you are referring to in function(client) is essentially a server socket and the term used 'client' is misleading. Please see the Node.js 'net' module example of createServer. For a server socket all the events listed (e.g data, timeout, end, close etc) are valid except the 'connect' event.
On the contrary to a server socket, if you are to create a client program where you called net.createConnection that will return you a client socket. Of that socket you subscribe to the 'connect' event.

Resources