I've ran into a fairly difficult to debug error with my node web server.
Background
I'm creating a node server with socket.io to provide a restful service, connected to mongodb which use web sockets(socket.io) for server-client messages.
Issue
In my node app, I've used an npm package called node-scheduler, in which I do some processing at set times(these are very dynamic times but work fairly well to date).
So I'll set off a job, using node-scheduler and when it ends you can provide a function.
In this function I emit a web socket message, exactly how I emit messages in the rest of the application but my client side never receives the message.
Checking the logs the client disconnections then re connections after the function has completed.
I've debugged a little further, and I send two messages to the client in this function. Only one of them is processed by the client. May be a client issue not a server issue.
Any ideas for solutions or suggestions would greatly be appreciated.
Well generally socket.io is only meant to be used as a "channel". You should have the Client exist as a separate entity in memory or something, and update the socket if and when it reconnects. Otherwise you're just sending to the past (disconnected) sockets.
Using passport you can identify a client as a user.
app.get('/', function(req, res){
// req.user;
});
Using passport.socketio you can get the same user in your socket
io.on('connection', function(socket){
// socket.request.user;
socket.request.user.socket = socket;
// this will be updated with the latest socket in case of a future reconnection
// So now you can be sure that user object will always have the latest socket
nodeScheduler(function(){
carryOutJobs(function callback(){
socket.request.user.socket.emit('done');
// will always emit to the "latest" socket.
});
});
});
Related
I'm working on a Node/Express backend that uses the socket.io library to send events to the frontend. We've decided to upgrade it (both in the backend and frontend) to version 3 and we're having an issue that we can't solve so far. I'll try to explain it:
Whenever a client authenticates with the backend, it opens a socket connection between them
We use the express-session library to save the new socket ID into the session with the following code:
this._socketIO.on('connection', socket => {
// Add the socket id to the session
socket.request.session.socketId = socket.id;
socket.request.session.save();
logger().info(`WebSocket: User connected (ID: ${socket.id})`);
socket.on('disconnect', () => {
logger().info(`WebSocket: User disconnected (ID: ${socket.id})`);
});
});
After updating the socket library to v3.0.4, the session doesn't have the socketId field anymore.
I've realized that if I restart the backend server, while there's a logged client in the UI (and, so, there's an existing session file), it works fine. If I log out from the UI and log in again (so it creates a new session file), it doesn't work again.
I've been checking the migration documentation from the official page (https://socket.io/docs/v3/migrating-from-2-x-to-3-0) but I couldn't find an answer for this problem.
Any clues? Thanks in advance,
We've found out that the problem is because we've registered 2 session middlewares, one for the regular requests and another one for the socket requests (to save the associated socket ID) and sometimes it fails due to a race condition. Depending on which requests is managed first, the session is saved with the socket id or not.
We've delayed the socket connection after all the needed login requests are done and now it's working fine.
Cheers,
I want to emit an event to the client when a long fucntion comes to an end.
This will show a hidden div with a link - on the client side.
This is the approach i tested:
//server.js
var express = require('express');
var app = express();
var http = require('http').Server(app);
var io = require('socket.io')(http);
require('./app/routes.js')(app, io);
//routes.js
app.post('/pst', function(req, res) {
var url = req.body.convo;
res.render('processing.ejs');
myAsyncFunction(url).then(result => {
console.log('Async Function completed');
socket.emit('dlReady', { description: 'Your file is ready!'});
//do some other stuff here
}).catch(err => {
console.log(err);
res.render('error.ejs');
})
});
I get this
ERROR: ReferenceError: socket is not defined
If i change the socket.emit() line to this:
io.on('connection', function(socket) {
socket.emit('dlReady', { description: 'Your file is ready!'});
});
Then i don't receive an error, but nothing happens at the client.
This is the client code:
<script>
document.querySelector('.container2').style.display = "none";
var socket = io();
socket.on('dlReady', function(data) { //When you receive dlReady event from socket.io, show the link part
document.querySelector('.container1').style.display = "none";
document.querySelector('.container2').style.display = "block";
});
</script>
This whole concept is likely a bit flawed. Let me state some facts about this environment that you must fully understand before you can follow what needs to happen:
When the browser does a POST, there's an existing page in the browser that issues the post.
If that POST is issued from a form post (not a post from Javascript in the page), then when you send back the response with res.render(), the browser will close down the previous page and render the new page.
Any socket.io connection from the previous page will be closed. If the new page from the res.render() has Javascript in it, when that Javascript runs, it may or may not create a new socket.io connection to your server. In any case, that won't happen until some time AFTER the res.render() is called as the browser has to receive the new page, parse it, then run the Javascript in it which has to then connect socket.io to your server again.
Remember that servers handle lots of clients. They are a one-to-many environment. So, you could easily have hundreds or thousands of clients that all have a socket.io connection to your server. So, your server can never assume there is ONE socket.io connection and sending to that one connection will go to a particular page. The server must keep track of N socket.io connections.
If the server ever wants to emit to a particular page, it has to create a means of figuring out which exact socket.io connect belongs to the page that it is trying to emit to, get that particular socket and call socket.emit() only on that particular socket. The server can never do this by creating some server-wide variable named socket and using that. A multi-user server can never do that.
The usual way to "track" a given client as it returns time after time to a server is by setting a unique cookie when the client first connects to your server. From then on, every connection from that client to your server (until the cookie expires or is somehow deleted by the browser) whether the client is connection for an http request or is making a socket.io connection (which also starts with an http request) will present the cookie and you can then tell which client it is from that cookie.
So, my understanding of your problem is that you'd like to get a form POST from the client, return back to the client a rendered processing.ejs and then sometime later, you'd like to communicate with that rendered page in the client via socket.io. To do that, the following steps must occur.
Whenever the client makes the POST to your server, you must make sure there is a unique cookie sent back to that client. If the cookie already exists, you can leave it. If it does not exist, you must create a new one. This can be done manually, or you can use express-session to do it for you. I'd suggest using express-session because it will make the following steps easier and I will outline steps assuming you are using express-session.
Your processing.ejs page must have Javascript in it that makes a socket.io connection to your server and registers a message listener for your "dlready" message that your server will emit.
You will need a top-level io.on('connection', ...) on your server that puts the socket into the session object. Because the client can connect from multiple tabs, if you don't want that to cause trouble, you probably have to maintain an array of sockets in the session object.
You will need a socket.on('disconnect', ...) handler on your server that can remove a socket from the session object it's been stored in when it disconnects.
In your app.post() handler, when you are ready to send the dlready message, you will have to find the appropriate socket for that browser in the session object for that page and emit to that socket(s). If there are none because the page you rendered has not yet connected, you will have to wait for it to connect (this is tricky to do efficiently).
If the POST request comes in from Javascript in the page rather than from a form post, then things are slightly simpler because the browser won't close the current page and start a new page and thus the current socket.io connection will stay connected. You could still completely change the page visuals using client-side Javascript if you wanted. I would recommend this option.
I am trying to use Socket.io and Sequelize to create a chat app. Socket.io will handle the socket to allow for instant messaging. Sequelize will handle storing the messages so when you refresh the screen you still have your messages.
What is happening is that on localhost my socket works, but it does not send the messages to the database. When I put it onto Heroku, my database worked, but it does not use the sockets.
My socket is located in app.js and my database route is located in routes/messages.js.
I have been working on this bug for a while now and I have been trying to get help with it. I think the best way to share this is with my markdown I created detailing my efforts to fix my bug that can be found at here. My repo for this can be found here.
There are a few different parts that you need to distinguish:
the HTTP server, in your code represented by the variable http
the Express app, represented by app
the Socket.IO server, represented by io
a Socket.IO (client) connection (see below)
The HTTP server directs "normal" HTTP requests to the Express app, which will handle them according to the middleware and routes that are set up. A router handler gets called with (at least) two arguments, generally called req and res, to represent the (incoming) HTTP request and the (outgoing) HTTP response.
The Socket.IO server gets to handle specific Socket.IO requests, which get sent to the server by the Socket.IO client (running in the browser). When such a client sets up a connection with the server, the connection event gets triggered on the server. Any handlers for this event will get passed an argument, generally called socket, that represents the (bidirectional) connection with that client.
That Socket.IO connection can receive messages (sent from the client running in the browser), which trigger events on the socket. You can install a handler to listen for particular messages (like "chat message"), which will receive, as argument, the data that was sent to it by the client.
The issue in your code seems to be with setting up everything to handle those chat messages. The correct setup order would be:
listen on the Socket.IO server for connection events
when such an event is received, add a listener for the chat message event on the connection
when such an event is received, write the data to the database.
In code:
// Listen for new client connections.
io.on('connection', function(socket) {
// Listen for the client to send a _"chat message"_ message.
socket.on('chat message', function(data) {
// Store the data in the database.
models.Messages.create({
message : data.message,
username : data.username
});
});
});
As you can see, req and res aren't available inside of those Socket.IO event handlers, because those are only used for normal HTTP requests.
Also, as opposed to HTTP, you don't necessarily have to send anything back to the client when you have received a message, so I left that part out. The handler above only writes the message data to the database (it also doesn't check for, or handle, errors, which eventually you should add).
My goal is to build a chat application - similar to whatsapp
To my understanding, socket.io is a real-time communication library written in javascript and it is very simple to use
For example
// Serverside
io.on('connection', function(socket) {
socket.on('chat', function(msg) {
io.emit('chat', msg);
});
});
// ClientSide (Using jquery)
var socket = io();
$('form').submit(function(){
socket.emit('chat', $('#m').val());
$('#m').val('');
return false;
});
socket.on('chat', function(msg){
$('#messages').append($('<li>').text(msg));
});
1) do I always need to start an io.on('connection') to use the real-time feature or i could just start using socket.on object instead? for example i have a route
app.post('/postSomething', function(req, res) {
// Do i need to start an io.on or socket.on here?
});
because i want the real-time feature to be listen only on specific route.
2) Redis is a data structure library which handles the pub/sub, why do we need to use pub/sub mechanism?
I read alot of articles but couldn't grasp the concept. Article example http://ejosh.co/de/2015/01/node-js-socket-io-and-redis-intermediate-tutorial-server-side/
for example the code below
// Do i need redis for this, if so why? is it for caching purposes?
// Where does redis fit in this code?
var redis = require("redis");
var client = redis.createClient();
io.on('connection', function(socket) {
socket.on('chat', function(msg) {
io.emit('chat', msg);
});
});
3) Just wondering why I need nginx to scale node.js application? i found this stackoverflow answer:
Strategy to implement a scalable chat server
It says something about load balancing, read that online and couldn't grasp the concept as well.
So far I have only been dealing with node.js , mongoose simple CRUD application, but I'm willing work really hard if you guys could share some of your knowledge and share some useful resources so that I could deepen my knowledge about all of these technologies.
Cheers!
Q. Socket.on without IO.on
io.on("connection" ... )
Is called when you receive a new connection. Socket.on listens to all the emits at the client side. If you want your client to act as a server for some reason then (in short) yes io.on is required
Q. Redis pub/sub vs Socket.IO
Take a look at this SO question/anwer, quoting;
Redis pub/sub is great in case all clients have direct access to redis. If you have multiple node servers, one can push a message to the others.
But if you also have clients in the browser, you need something else to push data from a server to a client, and in this case, socket.io is great.
Now, if you use socket.io with the Redis store, socket.io will use Redis pub/sub under the hood to propagate messages between servers, and servers will propagate messages to clients.
So using socket.io rooms with socket.io configured with the Redis store is probably the simplest for you.
Redis can act like a message queue if it is a requirement. Redis is a datastore support many datatypes.
Q. Why Nginx with Node.js
Node.js can work standalone but nginx is faster to server static content.
Since nginx is a reverse proxy therefore servers are configured with nginx to handle all the static data (serving static files, doing redirects, handling SSL certificates and serving error pages.
) and every other request is sent to node.js
Check this Quora post as well: Should I host a node.js project without nginx?
Quoting:
Nginx can be used to remove some load from the Node.js processes, for example, serving static files, doing redirects, handling SSL certificates and serving error pages.
You can do everything without Nginx but it means You have to code it yourself, so why not use a fast and proven solution for this.
Currently, I am implementing an API using nodejs express, then it needs to connect to socket.io and send event.
The API is located in socket.io-client (client), and it connects to socket.io (server)
1st API call: success
The connection is made for the 1st call of the API, message is sent and socket can be disconnected, with the 'disconnect' callback is invoked both on client and server side.
2nd API call: failure
When the API is invoked the 2nd time, the connection to server cannot be made, 'client' callback on client side is not called.
3rd API call: success
Then I tried to restart the client side, keeping other things unchanged. The API is called again, and the connection to socket.io is made successfully and everything is fine.
Can anyone explain the logistics behind this?
Updated
client.js
App.getByUserId(message.to_id, function(error, app) {
var socket = io.connect('http://127.0.0.1:9002');
socket.on('connect', function(){
console.log("client connect socket id:" + socket.id);
console.log("appkey:" + app.private_token);
socket.emit('appkey.check',{appkey: app.private_token, uuid: message.to_id.uuid}, function(data){
socket.emit("forceDisconnect");
socket = null;
});
});
You just hit one of Socket.IO's many "features" or "bugs" depending how you see this. Socket.IO tries to be smart and re-use connections (which causes a lot of connection issues actually) The way around this is use the force new connection option in your io.connect:
io.connect('http://127.0.0.1:9002', { 'force new connection': true });
What you could also do is use https://github.com/primus/primus which wraps Socket.IO if you use the socket.io transformer. Internally, it completely removes the use of the io.connect and uses the much more lower level io.Socket constructor to create more stable connections that you would get with a stock socket.io.
With socket 1.0+, you have to use this for forcing new connection.
io.connect(SERVER_IP, { 'forceNew': true });