Is it possible to detect new data from the server as it is sent? For example, with express.js:
res.write('Processing 14% or something');
and then display that on the page with a progress bar.
Edit:
My original question was a bit confusing so let me explain the situation. I have created a page where users can upload song files. These files are then converted (using ffmpeg) to .ogg and .mp3 files for the web. The conversion takes a long time. Is it possible to send real time data about the conversion back to the client using the same XMLHttpRequest that sent the files?
If i understand correctly you are trying to implement event based actions. Yes node.js has got some excellent web socket libraries such as socket.io and sack.js
You need to understand nodejs event driven pattern.
Websocket protocol helps maintain full duplex connection between server and client. You can notify clients when any action happens in server and similar you can notify server when any action happens in client. Libraries provide flexibility to broadcast event to all connected client or selected ones.
So it is basically emit and on that you will be using often.
Go through the documentation, it will not take much time to learn. Let me know if you need any help.
Related
I wonder how do I use socket.io properly with my express app.
I have a REST API written in express/node.js and I want to use socket.io to add real-time feature for my app. Consider that I want to do something I can do just by sending a request to my REST API. What should I do with socket.io? Should I send request to the REST API and send socket.io client the result of the process or handle the whole process within socket.io emitter and then send the result to socket.io client?
Thanks in advance.
Question is not that clear but from what I'm getting from it, is that you want to know what you would use it for that you cant already do with your current API?
The short answer is, well nothing really.. Websockets are just the natural progression of API's and the need for a more 'real-time' interface between systems.
Old methods (and still used and relevant for the right use case) is long polling where you keep checking back to the server for updated items and if so grab them.. This works but it can be expensive in terms of establishing a connection, performing a lookup, then closing a connection.
websockets keep that connection open, allowing both the client and server to communicate real time. So for example, lets say you make an update to your backend data and want users to get that update, using long polling you would rely on each client to ping back to the server, check if there is an update and if so grab it. This can cause lags between updates, some users have updated data while other do not etc.
Now, take the same scenario with websockets, you make an update to the backend data, hit submit, this then emits to your socket server. Socket server takes the call, performs the task ( grabs updated data ) and emits it to the users, each connected user instantly gets that update.
Socket servers are typically used for things like real time chats or polling where packets are smaller but they are also used for web games etc. Depending on the size of your payloads will determine how best to send data back and forth because the larger the payload the more resources / bandwidth it will take on the socket server so its something to consider.
I'm working on a chat application and using socket.io / node for that. Basically I came up with the following strategies:
Send message from the client which is received by the socket server which then sends it to the receiving client. On the background I store that to the message on the DB to be retrieved later if the user wishes to seee his old conversations.
The pros of this approach is that the user gets the message almost instantly since we don't wait for the DB operation to complete, but the con is that if the DB operation failed and exactly that time the client refreshed its page to fetch the message, it won't get that.
Send message form the client to the server, the server then stores it on the DB first and then only sends it to the receiving client.
The pros is that we make sure that the message will be received to the client only if its stored in the DB. The con is that it will be no way close to real time since we'll be doing a DB operation in between slowing down the message passing.
Send message to the client which then is stored on a cache layer(redis for example) and then instantly broadcast it to the receiving client. On background keep fetching records from redis and updating DB. If the client refreshes the page, we first look into the DB and then the redis layer.
The pros is that we make the communication faster and also make sure messages are presented correctly on demand. The con is that this is quite complex as compared to above implementations, and I'm wondering if there's any easier way to achieve this?
My question is whats the way to go if you're building a serious chat application that ensures both - faster communication and data persistence. What are some strategies that app like facebook, whatsapp etc. use for the same? I'm not looking for exact example, but a few pointers will help.
Thanks.
I would go for the option number 2. I've been doing myself Chat apps in node and I found out that this is the best option. Saving in a database takes few milliseconds, which includes the 0.x milliseconds to write in the databse and the few milliseconds of latency in communication ( https://blog.serverdensity.com/mongodb-benchmarks/ ).
SO I would consider this approach realtime. The good thing with this is that if it fails, you can display a message to the sender that it failed, for whatever reason.
Facebook, whatsapp and many other big messaging apps are based on XMPP (jabber) which is a very, very big protocol for instant messaging and everything is very well documented on how to do things but it is based in XML, so you still have to parse everything etc but luckily there are very good libraries to handle with xmpp. So if you want to go the common way, using XMPP you can, but most of the big players in this area are not following anymore all the standards, since does not have all the features we are used to use today.
I would go with doing my own version, actually, I already something made (similar to slack), if you want I could give you access to it in private.
So to end this, number 2 is the way to go (for me). XMPP is cool but brings also a lot of complexity.
I want to monitor in real time the data that users enter in comments table.
I have an Apache server running, and suppose that has a node server on port 1337.
How would I do that every time someone save new data, eg return me the total number of table rows in comment and show it in a view?
Maybe way is to make the $this->Comment->save($this->request->data); using a different port using Httpsockect?
Yes, it is possible.
You have multiple ways of solving this, let me give you my ideas
You can simply use long-polling and don't use Node.js at all. It's a suitable solution if there won't be too much traffic there, otherwise you will have a bad time.
You can use websockets and don't use Node.js at all. Here you have a basic guide about websockets and PHP. Although, I am almost sure you won't be able to create "rooms", that is, sending notifications for specific comments.
You can also use Ratchet. This is a more sophisticated library to handle websockets and it supports rooms.
Lastly, if you want to full dive in with Node.js and CakePHP, I would suggest start by watching this talk given on Cakefest 2012 which exactly describe your scenario.
After you have watched that, you might want to learn a little about Socket.io. This is a more complex solution, but it's what I have used when integrating CakePHP and Node.js to create real time applications.
The strategy here is to have the users join a room when they visit /article/view/123, let's say the room name is the articleID, then socket.io will be listening for events happening in this room.
You will have a Cakephp method that handles the save. Then, when user submits the form you don't call directly the Cake action, you have socket.io to dispatch an event, then in your event you pass the data to the server (Node.js) and nodejs will call your cakephp function that saves the data. When Nodejs receives confirmation from CakePHP then you broadcast an event (using socket.io), this event will let know all users connected to that room that a comment has been made.
You have basically the choice between Websockets and long polling.
Websockets (with Ratchet and Autobahn.js)
Long Polling Using Comet
Decide which technology you want to use and start implementing your use case. Consider that Websockets are more or less new. Depending on your requirements you might not be able to use Websockets because you might have to support crappy browsers. See this page.
I am writing a multiplayer real time game for the browser with the server as a master instance and the clients as input devices and slaves to show the graphics.
I have to send out changes in the game world very often and very fast and it doesn't matter if some of the data sometimes gets lost on the way because a couple of milliseconds later there will be the next update anyway.
Right now I am using Socket.io to talk between the server and the clients but this uses TCP which makes the update come in unnecessary late sometimes.
I know that there is WebRTC with data channels where I would be able to send my updates through wit UDP which would be very awesome and exactly what I want. And it even seems to be implemented in Firefox and Chrome already https://stackoverflow.com/a/12864512/63779
What I now need is some Node library which would allow me to use data channels to send my data (for now just JSON strings) with help of UDP to the clients which are browsers. On the browser I would be able to use webkitRTCPeerConnection() but I have no idea how to start something like that on the Node server. Any suggestions? If there is no Node module for that, would it be possible to write something in some other language and just send the data via Unix domain sockets or something?
I have a java based web application(struts 1.2). I have a requirement to display a status on the frontend (jsp). Now the status might change which my server gets notified by another server. But I want this status change to be notified to the browser.
I don't want to make a refresh at intervals. Rather I have to implement something like done in gmail chat, ie. the browser gets notified by changing events on the server.
Any ideas on how to go about this?
I was thinking on lines of opening a request to server for status, and at the server end I would hold the request and wouldn't respond back until there is a status change. Any pointers, examples on this?
Best possible solution will be to make use of XMPP protocol. It's standardized and a lot of open source solutions will get you started within minutes. You can use combination of Smack, StropheJS and Openfire to get your java based app work as desired.
There's a method called Long Polling (Comet). It basically sends a request to the server. The request thread created on the server simply waits for new data for the user, with a time limit of maybe 1 minute or more. When new data is available it is returned.
The main problem is to tackle the server-side issue, you don't want to have one thread for every user just waiting for new data. Of course you could use some asynchronous methods depending on your back-end.
Ref: http://en.wikipedia.org/wiki/Push_technology
Alternative way would be to use WebSockets. The problem is that it's not supported by all browsers today.