I am working on a feature where the admin creates a course and the user gets notified. I am thinking of using MongoDB change stream but I need sockets to send the changed data to the frontend.
Now the scenario: Admin creates a course, change stream notifies and the socket.io emits that to the frontend.
My question is, if we are using socket.io to send the data to the frontend then why do we need MongoDB change stream? What we can do is simply emit the event when admin success in creating a course.
I have gone through multiple articles and they are doing the same- calling an API to create and update the document, a change stream to watch, and socket.io emit the event.
If we can send the data to the frontend once the course is created then why do we need MongoDB Change Stream?
You server side code may want to send this event to multiple destinations and/or trigger additional processes.For security reasons you need this event to be controlled by the server side. Of course you could always rely on the client to emit an event that the db update was successful, but I would still check that on the server side. Mongodb streams would allow you to listen on those events and take action without relying on the client.
I want to use NodeJS for live graphs for that reason, I want to pull latest data from database using NodeJS and then push to UI. Is there any possible solution in NodeJS.
I am using Cassandra database.
This question is very broad. If you want to poll your database on a timer you could use
setInterval(() =>{
// Read data from database here
// Send data to client
});
To send data to clients you could use a tool like socket.io which is easy to implement and works great with real time data! There is a great example here how to us socket.io to send data between a server and a client.
As mentioned in other answers you could set a timer that polls the database every X milliseconds and send out a update if there is new information. This will not be "real-time" but could be enough for your need. This will also waste som performance because of polling.
What you could do to solve this is to send out updates to the client at the same time you add content to your database. You could use http://socket.io/ to push updates out to the client.
So when you do something like:db.insert(data) you also io.emit(data) and send it to your clients at "real-time"
This is my first time developing a web application that requires chat functionality. There are multiple rooms and there are no restrictions as to how many people can join a room. The part that I can't get my head around is the actual approach to storing the messages. The question is more so in two parts...firstly am I correct to assume that unless all the messages are stored in a database, a newly connected user cannot retain all the previous messages? Secondly is it not recommended to save every message to the database as they are sent ? How else could I keep track if they aren't saved in real time ?
Thanks for any advice, appreciate it
If you want to store the data in memory and persist at a time interval, you can use redis for this. Also if all the data are required to store for future reference, you can implement scheduler like resque to transfer the data from redis to your db and free redis for application state data (i.e. more recent data).
Hope this will help you...
Thanks
Persistence is a business requirement, and yes a new user cannot see previous messages unless you have some sort of persistence storage. You can of course store messages permanently, and many apps out there do that. You can also store them temporarily and let the messages expire and be erased after a certain time period. All of this is easily accomplished using common tools such as MongoDB and Redis. If you do not need to persist messages, then you'll need to use web sockets to send messages between clients. Probably the most famous Node library for that is Socket.io
I am new to server-side development. I'm trying to learn by doing so I'm building an application with express on the server, mongodb as my database and angularjs with twitter bootstrap on the client-side.
I dont know if this is the most practical way but when thinking about how to implement messaging between users I thought of a mongodb model called Conversation with an id and an array of the ids of every user in the conversation and another array of strings that correspond to messages. And then add this model to my REST API.
But lets say all/some of the users in the conversation are online, why not benefit from socket.io. So how can i switch from this to real time chat? Does the interaction with mongodb occure exactly as explained and socket.io just notifies every online user that an interaction has occured? If yes, how? Or is it something else?
socket.io can send real time events to connected sockets, you can use a database for storing messages that are failed to deliver and for offline users.
also, you might want to use something like Redis for this as it has channels with subscribe and publish capabilities.
I have a good old-style LAMP webapp. A week ago I needed to add a push notification mechanism to it.
Therefore, what I did was to add node.js+socket.io on the server and poll the MySQL database every 10 seconds using node.js to check whether there were new items: if so, I would have sent them to the client(s) with socket.io.
I was pretty happy with the result, even if that is not a proper realtime notification (as there is a lag of up to 10 secs).
Now, I am about to build a new webapp which will need push notifications, too. I am wondering whether to go with the same approach as the first one (that I believe is more stable and mature) or to go totally Node.js, without PHP and Apache. As for the database, I have already decided to go for MongoDB.
Finally, my question is: if I go for Node.js+Socket.io+MongoDB will I get a truly near-real-time webapp? I mean, as soon as a new record is inserted into MongoDB, will there be some sort of event triggered that I can catch via node.js, do some checking on it and, if relevant, send the notification to the client? Or will there be anyway some sort of polling on the db server-side and lag, as with my first LAMP webapp?
A related question: can you build a realtime webapp on MySQL without doing any polling as I did with my first app. Or do you need MongoDB (or Redis)?
I hope this question is not too silly - sorry, I am just starting with Node.js and co.
Thanks.
I understand your problem because I switched to node.js from php/apache/mysql too.
Generally node.js is stable, modules and your scripts are the main reasons for errors
Real-time has nothing to do with database, it's all about client and server, you can query as many data as you want in your requests and push it to the other client.
Choosing node.js is very wise but it's harder to implement.
When you insert a new record to your db, the event is the request itself, you will make a push event along with the database query something like:
// Please note this is not real code, just an example of the idea
app.get('/query', function(request, response){
// Query your database
db.query('SELECT * FROM users', function(rows){
// Push notification to dan
socket.emit('database_query_executed', 'to_dan', rows);
// End request
response.end('success');
})
})
Of course you can use MySQL! And any database you want, as I said real-time has nothing to do with databases because the database is in the middle of the process and it's totally optional.
If you want to use node.js for push notifications and php/apache for mysql then you will need to create 2 requests for each server something like:
// this is javascript
ajax('http://node.yoursite.com/push', node_options)
ajax('http://php.yoursite.com/mysql_query', php_options)
or if you want just one request, or you want to use a form, you can call your php and inside php you can create an http or net request to node.js from php, something like:
// this is php
new HttpRequest('http://node.youtsite.com/push', HttpRequest::METH_GET);
Using:
A regular MongoDB Collection as the Store,
A MongoDB Capped Collection with Tailable Cursors as the Queue,
A Node worker with Socket.IO watching the Queue as the Worker,
A Node server to serve the page with the Socket.IO client, and to receive POSTed data (or however else the data gets added) as the Server
It goes like:
The new data gets sent to the Server,
The Server puts the data in the Store,
The Server adds the data's ObjectID to the Queue,
The Queue will send the newly arrived ObjectID to the open Tailable Cursor on the Worker,
The Worker goes and gets the actual data in the ObjectID from the Store,
The Worker emits the data through the socket,
The client receives the data from the socket.
This is 'push' from the initial addition of the data all the way to receipt at the client - no polling, so as real-time as you can get given the processing time at each step.
Re: triggers in MongoDB - please see this answer: https://stackoverflow.com/a/12405093/1651408
There are much more convenient triggers in MySQL, but to call Node.js from them would require a bit of work with MySQL UDFs (user-defined functions), for instance pushing data through a Unix socket. Please note that this is necessary only when other applications (besides your Node.js process) are updating the database, and be sure to choose InnoDB as storage in this case (row- vs. table-level locking).
Can see no big problem with your technology choice of sockets.io, even if client-side web sockets aren't supported, you'll fall back (gracefully, I hope) to polling.
Finally, your question is not silly at all, since push technology is definitely superior to the flood of polling requests - it scales better. EDIT: However, would not describe either technology as real-time.
Another EDIT: for a quite well-known and successful setup of this kind please read this: http://blog.fogcreek.com/the-trello-tech-stack/
Have you discovered Chole? It works separately from your web sever and interfaces with it by using HTTP POSTs. That way you can code your web app any which way you want.
Actually Using Push Technology like Socket.IO helps you to use
the server's resource efficiently and also helps you to leverage old browsers to modern browsers making websocket or websocket-like connection.
10 sec polling is a HTTP request which is expensive especially when a lot of users present.
Unlike polling technology, push technology is relatively cheap. Users' client is opening a dedicated socket(ie. websocket) to listen to the server's push notification.
And usually your client-side JavaScript do some actions when the push notification is received.
Using your LAMP stack and Socket.IO with different port (other than 80) will be good enough to implement what you need.
But using Node.js + MongoDB + Socket.IO actually helps you to manage your server's resource much efficiently.
Because those three have non-blocking nature.
If you understand non-blocking concept correctly and implement your app appropriately,
your identical app, an app with same feature but with different language and different database, would be able to handle a lot more requests than general LAMP stack.
Above picture is a famous chart of comparing Non-blocking vs Thread way to handle concurrency
Apache(Thread) vs Nginx(Non-blocking)
MySQL is a great database. I believe you won't need join and transactions for realtime notification.
MongoDB does not have those two features unless you implement similar features by yourself.
Because of not having those two and some characteristics of its own, MongoDB can store and fetch data much faster than traditional SQL databases.
Switching from MySQL to MongoDB will decrease the time taking to insert and fetch data.
with JS you can open a socket to your server (not old browser), the server will have a ah-hoc program (on an ad-hoc port, so you need the permission to open door and run program on your server) that will send data (almost) realtime from and to the client, and without the HTTP's protocol overhead.old browser will just fall-back to polling mechanism.
I can't see other way to do this (probably there are already "coocked" framework that do this)