Hey guys after all these year I can finally get into node.js and react.js so i'm a total beginner.
My first project will be to migrate a time booking system for our production from the current Qt C++ application to a web application.
The biggest issues at this point is to decide how to build the backend.
The application needs to handle several concurrent users with full duplex communication between the client sides.
The closes example would be a system like Trello. So a collaborative system where multiple users are able to move tasks around.
Once a user moves a task from one spot to another i have to instantly reorganize the tasks for every currently logged in user. So i can guarantee that everyone is up to date at any point in time.
How would you build this and what libraries are best suited for this case?
I hope you understand what I'm trying to say.
greets
Use websocket to keep server and client side in touch.
To prevent operation conflict, you need to
lock the task/resource when user start doing edit/move, and inform other clients that it has been locked.
write result into db/nosql.
inform all other user client change result and get ACK response.
release the lock and inform all clients.
to smooth the process, release lock when getting the very first ACK.
A change log with timestamp (monotonic time) might be helpful to prevent conflict.
Besides, Electron (based on NodeJs) is suitable for building client side as an alternative to Qt.
Related
I am using SailsJS for a web application which basically lets the user download and process any video from the internet(say youtube). The user enters a link to the video and my sails app downloads the video if available and then starts processing the downloaded video using a shell script(Come OpenCV processing to find different frames).
This process takes a very long time to complete, and the user can navigate away from the page and do whatever he wants. Now, to check on the progress by visiting this page later I need to be able to connect with the child process that was created earlier for this video file.
I have come up with two possible solutions:
1) Using gearman to implement a job server and connect to it every time the user navigates to the page and start getting the callback events and show the progress based on them. This is the first time I'll be using gearman.
2) Somehow storing the processID of the child process in the session/db and then using it to find the process using ps-node.
Which of these is the better approach(if you think they'll work fine)? Or is there any other solution I don't know about? Any pointers in the right direction will be appreciated.
Let's start with the second option. Don't use it. Simply because this way your site users will have sort of more control over the number of processes running on your server then you will.
Number one is way better, but using a separate job server seems like a bit of overkill to me (I have to admit though that I'm not fully informed the scale of your plans).
Bottom line, I would use a message/job queue (kue seems like a perfect fit to me) and store the progress in DB or (preferably) Redis (or whatever cache you are using).
I am using Webrtc, nodejs, Expressjs as the framework to create a audio, video and chat application. I have used Forever so that the script runs continuously.
As my application deals with audio, video and chat. User presence plays an important role. We need to have the system up and running always and avoid system crashes and restart. If it happens we are going to loose all information regarding the users who were online.
Need Suggestions what are the best approaches to avoid such situations.
Also, while moving new features to the production server, what steps should we take into consideration so that the application doesn't stop and henceforth we don't loose user information.
What if the server went down or we had to make it down. What are the different techniques that can be used so that we don't loose the presence information of the online users in the system and restore them back(if necessary).
1) use node cluster to fork multiple process per core. So if one process died, another process will be auto boot up. Check out: http://nodejs.org/api/cluster.html
2) use domain to catch asyn operation instead of using try-catch or uncaught http://nodejs.org/api/domain.html. I'm not saying that try-catch or uncaught is bad thought!
3) use forever/supervisor to monitor your services
4) add daemon to run your node app: http://upstart.ubuntu.com
hope this helps!
I also answered at this post: How do I prevent node.js from crashing? try-catch doesn't work
So the first app that people usually build with SocketIO and Node is usually a chatting app. This chatting app basically has 1 Node server that will broadcast to multiple clients. In the Node code, you would have something like.
//Psuedocode
for(client in clients){
if(client != messageSender){
user.send(message);
}
}
This is great for a low number of users, but I see a problem with this. First of all, there is a single point of failure which is the Node server. Second of all, the app will slow down as the number of clients grow. What is there to do then when we reach this bottleneck? Is there an architecture (horizontal/vertical scaling) that can be used to alleviate this problem?
For that "one day" when your chat app needs multiple, fault-tolerant node servers, and you want to use socket.io to cross communicate between the server and the client, there is a node.js module that fits the bill.
https://github.com/hookio/hook.io
It's basically an event emitting framework to cross communicate between multiple "things" -- such as multiple node servers.
It's relatively complicated to use, compared to most modules, which is understandable since this is a complex problem to solve.
That being said, you'd probably have to have a few thousand simultaneous users and lots of other problems before you begin to have problems with this.
Another thing you can do, is try to develop your application in a way so that if a connection is lost (which happens all the time anyway), eg. server goes down, client has network issues (eg. mobile user), etc, your application should be able to handle that and recover from such issues gracefully.
Since Node.js has a single event-loop thread, this single point of failure is written into its DNA. Even reloading a server after code changes require this thread to be stopped.
There are however a lot of tools available to handle such failures gracefully. You could use forever; a simple CLI tool for ensuring that a given script runs continuously. Other options include distribute and up. Distribute is a load balancing middleware for Node. Up builds on top of Distribute to offer zero downtime reloads using either a JavaScript API or command line interface:
Further reading I find you just need to use Redis Store with Socket.io to maintain connection references between two or more processes/ servers. These options have already been discussed extensively here and here.
There's also the option of using socket.io-clusterhub if you don't intend to use the Redis store.
I want to update my node application on production, but users are using it for things like credit card transactions.
I run supervisor, but I would like to wait until all critical sections (like saving data or sending important information) are complete before it restarts.
Check out up by LearnBoost.
Zero-downtime reloads built on top of the distribute load balancer.
Read more from here:
http://www.devthought.com/2012/01/29/staying-up-with-node-js/
Another one is ncluster.
Creating a programmed dowtime seems the most straightforward thing to do, just notify the users and stop critical transactions a few minutes before the downtime, always choose the right time to go offline and be sure to be only a small timeframe away from your users.
You could also delegate to more applications the various sections of your app, for example process payments in a separate process you can message with a queue.
This clearly depends on your needs, by te way be sure to disclose a programmed downtime to your users, they will be happy to come back later.
We are having a web application build using asp.net 3.5 & SQL server as database which is quite big and used by around 300 super users for managing around 5000 staffs.
Now we are implementing SMS functionality into the application which means the users will be able to send and receive SMS. Every two minute the SMS server of the third party is pinged to check whether there are any new messages. Also SMS are hold in queue and send every time interval of 15 to 30 minutes.
I want this checking and sending process to run in the background of the application all the time, even if the user closes the browser window.
I need some advice on how do I do this?
Will using thread will achieve this or do I need to create a windows service for it or are there any other options?
More information:
I want to execute a task in a timer, what will happen if I close the browser window, the task wont be completed isn't it so.
For example I am saving 10 records to the database in a time interval of 5 minutes, which means every 5 minutes when the timer tick event fires, a record is inserted into the database.
How do I run this task if I close the browser window?
I tried looking at windows service but how do I pass a generic collection of data to it for processing.
There really is no thread or service choice, a service can (and usually is!) multi threaded, a thread can start a service.
There are three basic choices you can:-
Somehow start another thread running when a user logs in -- this is probably a very poor choice for what you want, as you cannot really keep it running once the user session is lost.
Write a fully fledged windows service which is starts on OS startup and continues running unitl the server is shutdown. You can make this dependant on the SQLserver service, so it starts after the DB is available. This is the "best" solution but may be overkill for your purposes. Aslo you need to know the services API to write it properly as you need to respond correctly to shutdown and status requests.
You can schedule your task periodically using either the Windows schedular, or, preferably the schedular which is built in to SQLServer, I think this would be the most suitable option for your needs.
Distinguish between what the browser is doing and what's happening server-side.
Your Web App is sitting server-side waiting for requests from whatever browsers may be running, and servicing those requests, in servicing those requests I guess it may well put messages on a queue and have a look in a database for any new messages.
You want the daemon processor, which talks to the third-party SMS, to be triggered by time rather than by browser function. Either of your suggestions would work:
A competely independent service could run and work against the queues and database.
Your web app, which I assume is already a service, could spawn a thread
In either case we have a few technical questions of avoiding any race conditions between the browser-request processing and the daemon - but databases and queueing systems can deal with that.
So I would decide between stand-alone daemon and background thread like this:
Which is easier to implement? I'm a Java EE developer, I know in my app server I have an API for specifying code to be run according to a timer, the API deals with the threading issues. So for me that's very easy. I don't know what you have available. Timers are not quite as trivial as they may appear - so having a reliable API is beneficial. If this was a more complex requirement, where the daemon code were gnarly and might possibly interfere with the WebApp code then I might prefer to keep it conspicuously separate.
Which is easier to deploy and administer? Deploy separate Web App and daemon, or deploy one thing. In the Java EE world we could have a single Enterprise Application with all the code, so that's a single thing to deploy, start and control.
One other thing to consider: Scaling and Resilience. You might choose to have more than one copy of your web app running, either to provide fail-over capabilities or just because you need the extra power. In which case how many daemons would you have? Would it be a problem to have two daemons running? You might need some extra code to mediate between two daemons, for example log in the database the time of last work, each daemon can say "Oh, my buddy balready did the 10:30 job, I'll go back to sleep"