Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I'm using node.js server and GCloud. What is exactly the GCloud's pub/sub? Does it work like socket or tcp? My server is using socket.io and node-ipc for communicating, can GCloud's pub/sub be the alternative?
PubSub is a messaging service. These allow asynchronous communication between two applications; one "publishes a message" to a message service and then some other process reads that message from the message service at a later time - seconds, minutes, or hours later. The application that published the message does not need to "stay connected".
That's really useful for scalable and reliable communication between applications - but quite different from socket-based communication which is point-to-point between a client and server process. Implementing request/response type communications is difficult over a messaging service - "send and forget" is the usual model. As #komarkovich noted, a message can also be received by many applications if that is appropriate.
Google Cloud Pub/Sub is an asynchronous publish/subscribe messaging service.
Publisher creates and sends messages to a topic. Subscriber creates a subscription to a topic to receive messages from it. Communication can be one-to-many, many-to-one, and many-to-many.
Pub/Sub has two endpoints:
Publisher: Any application that can make HTTPS requests to googleapis.com.
Subscriber:
Pull subscriber: Also any application that can make HTTPS requests to googleapis.com.
Push subscriber: Webhook endpoints that can accept POST requests over HTTPS.
You can check the Cloud Pub/Sub Client Libraries and review the example for Node.js to get you started using Google Cloud Pub/Sub API.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 4 days ago.
Improve this question
We are using node.js, socket.io, express and Mongo database to build an chat application like Teams.
When user text some message then
Data need to be saved
Respective users will be notified over app if they were actively connected.
Else we need to send firebase notification (It's to notify mobiles)
We are facing performance issue when different users of a group texts messages in a group at a time.Here
we are iterating over list of recipients to store data against each as we are also tracking later on to
know when it is delivered and read by that recipient. We are notifying over socket or firebase.
Due to above computation, performance is degrading and entire app get struck at server side till it finishes.
Note: While saving we are using async function. To return saved reference id (which is auto generated for
each entry) we are using Promise.
I am trying to use worker threads to save and notify but could not able to change code
to use worker thread due to code complexity. Kindly suggest any proper workaround to fix it.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm learning about server side websockets and have a question about a certain implementation and whether it's simply not a good idea or if it could be made to work.
Suppose a user is looking at their profile for a website they've joined and for some reason, they're expecting something specific to their account to change, e.g. their karma points to increase based on a funny post they made (Suppose their karma is private).
You could have a /karma websocket endpoint, and post to it whenever anyone's karma changes, but then everyone could potentially see everyone else's karma and in this scenario it's supposed to be private (Filtering it out on the client side is unacceptable, as I wouldn't want anyone except the target user to even receive a notification).
You could somehow store the userId against the web socket connection, and only send the karma notification to the intended user, but that doesn't feel scalable (It might work for this particular example, but probably not in cases where it's something like "send this message to everyone called john", as you could have a very large foreach loop to process).
The alternative is simply long polling with an ajax request, with a simple "get karma for this user" on a timer, which would work just fine, but be less fancy and require more requests to the server than is actually necessary (e.g. you might poll 1000 times but your karma changes just once during that time).
What's a good way of addressing such a requirement, keep it simple with long polling, or is there a websocket'y way that I'm missing?
If the requirement was "publicly get latest karma changes for all users" then websockets would be ideal, but I just can't see an obvious and simple way to make it work for more granular requirements.
Thanks
First off, I know of no circumstance where long polling is more efficient than a webSocket connection for server push. So, if you want to push data from server to client and have the client get it pretty much real time, you will want to use a webSocket connection from client to server and then you can send data to the client at any time and the client will receive and process it immediately.
The rest of your question is a little hard to understand what your objection is to using a webSocket.
If you're concerned about data being kept private (only the specific user can see their karma value), then that's just a matter of proper implementation to enforce that. When a webSocket connection is established, the user has to be authenticated so you know exactly which user is making the webSocket connection. Assuming your web pages have some sort of user authentication already, you can piggy back on that same auth when the webSocket connection is established because cookies are passed with the http request that starts a webSocket connection. So, now let's assume you have an authenticated webSocket connection and the server knows which webSocket belongs to which user.
So, now it's a matter of only sending appropriate data on each webSocket. Your server needs to implement the correct logic for taking a given karma change for a given user, find the authenticated webSocket connections belonging to that user and then sending a message over only those webSocket connections to alert the client that there's a new karma value.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I am making a chat app and would like to add the functionality of showing an online/offline symbol next to users. How can I do this reliably with minimum number of srever requests and database writes?
One way I found was to update a lastSeenAt field in the user document every time that user requests a page and use this to indicate whether the user is online/offline. Another way is to ping the server from the client side at fixed intervals of time and then update the lastSeenAt field.
Both theses ways would require a lot of database writes and/or server requests. Is there a way to do this more efficiently?
You have to push data from your server to the clients upon change, with something like web-sockets or server sent events. socket.io is a popular tool for implementing such functionally.
With socket.io or specifically websockets you could track the if a client is connected or not and have a flag in database that tracks whenever a client connects or
disconnects(keep in mind, the client might be having multiple connections(multiple devices, or even browser tabs), so if one connection disconnect he still might be online.
Assuming you will have multiple state-less webservers(common practice), then once a client connects or disconnects you want to notify other interested clients, then you should a use a pubsub pattern to notify other servers which will notify their connected clients respectively. There's a lot of implementations such as Redis, zeromq, AWS SNS, GCP Cloud Pubsub. You could even use MongoDB with tailable cursors as your pubsub.
However this might mean a lot of constant connections to your server, so this might hurt your scaliblity. If it proves to be to expensive for you then your lastSeenAt approach with some sane polling. You can find out which works better with your setup by just running some experiments.
If database writes and requests worries, you could always throw more servers at it. You could have a microservice specifically for this so your main server wouldn't be affected much and also you could have a database specifically for this. If performance worries you, you can use a in-memory database.
Also you can also have some caching in your servers with simple time based invalidation or you can sync the online/offline data between your servers with a pubsub pattern.
I would suggest try to make your setup as simple as possible, and you can run experiments to see how it handles your expected traffic.
Look into Express.io where you can combine HTTP calls with sockets. (the best method for real-time communication is Sockets).
In my current project, we are trying to implement a functionality using Node-RED for experiments and exploring new technologies.
The functionality is shown as follows. Here, the BadgeReader publishes its data using publish-subscribe to Proximity(it can be easily implemented using MQTT in Node-Red). The Proximity component receives data from BadgeReader and using that data it interact with ProfileDB using request/response interaction mode. Now, my question is--- how can we implmement request/response interaction in Node-RED? (Note that - Request/Response can be implemented using MQTT, but this question is related to dedicated request-response functionality in Node-RED?)
All the available database nodes will allow you to send a query and receive a reply before moving on to the next node in the flow.
There is also the http-request node that will do the same for a HTTP call to a remote service.
You can't do this with the Node-RED MQTT nodes because they either start a flow or end a flow. MQTT is asynchronous and publishers should be totally decoupled from subscribers so there is no way to know if a message ever reaches a subscriber, so no way to handle error cases or time outs properly. While it is possible to do request/response with MQTT it is not best suited this task.
If you want to do this with MQTT or something else then you may have to look at writing your own node, there is no generic request/response capabilities built into Node-RED.
P.S. given your flow of questions over the last few days you should probably look at the Node-RED mailing list here:
https://groups.google.com/forum/#!forum/node-red
It will be better suited to answering your questions than Stack Overflow
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
Looking to use a message queue in a small web app I'm building with node.js. I looked at resque but not sure that's appropriate. The goal is to push notifications to clients based on backend and other client actions with socketio. I could do this with just socketio but I thought maybe a proper message queue would make this cleaner and I wouldn't have to reinvent the wheel.
What are the options out there?
you could use redis with the lightning fast node_redis client. It even has built-in pubsub semantics.
You could use the node STOMP client. This would let you integrate with a variety of message queues including:
ActiveMQ
RabbitMQ
HornetQ
I haven't used this library before, so I can't vouch for its quality. But STOMP is a pretty simple protocol so I suspect you can hack it into submission if necessary.
Another option is to use beanstalkd with node. beanstalkd is a very fast "task queue" written in C that is very good if you don't need the feature flexibility of the brokers listed above.
Shameless plug: I'm working on Bokeh: a simple, scalable and blazing-fast task queue built on ZeroMQ. It supports pluggable data stores for persisting tasks, currently in-memory, Redis and Riak are supported. Check it out.
Here's a couple of recommendations I can make:
node-amqp: A RabbitMQ client that I have successfully used in combination with Socket.IO to make a real-time multi-player game and chat application amongst other things. Seems reliable enough.
zeromq.node: If you want to go down the non-brokered route this might be worth a look. More work to implement functionality but your more likely to get lower latency and higher throughput.
Take a look at node-busmq - it's a production grade, highly available and scalable message bus backed by redis.
I wrote this module for our global cloud and it's currently deployed in our production environment in several datacenters around the world. It supports named queues, peer-to-peer communication, guaranteed delivery and federation.
For more information on why we created this module you can read this blog post: All Aboard The Message Bus
I recommend trying Kestrel, it's fast and simple as Beanstalk but supports fanout queues. Speaks memcached. It's built using Scala and used at Twitter.
kue is the only message queue you would ever need
You might want to have a look at
Redis Simple Message Queue for Node.js
Which uses Redis and offers most features of Amazons SQS.
Look at node-queue-lib. Perhaps it is enough that you.
It support node.js and browsers. Has two delivery strategies: broadcast and round-robin.
Only javascript.
Quick example:
var Queue = require('node-queue-lib/queue.core');
var queue = new Queue('Queue name', 'broadcast');
// subscribe on 'Queue name' messages
queue.subscribe(function (err, subscriber) {
subscriber.on('error', function(err){
//
});
subscriber.on('data', function (data, accept) {
console.log(data);
accept(); // accept process message
});
});
// publish message
queue.publish('test');
How about Azure ServiceBus? It supports nodejs.
I used KUE with socketIO like you described.
I stored the socketID with the job and could then retreive it in the Job Complete..
KUE is based on redis and has good examples on github
something like this....
jobs.process('YourQueuedJob',10, function(job, done){
doTheJob(job, done);
});
function doTheJob(job, done){
var socket = io.sockets.sockets[job.data.socketId];
try {
socket.emit('news', { status : 'completed' , task : job.data.task });
} catch(err){
io.sockets.emit('news', { status : 'fail' , task : job.data.task , socketId: job.data.socketId});
}
job.complete();
}
You might also want to check out ewd-qoper8: https://github.com/robtweed/ewd-qoper8