I have a number of machines that can do some actions. To initiate an action, I want to send a message to a queue named by the machine ID, "12345" for instance.
To avoid anyone else consuming these messages, the queue should be exclusive, I think.
But from my controller, I cannot assert the queue if it's declared as exclusive on the machine that consumes it. However, I can send to the queue by specifying its name, without asserting, but I guess that's bad practice?
The machine is listening for commands like this:
ch.assertQueue(machineID, {exclusive: true, durable: false}, function (err, q) {
if (err) console.log(err)
console.log("opened queue")
ch.consume(q.queue, function (message) {
console.log(message.content.toString())
})
})
When I send to the queue, it will fail due to exclusive issues, at this line in the client:
ch.assertQueue(machineID, {})
But it will send the message if I comment the line above.
ch.sendToQueue(machineID, new Buffer(id.toString()))
What is the most elegant way to get around this?
One way is to create an exchange of type 'direct' and bind your queue to this exchange. So if you pass any message to this exchange then it will only be routed to that queue whose name is same as the routing key you pass to the exchange. For example, all the messages with routing key as '12345' will be routed to queue named as '12345' by such an exchange.
Related
I'm trying to create a private messaging functionality using socket.io with React and Node.
I am able to send a message to a particular socket like so:
io.to(targetSocketID).emit('privateMessage' message)
it successfully sends it to that specific user but not the sender. is there a way to just emit a message to sender and the target user to make this simple? From what I can see there is two approaches here.
When a message is created push the sender messages into a messages array setMessages([...messages, senderMessages]) and also push the socket message into that array setMessages([...messages, receivedMessages]). this approach seems sloppy and like to avoid this route as it can become problematic.
generate a unique room for each user and send the room to the server and join it:
//server
socket.on('joinRoom', room => {
socket.join(room)
socket.on('privateMessage', message => {
socket.on(room).emit('messageResponse', message)
})
})
I would like to know if there is a better way to do this.
that allows me to emit a message to just sender AND targeted receiver.
problem: remote systems reconnect to multiple nodes websocket server, for each system a dedicated queue in RabbitMQ is created/used. The queues should be automatically removed if no active connections exist. Websocket connect/disconnect events handlers are asynchronous, quite heavy, observed problem that a disconnect event handler finished after reconnect, making system inconsistent.
The main issue is with RabbitMQ queues - initial solution was to create unique queues for each connection and remove them on disconnect. Appeared to be heavy.
Second approach was to keep a dedicated queue per remote system (same queue name for any connection), the problem was that assertQueue added consumers for the same queue. Need to find way to remove stale queue consumers without removing the queue itself.
Solution is to store list of consumers per remote system and on disconnect event trigger cancel function with the olderst consumerTag, then update the list of queue consumers for the given remote system.
on remote system connect event
import { Replies } from "amqplib";
// bind callback function for queue-specific messages and store returned consumer description
const result: Replies.Consume = await channel.consume(queueName, this.onSomeMessage.bind(this));
// update consumers list for the connected remote system
const consumers: Array<string> | undefined = this.consumers.get(remoteId);
if (consumers === undefined) {
const consumersList: Array<string> = new Array();
consumersList.push(result.consumerTag);
this.consumers.set(remoteId, consumersList);
} else {
consumers.push(result.consumerTag);
}
on remote system disconnect event
// remove the oldest consumer in the list and update the list itself
// use cancel method of the amqp channel
const consumers = this.consumers.get(remoteId);
if (consumers === undefined) {
// shouldn't happen
console.error(`consumers list for ${remoteId} is empty`);
} else {
const consumerTag = consumers[0];
await this.rxchannel.addSetup(async (channel: ConfirmChannel) => {
await channel.cancel(consumerTag);
consumers.shift();
});
}
The code snippets are from some class' methods implementation (if you're wondering about "this").
Copyright notice (especially for German colleagues): the code from this answer can be used under Beerware (https://en.wikipedia.org/wiki/Beerware) or MIT license (whatever one prefers).
I am using Rascal.Js(it uses amqplib) for my messaging logic with rabbitMq on node.js app.
I am using something similar to their example on my project startup, which creates a permanent instance and "registers" all of my subscribers and redirects messages when they arrive to the queue (in the background).
My issue is with the publishers. There are http requests from outside which should trigger my publishers. A user clicks on create button of sorts which leads to certain flow of actions. At some point it reaches the point at which I need to use a publisher.
And here I am not sure about the right approach. Do I need to open a new connection every time I need to publish a message? and close it after it ends? Or maybe I should implement this in a way that it keeps the same connection open for all of the publishers? (I actually not so sure how to create it in a way that it can be accessed from other parts of my app).
At the moment I am using the following :
async publishMessage(publisherName, message) {
const dynamicSettings = setupDynamicVariablesFromConfigFiles(minimalPublishSettings);
const broker = await Rascal.BrokerAsPromised.create(Rascal.withDefaultConfig(dynamicSettings.rascal));
broker.on('error', async function(err) {
loggerUtil.writeToLog('error', 'publishMessage() broker_error_event: ' + publisherName + err + err.stack);
await broker.shutdown();
})
const publication = await broker.publish(publisherName, message);
try {
publication.on('error', async function(err) {
loggerUtil.writeToLog('error', 'publishMessage() publish_error_event: ' + err + err.stack);
await broker.shutdown();
}).on("success", async (messageId) => {
await broker.shutdown();
}).on("return", async (message) => {
loggerUtil.writeToLog('error', 'publishMessage() publish_return_event: ' + err + err.stack);
await broker.shutdown();
})
}
catch(err) {
loggerUtil.writeToLog('error', 'Something went wrong ' + err + err.stack);
await broker.shutdown();
}
}
I use this function from different parts of my app when I need to publish messages.
I thought to just add the broker.shutdown() for all of the endpoints but at some point after an error, I got an exception about closing a connection which was already closed, and this got me worried about the shutdown approach (which probably not a good one). I think it is related to this -
I tried doing that (the commented code) but I think it isnt working well in certain situations. If everything is ok it goes to "success" and then I can close it.
But one time I had an error instead of success and when I tried to use broker.shutdown() it gave me another exception which crashed the app. I think it is related to this -
https://github.com/squaremo/amqp.node/issues/111
I am not sure what might be the safest way to approach this?
Edit:
Actually now that I think about it, the exception might be related to me trying to shutdown the broker in the catch{} area as well. I will continue to investigate.
Rascal is designed to be initiated once at application startup, rather than created per HTTP request. Your application will be extremely slow if you use it in this way, and depending on how many concurrent requests you need to handle, could easily exceed max number of connections you can make to the broker. Furthermore you will get none of the benefits that Rascal provides, such as failed connection recovery.
If you can pre-determine the queue or exchange you need to publish to, then configure Rascal at application start-up (prior to your http server), and share the publisher between requests. If you are unable to determine the queue or exchange until your receive the http request, then Rascal is not an appropriate choice. Instead you're better off using amqplib directly, but should still establish a shared connection and channel. You will have to handle connection and channel errors manually though, otherwise they will crash your application.
I'm using amqplib with node.js and I'm trying to make sure I understand the concept of channels.
This is from the amqplib documentation: Channels are multiplexed over connections, and represent something like a session, in that most operations (and thereby most errors) are scoped to channels.
Here is some basic code where I'll open a amqp connection, create a channel, an exchange and a queue:
var amqp = require('amqp/callback_api');
var connection = amqp.createConnection({ host: "localhost", port: 5672 });
connection.on('ready', function () {
connection.createChannel(function(err, ch) {
ch.assertExchange('1', 'fanout', function(err, ok) {});
ch.assertQueue('a', {
exclusive: true,
durable: true
}, function(err, ok) {
});
});
In the above code do exchange '1' and queue 'a' only exist on the channel for which they were defined? By this I mean, if I were to publish a messages to exchange a from another channel would exchange a still route the messege?
All entities like exchanges, queues, messages exists globally on the broker and visible to all connections and channels inside single vhost. There are no exceptions from that.
Queues may be defined as exclusive, then they are exists only within same connection and when it closed they are destroyed. This is special case while as they still visible, they are not accessible from other connections.
There are auto-delete option for both queues and exchanges, which is by default set to true. It means that they will be removed after usage (see exchange and queue auto-delete docs for details).
I use rabbitMq, nodeJs(with socet.io, amqp modules), ZF2 for development chat
By default RabbitMq send message from queue at help Round-robin.
Does RabbitMq opportunity to send all subscriber queue the same message?
For example:
If i make for each connection its queue, that is work correct, but if user open 2 tabs on him browser, then will make 2 queue. I think its not good.
I want have one queue for each users(if i make that, than first message send to first tab, second message - to second tab)
My code:
var exchange = connectionAmqp.exchange('chat', {type: 'direct', passive: false, durable:false, autoDelete: false});
console.log(' [*] Client connected')
connectionAmqp.queue('chat'+userId.toString(), {
passive : false,
durable : false,
exclusive : false,
autoDelete: false
}, function(queue) {
//Catch new message from queue
queue.bind(exchange, userId.toString());
queue.subscribe(function(msg){
socket.emit('pullMessage', msg); //Emit message to browser
})
});
From other script i push message
var exchange = connectionAmqp.exchange('chat', {type: 'direct', passive: false, durable:false, autoDelete: false});
var data= {chatId:70,msg:"Text",time:1375333200}
exchange.publish('1', data, {contentType: 'application/json'});
Make sure the queues are not exclusive. Then make sure the client connects to the same queue. This can be done but having the client create the queue and specifying the name of that queue. The naming algorithm will make sure that the queue name is unique per client, but for the same client it will produce the same name. Both tabs will read in turn from the same queue ensuring the round robin effect that you are looking for.
If you want to send a message to all queues, you can use an exchange of type fanout. See here! It will broadcast a message to each queue bound to it. However, if you are attaching two consumers (callbacks) on one queue, those two consumers (callbacks) will still be fed round-robin wise.
Queues are very lightweight and RabbitMQ is build to handle many queues, so it's ok to create a queue for each tab. If you are still unsure, this post may be of your interest. The author build a simple chat system and stress tested it, showing that RabbitMQ easily handles thousands of queues and messages per second.
Although it is possible to do this with just one queue per user, it will be far easier with one queue per tab...and when using RabbitMQ there is usually no need to do such optimizations*.
*(of course there are exceptions)