I'm using amqplib with node.js and I'm trying to make sure I understand the concept of channels.
This is from the amqplib documentation: Channels are multiplexed over connections, and represent something like a session, in that most operations (and thereby most errors) are scoped to channels.
Here is some basic code where I'll open a amqp connection, create a channel, an exchange and a queue:
var amqp = require('amqp/callback_api');
var connection = amqp.createConnection({ host: "localhost", port: 5672 });
connection.on('ready', function () {
connection.createChannel(function(err, ch) {
ch.assertExchange('1', 'fanout', function(err, ok) {});
ch.assertQueue('a', {
exclusive: true,
durable: true
}, function(err, ok) {
});
});
In the above code do exchange '1' and queue 'a' only exist on the channel for which they were defined? By this I mean, if I were to publish a messages to exchange a from another channel would exchange a still route the messege?
All entities like exchanges, queues, messages exists globally on the broker and visible to all connections and channels inside single vhost. There are no exceptions from that.
Queues may be defined as exclusive, then they are exists only within same connection and when it closed they are destroyed. This is special case while as they still visible, they are not accessible from other connections.
There are auto-delete option for both queues and exchanges, which is by default set to true. It means that they will be removed after usage (see exchange and queue auto-delete docs for details).
Related
problem: remote systems reconnect to multiple nodes websocket server, for each system a dedicated queue in RabbitMQ is created/used. The queues should be automatically removed if no active connections exist. Websocket connect/disconnect events handlers are asynchronous, quite heavy, observed problem that a disconnect event handler finished after reconnect, making system inconsistent.
The main issue is with RabbitMQ queues - initial solution was to create unique queues for each connection and remove them on disconnect. Appeared to be heavy.
Second approach was to keep a dedicated queue per remote system (same queue name for any connection), the problem was that assertQueue added consumers for the same queue. Need to find way to remove stale queue consumers without removing the queue itself.
Solution is to store list of consumers per remote system and on disconnect event trigger cancel function with the olderst consumerTag, then update the list of queue consumers for the given remote system.
on remote system connect event
import { Replies } from "amqplib";
// bind callback function for queue-specific messages and store returned consumer description
const result: Replies.Consume = await channel.consume(queueName, this.onSomeMessage.bind(this));
// update consumers list for the connected remote system
const consumers: Array<string> | undefined = this.consumers.get(remoteId);
if (consumers === undefined) {
const consumersList: Array<string> = new Array();
consumersList.push(result.consumerTag);
this.consumers.set(remoteId, consumersList);
} else {
consumers.push(result.consumerTag);
}
on remote system disconnect event
// remove the oldest consumer in the list and update the list itself
// use cancel method of the amqp channel
const consumers = this.consumers.get(remoteId);
if (consumers === undefined) {
// shouldn't happen
console.error(`consumers list for ${remoteId} is empty`);
} else {
const consumerTag = consumers[0];
await this.rxchannel.addSetup(async (channel: ConfirmChannel) => {
await channel.cancel(consumerTag);
consumers.shift();
});
}
The code snippets are from some class' methods implementation (if you're wondering about "this").
Copyright notice (especially for German colleagues): the code from this answer can be used under Beerware (https://en.wikipedia.org/wiki/Beerware) or MIT license (whatever one prefers).
I have a number of machines that can do some actions. To initiate an action, I want to send a message to a queue named by the machine ID, "12345" for instance.
To avoid anyone else consuming these messages, the queue should be exclusive, I think.
But from my controller, I cannot assert the queue if it's declared as exclusive on the machine that consumes it. However, I can send to the queue by specifying its name, without asserting, but I guess that's bad practice?
The machine is listening for commands like this:
ch.assertQueue(machineID, {exclusive: true, durable: false}, function (err, q) {
if (err) console.log(err)
console.log("opened queue")
ch.consume(q.queue, function (message) {
console.log(message.content.toString())
})
})
When I send to the queue, it will fail due to exclusive issues, at this line in the client:
ch.assertQueue(machineID, {})
But it will send the message if I comment the line above.
ch.sendToQueue(machineID, new Buffer(id.toString()))
What is the most elegant way to get around this?
One way is to create an exchange of type 'direct' and bind your queue to this exchange. So if you pass any message to this exchange then it will only be routed to that queue whose name is same as the routing key you pass to the exchange. For example, all the messages with routing key as '12345' will be routed to queue named as '12345' by such an exchange.
I am trying to use the blocked and unblocked channel events using Nodejs module amqplib on RabbitMQ. My understanding is RabbitMQ will send a connection blocked command to the producers if the system resources have reached an "alarm" state. My intention is to utilize this to determine if the producer should proceed in creating a job, or to respond to with a "try again later".
amqplib docs: http://www.squaremobius.net/amqp.node/channel_api.html#model_events
Here are the versions of software I am using:
RabbitMQ 3.6.6, Erlang R16B03
NodeJS 6.9.2
amqplib ^0.5.1 (node module)
Ubuntu 14.04
Things I have tried
My nodejs code:
var amqp = require('amqplib');
amqp.connect('amqp://localhost').then((connection) => {
return connection.createChannel();
}).then((channel) => {
channel.on('blocked', (reason) => {
console.log(`Channel is blocked for ${reason}`);
});
channel.on('unblocked', () => {
console.log('Channel has been unblocked.');
});
return channel.assertQueue('myQueue', {
durable : true,
// This doesn't seem to do anything
arguments : {
"connection.blocked" : true
}
}).then(() => {
channel.sendToQueue('myQueue', new Buffer('some message'), {
persistent : true
});
});
});
I understand that this particular feature is an extension to the AMQP protocol and needs to be enabled/declared. I'm not very familiar with the erlang config syntax. Using an example config, I built a config that looks something like:
[{rabbit, [
{"server-properties", [
{"compatibilities", {
{ "connection.blocked", true }
}}
]}
As per the docs here: https://www.rabbitmq.com/consumer-cancel.html#capabilities
"To receive these notifications, the client must present a
capabilities table in its client-properties in which there is a key
connection.blocked and a boolean value true. See the capabilities
section for further details on this. Our supported clients indicate
this capability by default and provide a way to register handlers for
the connection.blocked and connection.unblocked methods."
And then restarted the server using service rabbitmq-server restart.
This does not crash the server, but the events do not fire either. I'm expecting that the connection should become block in the event that system resources are low. The RabbitMQ docs has a link for more info on Capabilities but the link is dead, and I'm not sure what else to try.
blocked and unblocked are on connection not on channel.
In your example: connection.on('blocked', console.log) and connection.on('unblocked', () => console.log('unblocked')).
To test it just run: rabbitmqctl set_vm_memory_high_watermark 0.000001 and send a new message to your queue.
compatibilities are enabled by default, and amqplib informs server about connection.blocked. Nothing should be done here.
I use rabbitMq, nodeJs(with socet.io, amqp modules), ZF2 for development chat
By default RabbitMq send message from queue at help Round-robin.
Does RabbitMq opportunity to send all subscriber queue the same message?
For example:
If i make for each connection its queue, that is work correct, but if user open 2 tabs on him browser, then will make 2 queue. I think its not good.
I want have one queue for each users(if i make that, than first message send to first tab, second message - to second tab)
My code:
var exchange = connectionAmqp.exchange('chat', {type: 'direct', passive: false, durable:false, autoDelete: false});
console.log(' [*] Client connected')
connectionAmqp.queue('chat'+userId.toString(), {
passive : false,
durable : false,
exclusive : false,
autoDelete: false
}, function(queue) {
//Catch new message from queue
queue.bind(exchange, userId.toString());
queue.subscribe(function(msg){
socket.emit('pullMessage', msg); //Emit message to browser
})
});
From other script i push message
var exchange = connectionAmqp.exchange('chat', {type: 'direct', passive: false, durable:false, autoDelete: false});
var data= {chatId:70,msg:"Text",time:1375333200}
exchange.publish('1', data, {contentType: 'application/json'});
Make sure the queues are not exclusive. Then make sure the client connects to the same queue. This can be done but having the client create the queue and specifying the name of that queue. The naming algorithm will make sure that the queue name is unique per client, but for the same client it will produce the same name. Both tabs will read in turn from the same queue ensuring the round robin effect that you are looking for.
If you want to send a message to all queues, you can use an exchange of type fanout. See here! It will broadcast a message to each queue bound to it. However, if you are attaching two consumers (callbacks) on one queue, those two consumers (callbacks) will still be fed round-robin wise.
Queues are very lightweight and RabbitMQ is build to handle many queues, so it's ok to create a queue for each tab. If you are still unsure, this post may be of your interest. The author build a simple chat system and stress tested it, showing that RabbitMQ easily handles thousands of queues and messages per second.
Although it is possible to do this with just one queue per user, it will be far easier with one queue per tab...and when using RabbitMQ there is usually no need to do such optimizations*.
*(of course there are exceptions)
I have a function which I will call from time to time.
function blah() {
sub.unsubscribe();
sub.subscribe("a");
sub.subscribe("b");
}
Above results in error message "Error: node_redis command queue state error."
Same as https://github.com/mranney/node_redis/issues/137
Once a connection is subscribed, it can only issue subscription related commands (subscribe, psubscribe, etc)
Might another part of your code be using the same connection?
You could also try and subscribe to multiple channels with one call sub.subscribe("a", "b") or subscribe to a pattern that matches your need?
I am currently using redis.I was also facing the same issue in which i was trying to subscribe a set of channels.
var redis = require('redis')
const subscribe = redis.createClient({
host: 'localhost',
port: 6379
})
subscribe.psubscribe(`user:chat:*`)
subscribe.on('pmessage', function(pattern, channel, message) {
console.log(channel, message, pattern)
// Write Your Awesome Code here.
})
Psubscribe used for subscribing for multiple channels using redis. You can subscribe multiple channel on the basis of pattern.