Imagine a scenario where we have many producers and one consumer. The consumer subscribes to a channel group named "consumerGroup". Each time a producer enters the system, he adds the channel "Producer-ID-Channel" to the consumerGroup, subscribes to the channel and publishes a message.
Does the consumer gets automatically subscribed to this channel since he has already subscribed to the group it belongs to? Even if the producer's channel was added to the group after the time he initially subscribed to the group?
Yes! All the client needs to do is subscribe to the channel group, and as channels are added/removed to the channel group by whatever means, the client will receive them.
On JavaScript, if your message callback looks like:
function cb(message, envelope, channel){
/* message will contain the message
envelope will contain the server-response, in which data like message and channel are
derived from.
element 0 will be the current message in the callback cycle
(also accessible via 'message')
element 1 will be the timetoken of the server response
element 2 will be the channel group name the message arrived on
element 3 will be the actual channel within the channel group the msg arrived on */
}
geremy
Related
In rabbitmq message from a named queue is displayed as ready and it is ackowledged by another queue which has topic exchange.
In the following image I have 2 different queues, one named one generated randomly by rabbtmq:
I know that I send one message from the named queue (6.30189.facility_status) which has topic exchange and a specific routing key. In the consumer I create a queue without giving it a specific name, in the picture is the second queue (amq.gen-....). The consumer has a topic exchange and can get message from many queues.
How can I delete the messaged which is read from the named queue after I consume it in the second queue?
I have a web client using pubnub, and there are 1000 channels need to be subscribed. But once I call pubnub.subscribe(allChannels), the server returns 500.
I did not find any limit saying why it cannot.
Actually, there is no hard limit on the number of channels you can subscribe to. But if you are going to subscribe to more than 20-30 channels, it's usually recommended to use Channel Groups instead, which allows you to create a collection of channels and give that collection a name. Then, your client simply subscribes to the Channel Group instead of the individual channels. https://www.pubnub.com/developers/tech/key-concepts/stream-controller/channel-groups/
There is a limit number of subscribed channels, that's what I have tested: 640 channels. And once subscribed 640 channels, it is still successful, but if you subscribe one more, errors will start happening (server returns 500) and all your pubnub connection will get destroyed.
It seems you can subscribe many more channels at one time (Its official website suggest 50 channels, but I have tried subscribed 400 channels at one time and it is successful)
So total limit 640 is what I have found.
Subscribe to 1,000 Channels and Beyond
You already know the list of channels you want to subscribe to, Channel Groups will help you get to 2,000 channels per group. There are also some options to subscribe to an unlimited number of channels using wildcards!
You can have 2,000 channels per channel group. This matches your need for 1,000 channel subscription.
🔐 Also remember that Security is important. Remember that you should secure your data.
More channel subscription options below for your consideration.
Channel Subscription Options
For the PubNub SDKs 4.0 and higher. Multiplexing the SDK supports up to around 50 channels practically. You can increase this number further but device performance will be impacted. It is better to utilize the Channel Group feature of the core PubNub Stream Controller product. Stream Controller includes the following channel subscription enhancements.
Wildcard Subscribe
Maximum Addressable Channels: ( Unlimited )
PubNub Also supports Wildcard Subscriptions using a Hierarchical dot notation. Today in PubNub you can subscribe to channel "a.b.*" and receive messages at any channel below a.b. You could publish to channel "a.b.c" and receive the message in your wildcard subscribe. Note that the maximum depth supported of the hierarchy is currently a depth of three. For example "a.b.*" is good but "a.b.c.*" is not supported.
Channel Groups
Maximum Addressable Channels: ( 2,000 ✕ 10 ＝ 20,000 )
Dynamically control the device's data stream feed with PubNub Channel Groups. You can remotely control which streams the device is subscribed to and dynamically add and remove channels from the list of channels. You can multiplex subscribe up to 10 channel groups and each channel group can contain up to 2,000 channels.
Multiplexing
Maximum Addressable Channels: ( ~50-500 )
You can multiplex your connection by subscribing to a combination of Wildcard Channels, Channel Groups, Presence Event Stream Channels and more. It is recommended to keep your multiplexed channels below 50 for best device performance.
Stream Filtering
Maximum Addressable Channels: ( Unlimited )
Stream Filter allows a subscriber to apply a filter to only receive messages that satisfy the conditions of the filter. The message filter is set by the subscribing client(s) but it is applied on the server side thus preventing unwanted messages (those that do not meet the conditions of the filter) from reaching the subscriber. Stream Filters are implemented with two components: meta dictionary on publish and filter expression on subscribe. Filters are applied to all channels that the client is subscribed to. When messages are encrypted (using crypto key when initializing PubNub), the meta dictionary is plain text, so that the PubNub Network can properly apply the filters as required. It is important to only include information that is not confidential or otherwise requiring encryption.
you need to correct the syntax.
pubnub.subscribe({
channels:allChann
})
I have two types of users 'Managers' and 'Students' which subscribe to a channel 'all.channel', I want the following behavior to be applied:
a. Users from 'Managers' group will get all messages published from both user groups 'Managers' and 'Students'.
b. Users from group 'Students' will get publish messages only from user of group 'Managers' (will not see 'Students' publish messages).
My idea was to create two channels:
a. 'all.channel' - 'Managers' and 'Students' users will publish to it. only 'Managers' users will subscribe to it. A pubnub 'After Publish' function will chain only 'Managers' message to another channel called 'all.student.channel'
b. 'all.student.channel' - only 'Students' will subscribe to it.
My question is if there're any build-in tools or capabilities in PubNub to do it less complex, or even with only one channel?
Manager & Student Configuration
🔐 Security is important. Students should not see Manager's private messages.
You'll need to consider security. Only managers should be able to see manager messages. Consider multiplexing your channels for Managers. A manager will subscribe to both root.managers channel and root.students channel too. While students only subscribe to root.students channel. This way you can grant read/write permission to managers on the root.*. You can grant access to students on root.students.roomID to read/write.
You may also consider 💬 ChatEngine an open and extensible chat SDK and APIs for building group chat.
More channel configuration options below for your consideration.
Channel Configuration Options
For the PubNub SDKs 4.0 and higher. Multiplexing the SDK supports up to about 50 channels practically. You can increase this number further but device performance will be impacted. It is better to utilize the Channel Group feature of the core PubNub Stream Controller product. Stream Controller includes the following channel subscription enhancements.
Wildcard Subscribe
Maximum Addressable Channels: ( Unlimited )
PubNub Also supports Wildcard Subscriptions using a Hierarchical dot notation. Today in PubNub you can subscribe to channel "a.b.*" and receive messages at any channel below a.b. You could publish to channel "a.b.c" and receive the message in your wildcard subscribe. Note that the maximum depth supported of the hierarchy is currently a depth of three. For example "a.b.*" is good but "a.b.c.*" is not supported.
Channel Groups
Maximum Addressable Channels: ( 2,000 ✕ 10 ＝ 20,000 )
Dynamically control the device's data stream feed with PubNub Channel Groups. You can remotely control which streams the device is subscribed to and dynamically add and remove channels from the list of channels. You can multiplex subscribe up to 10 channel groups and each channel group can contain up to 2,000 channels.
Multiplexing
Maximum Addressable Channels: ( ~50 )
You can multiplex your connection by subscribing to a combination of Wildcard Channels, Channel Groups, Presence Event Stream Channels and more. It is recommended to keep your multiplexed channels below 50 for best device performance.
Stream Filtering:
Maximum Addressable Channels: ( Unlimited )
Stream Filter allows a subscriber to apply a filter to only receive messages that satisfy the conditions of the filter. The message filter is set by the subscribing client(s) but it is applied on the server side thus preventing unwanted messages (those that do not meet the conditions of the filter) from reaching the subscriber. Stream Filters are implemented with two components: meta dictionary on publish and filter expression on subscribe. Filters are applied to all channels that the client is subscribed to. When messages are encrypted (using crypto key when initializing PubNub), the meta dictionary is plain text, so that the PubNub Network can properly apply the filters as required. It is important to only include information that is not confidential or otherwise requiring encryption.
I am writing a small distributed program. The program consists of two publishers and two subscribers. Each subscriber is multi-threaded with two threads of control. Each thread subscribes to one of the two subscribers. This means each subscriber will subscribe to both publishers, one in each thread.
When a thread in one of the subscriber receives a specific message from the current subscribing publisher, it should stop subscribing to the current publisher, and start subscribing to the other publisher.
How does a subscriber unsubscribe to the current publisher? I change the topic the subscriber associated but failed.
What is the mechanism to construct and destroy a connection between a publisher and a subscriber? Topic match? QoS policy match? Which level of match is key to the problem?
Could you recommend some useful tutorials to learn the mechanism of Open DDS? I read the developer guide from official site (http://opendds.org/) but still cannot figure out the mechanism.
Quite new to RabbitMQ and I'm trying to see if I can achieve what I need with it.
I am looking for the Worker Queues pattern but with one caveat. I want to have only a single worker running concurrently per routing key.
An example for clarification:
If i send the following messages with routing keys by order: a, a, b, c, I want to have only 3 workers running concurrently. When the first a message is received a worker picks it up and handles it.
When the next a message is received and the previous a message is still handled (not acknowledged) the new a message should wait in queue. When the b and c messages are received they each get a worker handling them. When the first a message is acknowledged any worker can pick up the next a message.
Would that pattern be possible using RabbitMQ in a natural way (without writing any application code on my side to handle the locking and stuff...)
Edit:
Another clarification. All workers can and should handle all messages, and I don't want to have a queue per Worker as I want to share the load between them, and the Publisher doesn't know which Worker should process the message. But I do want to make sure that no 2 Workers are working on messages sharing the same key at the same time.
For example, if I have a Publisher publishing messages with a userId field, I want to make sure no 2 Workers are handling messages with the same userId at the same time.
Edit 2
Expanding on the userId example. Let's say I have a single Publisher and 3 Workers. The publisher publishes messages like these: { userId: 1, text: 'Hello' }, with varying userIds. My 3 Workers all do the same thing to this messages, so I can have any of them handle the messages coming in. But what I'm trying to achieve is to have only a single worker processing a message from a certain user at the same time. If a Worker has received a message with userId 1 and is still processing it, and another message with userId 1 is received I want to make sure no other Worker picks up that message. But other messages coming in with different userIds should be processed by other available Workers.
userIds are not known beforehand, and the publisher doesn't know how many workers are or anything specific about them, he just wants to schedule the messages for processing.
what your asking is not possible with routing keys, but is built into queues with a few settings.
if you define "queue_a" for a messages, "queue_b" for b messages, etc, you can then have as many consumers connect to it as you want.
RabbitMQ will only deliver a given message to a single consumer of a given queue.
The way it works with multiple consumers on a single queue is basic round-robin style dispatch of the messages. that is, the first message will be delivered to one of the consumers, and the next message (assuming the first consumer is still busy) will be delivered to the next consumer.
So, that should satisfy the need to deliver the message to any given consumer of the queue.
To ensure your messages have an equal chance of getting to any of the consumer (and are not all delivered to the same consumer all the time), there are a few other settings you should put in place.
First, make sure to set the message consumer no ack setting to false (sometimes called "auto ack"). This will force you to ack the message from your code.
Lastly, set the "consumer prefetch" limit of the consumer to 1.
With this combination of settings, a single consumer will retrieve a single message and begin working on it. While that consumer is working, any message waiting in the queue will be delivered to other consumers if any are available. If there are none available, the message will wait in the queue until a consumer is available.
With this, you should be able to achieve the behavior you are wanting, on a given queue.
...
Keep in mind this only applies to queues, though. routing keys cannot be managed this way. all matched routing keys from an exchange will cause a copy of the message to be sent to the destination queue.