I have a web client using pubnub, and there are 1000 channels need to be subscribed. But once I call pubnub.subscribe(allChannels), the server returns 500.
I did not find any limit saying why it cannot.
Actually, there is no hard limit on the number of channels you can subscribe to. But if you are going to subscribe to more than 20-30 channels, it's usually recommended to use Channel Groups instead, which allows you to create a collection of channels and give that collection a name. Then, your client simply subscribes to the Channel Group instead of the individual channels. https://www.pubnub.com/developers/tech/key-concepts/stream-controller/channel-groups/
There is a limit number of subscribed channels, that's what I have tested: 640 channels. And once subscribed 640 channels, it is still successful, but if you subscribe one more, errors will start happening (server returns 500) and all your pubnub connection will get destroyed.
It seems you can subscribe many more channels at one time (Its official website suggest 50 channels, but I have tried subscribed 400 channels at one time and it is successful)
So total limit 640 is what I have found.
Subscribe to 1,000 Channels and Beyond
You already know the list of channels you want to subscribe to, Channel Groups will help you get to 2,000 channels per group. There are also some options to subscribe to an unlimited number of channels using wildcards!
You can have 2,000 channels per channel group. This matches your need for 1,000 channel subscription.
🔐 Also remember that Security is important. Remember that you should secure your data.
More channel subscription options below for your consideration.
Channel Subscription Options
For the PubNub SDKs 4.0 and higher. Multiplexing the SDK supports up to around 50 channels practically. You can increase this number further but device performance will be impacted. It is better to utilize the Channel Group feature of the core PubNub Stream Controller product. Stream Controller includes the following channel subscription enhancements.
Wildcard Subscribe
Maximum Addressable Channels: ( Unlimited )
PubNub Also supports Wildcard Subscriptions using a Hierarchical dot notation. Today in PubNub you can subscribe to channel "a.b.*" and receive messages at any channel below a.b. You could publish to channel "a.b.c" and receive the message in your wildcard subscribe. Note that the maximum depth supported of the hierarchy is currently a depth of three. For example "a.b.*" is good but "a.b.c.*" is not supported.
Channel Groups
Maximum Addressable Channels: ( 2,000 ✕ 10 ＝ 20,000 )
Dynamically control the device's data stream feed with PubNub Channel Groups. You can remotely control which streams the device is subscribed to and dynamically add and remove channels from the list of channels. You can multiplex subscribe up to 10 channel groups and each channel group can contain up to 2,000 channels.
Multiplexing
Maximum Addressable Channels: ( ~50-500 )
You can multiplex your connection by subscribing to a combination of Wildcard Channels, Channel Groups, Presence Event Stream Channels and more. It is recommended to keep your multiplexed channels below 50 for best device performance.
Stream Filtering
Maximum Addressable Channels: ( Unlimited )
Stream Filter allows a subscriber to apply a filter to only receive messages that satisfy the conditions of the filter. The message filter is set by the subscribing client(s) but it is applied on the server side thus preventing unwanted messages (those that do not meet the conditions of the filter) from reaching the subscriber. Stream Filters are implemented with two components: meta dictionary on publish and filter expression on subscribe. Filters are applied to all channels that the client is subscribed to. When messages are encrypted (using crypto key when initializing PubNub), the meta dictionary is plain text, so that the PubNub Network can properly apply the filters as required. It is important to only include information that is not confidential or otherwise requiring encryption.
you need to correct the syntax.
pubnub.subscribe({
channels:allChann
})
Related
I have 16 queues & mulitple consumer servers for those queues. I have created one dedicated channel for each queue to consume messages. Consumer & dispatch channels on each server share same connection.
When I dispatch messages to each queue, I do the following:
create a new channel
bind channel to the queue with proper routing
dispatch the message
close the channel
I have lots of incoming webhooks from Shopify & these webooks contents are dispatched to specific queues.
While processing each message, I need make an API call to Shopify. Shopify API has rate limit. If I hit rate limit once, I redispatch all messages from the consumer back to rabbitmq with a delay header of 1 minute(time required to clear the API rate limit).
Now, when I have several consumers running with lots of messages in the queue & I re-dispatch those messages, I get too many channels error for a period of time. How can I avoid this error?
I tried to keep 2 dedicated channels per queue:
for conusmer purpose only
for dispatch purpose only
For, 16 queues, & around 11 consumer servers. This way, I always have to keep 352 channel open. This caues CPU utilization on rabbitmq host server to reach >90% which is also an issue. As the server can crash any time.
I found the solution to the problem after digging through the RabbitMQ documentation.
Instead of creating a new channel for each dispatch, I created a single channel & kept it alive for the entire connection session. When creating the channel, I asserted all the exchanges that would be used by my queues.
Then I just publish the messages to the desired exchange with the routing key. As my queues are already bonded with the exchanges & listen for messages with a given routing key, the messages end up in the correct queue!
This way I can maintain just 01 connection & only 01 channel per server!
Maximum number of subscription possible by paho-mqtt python library.
Normally subscriptions are held by the broker, not the client. The broker just forwards messages to the client that match a topic pattern and the client passes that received message to the callback.
How those topic patterns are stored will differ from broker to broker, but assuming even the most naive implementation of an array then the limit would likely to be the size of an int on the platform the broker was running on, which is likely to be larger than any sensible system would ever hit.
If the client library is keeping track of the list of subscribed topics (which I don't believe the Paho libraries do as there is no need), then the list is likely to be on the same scale as the broker.
Also be aware that you can subscribe to wildcard topics, this would hold a single slot in any list, but could match any number of actual topic a message is published on.
I have two types of users 'Managers' and 'Students' which subscribe to a channel 'all.channel', I want the following behavior to be applied:
a. Users from 'Managers' group will get all messages published from both user groups 'Managers' and 'Students'.
b. Users from group 'Students' will get publish messages only from user of group 'Managers' (will not see 'Students' publish messages).
My idea was to create two channels:
a. 'all.channel' - 'Managers' and 'Students' users will publish to it. only 'Managers' users will subscribe to it. A pubnub 'After Publish' function will chain only 'Managers' message to another channel called 'all.student.channel'
b. 'all.student.channel' - only 'Students' will subscribe to it.
My question is if there're any build-in tools or capabilities in PubNub to do it less complex, or even with only one channel?
Manager & Student Configuration
🔐 Security is important. Students should not see Manager's private messages.
You'll need to consider security. Only managers should be able to see manager messages. Consider multiplexing your channels for Managers. A manager will subscribe to both root.managers channel and root.students channel too. While students only subscribe to root.students channel. This way you can grant read/write permission to managers on the root.*. You can grant access to students on root.students.roomID to read/write.
You may also consider 💬 ChatEngine an open and extensible chat SDK and APIs for building group chat.
More channel configuration options below for your consideration.
Channel Configuration Options
For the PubNub SDKs 4.0 and higher. Multiplexing the SDK supports up to about 50 channels practically. You can increase this number further but device performance will be impacted. It is better to utilize the Channel Group feature of the core PubNub Stream Controller product. Stream Controller includes the following channel subscription enhancements.
Wildcard Subscribe
Maximum Addressable Channels: ( Unlimited )
PubNub Also supports Wildcard Subscriptions using a Hierarchical dot notation. Today in PubNub you can subscribe to channel "a.b.*" and receive messages at any channel below a.b. You could publish to channel "a.b.c" and receive the message in your wildcard subscribe. Note that the maximum depth supported of the hierarchy is currently a depth of three. For example "a.b.*" is good but "a.b.c.*" is not supported.
Channel Groups
Maximum Addressable Channels: ( 2,000 ✕ 10 ＝ 20,000 )
Dynamically control the device's data stream feed with PubNub Channel Groups. You can remotely control which streams the device is subscribed to and dynamically add and remove channels from the list of channels. You can multiplex subscribe up to 10 channel groups and each channel group can contain up to 2,000 channels.
Multiplexing
Maximum Addressable Channels: ( ~50 )
You can multiplex your connection by subscribing to a combination of Wildcard Channels, Channel Groups, Presence Event Stream Channels and more. It is recommended to keep your multiplexed channels below 50 for best device performance.
Stream Filtering:
Maximum Addressable Channels: ( Unlimited )
Stream Filter allows a subscriber to apply a filter to only receive messages that satisfy the conditions of the filter. The message filter is set by the subscribing client(s) but it is applied on the server side thus preventing unwanted messages (those that do not meet the conditions of the filter) from reaching the subscriber. Stream Filters are implemented with two components: meta dictionary on publish and filter expression on subscribe. Filters are applied to all channels that the client is subscribed to. When messages are encrypted (using crypto key when initializing PubNub), the meta dictionary is plain text, so that the PubNub Network can properly apply the filters as required. It is important to only include information that is not confidential or otherwise requiring encryption.
I am looking at building an app that monitors the public transport buses for a major city:
I did a quick prototype using pubnub. The buses have a phone transmitting gps signals to a channel and bus users have phones subscribed to channels. I have questions:
I am planning for each bus route there is a channel. The city has 50 routes so there will be 50 routes. Does this adhere to the best practice?
Is there an api to list channels ?
I am sending a message to a channel every second. Assume, there are 50 routes with 5 buses each running 24 hours. There will be 216000000 daily messages. what will i be charged for a day?
Does your Android client open a network connection everytime a publish is call? I want to minimize the bandwith used by the phone that is transmitting the GPS signal.
Bus users may want to see location of multiples buses. I know best practice is to subscribe to one public and one private channel. What is the best way to do it?
I would appreciate if you could answer the above questions.
Full disclosure up front - I work for PubNub Customer Success so responses for pricing related questions are informational in nature only and not to be construed as a promotional. Asker specifically mentions PubNub and the information provided below is publicly available from the PubNub website.
Anant, also as an FYI StackOverflow would normally ask that each of these questions gets asked as a separate thread. Moving forward please do your best to adhere to community guidelines.
1 Every implementation will be different as far as the specific architecture and design pattern strategy though your proposed approach seems to be a sensible utilization of channel methodology. PubNub does not limit the total number of channels in use, however as a practical limitation for most mobile development frameworks subscribing to more than 50 channels simultaneously would be around the upper limit. Adding more than that and both iOS and Android will begin exhibiting performance limitations. If new bus lines are added the subscriptions can be managed to only subscribe to nearby routes, etc.
Question 1 the second with the indent. Yes that can be done with the here_now API
2 PubNub charges $1 per million messages (without SSL enabled) so based on your hypothetical your message charges would be $216 per day. That being said, there is significant room here for design pattern optimization so that busses only publish a new location whenever there is a change - repeated publishes while the bus is standing still are unnecessary. This optimization on it's own will bring the message usage figure down significantly, and there are other strategies which can be utilized to further optimize depending on your specific implementation approach. If you anticipate needing more than 1 billion messages per month, a deployment to Global Cloud would make sense so as to avail yourself of volume discount pricing not otherwise available on Go Cloud.
3 Rather than opening a new connection with every publish, PubNub keeps an active socket connection open until unsubscribed or disconnected via loss of network connection/app force close. The bandwidth utilization to keep this connection active over a period of several hours and absent any other publish/subscribe activity typically measures less than 1K depending on your configuration parameters. Android supports background threading so even when the app is not in focus the connection can remain open to facilitate data push alerts which can be used to prompt the user to bring the app back into the foreground to review any updated information.
4 This question is not clear, assuming that the bus locations are published to the public channel what would the purpose of the private channel serve? If you meant a private channel to receive alerts for the arrival of the user's selected bus, then yes that would be an appropriate implementation strategy. Please clarify if you meant something different.
I could not find any explicit information about the creation of channels with Pusher.
Is that simply an implicit action when either subscribing on the client or pushing events on the server?
class HelloController < ApplicationController
def hello
#does this create a channel "named 'test-channel'"?
Pusher['test-channel'].trigger('test_event', { :hello => 'world' })
end
end
If so is there a limit to the number of channels available?
The reason for my question is that I'd like to create a unique channel for every user and after the client side has closed down that channel.
But probably that is not really a good idea ;-)
thanks
Channels are really just a way of routing or filtering data. They exist by simply being subscribed to or having data published to them. So, it is an implicit action.
There are no limits to the number of channels you use and a unique channel per user is a nice solution for targeted messaging.