I implemented Pusher API in a live chat recently.
I launched a Startup package of Pusher yesterday. After 4 hours of being live, I receive an email that my account is reaching the cap on usage.
I logged in and looked at the stats, to discover that the Messages per Minute were between 5,000 and 20,000.
I don't understand how this is possible. I have around 100-150 connections open.
Why is the message count so high?
Armin
Found the answer myself! :)
Here is the link for anyone who may have the same problem:
https://pusher.tenderapp.com/kb/accountsbillingplanspricing/how-is-my-message-count-calculated
Basically, if you have 100 users subscribed into a channel, and 1 message is sent, it counts are 100 messages being sent since each user would have to be notified.
Bottom line is to properly filter your channels.
Related
I'm writing bot for telegram to gather some stats from group chat. I need to get info about every message (from the beginning of chat). I know how can i do it, but it's a quite bad idea. I can use forwardMessage method, but i need second acc for it and i'm getting timeouted when i'm sending messages too fast (for one hour), so it's a bit long way to collect stats for conversation that has over 2 million messages ;s I tried to set limit on 10 messages per second but i'm still getting timeouted, so idk how it works.
There must be other way to get JUST message info by id without forwarding it ;v I can't find it in API.
There has no API to do this at this time, you can suggest this idea to #BotSupport, before them added this feature, I am doing same thing like you.
According to Bot FAQ, Telegram API rate limit 1/s pre chat, and global limit is 30/s.
There is no way to do this with Telegram bot api, you can use ReadHistory Method of MadelineProto without the necessity to use forward message method
I recently deployed a bot using Azure and BotFramework to Skype, Slack, Telegram and some other platforms.
They all seem to work fine, except in Kik, where the bot will suddenly stop responding. The error message in BotFramework reads:
{"message":"Too many requests for user: 'redacted_user_name'","error":"TooManyRequests"}
The Kik tester is triggering this error through regular use, though when I test it on my (Android) phone, it works just fine.
Any idea what might be causing this?
EDIT:
After contacting Kik, I was told that my Bot was sending more messages than it was recieving, and they only allow a surplus of 20 before a bot becomes banned.
They say the solution is to implement batching, which BotBuilder says is built in. (My bot uses session.send("text") followed by a prompt.) However, Kik does not see my messages as a batch, and every couplet is counting as 2 messages.
I tried adjusting the autoBatchDelay to see if 0 would work better than the default and noticed that it did not make a difference. Furthermore, changing it to 2000 also made no difference, and did not delay 2000ms between messages.
var bot = new builder.UniversalBot(connector, {autoBatchDelay: 0});
Is it possible my bot is not batching properly? What steps could I take to address this?
Batching for Kik is currently on our backlog. In the mean time, is there any reason you can't send your text and prompt in the same message (with carriage returns in between if needed)? That should resolve your issue (as I understand it).
Also worth noting that the Kik rules for recovering from a throttling deficit are somewhat complex.
• In any given send message request, a bot can send up to 25 messages in a single POST request. Within the 25 messages, a bot is allowed to have up to 5 messages directed to a single user.
• Whether you send 1 message or 5 messages, that collection of requests is considered a “batch” of messages to a user.
• A bot is allowed 20 unsolicited batches to a user a day.
• This means you could be sending between 20-100 unsolicited messages to a user a day depending on how many messages you have in a batch. How the bot platform determines unsolicitation works like a debit/credit system that resets at the end of a day. e.g. Julie sends the bot a message, the balance becomes +1. The bot responds with 3 messages in one batch, the balance becomes 0. Julie sends the bot 1 message, the balance becomes +1. The bot responds with 5 messages in separate batches, the balance becomes -4. Julie sends the bot a message, the balance becomes +1. The bot responds with 5 messages in separate batches, the balance becomes -9.
• If this deficit continues to -20, the daily user rate limit will have reached, and the bot will NOT be able to send any more messages to that user. There are different methods to work with this rate limit, e.g. using batches more efficiently or building a UX that encourages more user interactivity.
We are facing a problem with Service Bus.
We have a topic, with two subscriptions.
We have enabled Duplicate Detection on those, with 1 minutes window (tried with 2 seconds first). We are using Duplicate Detection to avoid multiple messages processed in short interval (to maintain the interval between the messages)
We are using the message scheduling (ScheduledEnqueueTimeUtc) to repeat the messages to appear after 5 minutes, with same message ID (every time a new message is created with schedule, and old message is completed)
The workflow is as follows (problem):
First time a message is published (without scheduling)
This message is immediately consumed by the message pump, and a new message with same details and a schedule time of 5 minutes is send to the topic (UTC), expecting it to appear after 5 minutes
The message is not appearing in the subscription
When debugged, this issue doesn’t come up
When we send the First message with at least 30 second delay (scheduled), then it is working fine
If we recreate the topic and subscription with Duplicate Detection turned off, we are able to get the message using the above workflow
Since we have no clue on what is happening to the published message, we need help to identify the root cause of the issue.
This is an expected behavior of the ASB.
When a message is scheduled, it's actually enqueued on the broker with delayed appearance. ASB on the server side de-duplicates messages upon arrival and uses message ID for de-dup.
In your case, if you delay dispatch of the second message and the original message is processed, there will be nothing to de-dup and the second message will be enqueued. If you don't delay, then the broker will see an identical ID to the previously sent message that has not been completed or DLQed yet, and it will be de-duped.
Possible way to go about is not reuse the same transport message ID (ID used for the BrokeredMessage). In case you need to associate messages, you can use Properties for that.
I have the following dilemma:
I need to send a heartbeat message every 5 minutes (or less) to all users of my app
I thought about topic messaging, but the 1 million subscriber limit is not acceptable for my application
So: the only possibility left is sending out the message in batches of 1000
This is really resource intensive
Now my question:
How can I make this process of batching and sending really efficient? Is there a good solution already made, preferably in node.js?
Thank you,
Sebastian
You may use XMPP, instead of HTTP.
As google says, it is less resource intensive in respect to HTTP:
The asynchronous nature of XMPP allows you to send more messages with
fewer resources.
Also you can have 1000 similtanouis connection per app (sender ID):
For each sender ID, GCM allows 1000 connections in parallel.
Also there exists a node-xmpp solution available for that.
I am looking at building an app that monitors the public transport buses for a major city:
I did a quick prototype using pubnub. The buses have a phone transmitting gps signals to a channel and bus users have phones subscribed to channels. I have questions:
I am planning for each bus route there is a channel. The city has 50 routes so there will be 50 routes. Does this adhere to the best practice?
Is there an api to list channels ?
I am sending a message to a channel every second. Assume, there are 50 routes with 5 buses each running 24 hours. There will be 216000000 daily messages. what will i be charged for a day?
Does your Android client open a network connection everytime a publish is call? I want to minimize the bandwith used by the phone that is transmitting the GPS signal.
Bus users may want to see location of multiples buses. I know best practice is to subscribe to one public and one private channel. What is the best way to do it?
I would appreciate if you could answer the above questions.
Full disclosure up front - I work for PubNub Customer Success so responses for pricing related questions are informational in nature only and not to be construed as a promotional. Asker specifically mentions PubNub and the information provided below is publicly available from the PubNub website.
Anant, also as an FYI StackOverflow would normally ask that each of these questions gets asked as a separate thread. Moving forward please do your best to adhere to community guidelines.
1 Every implementation will be different as far as the specific architecture and design pattern strategy though your proposed approach seems to be a sensible utilization of channel methodology. PubNub does not limit the total number of channels in use, however as a practical limitation for most mobile development frameworks subscribing to more than 50 channels simultaneously would be around the upper limit. Adding more than that and both iOS and Android will begin exhibiting performance limitations. If new bus lines are added the subscriptions can be managed to only subscribe to nearby routes, etc.
Question 1 the second with the indent. Yes that can be done with the here_now API
2 PubNub charges $1 per million messages (without SSL enabled) so based on your hypothetical your message charges would be $216 per day. That being said, there is significant room here for design pattern optimization so that busses only publish a new location whenever there is a change - repeated publishes while the bus is standing still are unnecessary. This optimization on it's own will bring the message usage figure down significantly, and there are other strategies which can be utilized to further optimize depending on your specific implementation approach. If you anticipate needing more than 1 billion messages per month, a deployment to Global Cloud would make sense so as to avail yourself of volume discount pricing not otherwise available on Go Cloud.
3 Rather than opening a new connection with every publish, PubNub keeps an active socket connection open until unsubscribed or disconnected via loss of network connection/app force close. The bandwidth utilization to keep this connection active over a period of several hours and absent any other publish/subscribe activity typically measures less than 1K depending on your configuration parameters. Android supports background threading so even when the app is not in focus the connection can remain open to facilitate data push alerts which can be used to prompt the user to bring the app back into the foreground to review any updated information.
4 This question is not clear, assuming that the bus locations are published to the public channel what would the purpose of the private channel serve? If you meant a private channel to receive alerts for the arrival of the user's selected bus, then yes that would be an appropriate implementation strategy. Please clarify if you meant something different.