I am currently creating a web-dashboard for a Discord bot I have made previously.
I am using express.js as my backend server, and in order to validate user's credentials, almost every request made whether it is a get or post request, has some sort of call to the Discord API originating from the backend of the dashboard before the frontend is served to the user (as I do not know how I can securely make API requests from the frontend as user tokens are required)
As you can probably guess from the previous paragraph, I have been temporarily banned from using the API (below), meaning I can no longer use my Discord Account and my bot has gone offline.
Image showing API error (Code 429)
I understand that the API request sends headers regarding the rate limits and remaining quota etc, I just don't know what to do with that data. What would be the best programming practices in order to solve this rate-limiting issue?
I have tried using the refresh tokens initially, as I thought new tokens would stop users having to re-authenticate. These tokens were stored in a mongodb database (to prevent me from sending a token request every time a user wanted to make a request), but these measures did not help as much as I would have liked them to.
You shouldn't be experiencing rate limits that quickly, why are you making so many requests to the API? The access token provided from OAuth2 flow works for multiple requests (it stays valid for a full week if you don't ask for a new token), so make sure you aren't requesting a new token every time the client loads your app.
If the problem is that you're using the access token to make a request to the Discord API for user info every time the user reloads your app, then you need to change that. Just save user info on your backend, and every once in a while (maybe a minute or two), "refresh" their data automatically by making just one request. That way if somebody spam reloads your app, your app won't then spam request the Discord API for new info every time.
If you are ever ratelimited by Discord because your users spam reloaded your app, that's a vulnerability on your end and you need to handle it.
Also, you mentioned using the rate limiting info provided in the request headers. That info can be useful so that you know how long to wait before making another request, and stop Discord's API from acting up towards your app. But it definitely wouldn't help in actually stopping your app from getting ratelimited, that can only be helped by managing how you make requests to the Discord API better.
Related
I am developing a public website
One of the page is to let people submit opinions.
They need to enter email, phone, message ,etc, in order to submit the opinion form.
This means I need to make an api for people, so that after they press the submit button, the POST request will be sent.
But some people might abuse this function(e.g. create bot to send massive amount of POST request).
Since the frontend design is already finished and no recapcha is used in frontend, I was thinking if there are backend ways to do it.
Is there any ways/best practice to prevent this kind of things?
I am using node.js(aws lambda) for my api function
Yes, if there is still a way to implement captchas, use package "svg-captcha". If you only want to deal with the backend, refer to this header: req.headers['user-agent'];, so you can track down those who abuse the requests and limit their access for a while.
I am planning to create a mobile application for android and ios users, i think i will take a try with xamarin since i will be alone on this project and i don't have a lots of time.
I want that the mobile app for both platform get datas from the api, then if there is new datas available we notify the user by a notification.
How the mobile will work in that kind of project? I mean should i make a background service then check every x seconds/minutes by http request? In that case which time interval? Should i use websockets instead for this case?
The app might be used by many people, so i would to know the scenario in this kind of project: Getting very fast changes, without overload the server due to too many connects or whatever else.
I'm confused about this and i need some lights around, any mobile application/server experiences related would be apprecied!
EDIT:
As suggered by an user, here additional infos:
The api is homemade, restful using JWT made in NodeJS.
Each users on their device should get messages from server asap, even when the app is in background/closed.
Maybe in the future a way to send messages between users themselves.
You have to implement push notification.
It is quite easy to implement this in xamarin. just send the push notification to the device and on the notification received call back send the API request to retrieve the updated data.
Here is the document for sending push notification from custom API.
https://learn.microsoft.com/en-us/appcenter/push/pushapi
I'm not a mobile developer, so take this with a grain of salt.
The answer to this really depends on what you're doing, which informs how often to check the API. If it's a messaging app, for example, you could have it check every couple minutes to see if there are undelivered messages, then check more frequently for the next X minutes (to facilitate a conversation in real time).
If it's a GPS navigation app to be used while driving, you'd need much more frequent requests.
As for the API, that also depends on what type of API and the number of requests you can make to it. Is it a commercial API that you get x number of calls per hour on? Is it an API that you built? Etc.
Basically, you need to give more information in order to get more specific answers.
I have developed a chatbot using Microsoft BotFramework and node.js and deployed it in webchat. As per this documentation, it is written that the bot's secret 's' can be replaced with a temporary token 't' which is valid for one conversation only.
But the problem is the life span of this token is 30 minutes and within this time window, if some intruder accesses the entire URL: "https://webchat.botframework.com/embed/YOUR_BOT_ID?t=YOUR_TOKEN_HERE&userid=some_user_id" then it is just a child’s play for him to get all of the user’s data because it mimics the chat of the actual user in the other machine.
Is there anything in the BotFramework (apart from DirectLine) that can be done that restricts the URL with the same token to be opened in another machine?
There is an issue on GH which is facing the same event with you, and with the comments, we can found that this situation will not be changed currently.
However, we can get some hints from the comment:
Ultimately, you can't hide the secret/token from clients.
If you want to remove it from the URL, you can host the JS control on your own.
If you want to remove it from page source, you can pass the value in a cookie >and read it in JS in your webpage.
However, in all cases, the value will be available in memory.
I think you can build another simple web site yourself as the bridge from the iframe and yout bot application. You can restrict your user whether is unique in this website's session. And also you can verify your user before instantiate the Bot WebChat.
I have a webhook designed in Node.js for API.ai that interacts with multiple API's to gather information and give response to user.
Since, I am interacting with multiple API's the response time taken is more than 5 secs which is causing the API.ai request to timeout.
To overcome this, I am trying to implement caching into the node.js webhook which saves the response from API's until a certain amount of time. This will remove timeout until the max-age header time is reached.
Edit: What is the best node module that I can use to cache the API responses for subsequest requests.
Note: I am using request node module for http requests but it doesnt seem to provide a way to cache the response.
All of the answers given are reasonable for tackling the cache problem on the request side. But since you specified API.AI and Actions, you might also be able to, or need to, store information while the conversation is in progress. You can do this using an API.AI context.
It may even be that if you limit it to just one remote call for each response from the user, you might be able to fit it in the timeframe.
For example, if you were having a conversation about movie times and ticket ordering, the conversation may go something like:
User: "I want to see a movie."
[You use an API to lookup the nearest theater, store the theater's location in a context and reply] "Your nearest theater is the Mall Megaplex. Are you interested in one there?"
User: "Sure"
[You now already have the theater, so you query for what it is playing with another API call and store it in a context] "There are seven different movies playing, including Star Wars and Jaws. Do those sound interesting?"
User: "No"
[You already have the data in the context, so you don't need another call.] "How about Rocky or..."
In this way you're making the same number of calls (generally), but storing the user's results in the session as you go as opposed to collecting all the information for the user, or all the possible results, and then narrowing them.
Finally decided to use the below module:
https://www.npmjs.com/package/memory-cache
This served my scenario better. Might try using Redis soon when i get some time.
I created an iOS app that using Instagram API and have a critical issue posting likes to Instagram.
Using endpoint – POST /media/media-id/likes, with user-specific access token. I request scope – likes from users.
Instagram API works fine for all requests except saving likes. We can only save/post a couple hundred likes to Instagram, and then response is "We've taken extra measures to protect the Instagram API from abnormal activity. Since you have recently created your API application, please contact apidevelopers#instagram.com to receive whitelisting for a higher rate limit."
There is other developers with same issue with no support or any feedback waiting for months.
Please, let me know if there is limits and how it works, or how to become whitelisted
I don’t want to create many Instagram API clients and assign users among them. It seems like bad acting for me, if there is official API with strict limits I want to work with it.
Instagram has new API limits: 100 likes/hr 60 follows/unfollows/hr per access token. That is only if you authenticate serverside and sign your headers. Check out the new authentication: http://instagram.com/developer/authentication/
I'm using it for my new site that automates likes/follows/unfollows, haven't had any problems works great. Check it out: http://instapromobiz.com
Instagram takes increased measures to limit your API requests based on IP address. This is not common knowledge, it's just something I've come to know from working with the API extensively.
They've built an API that is FAR from perfect, but have been upgrading it to be "smart". Smart enough to know that a newer client, pinging the API many times just for one user, may very well be a script kiddie trying to bot the system. For this reason, they do not give you a rate limit error, they politely say "we have taken extra measures to protect our API from being spammed by bots".
The good news is, other users should still be able to access your liking mechanism. Just not you, from that IP.