I would like to take a url input from a user, and serve that url to all other users in some context. I will show this url to users within my website as a link with a message of More Details. I am aware that when you point the cursor to the link, it shows the url and give or take it can be understood whether it is real or malicious, but 99.9% of the people won't think about such thing and will just click it right away.
So my question is, can I detect whether the inputted link is real or malicious, and if so how, and if not so, what can I do to at least improve the security to some extent? I am using frontend Reactjs , backend Nodejs, and multiple AWS Resources for data and API management.
As far as I am aware, there's nothing specific that can be done on the AWS side to achieve this since this is specific to your backend implementation.
I am no expert on security, but maybe using VirusTotal API to check if a given URL is malicious? There are limits on the allowed number of requests. Also, as stated:
The public API is a free service, available for any website or application that is free to consumers. The API must not be used in commercial products or services
If you want to commercialize your service, you may get banned from using VirusTotal if you do not go with the paid route.
Maybe there are alternative solutions that are free for commercial use. But using such a service is your only route if you want to delegate URL security checks to a third-party service since AWS does not offer anything similar.
Related
I am currently working on a web application. The client is designed in Vue.js and the server application is made with node.js and express.
As of now I plan to deploy both the client-website and the node.js-app on the same server. Both will be adressed via two different, unique domains. The server will be set up manually with nginx.
The problem now is that this solution won't prevent a user from being able to send requests to the server outside the client that was made for it. Someone will be able to call the /register route (with postman, curl etc.) to create an account an 'unofficial' way. I think the only clean solution is that only my Vue.js-app would be able to perform such actions. However, since both the server and the client are two different environments/applications, some sort of cross-origin-request mechanism (cors for instance) must be set up.
So I'm wondering, is this bad by design or is it usual that way? If I wanted this not to be possible, should I see to that issue and try to make the express-API as private as possible? If so, what are usual best practices for development and deployment / things to consider? Should I change my plan and work on a complete different architecture for my expectations instead / How do 'bigger' sites manage to allow no requests outside the official, public developer API's?
I think the only clean solution is that only my Vue.js-app would be able to perform such actions.
An API that is usable from a browser-based application is just open to the world. You cannot prevent use from other places. That just how the WWW works. You can require that a user in your system is authenticated and that auth credential is provided with each request (such as an auth cookie) before the API will provide any data. But, even then, any hacker can sign up for your system, take the auth credential and use your API for their own uses. You cannot prevent that.
If I wanted this not to be possible, should I see to that issue and try to make the express-API as private as possible?
There is no such thing as a private API that is used from a browser-based application. Nothing that runs in a browser is private.
If you were thinking of using CORs protections to limit the use of your API, that only limits it from other browser-based applications as CORs protections are enforced inside the browser. Any outside script using your API is not subject to CORs at all.
How do 'bigger' sites manage to allow no requests outside the official, public developer API's?
Bigger sites (such as Google) have APIs that require some sort of developer credential and that credential comes with particular usage rules (max number of requests over some time period, max data used, storage limits, etc...). These sites implement code in their API servers to verify that only an authorized client (one with the proper developer credential) is using the API and that the usage stays within the bounds that are afforded that developer credential. If not, the API will return some sort of 4xx or 5xx error.
Someone will be able to call the /register route (with postman, curl etc.) to create an account an 'unofficial' way.
Yes, this will likely be possible. Many sites nowadays use something like a captcha to require human intervention before a request to create an account can succeed. This can be successful at preventing entirely automated creation of accounts. But, it still doesn't stop some developer from manually creating an account, then grabbing that accounts credentials and using them with your API.
When talking about web applications, the only truly private APIs are APIs that are entirely within your server (one part of your server calling something in another part of your server). These private APIs can even be http requests, but they must either not be accessible to the outside world or they must require credentials that are never available to the outside world. Since they are not available to the outside world, they cannot be used from within a browser application.
OK, that was a lot of things you cannot do, what CAN you do?
First and foremost, an application design that keeps private APIs internal to the server (not sent from the client) is best. So, if you want to implement a piece of functionality that needs to call several APIs you would like to be private, then don't implement that functionality on the client. Implement that functionality on the server. Have the client make one request and get some data or HTML back that it can then display. Keep as much of the internals of the implementation of that feature on the server.
Second, you can require auth credentials for a user in your system for all API usage. While this won't prevent rouge usage, it will give you a bit more control because you can track usage, suspend user accounts when you find abuse, etc...
Third, you can implement usage rules for your public-facing APIs such as requests per minute, amount of data, etc... that your actual web application would never exceed so if they are exceeded, then it must be some unintended usage of the API. And, you could go further than that and detect usage patterns that do not happen in your client. For example, if you see an API user cycling through dozens of users, requesting all their profiles and you know that is something your regular client never does, you could detect that type of usage and block it.
I have cors installed and only my website is whitelisted, how reliable is this? Can bad actors still call my api if they are not calling it from my website?
Next I want to rate limit each user on my website, (the users are not registered or signed in),
I want to restrict each user to make no more than 1 request per second.
How can each user be identified? and then how can each user be limited?
Too many separate questions packaged together here. I'll tackle the ones I can:
I have cors installed and only my website is whitelisted, how reliable is this? Can bad actors still call my api if they are not calling it from my website?
CORS only works with cooperating clients. That means browsers. Your API can be used by anybody else with a scripting tool or any programming language or even a tool like CURL. So, CORS does not prevent bad actors at all. The only thing it prevents is people embedding calls to your API in their own web page Javascript. It doesn't prevent anyone from accessing your API programmatically from whatever tool they want. And, they could even use your API in their own web-site via a proxy. It's not much protection.
How can each user be identified? and then how can each user be limited?
Rate limiting works best when there's an authentication credential with each request because that allows you to uniquely identify each request and/or ban or delay credentials
that misbehave. If there are no credentials, you can try to cookie them to track a given user, but cookies can be blocked or thrown away even in browsers to defeat that. So, without any sort of auth credential, you're stuck with just the requesting IP address. For some users (like home users), that's probably sufficient. But, for corporate users, many, many users may present as the same corporate IP address (due to how their NAT or proxy works), thus you can't tell one user at a major company from another purely by IP address. If you had a lot of users from one company simultaneously using the site, you could falsely trigger rate limiting.
I've been looking at node.js, REST APIs and WebSockets lately to further my knowledge about backend and frontend web development. Trying to go with best practices I see REST API comes up all the time. Now my problem which I don't seem to understand how to properly solve.
Say for example I'd like to have client / server decoupled and for this I implement a REST API in the backend so that my frontend will access and get data to render. Specific (imaginary) example: lets say I want to build a rental service website. Now I would like to have an endpoint for my frontend to access information about certain products, let's say the number of bikes that have been rented so far. I'd like to be able to show this on my frontend (through the help of the REST API) but I wouldn't like for other people who call this REST API to be able to get the data (because espionage is a serious business and I'd like to keep the evil ones away, yes they can webcrawl but bla bla). So in essence I'd like for the localhost machine to be able to access (part of) the REST API but not anyone else. Things get complicated because I'd also like people to be able to create a user on my website so then I'd like to have other endpoints which can then be accessed without restriction because I'm thinking, what if at some point I'd like to have a mobile app that is integrated with the service. Then it will be unfeasible to restrict all requests to localhost.
How would you architect a secure server / client as this one? Or in your opinion is it not that big of a deal to have the REST API exposed to others (the evil ones)?
The above goes for WebSockets as well. I know REST APIs are all nice and neat but I think the future lies in near-realtime connections and so I'm likewise as interested in WebSockets (through higher level modules of course, Socket.io, SockJS etc.).
There are many solutions to secure your API out there and many of them are opensource. Which one you'll use really depends your detailed needs.
But to get you started I will mention a solution that is very accepted and supported by a large community:
Have a look at JSON Web Token, which are for example explained in this Article.
Basically your client requests an authentication token from the server and then stores it locally to reuse it for every request to your API.
The Server on the other hand may protect your API as needed. That means you may also have a public API that does not expect a token in the HTTP Header.
Tokens may also expire. That is handy if you, for example, will allow a new user for registering on your site for a limited time.
Here is another article that explains things.
Now on to the websocket part of your question. YES, you definetly want to protect your server side sockets as well. So look out for a library that supports authentication. Again, I think there are a number of opensource libraries out there.
To mention one: Primus.
Primus is an abstraction layer for many socket libraries underneath and lets you quickly change the socket provider. But it also has an authentication hook that you can implement.
And guess what.. you can use it to check for a JSON Web Token!
Hope this gets you started.
What algorithm or set of heuristics can a server and a mobile app use so that the server can always be fairly certain that the app is used within the boundaries of a given geographic region (e.g. a country)? How can the server ensure that app users outside of the defined region can not falsely claim that they are inside the region?
You can't be 100% sure that user isn't reporting a fake location, you can only make the process of faking it as difficult as possible. You should implement several checks depending on the data you have access to:
1) user's IP address (user can use a proxy)
2) device's gps coordinates (they can be spoofed)
3) the locale of the device (isn't a reliable indicator)
One of the most secure checks (but also not 100%) is sending user an SMS with the confirmation code, which he has to type in the app.
One of the most sophisticated algorithms known to me is in the Google Play (so some apps can only be available only certain countries). It checks such parameters as IP address, user's mobile operator and several others, but there are tools (like Market Enabler) and techniques that can trick the system.
If you dont want to use Google Play or other ways, the best way (I say best because it first costs nothing performance-wise and cost-wise, and secondly it is easy to use and and thirdly you need it anyway if you expect large number of users - it provides nice tools and static cache, optimizer, analytics, user blocking, country blocking etc) is to use cloudflare.
Once you signup for a free cloudflare account, you can set up your server public IP address there so that all traffic is coming through cloudflare proxy network.
After that everything is pretty straightforward, you can install cloudflare module in your server .
In your app, you can get country code of the visitor in the global server request variable HTTP_CF_IPCOUNTRY - for example,
$_SERVER['HTTP_CF_IPCOUNTRY'] in PHP. It will give you AU for Australia. (iso-3166-1 country codes). It doesnt matter what language you use.
Coudflare IP database is frequently updated and seems very reliable to detect user's geolocation without performance overhead.
You also get free protection from attacks, get free cache and cdn features for fast-loading etc.
I had used several other ways but none of them was quite reliable.
If you app runs without a server, you cstill pout a file to a server and make a call to the remote url to get country of the user at each request.
apart from things that #bzz mentioned. you can read the wifi SSID of user wifi networks, services like http://www.skyhookwireless.com/ provides api( i think with browser plugins, i am not sure) which you can use to get location by submitting the wifi SSID.
if you need user to be within specific region all the time when using the app you ll probably end up using all the options together, in case you just need one time check, SMS based approach is the best one IMO.
for accessing wifi SSID , refer to this, still you can not be 100% sure.
I want to create a SaaS extension for chrome.
How do I ensure that they cannot use my extension's functionality when their subscription is no longer current?
My basic idea is that whenever they want to use my chrome extension's functionality, the extension makes an ajax request to my server to check to see if today's date is before the subscription's ending date in my DB.
The extension is obviously is client based, so even if I have code on the client side that's only executed if my ajax request returns that they have a current subscription, couldn't an enterprising individual just look at my code and run it via console in a way that gets past my ajax request requirement?
Is there a way to enforce the subscription?
Edit:
This is mostly a conceptual question, but I'll try be clearer.
All the javascript code needed for my app to function is on their local machine, in their source files (to work it doesn't require access to my database).
so you could think of my code on their local machine as looking like this:
if (usersSubscriptionIsCurrent) {
runFeature()
}
And usersSubscriptionIsCurrent is true if the Ajax request to my server returns that their subscription is current.
Someone could still run my feature just by looking at the source code, and then typing runFeature() into their console.
I want to prevent that.
My extension relies on sending data from the extension to a related chrome app, so I just had the idea that I could also send the data to my server, which could then forward the data to user's chrome app if they have a current subscription. But yikes.
The more I think about it, the less I think it's possible for me to prevent, but I figured I'd ask in case anyone has a clever idea.
I think you are slightly confused about what counts as SaaS. Wikipedia:
Software as a service is a software licensing and delivery model in which software is licensed on a subscription basis and is centrally hosted. SaaS is typically accessed by users using a thin client via a web browser.
Emphasis mine.
If your app / extension contains all the logic required, it does not qualify as SaaS. Futhermore, as it is always possible to copy/dissect your app, taking out all license checks, you can't protect it against a determined attacker.
There are ways to protect your code to some degree, via obfuscation, offloading logic to (P)NaCl modules, native host modules, or, as Alex Belozerov suggested, load the code on runtime. Again, all of that can be broken by a determined attacker.
But if you truly have SaaS in mind (and not just subscription-based licensing), your client app should be a thin client: that is, your app logic should be processed on a server, with code safely away from clients. That is the only "sure" way to protect it, but incurs processing costs to you, but that's what subscription is supposed to cover in the first place.
You can get part of code needed from server side. So if user's subscription is over, he won't be apple to run your feature as part of code is missing. Concept of my idea:
var subscriptionStatusResponse = makeAjaxCall();
if(subscriptionStatusResponse.usersSubscriptionIsCurrent) {
runFeature_localCode(); // only part of functional
subscriptionStatusResponse.remoteCode(); // second part
}
Maybe the best solution is to check if their subscription is current as soon as the extension starts, and then use the chrome management API to uninstall or disable it if their subscription is over.
I'd love to hear better ideas though.