Related
It's a hot debate on how to securely handle API Keys.
And almost all of us know that it's the best solution to have them stored on the server side, and never exposed in client-side applications.
Our clients can send requests to our APIs, and our APIs can act as proxy to send/receive data to/from the third party API and return the response back to our client.
However, there are some third party SDKs that you can integrate into your client-side app and they also have their API Keys.
For example, Zoom has SDKs for Web, Android, iOS, Windows, etc., or Pusher has Pusher Key.
When you want to work with these libraries, you CAN NOT send request to your API to hide API Key. You have to initialize these libraries in your client-side code (react for example).
An example from Zoom to join a meeting inside your web app:
client.join({
apiKey: apiKey,
signature: signature,
meetingNumber: meetingNumber,
password: password,
userName: userName
})
What are the best practices to secure API Keys for client-side SDKs and libraries?
Your Problem
When you want to work with these libraries, you CAN NOT send request to your API to hide API Key. You have to initialize these libraries in your client-side code (react for example).
What are the best practices to secure API Keys for client-side SDKs and libraries?
Well you found yourself a very hard problem to solve (but not impossible to some degree), because once the API Key is in the client side it's public. So, no matter how well you hide it will always be possible to retrieve it on a browser or mobile app.
Web Apps
On browsers is very trivial to get hands on the API key, just open the developer tools and in the network tab look for the request you are interested in extracting the API key and click on it to inspect the request headers.
Mobile Apps
In mobile devices id more laborious to extract an API key from a mobile app, but not that difficult has many may think.
JNI/NDK - Hide API Key in Native C Code
For example, you can hide the the API key in C native code via JNI/NDK:
Using Android Studio 2.2 and higher, you can use the NDK to compile C and C++ code into a native library and package it into your APK using Gradle, the IDE's integrated build system. Your Java code can then call functions in your native library through the Java Native Interface (JNI) framework.
This approach aims to protect the AP/i key from being extracted from your mobile app binary via static binary analysis, as exemplified in this repo and blog post I wrote:
During this article we will use the Android Hide Secrets research repository that is a dummy mobile app with API keys hidden using several different techniques.
Extract the API Key hidden in Native C Code with a MitM Attack
In the above blog post the API key hidden in the source code with JNI/NDK interface was not possible to extract via static binary analysis, but it was easy to extract with a MitM attack as I demo in the article Steal that Api Key with a Man in the Middle Attack:
In order to help to demonstrate how to steal an API key, I have built and released in Github the Currency Converter Demo app for Android, which uses the same JNI/NDK technique we used in the earlier Android Hide Secrets app to hide the API key.
So, in this article you will learn how to setup and run a MitM attack to intercept https traffic in a mobile device under your control, so that you can steal the API key. Finally, you will see at a high level how MitM attacks can be mitigated.
Prevent MitM Attack with Certificate Pinning
The first thing one can do to prevent a MitM attack is to use certificate pinning and I wrote about how to do it in the article Securing HTTPS with Certificate Pinning:
In order to demonstrate how to use certificate pinning for protecting the https traffic between your mobile app and your API server, we will use the same Currency Converter Demo mobile app that I used in the previous article.
In this article we will learn what certificate pinning is, when to use it, how to implement it in an Android app, and how it can prevent a MitM attack.
I see the smile on your face now, but will not be for long because certificate pinning can be bypassed.
Bypassing Certificate Pinning
You can do it repackaging the mobile app without pinning or by using an instrumentation framework at runtime to disable it.
Repackaging the mobile app to bypass pinning
This is not hard to achieve when you have the correct tools and open source is full of them. I wrote how to do it in the article Bypassing Certificate Pinning
In this article you will learn how to repackage a mobile app in order to make it trust custom ssl certificates. This will allow us to bypass certificate pinning.
Using an Instrumentation Framework to bypass pinning
This is my preferred method and my instrumentation framework of preference is Frida, and guess what, I also have an article on it with the title How to Bypass Certificate Pinning with Frida on an Android App to show you how to do it:
Today I will show how to use the Frida instrumentation framework to hook into the mobile app at runtime and instrument the code in order to perform a successful MitM attack even when the mobile app has implemented certificate pinning.
Bypassing certificate pinning is not too hard, just a little laborious, and allows an attacker to understand in detail how a mobile app communicates with its API, and then use that same knowledge to automate attacks or build other services around it.
Possible Solutions
You may employ an array of different approaches and techniques to defend your API server and mobile app, but give preference to use a security solution that spans both the mobile/web app and API server.
The solution(s) to use will depend on your threat model, your budget and your resources and I will give you below pointers to some options.
For Mobile Apps
I recommend you to read this answer I gave to the question How to secure an API REST for mobile app?, especially the sections Hardening and Shielding the Mobile App, Securing the API Server and A Possible Better Solution.
For Web Apps
You can learn some useful techniques to help your API backend to try to respond only to requests coming from what you expect, your genuine web app, and to do so I invite you to read my answer to the question Secure api data from calls out of the app, especially the section dedicated to Defending the API Server.
Do You Want To Go The Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
For Web Apps
The Web Security Testing Guide:
The OWASP Web Security Testing Guide includes a "best practice" penetration testing framework which users can implement in their own organizations and a "low level" penetration testing guide that describes techniques for testing most common web application and web service security issues.
What do you want to secure exactly? I assume you want to avoid someone from misusing your API key, but you should ask yourself, from what security risk or threat do you want to protect?
You should also understand what kind of API key are you dealing with, as not all of them are intended for the same use-case and could 'offer' different level of security.
For example, you could have an API key in the form of a personal access token (in GitHub, for instance), where the token is directly tied to a user/employee and should be considered a secret, or you may have an API key in the form of a machine token that it's tied to your organization or your repository (still in GitHub, for instance) and that can be configured with different permissions (read-only, read-write).
It is also possible that the API key can be configured and restricted to some clients, according to the possibility of the client itself.
For example some of the Google Maps APIs allow you to configure a trusted origin that it is allowed to perform requests using your specific API key, however this protection works by checking the referrer header of the request and it could be spoofed by arbitrary clients. Browsers should still honor the convention and send the correct referrer, protecting you from people that want to use your API key on their website.
Another example in the mobile application world: there are some vendors that allow you to bind your API key to a specific package name that it is then validated at runtime by the vendor's SDK, however this kind of protection is usually as a licensing mechanism, to avoid developers to configure for free an SDK with a leaked API key.
Most generally, if the API key is intended to be used on public clients, then the developers of the API already considered the threat of having this leaked and you should not have a repercussion. This means that you will be covered from huge API usage billings or from rate-limit/usage quota limit (but better check yourself!).
The general rule is to always check the developer's documentation of the application you're trying to configure and see how to create a proper API key for your use-case and if it's fine for you to have this 'leaked' from your client. Additionally, if the API key allows you to configure permission, remember to follow the Principle of Least Privilege.
Another golden rule is to always threat model your implementation:
What are the capabilities of the API key?
What is the worst an attacker can do if they access the API key?
Am I protecting myself from these threat? How? Can I put more controls or monitors on back-end side (i.e. notification on high usage, etc)?
Finally, if your API key need to be kept secret, then you must not use this on a public client, no matter how hidden it is. There will always be at least one person able to retrieve it (And don't rely on client-side check either!). In this case what you want is probably to have your own back-end service responsible both for querying the APIs using the secret API key and for authenticating and authorizing your customers/users, and also to implement additional security measures like rate limit.
One thing I found very helpful is to always document any generated/used API key and its capabilities along with the threats of having them leaked and some preventive measures to minimize the risk.
Thanks for your help in advance.
I'm using React Native and Node.js to deliver a product for my company.
I've setup the steps on the backend to retrieve a password, validate it and respond with a token. The only problem is - the password I use on the front end (mobile app) to be validated by the back end is hardcoded.
My question is:
How should I securely store this password on the mobile app so that it can not be sniffed out by a hacker and used to compromise the backend?
My research so far.
Embedded in strings.xml
Hidden in Source Code
Hidden in BuildConfigs
Using Proguard
Disguised/Encrypted Strings
Hidden in Native Libraries
http://rammic.github.io/2015/07/28/hiding-secrets-in-android-apps/
These methods are basically useless because hackers can easily circumnavigate these methods of protection.
https://github.com/oblador/react-native-keychain
Although this may obfuscate keys, these still have to be hardcoded. Making these kind of useless, unless I'm missing something.
I could use a .env file
https://github.com/luggit/react-native-config
Again, I feel like the hacker can still view secret keys, even if they are saved in a .env
I want to be able to store keys in the app so that I can validate the user an allow them to access resources on the backend. However, I don't know what the best plan of action is to ensure user/business security.
What suggestions do you have to protect the world (react- native apps) from pesky hackers, when they're stealing keys and using them inappropriately?
Your Question
I've setup the steps on the backend to retrieve a password, validate it and respond with a token. The only problem is - the password I use on the front end (mobile app) to be validated by the back end is hardcoded.
My question is:
How should I securely store this password on the mobile app so that it can not be sniffed out by a hacker and used to compromise the backend?
The cruel truth is... you can't!!!
It seems that you already have done some extensive research on the subject, and in my opinion you mentioned one effective way of shipping your App with an embedded secret:
Hidden in Native Libraries
But as you also say:
These methods are basically useless because hackers can easily circumnavigate these methods of protection.
Some are useless and others make reverse engineer the secret from the mobile app a lot harder. As I wrote here, the approach of using the native interfaces to hide the secret will require expertise to reverse engineer it, but then if is hard to reverse engineer the binary you can always resort to a man in the middle (MitM) attack to steel the secret, as I show here for retrieving a secret that is hidden in the mobile app binary with the use of the native interfaces, JNI/NDK.
To protect your mobile app from a MitM you can employ Certificate Pinning:
Pinning is the process of associating a host with their expected X509 certificate or public key. Once a certificate or public key is known or seen for a host, the certificate or public key is associated or 'pinned' to the host. If more than one certificate or public key is acceptable, then the program holds a pinset (taking from Jon Larimer and Kenny Root Google I/O talk). In this case, the advertised identity must match one of the elements in the pinset.
You can read this series of react native articles that show you how to apply certificate pinning to protect the communication channel between your mobile app and the API server.
If you don't know yet certificcate pinning can also be bypassed by using tools like Frida or xPosed.
Frida
Inject your own scripts into black box processes. Hook any function, spy on crypto APIs or trace private application code, no source code needed. Edit, hit save, and instantly see the results. All without compilation steps or program restarts.
xPosed
Xposed is a framework for modules that can change the behavior of the system and apps without touching any APKs. That's great because it means that modules can work for different versions and even ROMs without any changes (as long as the original code was not changed too much). It's also easy to undo.
So now you may be wondering how can I protect from certificate pinning bypass?
Well is not easy, but is possible, by using a mobile app attestation solution.
Before we go further on it, I would like to clarify first a common misconception among developers, regarding WHO and WHAT is accessing the API server.
The Difference Between WHO and WHAT is Accessing the API Server
To better understand the differences between the WHO and the WHAT are accessing an API server, let’s use this picture:
The Intended Communication Channel represents the mobile app being used as you expected, by a legit user without any malicious intentions, using an untampered version of the mobile app, and communicating directly with the API server without being man in the middle attacked.
The actual channel may represent several different scenarios, like a legit user with malicious intentions that may be using a repackaged version of the mobile app, a hacker using the genuine version of the mobile app, while man in the middle attacking it, to understand how the communication between the mobile app and the API server is being done in order to be able to automate attacks against your API. Many other scenarios are possible, but we will not enumerate each one here.
I hope that by now you may already have a clue why the WHO and the WHAT are not the same, but if not it will become clear in a moment.
The WHO is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
OAUTH
Generally, OAuth provides to clients a "secure delegated access" to server resources on behalf of a resource owner. It specifies a process for resource owners to authorize third-party access to their server resources without sharing their credentials. Designed specifically to work with Hypertext Transfer Protocol (HTTP), OAuth essentially allows access tokens to be issued to third-party clients by an authorization server, with the approval of the resource owner. The third party then uses the access token to access the protected resources hosted by the resource server.
OpenID Connect
OpenID Connect 1.0 is a simple identity layer on top of the OAuth 2.0 protocol. It allows Clients to verify the identity of the End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.
While user authentication may let the API server know WHO is using the API, it cannot guarantee that the requests have originated from WHAT you expect, the original version of the mobile app.
Now we need a way to identify WHAT is calling the API server, and here things become more tricky than most developers may think. The WHAT is the thing making the request to the API server. Is it really a genuine instance of the mobile app, or is a bot, an automated script or an attacker manually poking around with the API server, using a tool like Postman?
For your surprise you may end up discovering that It can be one of the legit users using a repackaged version of the mobile app or an automated script that is trying to gamify and take advantage of the service provided by the application.
Well, to identify the WHAT, developers tend to resort to an API key that usually they hard-code in the code of their mobile app. Some developers go the extra mile and compute the key at run-time in the mobile app, thus it becomes a runtime secret as opposed to the former approach when a static secret is embedded in the code.
The above write-up was extracted from an article I wrote, entitled WHY DOES YOUR MOBILE APP NEED AN API KEY?, and that you can read in full here, that is the first article in a series of articles about API keys.
Mobile App Attestation
The use of a Mobile App Attestation solution will enable the API server to know WHAT is sending the requests, thus allowing to respond only to requests from a genuine mobile app while rejecting all other requests from unsafe sources.
The role of a Mobile App Attestation service is to guarantee at run-time that your mobile app was not tampered, is not running in a rooted device and is not being the target of a MitM attack. This is done by running a SDK in the background that will communicate with a service running in the cloud to attest the integrity of the mobile app and device is running on. The cloud service also verifies that the TLS certificate provided to the mobile app on the handshake with the API server is indeed the same in use by the original and genuine API server for the mobile app, not one from a MitM attack.
On successful attestation of the mobile app integrity a short time lived JWT token is issued and signed with a secret that only the API server and the Mobile App Attestation service in the cloud are aware. In the case of failure on the mobile app attestation the JWT token is signed with a secret that the API server does not know.
Now the App must sent with every API call the JWT token in the headers of the request. This will allow the API server to only serve requests when it can verify the signature and expiration time in the JWT token and refuse them when it fails the verification.
Once the secret used by the Mobile App Attestation service is not known by the mobile app, is not possible to reverse engineer it at run-time even when the App is tampered, running in a rooted device or communicating over a connection that is being the target of a Man in the Middle Attack.
So this solution works in a positive detection model without false positives, thus not blocking legit users while keeping the bad guys at bays.
What suggestions do you have to protect the world (react- native apps) from pesky hackers, when they're stealing keys and using them inappropriately?
I think you should relaly go with a mobile app attestation solution, that you can roll in your own if you have the expertise for it, or you can use a solution that already exists as a SAAS solution at Approov(I work here), that provides SDKs for several platforms, including iOS, Android, React Native and others. The integration will also need a small check in the API server code to verify the JWT token issued by the cloud service. This check is necessary for the API server to be able to decide what requests to serve and what ones to deny.
Summary
I want to be able to store keys in the app so that I can validate the user an allow them to access resources on the backend. However, I don't know what the best plan of action is to ensure user/business security.
Don't go down this route of storing keys in the mobile app, because as you already know, by your extensive research, they can be bypassed.
Instead use a mobile attestation solution in conjunction with OAUTH2 or OpenID connect, that you can bind with the mobile app attestation token. An example of this token binding can be found in this article for the check of the custom payload claim in the endpoint /forms.
Going the Extra Mile
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
I've set up an API with authentication but I want to only allow certain applications and websites to access it. What do I do?
I've got authentication set up for users that are Logged in only being able to access the API, however, how do I prevent them from just logging in from anywhere?
Before I address your question, I think is important that first we clear a common misconception among developers, regarding WHO and WHAT is accessing an API.
THE DIFFERENCE BETWEEN WHO AND WHAT IS COMMUNICATING WITH YOUR API SERVER
To better understand the differences between the WHO and the WHAT are accessing your mobile app, let’s use this picture:
The Intended Communication Channel represents your mobile being used as you expected, by a legit user without any malicious intentions, using an untampered version of your mobile app, and communicating directly with your API server without being man in the middle attacked.
The actual channel may represent several different scenarios, like a legit user with malicious intentions that may be using a repackaged version of your mobile app, a hacker using the genuine version of you mobile app while man in the middle attacking it to understand how the communication between the mobile app and the API server is being done in order to be able to automate attacks against your API. Many other scenarios are possible, but we will not enumerate each one here.
I hope that by now you may already have a clue why the WHO and the WHAT are not the same, but if not it will become clear in a moment.
The WHO is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
OAUTH
Generally, OAuth provides to clients a "secure delegated access" to server resources on behalf of a resource owner. It specifies a process for resource owners to authorize third-party access to their server resources without sharing their credentials. Designed specifically to work with Hypertext Transfer Protocol (HTTP), OAuth essentially allows access tokens to be issued to third-party clients by an authorization server, with the approval of the resource owner. The third party then uses the access token to access the protected resources hosted by the resource server.
OpenID Connect
OpenID Connect 1.0 is a simple identity layer on top of the OAuth 2.0 protocol. It allows Clients to verify the identity of the End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.
While user authentication may let your API server know WHO is using the API, it cannot guarantee that the requests have originated from WHAT you expect, your mobile app.
Now we need a way to identify WHAT is calling your API server, and here things become more tricky than most developers may think. The WHAT is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
For your surprise you may end up discovering that It can be one of your legit users using a repackaged version of your mobile app or an automated script trying to gamify and take advantage of your service.
Well, to identify the WHAT, developers tend to resort to an API key that usually they hard-code in the code of their mobile app. Some developers go the extra mile and compute the key at run-time in the mobile app, thus it becomes a runtime secret as opposed to the former approach when a static secret is embedded in the code.
The above write-up was extracted from an article I wrote, entitled WHY DOES YOUR MOBILE APP NEED AN API KEY?, and that you can read in full here, that is the first article in a series of articles about API keys.
YOUR QUESTIONS
I've got authentication set up for users that are Logged in only being able to access the API, however, how do I prevent them from just logging in from anywhere?
If by logging in from anywhere you mean any physical location, then you can use blocking by IP address as already suggested by #hanshenrik, but if you mean blocking from logging from other applications, that are not the ones you have issued the API keys for, then you have a very hard problem in your hands to solve, that leads to your first question:
I've set up an API with authentication but I want to only allow certain applications and websites to access it. What do I do?
This will depend if WHAT is accessing the API is a web or a mobile application.
Web application
In a web app we only need to inspect the source code with the browser dev tools or by right click on view page source and search for the API key, and then use it in any tool, like Postman or in any kind of automation we want, just by replicating the calls as we saw them being made in the network tab of the browser.
For an API serving a web app you can employ several layers of dense, starting with reCaptcha V3, followed by Web Application Firewall(WAF) and finally if you can afford it a User Behavior Analytics(UBA) solution.
Google reCAPTCHA V3:
reCAPTCHA is a free service that protects your website from spam and abuse. reCAPTCHA uses an advanced risk analysis engine and adaptive challenges to keep automated software from engaging in abusive activities on your site. It does this while letting your valid users pass through with ease.
...helps you detect abusive traffic on your website without any user friction. It returns a score based on the interactions with your website and provides you more flexibility to take appropriate actions.
WAF - Web Application Firewall:
A web application firewall (or WAF) filters, monitors, and blocks HTTP traffic to and from a web application. A WAF is differentiated from a regular firewall in that a WAF is able to filter the content of specific web applications while regular firewalls serve as a safety gate between servers. By inspecting HTTP traffic, it can prevent attacks stemming from web application security flaws, such as SQL injection, cross-site scripting (XSS), file inclusion, and security misconfigurations.
UBA - User Behavior Analytics:
User behavior analytics (UBA) as defined by Gartner is a cybersecurity process about detection of insider threats, targeted attacks, and financial fraud. UBA solutions look at patterns of human behavior, and then apply algorithms and statistical analysis to detect meaningful anomalies from those patterns—anomalies that indicate potential threats. Instead of tracking devices or security events, UBA tracks a system's users. Big data platforms like Apache Hadoop are increasing UBA functionality by allowing them to analyze petabytes worth of data to detect insider threats and advanced persistent threats.
All this solutions work based on a negative identification model, by other words they try their best to differentiate the bad from the good by identifying WHAT is bad, not WHAT is good, thus they are prone to false positives, despite of the advanced technology used by some of them, like machine learning and artificial intelligence.
So you may find yourself more often than not in having to relax how you block the access to the API server in order to not affect the good users. This also means that this solutions require constant monitoring to validate that the false positives are not blocking your legit users and that at same time they are properly keeping at bay the unauthorized ones.
Mobile Application
From your reply to a comment:
What about for mobile applications?
Some may think that once a mobile app is released in a binary format that their API key will be safe, but turns out that is not true, and extracting it from a binary is sometimes almost as easy as extracting it from a web application.
Reverse engineering a mobile app is made easy by plethora of open source tools, like the Mobile Security Framework(MobSF), Frida, XPosed, MitmProxy, and many other more, but as you can see in this article, it can be done with MobSF or with the strings utility that is installed in a normal Linux distribution.
Mobile Security Framework
Mobile Security Framework is an automated, all-in-one mobile application (Android/iOS/Windows) pen-testing framework capable of performing static analysis, dynamic analysis, malware analysis and web API testing.
Frida
Inject your own scripts into black box processes. Hook any function, spy on crypto APIs or trace private application code, no source code needed. Edit, hit save, and instantly see the results. All without compilation steps or program restarts.
xPosed
Xposed is a framework for modules that can change the behavior of the system and apps without touching any APKs. That's great because it means that modules can work for different versions and even ROMs without any changes (as long as the original code was not changed too much). It's also easy to undo.
MiTM Proxy
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
Regarding APIs serving mobile apps a positive identification model can be used by using a Mobile App Attestation solution that guarantees to the API server that WHAT is making the requests can be trusted, without the possibility of false positives.
The Mobile App Attestation
The role of a Mobile App Attestation service is to guarantee at run-time that your mobile app was not tampered or is not running in a rooted device by running a SDK in the background that will communicate with a service running in the cloud to attest the integrity of the mobile app and device is running on.
On successful attestation of the mobile app integrity a short time lived JWT token is issued and signed with a secret that only the API server and the Mobile App Attestation service in the cloud are aware. In the case of failure on the mobile app attestation the JWT token is signed with a secret that the API server does not know.
Now the App must sent with every API call the JWT token in the headers of the request. This will allow the API server to only serve requests when it can verify the signature and expiration time in the JWT token and refuse them when it fails the verification.
Once the secret used by the Mobile App Attestation service is not known by the mobile app, is not possible to reverse engineer it at run-time even when the App is tampered, running in a rooted device or communicating over a connection that is being the target of a Man in the Middle Attack.
The Mobile App Attestation service already exists as a SAAS solution at Approov(I work here) that provides SDKs for several platforms, including iOS, Android, React Native and others. The integration will also need a small check in the API server code to verify the JWT token issued by the cloud service. This check is necessary for the API server to be able to decide what requests to serve and what ones to deny.
CONCLUSION
In the end the solution to use in order to protect your API server must be chosen in accordance with the value of what you are trying to protect and the legal requirements for that type of data, like the GDPR regulations in Europe.
So using API keys may sound like locking the door of your home and leave the key under the mat, but not using them is liking leaving your car parked with the door closed, but the key in the ignition.
Say I have an API where users can log in through OAuth2. What are my options for allowing only trusted applications to interact with parts of this API? For example, I would like to have a mobile app and a web app, but I don't want anyone else developing apps that would interact with this API.
In other words, how do Facebook, Twitter and Instagram keep people from just cloning the client id from their mobile apps and using their entire APIs as if they were the official app?
WHO AND WHAT IS ACCESSING THE API SERVER
Before I address your question I would like to make clear the distinction between WHO and WHAT is accessing the API server.
The WHO is the user of the mobile app that you can authenticate,authorize and identify in several ways, like using OpenID or OAUTH2 flows.
Now you need a way to identify WHAT is calling your API server and here things become more tricky than most developers may think. The WHAT is the thing making the request to the API server, is it really your genuine mobile app or is a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
Well to identify the WHAT developers tend to resort to an API key that usually they hard-code in the code of their mobile app and some go the extra mile and compute it at run-time in the mobile app, thus becomes a dynamic secret in opposition to the former approach that is a static secret embedded in the code.
THE PROBLEM
Say I have an API where users can log in through OAuth2. What are my options for allowing only trusted applications to interact with parts of this API?
You found yourself a very hard problem to solve...
If your API only serves mobile apps you can keep your wording, but from the moment you create an API that needs to server web apps you cannot apply anymore the wording allowing only trusted applications and you will have to satisfy yourself with something around preventing access of unauthorized applications.
The problem with web apps is that you only need to inspect the web page source code with the browser dev tools to be able to extract any secret used to identify the web app to the API server.
With mobile app some developers have the misconception that once they are released as binary that is not possible to extract secrets or is very hard... Well lets see how the strings command in Linux can help us:
$ strings -aw app-debug.apk | grep -C 1 '_API_' -
ic_launcher_round
GRADLE_API_KEY
GRADLE_ENV_API_KEY
abc_action_bar_home_description
You can prevent the above by obfuscating the code when you create the release, but then we just need to use some more sophisticated tooling like the Mobile Security Framework(MobSF) that uses under the hood a set of other open source tools to de-compile the binary and perform static analysis on it to enumerate all attack vectors and expose secrets embed on the code.
Mobile Security Framework
Mobile Security Framework is an automated, all-in-one mobile application (Android/iOS/Windows) pen-testing framework capable of performing static analysis, dynamic analysis, malware analysis and web API testing.
So anything that runs on the client side and needs some secret to access an API can be abused in different ways and you can learn more on this series of articles about Mobile API Security Techniques. This articles will teach you how API Keys, User Access Tokens, HMAC and TLS Pinning can be used to protect the API and how they can be bypassed.
PREVENT API ABUSE
In other words, how do Facebook, Twitter and Instagram keep people from just cloning the client id from their mobile apps and using their entire APIs as if they were the official app?
I don't know what they use specifically to prevent API abuse with the cloned identifiers, but anyone that wants to protect their APIs from being abused by non official clients will use Web Application Firewalls(WAF) and User Behavior Analytics(UBA) solutions that employ machine learning and artificial intelligence to detect bad behaviors and block their access to the API.
WAF - Web Application Firewall:
A web application firewall (or WAF) filters, monitors, and blocks HTTP traffic to and from a web application. A WAF is differentiated from a regular firewall in that a WAF is able to filter the content of specific web applications while regular firewalls serve as a safety gate between servers. By inspecting HTTP traffic, it can prevent attacks stemming from web application security flaws, such as SQL injection, cross-site scripting (XSS), file inclusion, and security misconfigurations.
UBA - User Behavior Analytics:
User behavior analytics (UBA) as defined by Gartner is a cybersecurity process about detection of insider threats, targeted attacks, and financial fraud. UBA solutions look at patterns of human behavior, and then apply algorithms and statistical analysis to detect meaningful anomalies from those patterns—anomalies that indicate potential threats. Instead of tracking devices or security events, UBA tracks a system's users. Big data platforms like Apache Hadoop are increasing UBA functionality by allowing them to analyze petabytes worth of data to detect insider threats and advanced persistent threats.
The problem with this approach is that is based on a negative detection model that tries to identify the bad guys based on patterns and it tends to have false positives, that leads to relax the blocking policies to avoid leaving out the legit users, meaning that some bad guys will always find their way through.
A POSSIBLE SOLUTION
For example, I would like to have a mobile app and a web app, but I don't want anyone else developing apps that would interact with this API.
For a Web App
For your web app I would use reCAPTCHA V3 across all pages of the website in order to let Google to distinguish humans from bots. This detection is done in the background without requiring interaction with the user.
Google reCAPTCHA V3:
reCAPTCHA is a free service that protects your website from spam and abuse. reCAPTCHA uses an advanced risk analysis engine and adaptive challenges to keep automated software from engaging in abusive activities on your site. It does this while letting your valid users pass through with ease.
...helps you detect abusive traffic on your website without any user friction. It returns a score based on the interactions with your website and provides you more flexibility to take appropriate actions.
This on its own will not be enough and will also have false positives, once is a negative detection model, but is a good start to leave at bay the script kids.
For the most determinate attackers you will need to employ the already mention WAFs and UBAs solutions, and this ones will be more complex to deploy and maintain, but once more they work in a best effort basis and will not be able to eliminate the API abuse.
For Mobile Apps
Limit and Hide Secrets
The first step is to limit secrets in your mobile to only one, the one used to access your API server and all other third part services you need to access from your mobile app you should delegate to the API server, thus not exposing in the mobile app the keys to access your third part services.
You can see this repo for a dummy Android APP that I created for my next blog post(not published yet), that shows the several techniques to hide secrets. Spoiler alert, the best approach in Android is this one:
#include <jni.h>
#include <string>
#include "api_key.h"
extern "C" JNIEXPORT jstring JNICALL
Java_com_criticalblue_androidhidesecrets_MainActivity_stringFromJNI(
JNIEnv *env,
jobject /* this */) {
// To add the API_KEY to the mobile app when is compiled you need to:
// * copy `api_key.h.example` to `api_key.h`
// * edit the file and replace this text `place-the-api-key-here` with your desired API_KEY
std::string JNI_API_KEY = ANDROID_HIDE_SECRETS_API_KEY_H;
return env->NewStringUTF(JNI_API_KEY.c_str());
}
Bear in mind that the above technique is good only to make very hard to extract the secret with static analysis of the binary.
The easiest way to extract secrets used by a mobile app is for an attacker to mount a man in the middle attack to the mobile app in a device he controls, with a tool like:
MiTM Proxy
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
With this tool the attacker will add a custom ssl certificate to the device and mobile app in order he can intercept and read all https traffic, thus being able to understand how the communication happens between the mobile app and the API server in order to mount an automated attack.
Code Obfuscation
When building the release binary always obfuscate the code, for example Android Studio
has built-in support for it or alternatively you can use commercial tooling to achieve even better results.
Certificate Pinning
When pinning the connection between the mobile app and the API server we prevent the man in the middle attacks from occurring, because the mobile app will refuse connections that are not using the certificate from the API server.
Certificate Pinning
Pinning is the process of associating a host with their expected X509 certificate or public key. Once a certificate or public key is known or seen for a host, the certificate or public key is associated or 'pinned' to the host. If more than one certificate or public key is acceptable, then the program holds a pinset (taking from Jon Larimer and Kenny Root Google I/O talk). In this case, the advertised identity must match one of the elements in the pinset.
Mobile App Attestation
The use of a Mobile App Attestation solution will enable the API server to know WHAT is sending the requests, thus allowing to respond only to requests from a genuine mobile app while rejecting all other requests from unsafe sources.
The role of a Mobile App Attestation service is to guarantee at run-time that your mobile app was not tampered or is not running in a rooted device by running a SDK in the background that will communicate with a service running in the cloud to attest the integrity of the mobile app and device is running on.
On successful attestation of the mobile app integrity a short time lived JWT token is issued and signed with a secret that only the API server and the Mobile App Attestation service in the cloud are aware. In the case of failure on the mobile app attestation the JWT token is signed with a secret that the API server does not know.
Now the App must sent with every API call the JWT token in the headers of the request. This will allow the API server to only serve requests when it can verify the signature and expiration time in the JWT token and refuse them when it fails the verification.
Once the secret used by the Mobile App Attestation service is not known by the mobile app, is not possible to reverse engineer it at run-time even when the App is tampered, running in a rooted device or communicating over a connection that is being the target of a Man in the Middle Attack.
So this solution works in a positive detection model without false positives, thus not blocking legit users while keeping the bad guys at bays.
The Mobile App Attestation service already exists as a SAAS solution at Approov(I work here) that provides SDKs for several platforms, including iOS, Android, React Native and others. The integration will also need a small check in the API server code to verify the JWT token issued by the cloud service. This check is necessary for the API server to be able to decide what requests to serve and what ones to deny.
CONCLUSION
In the end of the day is all about how much valuable is the data you have, what are the impacts for the business of a data breach, and what are the regulations you need to comply with for that data, like GDPR in Europe.
So based on this assessment you need to decide how many layers of defense you want to put around your API serving web and mobile apps.
I'm building an (amateur) application that uses the Twitter API, which supports authentication via the OAuth protocol.
Part of the OAuth sign-in process involves each application being assigned a Consumer Key and Consumer Secret (both strings), which are used to generate signatures for communication with the Twitter server.
The Twitter dev guide explicitly states that one should 'Keep the "Consumer secret" a secret. This key should never be human-readable in your application.'. This is obviously important, as if a malicious individual obtains your credentials, they can impersonate your app.
However, I do not see how this can be achieved. In order for the application to use the string, it must be accessible to the app somehow (either directly coded into the app, stored in a bundled database, or accessible via a linked web service) - and if it's accessible to the app, it must be accessible to the user. It can be obfuscated by splitting, character-shifting, etc., but not (as far as I can see) in any way that can't be undone.
This SO answer confirms my suspicions that this is a problem - I was wondering if there had been any progress since it was posted in December '09?
The problem with mobile devices is that is in the users hands. And with enough time/effort the user can pull any data out of the device. It isn't an OAuth security problem, it is an overall security problem that there really isn't an answer for.