I apologize for the question but I don't have the resources to figure it out myself.
I'm looking for features my next iOS / android app should have and as you can imagine, I'm interested on a "pinch of social" that's why getsream seems to be my saver.
After reading the getting started section and the documentation, I found this warning http://getstream.io/docs/#mobile that confuses me.
I supposed getstream is a managed service that takes care of everything letting me use the REST API to build my mobile community within their phones.
Could you please tell me where I'm wrong?
Many thanks
There are two main reasons we do not recommend integrating Getstream client side (i.e. in the browser or on mobile). First, it is hard to guarantee security when you integrate from the client-side, you have to somehow provision tokens for each user's feed (and feeds they want to target with activities http://getstream.io/docs/#targetting), you could also generate an application wide (read/write) token and ship this to all clients but this is also a bad idea for obvious reasons. Second, we do not recommend using Getstream to store all your activity data, you store references to objects in your local database and enrich the activities from getstream at read time (have a look at our integration libraries for Django/Rails).
Related
Is there any way to restrict post requests to my REST API only to requests coming from my own mobile app binary? This app will be distributed on Google Play and the Apple App Store so it should be implied that someone will have access to its binary and try to reverse engineer it.
I was thinking something involving the app signatures, since every published app must be signed somehow, but I can't figure out how to do it in a secure way. Maybe a combination of getting the app signature, plus time-based hashes, plus app-generated key pairs and the good old security though obscurity?
I'm looking for something as fail proof as possible. The reason why is because I need to deliver data to the app based on data gathered by the phone sensors, and if people can pose as my own app and send data to my api that wasn't processed by my own algorithms, it defeats its purpose.
I'm open to any effective solution, no matter how complicated. Tin foil hat solutions are greatly appreciated.
Any credentials that are stored in the app can be exposed by the user. In the case of Android, they can completely decompile your app and easily retrieve them.
If the connection to the server does not utilize SSL, they can be easily sniffed off the network.
Seriously, anybody who wants the credentials will get them, so don't worry about concealing them. In essence, you have a public API.
There are some pitfalls and it takes extra time to manage a public API.
Many public APIs still track by IP address and implement tarpits to simply slow down requests from any IP address that seems to be abusing the system. This way, legitimate users from the same IP address can still carry on, albeit slower.
You have to be willing to shut off an IP address or IP address range despite the fact that you may be blocking innocent and upstanding users at the same time as the abusers. If your application is free, it may give you more freedom since there is no expected level of service and no contract, but you may want to guard yourself with a legal agreement.
In general, if your service is popular enough that someone wants to attack it, that's usually a good sign, so don't worry about it too much early on, but do stay ahead of it. You don't want the reason for your app's failure to be because users got tired of waiting on a slow server.
Your other option is to have the users register, so you can block by credentials rather than IP address when you spot abuse.
Yes, It's public
This app will be distributed on Google Play and the Apple App Store so it should be implied that someone will have access to its binary and try to reverse engineer it.
From the moment its on the stores it's public, therefore anything sensitive on the app binary must be considered as potentially compromised.
The Difference Between WHO and WHAT is Accessing the API Server
Before I dive into your problem I would like to first clear a misconception about who and what is accessing an API server. I wrote a series of articles around API and Mobile security, and in the article Why Does Your Mobile App Need An Api Key? you can read in detail the difference between who and what is accessing your API server, but I will extract here the main takes from it:
The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
Think about the who as the user your API server will be able to Authenticate and Authorize access to the data, and think about the what as the software making that request in behalf of the user.
So if you are not using user authentication in the app, then you are left with trying to attest what is doing the request.
Mobile Apps should be as much dumb as possible
The reason why is because I need to deliver data to the app based on data gathered by the phone sensors, and if people can pose as my own app and send data to my api that wasn't processed by my own algorithms, it defeats its purpose.
It sounds to me that you are saying that you have algorithms running on the phone to process data from the device sensors and then send them to the API server. If so then you should reconsider this approach and instead just collect the sensor values and send them to the API server and have it running the algorithm.
As I said anything inside your app binary is public, because as yourself said, it can be reverse engineered:
should be implied that someone will have access to its binary and try to reverse engineer it.
Keeping the algorithms in the backend will allow you to not reveal your business logic, and at same time you may reject requests with sensor readings that do not make sense(if is possible to do). This also brings you the benefit of not having to release a new version of the app each time you tweak the algorithm or fix a bug in it.
Runtime attacks
I was thinking something involving the app signatures, since every published app must be signed somehow, but I can't figure out how to do it in a secure way.
Anything you do at runtime to protect the request you are about to send to your API can be reverse engineered with tools like Frida:
Inject your own scripts into black box processes. Hook any function, spy on crypto APIs or trace private application code, no source code needed. Edit, hit save, and instantly see the results. All without compilation steps or program restarts.
Your Suggested Solutions
Security is all about layers of defense, thus you should add as many as you can afford and required by law(e.g GDPR in Europe), therefore any of your purposed solutions are one more layer the attacker needs to bypass, and depending on is skill-set and time is willing to spent on your mobile app it may prevent them to go any further, but in the end all of them can be bypassed.
Maybe a combination of getting the app signature, plus time-based hashes, plus app-generated key pairs and the good old security though obscurity?
Even when you use key pairs stored in the hardware trusted execution environment, all an attacker needs to do is to use an instrumentation framework to hook in the function of your code that uses the keys in order to extract or manipulate the parameters and return values of the function.
Android Hardware-backed Keystore
The availability of a trusted execution environment in a system on a chip (SoC) offers an opportunity for Android devices to provide hardware-backed, strong security services to the Android OS, to platform services, and even to third-party apps.
While it can be defeated I still recommend you to use it, because not all hackers have the skill set or are willing to spend the time on it, and I would recommend you to read this series of articles about Mobile API Security Techniques to learn about some complementary/similar techniques to the ones you described. This articles will teach you how API Keys, User Access Tokens, HMAC and TLS Pinning can be used to protect the API and how they can be bypassed.
Possible Better Solutions
Nowadays I see developers using Android SafetyNet to attest what is doing the request to the API server, but they fail to understand it's not intended to attest that the mobile app is what is doing the request, instead it's intended to attest the integrity of the device, and I go in more detail on my answer to the question Android equivalent of ios devicecheck. So should I use it? Yes you should, because it is one more layer of defense, that in this case tells you that your mobile app is not installed in a rooted device, unless SafetyNet has been bypassed.
Is there any way to restrict post requests to my REST API only to requests coming from my own mobile app binary?
You can allow the API server to have an high degree of confidence that is indeed accepting requests only from your genuine app binary by implementing the Mobile App Attestation concept, and I describe it in more detail on this answer I gave to the question How to secure an API REST for mobile app?, specially the sections Securing the API Server and A Possible Better Solution.
Do you want to go the Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
No. You're publishing a service with a public interface and your app will presumably only communicate via this REST API. Anything that your app can send, anyone else can send also. This means that the only way to secure access would be to authenticate in some way, i.e. keep a secret. However, you are also publishing your apps. This means that any secret in your app is essentially being given out also. You can't have it both ways; you can't expect to both give out your secret and keep it secret.
Though this is an old post, I thought I should share the updates from Google in this regard.
You can actually ensure that your Android application is calling the API using the SafetyNet mobile attestation APIs. This adds a little overhead on the network calls and prevents your application from running in a rooted device.
I found nothing similar like SafetyNet for iOS. Hence in my case, I checked the device configuration first in my login API and took different measures for Android and iOS. In case of iOS, I decided to keep a shared secret key between the server and the application. As the iOS applications are a little bit difficult to reversed engineered, I think this extra key checking adds some protection.
Of course, in both cases, you need to communicate over HTTPS.
As the other answers and comments imply, you cant truly restrict API access to only your app but you can take different measures to reduce the attempts. I believe the best solution is to make requests to your API (from native code of course) with a custom header like "App-Version-Key" (this key will be decided at compile time) and make your server check for this key to decide if it should accept or reject. Also when using this method you SHOULD use HTTPS/SSL as this will reduce the risk of people seeing your key by viewing the request on the network.
Regarding Cordova/Phonegap apps, I will be creating a plugin to do the above mentioned method. I will update this comment when its complete.
there is nothing much you can do. cause when you let some one in they can call your APIs. the most you can do is as below:
since you want only and only your application (with a specific package name and signature) calls your APIs, you can get the signature key of your apk pragmatically and send is to sever in every API call and if thats ok you response to the request. (or you can have a token API that your app calls it every beginning of the app and then use that token for other APIs - though token must be invalidated after some hours of not working with)
then you need to proguard your code so no one sees what you are sending and how you encrypt them. if you do a good encrypt decompiling will be so hard to do.
even signature of apk can be mocked in some hard ways but its the best you can do.
Someone have looked at Firebase App Check ?
https://firebase.google.com/docs/app-check
Is there any way to restrict post requests to my REST API only to requests coming from my own mobile app binary?
I'm not sure if there is an absolute solution.
But, you can reduce unwanted requests.
Use an App Check:
The "Firebase App Check" can be used cross-platform (https://firebase.google.com/docs/app-check) - credit to #Xande-Rasta-Moura
iOS: https://developer.apple.com/documentation/devicecheck
Android: https://android-developers.googleblog.com/2013/01/verifying-back-end-calls-from-android.html
Use BasicAuth (for API requests)
Allow a user-agent header for mobile devices only (for API requests)
Use a robots.txt file to reduce bots
User-agent: *
Disallow: /
These days I am finding myself in the position of having to implement for one of my college courses a system that should act as a giant wrapper over many communications apps like Gmail , Facebook Messenger maybe even WhatsApp .To put it simply you should have a giant web interface where you can authorize Gmail , Messenger and use them at once when required. I am thinking of going with an REST API to manage the user's services authorized by OAuth2.Also I am thinking of using Node.JS and Express.js in the backend and React.js in the frontend. I found some sweet libraries in npm that should take care of interacting with the involved APIs(https://www.npmjs.com/package/node-gmail-api this one for instance), but I am also doubtful about this approach , for example I have no idea how to keep the use notified about its incoming mails or messages for example . I am in dire need of some expertise since I forgot to mention but I am quite the newbie in this field. To sum it up for once my question is how would you implement such an infrastructure ? Is it my approach viable or I am bound to hit some really hard to overcome obstacles?
As a college exercise, it would be a really fun experiment, so it definitely worth the time you want to put into it. However, once you want to add more features, the complexity will go up pretty fast.
Here are a couple of ideas you can think of:
It's pretty clear that your system can't do more things than the capabilities exposed by the APIs of communication apps (e.g. you can't have notifications in gmail if the API doesn't have this capability).
For that reason, you should carefully study the APIs and what functionalities they expose. They have public docs that you can check out: (Gmail API, Facebook Messanger API)
Some of the apps you want to communicate with may not have an official API (e.g. WhatsApp) - those kinds of details you definitely want to know from the start.
Based on the analysis of those APIs, you should lay out a list of requirements for your system, which can be extracted from all the APIs, for example: message notifications, file transfers, user profiles, etc.
In this way, you know exactly what capabilities your system should have, and you don't end up implementing a feature that is available only in 1 API out of 4.
Also, it would be a bit challenging to design your system from a user perspective, because the apps have different usage patterns - chat apps, where messages are coming in real-time, vs email, which is not real-time communication. That's just a detail anyway, the gist of your project is to play with those APIs.
Also, it may worth checking out the Gateway Aggergation Pattern, which is related to this project - you may want the user to send a message to multiple apps, by using a single request to your service.
I am working on a desktop application using GCP Speech-to-Text API to perform streaming recognition. I'm using Python 3 and Google client libraries, google.cloud.speech. I've started off the transcribe streaming infnite Python sample and built around those concepts, and everything works nicely. I'm now trying to get to the end user access question, as I'm currently using a service account file for development purposes, which I'm obviously not willing to distribute to all users.
Basically, how do I give access to my end users to the Speech-to-Text service for streaming recognition in the least intrusive way possible? I have no need for accessing storage or the like, as I don't access buckets contents and stream all audio directly. I actually don't need any user information, I only need GCP to process the STT requests and send me the responses.
I see 2 solutions that should work on paper, out of the three the documentation mentions (I leave out the service account file ones):
API key
OAuth2
API key
API keys sound like my dream option: it's simple, doesn't require user interaction past initial setup, I can manage such keys in GCP's console, and it should be able to grant access to what I need (as I effectively don't need any user info, an account is mostly irrelevant).
However, how to use an API key using the Google Speech client library totally eludes me. I can see a PUB/SUB Go example, but I can't find any mapping to Python. I'm not even 100% sure it can work, as the Go documentation for the option seems to note it only works for JSON-over-HTTP, and I believe the client library for Speech-to-Text is using gRPC.
Yet, at least with a JSON non-streaming recognize request, I can use such an API key, and successfully did so manually using cURL on the command-line. So I still have a little bit of hope, in case the gRPC restriction either isn't true or doesn't concern my use case.
This part's question would summarize as: "how do I specify an API key using the Python SpeechClient?".
OAuth2
This sounds like my second-best option, as it uselessly asks the user for authentication while I don't actually need any personal data. Yet, I still have serious issues I'm struggling to overcome:
How do I reliably store info that prevent me from having to re-ask the user for authorization every single time the app runs? google_auth_oauthlib's InstalledAppFlow doesn't seem to provide such feature, so I'm rolling my own based on google.oauth2.credentials.Credentials.from_authorized_user_file() after having saved them with the to_json() the first time I obtained them with InstalledAppFlow. I'm however confident this will not last, and I'm really not sure how to check whether the credentials are still good before I fail to use an API with them (e.g. I can't seem to be able to rely on Credentials.valid before they actually get used).
There seem to be no specific Speech-to-Text scopes, and the required one is way broader than what I need, leading to an overly complex and frightening authorization request. And no, without this scope I cannot access the Speech-to-Text API, I tried :)
Summary
To summarize: what is my best and least-intrusive option to provide credentials to be able to use GCP Speech-to-Text in my desktop application?
I'm currently assessing the pros and cons of going native vs. PhoneGap for an app I have in mind, and thus far, PhoneGap seems to be the ideal option since most of the data processing will be done on the server-side, while the app will merely be a means to get inputs from the user.
But I've been reading a lot about how all the PhoneGap .html and .js files would be easily accessible on a rooted/jailbroken phone. My concern is that my app will be using a couple of third party APIs, particularly Last.fm and Parse. Both these APIs come with an API secret. Would this not cause a security concern? While the maximum damage one could inflict with my Last.fm API secret key would be to exhaust the API limits, with Parse, it could be much more serious, especially if I plan to store user logins, passwords, emails etc. Anyone could simply grab my Parse Application ID and JavaScript Key and start querying away on Parse (and possibly (but not probably) a competitor or a troll (more likely) could push up the requests/second from my Application ID so that I end up with a big, fat US $10,000 bill).
Are there any methods to protect/encrypt/obfuscate these API secret keys while developing apps in PhoneGap? Does this problem go away if you go native?
First of all, I think the security issues you mention are not related to Phonegap only, same problems exist for native apps as well. I admit, it is harder to find these keys in native apps, but it's doable.
On a side note, both Android apk and Apple ipa files are actually zip archives, so you don't need a rooted phone to open up and peek inside. You can already do it for all the apps (not only Phonegap but native ones), open the archive, see the resources. But in native apps, you have a binary executable instead of html/js files. That's why it's not easy to figure out the data (keys), and app logic inside.
One approach is to implement a special scheme that will encrypt your js files during packaging (development), distribute it with the app and decrypt during runtime. It is totally possible (we did it in a project successfully), your Phonegap app becomes as difficult to hack as native apps. Note that I am not saying impossible, because you still have the problem of hiding the decryption key somewhere in your code/resources. Beware though, writing such a framework is not easy and requires some modification to Phonegap source code as well.
I think the safest solution to hide keys for Parse, etc. is to use your own server, implement server to server authentication and just pass a token to your client for client-server communication. In all other cases, you have to send the keys with the app, and however you hide it, there will be some guy who will find a way to unhide it.
This post is intended to start a deeper discussion on Single Page Applications for the web. There are questions that do not seem to have a clear answer in most resources on the subject.
They are in my mind
Authorization and authentication.
With entire web app being on the client, it may make calls to the server in any of its functions, even those that the user does not have rights to. The fact that the user cannot see a menu, does not preclude that person from invoking java script functions. This is easily handled in MVC app, for example, by using controllers that validate user rights to a specific function based on a cookie for example. However, some SPA apps just use single controller with Breeze or Web Api, which make authorization server side impossible.
Memory management on the client
For small sample apps this is not an issue, but imagine an app with 100's of screens or an app with a single screen that pulls thousands of records over the course of one day. With persistent caching one could imagine large memory issues, especially on under-powered devices with little RAM, like phones or tablets. How can a group of developers had SPA route without a clear vision of handling memory management?
Three Tier deployment
Some IT departments will never allow applications with a connection string to a database located on front end web servers. Every SPA demo I have seen is structured exactly like that, including Breeze or Web Api for that matter.
Unobtrusive validation.
It would require developers to use MVC partial views and controllers instead of just HTML files, which seems to fly in the face of SPA concepts, while it provides a very robust way to easily incorporate validation and UI to support it into web applications.
Exposing primary integer based keys in the url.
This is non-no in OWASP.
As a result, SPA applications "seem" to target areas with few security requirements and small feature sets. What do you think?
Thanks.
#Sergey - I think this is just too broad a question for StackOverflow. S.O. isn't a discussion forum; it's a place to go for specific answers. So while your questions are potentially valid, I don't think you should hold out much hope for deep substantive responses here.
May I add, in the friendliest possible way, that your sweeping, unsupported, and negative statements make you look like a troll. You're not a troll are you Sergey?
On the chance that you are in fact authentically concerned, I offer a few quick reactions, particularly as they pertain to Breeze.
Authorization. In Web API you can authorize at the method level. The ApiController base class has a User property that returns the IPrincipal. So whether you have one controller or many (and you can have many in Breeze if you want), the granularity is method level, not just class level.
Memory management. Desktop developers have coped with this concern for years. It may cause you some astonishment if you've always developed traditional web apps where process lifetimes are brief. But long-running processes are not news to those of us who built large apps in desktop technologies such as WinForms, WPF, and Silverlight. The issues and solutions are much the same in the land of HTML and JavaScript.
Layers on the backend. You've been looking at demos too long. Yes most demos dump everything into one project running on one server. We assume you know how to refactor the server to meet scaling, performance and security requirements for your environment. Our demos are concerned mostly with front-end SPA development. We do dabble at the service boundary to show how data flow through a service API, through an ORM, through to the database. We thought it sufficient to identify these distinct layers and leave as an exercise for the reader the comparatively trivial matter of moving these layers to different tiers. We may have to re-visit that assumption someday. But does anyone seriously believe that there are significant obstacles to distributing layers/responsibilities across server-side tiers? Really? Like what?
Unobtrusive validation. When most people start using the word "unobtrusive" in connection with HTML, they are usually making a point about keeping JavaScript out the HTML. Perhaps that's what you mean too, in which case SPA developers everywhere agree ... and that's why there are numerous "unobtrusive validation" libraries available. HTML 5 validation, jQuery validation and Knockout validation come to mind. All of them are in the SPA developer's toolkit and none of them "require developers to use MVC partial views and controllers". What gives you the impression that a SPA would need any server-side resources of any kind to implement validation with JavaScript-free HTML markup?
Ids as security risk. Really? This is bogus. The key value is no more a security risk than any other data value. Millions of applications - not just SPAs - communicate key values to the client, both in the URL and in the body. It's standard in REST APIs. It's standard in ODATA. And you want to dismiss them all by saying that they "target areas with few security requirements and small feature sets"? Good luck with that. I think you'll have to do better than rest your case on a link to a relatively obscure organization's entire web site.
I have built some SPA applications, ranging from small to large (over 100 scripts and views). Only a handful of them had every view accessible to the public. The rest went through a strict access structure. It was so simple to return a 401 unauthorized from the server and the client just handling the 401 to redirect it to the login screen. Mr. Ward and Mr. Papa put it right. Get out of the Demo mode and try to find solutions to the issues you come across. I have watched John Papa's SPA on pluralsight, gone through numerous articles and applications on Breeze and I have to tell you, none of my applications use Breeze to do queries from the client side, because YOU DON'T NEED TO!!
Moreover, I have only extended what I have learnt and come up with my own way of solving problems. This is not an answer to your queries, but I only can provide a short comment. No technique is perfect and there is no ONE way to do everything. My server side is locked down where it needs to be locked down, my routes on the client side are locked down (if using durandal take a look at guardRoute), my scripts are minified and my images are sprited (if there is a word like that). All in all, SPA is a great technique, you got to find solutions to the quirks!