I have a pretty standard application written in Java which also runs queries against a DB. The application resides on GCP and the DB on Atlas.
For understandable reasons, I don't want to keep the username and password for the DB in the code.
So option number 1 that I had in mind, is to pass the username and password as environment variables to the application container in GCP.
Option number 2 is using Secret Manager in GCP and store my username and password there, and pass the GCP Credentials as an environment variable to the application container in GCP.
My question is, what is the added value of option number 2 if it has any? It seems that option 2 is even worse from a security aspect since if some hacker gets the google credentials, it has access to all of the secrets stored in the Secret Manager.
I don't know what are the best practices and what is advised to do in such cases. Thank you for your help.
Having credentials in GCP secret manager will help you to keep track of all the secrets and changes in a centralized location and access globally from any of your app.
For a standard application where one JAVA is connecting to a DB, may not add much values.
You may look into kubernetes secret for that reason.
If your application resides in GCP, you don't need a service account key file (which is your security concern, and you are right. I wrote an article on this)
TL;DR use ADC (Application Default Credential) to automatically get the service account credential provided automatically on Google Cloud Component (look at metadata server for more details).
Then grant this component identity (by default or user defined, when supported), i.e. the service account email, to access to your secrets.
And that's all! You haven't secrets in your code and your environment variable, neither the login/password, nor the service account key file.
If you have difficulties to use ADC in Java, don't hesitate to share your code. I will be able to help your to achieve this.
To use Secret Manager on Google Cloud you need to install the Secret Manager Java SDK Libraries. This documentation shows how to get started with the Cloud Client Libraries for the Secret Manager API, you only need to go to the Java section.
This Libraries helps you to access your keys in order that it can be used by your app.
The following link shows how to get details about a secret by viewing its metadata. Keep in mind that viewing a secret's metadata requires the Secret Viewer role (roles/secretmanager.viewer) on the secret, project, folder, or organization.
I recommend you to create a special Service Account to handle the proper permissions for your app, because if you don’t have a SA defined, the default SA is what is going to generate the request, and it is not secure. you can learn more about how to create a service account in this link
On the other hand, you can find an example on how you can use the following guide that contains a good example of finding your credentials automatically, that's more convenient and secure than manually passing credentials.
Related
We're running a server on AWS that will be using a few constants. These constants may be details that are confidential like a few API tokens, Client secrets & even DB credentials. We have been saving these details in one of our files on the server itself (say Credentials.js). So,
What is the best possible way to store these Credentials and in a secure manner.
We were also planing to switch to AWS SSM parameter store. Is it worth considering it? It also provides KMS encryption to confidential parameters.
Even if we do switch to AWS SSM Parameter store, we will have to call them multiple times when we make requests to third-party application servers (as we'll need the API tokens for those apps). Does this justify the cost we'll pay for SSM (Considering we take Standard store with High throughput) ?
Also, Please let me know if there are there alternatives to securely store these Parameters.
Thanks.
Secret Manager
Secrets Manager enables you to replace hardcoded credentials in your code, including passwords, with an API call to Secrets Manager to retrieve the secret programmatically. This helps ensure the secret can't be compromised by someone examining your code, because the secret no longer exists in the code. Also, you can configure Secrets Manager to automatically rotate the secret for you according to a specified schedule. This enables you to replace long-term secrets with short-term ones, significantly reducing the risk of compromise.
To get an overview how it look like, see AWS Secrets Manager: Store, Distribute, and Rotate Credentials Securely.
Cost
See Pricing. $0.40 USD per secret per month and $0.05 per 10,000 API calls.
Documents
Tutorials - Start here to get the ideas
secrets_getsecretvalue.js - Example to get secrets in JS
JS SDK for Secret Manager - Look further here to know the JS SDK
CreatSecretAPI - AWS API to create a secret for the detailed references
Create a secret via the AWS console or using SDK. See Creating a secret. A secret is a key/value pair where the value is in JSON format.
Alternatives
Hashicorp Vault
Static Secrets: Key/Value Secrets Engine
Vault JS client
Lambda
Use a lambda which only accepts an access from those with a specific IAM role/permission attached to the IAM profile of an EC2 instance to run your app.
Others
Just Googling "parameter store for secret management" showed bunch of articles and how-to. Please do the research first.
I'm seeking your advice to piece together a mechanism that would facilitate authentication to Dialogflow ES and CX to allow running experiments on multiple agents (projects) from our workbench application in a smooth and error-proof manner. The workbench is an internal tool written in TypeScript (using the dialogflow RPC node module) running outside of GCP. Our users analyze the results of sending the same inputs (utterances) to multiple agents, usually going back and forth between them in the course of their work.
With proper IAM configuration, we have been able to detect intents successfully by doing a gcloud auth application-default login, however we haven't found a way to update the quota project programmatically or to specify the quota project through the google.cloud.dialogflow library, so we haven't been able to fix the "switch easily between projects" part. It looks like tampering with the quota_project_id property in application_default_credentials.json once authenticated is the way to go (gcloud auth application-default set-quota-project <project>) but we would have preferred doing this programmatically.
Using service account keys (JSON) works as expected and that's what we have been doing so far, that's also what we do in our CI/CD pipeline and in our agents running in production. But we aim at reducing the amount of service account credentials file that we share with individuals. Ideally, speech/data scientists would use their own end-user credentials to perform experiments.
We are looking for alternatives so that users would authenticate once with gcloud auth application-default login and the workbench would handle the rest behind the scenes, using only, as additional argument, the project-id against which the experiment must be run. This would eliminate the need to pause the experiment to update the quota project (using set-quota-project), or to update the GOOGLE_APPLICATION_CREDENTIALS variable when using service account keys.
Another thing we tried was Service Account Impersonation, unfortunately this does not seem supported by the google.cloud.dialogflow library, so even though we were able to successfully submit requests (with Curl/Postman) to the Dialogflow RESTful API using impersonation, we haven’t been able to leverage this mechanism in our code.
Has anyone been able to overcome a similar challenge? Is there any other authentication mechanism that could help us achieve this goal?
Is there a comprehensive list over which keys and UUIDs provided by Azure are sensitive information?
For context, I am setting up a react app using react-adal. This requires me to provide the tenantID and clientID to the app. It would be trivial for anybody with a little know-how to extract those UUIDs from the application given the way I currently have it setup. It's a static app with no back-end service, so it all runs in the browser. I have the application setup as a App registration (registered app?), in AzureAD to allow users to login with their Office365 account for my organization. Almost all the tutorials online show setting it up the same way I am. However, some tutorials with screenshots almost always have the clientID censored in the screenshots.
Edit: While this question does solve my immediate need, I am in search of documentation from Microsoft concerning security best-practices in regards to Azure keys, UUIDs, etc.
Your tenant ID is public and refers to the entire Azure AD directory. It must be in order to be used for authentication. You can find it here.
Your client app ID is not secret, as it must be used with the client secret in order to obtain a token.
I've built a simple messaging application with NodeJS on GCP that, at the moment consists of a single compute engine instance. I'm using this to learn the stack and how GCP sits together.
My application has it's own user/password registration system and allows users to message each other in 'groups'. These groups can consist of 1...n users and one user is responsible for adding/removing users from a group. They can do this at any time.
I want to allow the users to upload pics and ideally I will then store them in Google Storage. But I want to make sure that only users in a room where an image is uploaded can view that file.I know that GCP makes use of IAM roles etc. but with the authentication being in my system am I expected to update IAM policies every time via the API? In a scaled solution would this work?
My initial thought is that I should do the authentication at an app level. e.g my Compute Engine instance can talk to Storage so when a user requests to a view an image by its url - such as example.com/uploads/:id
I then validate that the current user can view the upload with id :id and if they can, then serve the image from the app. Would this work? Would it be compatible with utilising Google CDN? Is there a preferred solution for doing something like this bearing in mind I'm not using firebase (which I understand can use access tokens for auth) but I'm using my own authentication based on username/password combos with sessions?
For examples of sharing and collaboration scenarios that involve setting bucket and object ACLs, you may take a look at Sharing and Collaboration. As you mentioned and also noted here, you can create a service that authenticates users and redirects them to a URL signed by a service account and this solution helps the scaling amount of users as well.
You must give Cloud CDN permission to read the objects by adding the Cloud CDN service account to Cloud Storage's ACLs for restricting the users on reading the objects.
I should also add that Cloud Storage is integrated with Firebase and you may acquire Firebase Security Rules for Cloud Storage to authenticate and authorize the users.
So it looks like I actually have 2 options here.
I can use signed urls https://cloud.google.com/storage/docs/access-control/signed-urls#signed-urls and grant temporary access to the files to the users in question. I would just need to regenerate this whenever required.
My second option (event though I said I don't want to migrate) is to use Firebase Auth. I wasn't aware it actually supported email/password migration and validation and is actually free regardless of number of users. The only thing I'm not sure on here is how Storage is configured with relation to my current GCP project.
I am building an application that will run on Google App Engine (GAE). It will need access to data stored by the user in other systems (e.g. the user's Nest thermostat, Yahoo mail). The application running on GAE will allow the user to provide credentials for the other system. The application will store these credentials in Google Cloud (Datastore) for later use by an application running on Google Compute Engine on the users behalf. The application will also allow OAuth to allow the user to allow the application access the external system in the user's behalf. The application will need to store user credentials (username and passwords) or OAuth access tokens in the Google Cloud.
The application will need to encrypt the secrets before they are stored and be able to unencrypt the data to send it to the external systems. That is, the system will need to use symmetric encryption and therefor need to securely manage keys.
How can the application store these secrets in the Google Cloud Datastore (Datastore) securely? I think I am looking for something like the AWS CloudHSM for Google. That is, I would like to store each secret with a seed and key id and use the key id to get the key from a key management system. This implementation would also allow for key rotation and other standard security practices.
I think I am looking for a Google Cloud service or Google API that provides secrets management and only allows an app with the proper Google app identifier to access the secrets.
Is there a service within Google Cloud or Google APIs that will manage secrets? Is there another architecture that I should be considering?
By the way, the application uses Google Identity Toolkit (GitKit) to authenticate and authorize users to use the GAE hosted application. The application allows users to create accounts using either federate identities or username and passwords via GitKit.
Thanks,
chris
In the meantime, Google also added a Key Management Service: https://cloud.google.com/kms/
You could e.g. use it to encrypt your data before storing it in a database. Or, use KMS to encrypt an AES key to encrypt your data, and possibly keep a backup of your AES key somewhere in case you lose access to KMS.
App Identity Service might be what you are looking for https://cloud.google.com/appengine/docs/java/appidentity/#Java_Asserting_identity_to_other_systems
It lets you sign content with an application-specific private key, and provides rotating certificates to validate signed content.
So as far as I can tell the answer is that you can't. What you are looking for is an equivalent to KMS. That service let's you create and manage keys and do a bunch of your own crypto stuff. It's really cool and it will allow you to quickly do incredibly strong crypto with just a few simple lines of code. Azure has a similar service called KeyVault. It lacks automated key generation and rotation as far as I can tell, but other than that it's good. At the time of this response there was not an equivalent service for Google. They have an internal KMS which they used for crypto operations and you can provide your own keys, but that's pretty much it. Not quite the same thing that you get on KeyVault, and nothing like KMS.
That said there is hope. You can do one of two things:
Create a VPC and use an HSM from somewhere else. You could use RackSpace, or you could simply use AWS KMS. That sounds crazy but it's actually a good idea and the extra management is worth it. In general the most secure solution separate the keys from the encrypted data, particularly at rest. That means that keys in one data center and encrypted data stored in another data center is the most secure solution. That sounds like hard stuff, but thankfully I've made an opensource project which makes it very easy for you called KeyStor. With KeyStor you can get a data center that deals with encryption services set up in a day, no problem, and you can use AWS very cost effectively.
Set up your own cypto service, skip the HSM integration and simply be careful about who has access to the machines that maintain your keys. You can do this with KeyStor as well, and if KeyStor doesn't quite do what you want, that's why it's open-source. Take the code and build what you need to build.
You could store secrets in storage (e.g., in Datastore, Google Cloud Storage, or another storage system of your choice) and encrypt those with a key from Google's Cloud KMS.
Here's some documentation from Google on secret management, and here's a codelab on specifically encrypting data in Google Cloud Storage at the application layer using Cloud KMS.
For the Google Cloud managed service that provides the API for secure storage of secrets, see Google Cloud Secret Manager for more details.
Secret Manager is a secure and convenient storage system for API keys, passwords, certificates, and other sensitive data. Secret Manager provides a central place and single source of truth to manage, access, and audit secrets across Google Cloud.