I am devops engineer and project admin in Google Cloud project.
Organization has some super sensitive keys that should be accessed by containers running in Kubernetes or Cloud Run, but they should not be accessible by me.
So far I see Secrets Manager and KMS as possible solutions. I have admin permissions for this project, so probably these secrets should be stored in another project owned by CTO and cross-referenced.
Also I have all permissions for monitoring running containers, is there any way to prevent secrets from leaking to me in runtime? And what solution should be used?
If you are project owner, you can't technically prevent the secret leak.
If the secret is stored in another project, you can't access the secret through the UI. However, that also means a service account has access to the secret stored in another project. Because you are project owner, you can impersonate the service account which has the right to access the secret, and access it also.
The only way to solve that issue, is the least privilege principle. Have only the minimal permissions on the project, and not the security admin role.
When you have to grant permission on service account, you need to use automated system, like CI/CD and terraform for example. And of course, each update need to be reviewed by another person, you can't update the CI/CD pipeline and grant permission to yourselves (such as service account impersonation) without peer review.
Related
I have a rather (hopefully) theoretical question regarding the secure usage of Service Pricipals in Azure (Enterprise Applications)
Introduction
we currently deploy our DevOps Code via Azure Service Principals.
AppRegistration/Enterprise App is created
Secret is generated
Permission (i.e. Contributor) to the Ressource Group is granted in Azure
Service Connection is made in Devops
everything works fine.
Assumption
By default the Service Principal (Enterprise Application) is not restricted to a specific user/group (Assignment Required => "no").
My assumption is now, that every user in the AAD-Tenant is able to login to the Enterprise Application as well.
I i.e. do this by using the "Graph Powershell API"-EnterpriseApp.
I can either use a Secret or use my User Credentials to access the Service Principal and its permissions
Security issue?
coming back to our DevOps configuration:
The Service Principal has Contributor Permission on the dedicated Resource Group
Assignment Required is set to no (default configuration)
if I (as a malicious user) have the Application ID, i could simply logon to the Service Principal and receive the Token.
Question:
With this token and my login to the App, do i also have the Contributor Permissions of the App and could now manipulate the whole Resource Group?
Since i'm not an Azure Developer - but only an Azure AD Admin - my knowledge regarding this is limited,
so i'm not able to test it.
Can someone maybe either provide code or prove that my assumptions are wrong or correct?!
Thanks
Yes, the SPN can manage the resources within the resource group if it has Contributor - it is no different than a normal (human) identity.
Consider if the SPN actually needs Contributor or if you can limit it with another role or even make a custom role.
Furthermore, monitor the sign-ins using the Azure AD sign-in logs:
https://learn.microsoft.com/en-us/azure/active-directory/reports-monitoring/concept-all-sign-ins
You can also use CanNotDelete resource lock, which means that the service principal cannot delete resource as it is only Contributor:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/lock-resources?tabs=json
You might want to look into the Conditional Access to strengthen your environment:
https://learn.microsoft.com/en-us/azure/active-directory/develop/workload-identities-overview
https://learn.microsoft.com/en-us/azure/active-directory/conditional-access/workload-identity
https://learn.microsoft.com/en-us/azure/active-directory/privileged-identity-management/pim-create-azure-ad-roles-and-resource-roles-review
Take a look here:
https://infosecwriteups.com/a-lab-for-practicing-azure-service-principal-abuse-bd000e6c48eb
https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/disable-user-sign-in-portal
https://learn.microsoft.com/en-us/powershell/module/az.accounts/connect-azaccount?view=azps-9.3.0#example-3-connect-to-azure-using-a-service-principal-account
My assumption is now, that every user in the AAD-Tenant is able to login to the Enterprise Application as well.
No. They would need the client secret or the rights to generate a new one. Which requires that they are owners of the App Registration. In the App Registration on the Owners tab it says:
The users listed here can view and edit this application registration. Additionally, any user (may not be listed here) with administrative privileges to manage any application (e.g., Global Administrator, Cloud App Administrator etc.) can view and edit the application registrations.
Is there any point in using Azure Key Vault over App Configuration?
Yes, yes, I know - they are complimentary, key vault for secrets, app config for... well, app config.
But, considering they are both encrypted, basically for someone to see either a secret or a config value they'd have to have access to your azure portal (this is a low-level bad guy scenario).
The ONLY difference I see is that you can control permissions differently between the vault and config but apart from that if someone unauthorized has access to your portal you've got bigger problems.
So - why? and please only good and real arguments no "because you should" or "because person X said so", what benefits would I reap with key vault that I don't have with app config?
I appreciate your question. I'd re-phrase it to this:
Q) How are are Key Vault and App Configuration designed differently supporting different purposes? And where can I find a clear comparison table of features and benefits?
I also appreciate your aside:
please only good and real arguments no "because you should" or
"because person X said so", what benefits would I reap with key vault
that I don't have with app config?
Here is what I found to understand benefits as a contrast:
Article 1: What is Azure App Configuration?
App Configuration complements Azure Key Vault, which is used to store application secrets. App Configuration makes it easier to implement the following scenarios:
Centralize management and distribution of hierarchical configuration data for different environments and geographies
Dynamically change application settings without the need to redeploy or restart an application
Control feature availability in real-time
Article 2 : Key management with Key Vault
Key management with Key Vault
Without proper protection and management of the keys, encryption is rendered useless. Key Vault is the Microsoft-recommended solution for managing and controlling access to encryption keys used by cloud services. Permissions to access keys can be assigned to services or to users through Azure Active Directory accounts.
Key Vault relieves organizations of the need to configure, patch, and maintain hardware security modules (HSMs) and key management software. When you use Key Vault, you maintain control. Microsoft never sees your keys, and applications don’t have direct access to them. You can also import or generate keys in HSMs.
Article 3: Azure Key Vault recovery management with soft delete and purge protection
Soft delete and purge protection are two different key vault recovery features.
I think you should not neglect the fact that someone that have the configuration of your App Service can see the secrets. A developer of your company could have access a production App Service for bug investigation but it should not mean he has access to production secrets. The fact of having a single employee's laptop (with access to Azure Portal) hacked should not necessarily mean "access to every secret of your application".
But appart from (as you already know the above), what differences I see:
Better governance and acess monitoring : you have logs to see who try to access each secret, when and how, something you cannot do in an App Service
Better secret management: all secrets stored in one place, you only modify them in key vault once instead of on each app service configuration when some secrets are shared in (used by) multiple app services
Some advanced mechanism like recovery management and purge protection
Better development experience : when debugging an app, if your user has access to the keyvault he can just run the app that will load secrets from keyvault into configuration instead of manually copying secrets from azure app settings locally
1-Data stored in Azure Key Vault is encrypted (App Configuration is not)
2-If a person is associated with Contributor role, he/she can see the configurations in your App Configuration, on Key Vault, only allowed Principals.
3-You can rotate secrets stored in Azure Key Vault, and there won't be any downtime in your app (unless you're caching it on your App Service and need to restart it, in order to refresh the cache)
4-Azure Key Vault is the Microsoft recommended service to store Secrets, Keys and Certificates
Just conceptual questions because I apparently doesn't feel the articles already available #google in this topic.
The story:
I have 2 git projects for IaC which create and configure network and create and configure projects. Both of those projects use different service accounts to perform operation on the cloud (different permissions are needed by them and I don't want to have one super account with all permissions). I would like to perform CICD pipeline which will allow to perform in general teraform apply.
The question:
I would like to not store keys for such accounts at GitLab (gitlab variables - I would like to store them in Hashicorp Vault - The goal is to make sure that it will only be avaliable for CICD. No human should be allowed to access those keys. How to make it work? To cloud access keys stored at vault I still need vault key to be stored on runner machine or in gitlab variables both not considered as safe for me, if anyone will get access to vault key it is really easy to access service account keys by Vault Rest API
One way we solve this is by leveraging Gitlab’s identities and Vault’s JWT authentication method.
You can read more about this here: https://docs.gitlab.com/ee/ci/examples/authenticating-with-hashicorp-vault/ for Gitlab and https://www.vaultproject.io/docs/auth/jwt for the Vault auth method.
Why would this solve the issue (if I understood your challenge correctly)?
Once you have this setup, you can
create an identity entity in Vault to represent your CI service account
create an alias to bind this identity entity to the identity Gitlab will provide in CI_JOB_JWT
create a policy to allow this identity entity (directly or via a group) to read the secrets)
Hence anyone running the pipeline from another identity won’t be able to read the secrets (because it would map to another identity entity in Vault, so they won’t get a token with that policy)
Any one who can hack into my code and get the client id, tenant id and cert thumbprint can access the secrets stored in my key vault. So what’s the difference between this and just storing the secret values directly in my code where any one who hacks into my code could read?
Even if someone was able to get that info, your vault will still be secure because in order to access your Key Vault and view the contents inside, a user, app, or identity will need to be added to your Key Vault's management plane (RBAC), and data plane (access policies) to view anything within your vault.
Therefore, if a user has a TenantID they still won't have access to your AKV unless they have the credentials of a user, app, or identity that has the correct RBAC and access policies permissions.
For more info: Management plane and Azure RBAC Data plane and access policies
For applications hosted in Azure, using Managed Identity is preferred. This is a separate principal where a password is never made available to even steal or leak.
If you use the DefualtAzureCredential in the Azure SDKs, you can better configure your vault for development and production environments. You can give limited permissions to AAD users, who can log in via Visual Studio, Azure CLI, Az PowerShell Module, and more. Without modifying the code, your application can rely on Managed Identity with its own set of permissions. For example, maybe you use different vaults for production and development configured by a setting where Managed Identity has read access to secrets/keys, but your devs don't - not in the production vault. Role-based Access Control is also coming to Managed HSM (a more secure hardware-based Key Vault) to have principal granularity even in the same vault.
Service principals using environment variables can be used in lieu of Managed Identity but should be reserved only for your production environment, ideally. Devs could also have one, but that probably means sharing of client IDs and secrets, which provides little to no mitigation if someone leaves or information is leaked - no easy mitigation, anyway.
I run a very simple nodejs web app on Azure App Services. I need to pass a secret into the app, for which I use dotenv locally. I can set these environment variables in Azure configuration Application settings. Wonder if the secrets are secure there I found I can also store them in Key Vault referencing to them in the environment variables using #Microsoft.KeyVault({referenceString}). In both places I can "unhide" the secret, so I wonder what the advantage of the one over the other is in my scenario?
In the Application settings, the app setting is not really hidden, as you know, simply click Show values then you can see it. Also, anyone has the read permission at your web app scope, he will always be able to check this value, not only the portal UI, but also
azure powershell, cli, resource explorer, etc.
To store the it as a secret in keyvault is secure, just the one(in your case, the system-assigned identity of the web app) who has the correct permission in the Access policies can access the secret. The other people will not be able to access the secret(they are not the RBAC roles e.g. Owner, Contributor in the subscription/keyvault, otherwise they can add themselves to the Access policies).
For more details, see Secure access to a key vault.