I have some secret which is created using terraform , due to some mistake I had commented and applied tf so the reource marked for deletion, but now if I enable it and apply it is saying the resource is marked for deletion.
resource "aws_secretsmanager_secret" "rotation-example" {
name = "mysecret"
description ="sccretatexample"
recovery_windows_in_days = 7
}
I can't change the name and create other resource, and also I dont have access to aws console/cli . pls guide me how to create again or is it possible to use the old one by overriding
As of now there is no functionality available to retrieve deleted secret using terraform. Check this open issue -
https://github.com/terraform-providers/terraform-provider-aws/issues/10259
But you can do it using some manual work but either you will require help from your AWS administrator or AWS access key should be having below permission.
To restore a secret and the metadata in the console, you must have these permissions:
secretsmanager:ListSecrets – Use to navigate to the secret you want to restore.
secretsmanager:RestoreSecret – Use to delete any versions still associated with the secret.
if AWS access key have above permission use below cmd to restore password.
aws secretsmanager restore-secret --secret-id mysecret
follow this AWS document to restore secret.
https://docs.aws.amazon.com/secretsmanager/latest/userguide/manage_delete-restore-secret.html
once secret is restored you can use "terraform import" as below to updated you state file with existing secret details.
terraform import aws_secretsmanager_secret.rotation-example mysecret
In addition, if you want to create and delete secret frequently use below.
recovery_windows_in_days = 0
Related
I am working on a project that uses a Jenkinsfile and given the name of a lambda it creates this lambda in AWS along with its terraform configuration, and uses AWS Secrets Manager to grab the secrets.
I have created the secrets via terraform and essentially want to keep all of the secrets for each of the lambdas centralized in one location ("project_lambda")
The tf looks like this (there is a policy as well, but has been omitted):
resource "aws_secretsmanager_secret" "project_lambda" {
name = "project_lambda"
description = "Secrets for project"
recovery_window_in_days = 0
}
resource "aws_secretsmanager_secret_version" "sversion" {
secret_id = aws_secretsmanager_secret.project_lambda.id
secret_string = jsonencode(var.map_of_secrets)
}
The pipeline generated the secrets fine, and re-ran fine when it was only one lambda. But when I added in another (they have seperate state), this error comes up!
Error: error creating Secrets Manager Secret: ResourceExistsException: The operation failed because the secret project_lambda already exists.
I tried commenting out the code, but then it marked the secret for deletion and I had to manually delete it.
Any ideas for what the approach should be to solve this? Can I force recreation of the secret, delete then create, or delete that code and somehow have it not marked for deletion?
I am creating a script using Azure CLI that will automatically generate an App Registration (service principal), and then use that App Registration to create a secret that will be stored in Azure Key Vault.
However, I am getting the following error:
The user, group or application 'appid=04b07795-8ddb-461a-bbee-02f9e1bf7b46;oid=0ec2b0e8-daeb-46a8-b627-0d4f61f87157;numgroups=134;iss=https://sts.windows.net/72f988bf-86f1-41af-91ab-2d7cd011db47/' does not have secrets set permission on key vault 'asakeyabcfelaqpgsfnxcy;location=eastus'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287
Can anyone provide guidance on what this ID is and how to resolve this error? This is not my App Registration Object ID or App ID.
I think there're 2 points you're now concerning, one is you failed to add secret, another is the app id in the error message is not the one you registered.
I think you've followed the document to execute the cli command, so I just want to explain the second point. Pls allow me show you a code sample for a better explanation.
using Azure.Identity;
using Azure.Security.KeyVault.Secrets;
public async Task<IActionResult> PrivacyAsync()
{
var kvUri = "https://your_vault_name.vault.azure.net/";
var client = new SecretClient(new Uri(kvUri), new DefaultAzureCredential());
_ = await client.SetSecretAsync("test0311", "hello");
return View();
}
When we want to add key vault secret to azure, we need to provide a credential so that our operations are authenticated. This is the DefaultAzureCredential() here, and it has several sources to get the authentication like screenshot below.
That means if someone sets the environment variables for authentication, then it will cover the information you entered when executing cli command, this may usually cause the issue that the app is different from what you set. I think you may follow this document to check all your configurations and try again, or you can directly add environment variables with the app you registered on your computer.
By the way, pls don't forget to add access policy in azure portal for the azure ad app you registered.
I have a secret personal access token (only for building purposes) in my .npmrc file. As this secret is exposed, I thought of replacing this using Azure Key Vault. I haven't found any documentation around it. When I created the personal token before, I had given it only packaging/building access. How can I achieve this, please help me with this? Or is there any better way to include the personal access token in the .npmrc file?
Since you confirmed you are using Azure DevOps for your build, you don't need to maintain PAT in the .npmrc file. Just keep your npm registry URL there (I assume the private npm registry is also in the Azure DevOps) like below:
registry={your npm registry URL}
always-auth=false
Now, in the build pipeline, add npm Authenticate task before npm install.
- task: npmAuthenticate#0
inputs:
workingFile: <relative path to your .npmrc file>
Providing secrets to your resource can be done in many ways.
Some resources in Azure allow you to specify environment variables through the Azure CLI. Here's an example with the Azure container instances: link.
On Azure, once you have a Key Vault instance, you can use your Key Vault to provide secrets to your App Service and Azure Function instances. This is documented here: link, with a focus for Azure Resource Manager templates, which is specially useful for automated deployments.
Although the following is explained in the documentation link above, the general picture on how to use Key Vault secrets from other Azure resources requires the following:
Make a user assigned identity or Azure Active Directory application.
Grant access to this identity (or AAD app) by going to the Access Policies of your Key Vault (this can be done through the portal, of course), and giving your identity at least read access to your Key Vault.
After that, create a secret on your Key Vault, go to the secret details and copy the "Secret Identifier". This will be a URI similar to: https://myvault.vault.azure.net/secrets/mysecret/.
That's the URI you can use to bring Key Vault secrets to other resources.
You'll be able to access this secret from other resources by ensuring the resource has access to the same identity, and by providing the URI through a syntax similar to: #Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/).
For example, if you link an Azure Function to the same identity you granted read access to your Key Vault, you can provide a secret through environment variables by setting configuration properties in your resource. By going to the Azure Portal, locating your resource, then going to Configuration, then to Application settings, if you proceed to add the name of your environment variable, and as the value something similar to: #Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/), you'll be providing the expected environment variable with the expected secret value to your resource.
The final approach I can think of is by using the #azure/keyvault-secrets client. If using an NPM library to retrieve Key Vault secrets sounds interesting, this is the dependency for you. All the information needed to work with this library should be available on NPM: same link. But in any case, a sample using this client would look as follows:
const { DefaultAzureCredential } = require("#azure/identity");
const { SecretClient } = require("#azure/keyvault-secrets");
const credential = new DefaultAzureCredential();
const client = new SecretClient(`https://my-key-vault.vault.azure.net`, credential);
async function main() {
const secretName = "MySecretName";
const latestSecret = await client.getSecret(secretName);
console.log(`Latest version of the secret ${secretName}: `, latestSecret);
}
main();
You could use this library to load your secrets at any point while your service or program is running.
Please let me know if this information is useful for you. I'm here to help!
So I am trying to find someway to hide a secret in Terraform. The caveat is the secret is a Service Principal that is used to connect to our Key Vault. I can't store the secret in the Key Vault as it hasn't connected to the Key Vault yet at that point. This is part of my main tf file.
provider "azurerm" {
alias = "kv_prod"
version = "1.28"
tenant_id = "<tenant id>"
subscription_id = "<sub id>"
client_id = "<SP client id>"
client_secret = "<SP secret>"
}
This is used further down my module to store Storage Account keys and other secrets. It just happens to be in a Prod subscription that not everyone has access to.
Has anyone run into something like this? If so, how would you go about securing that secret?
#maltman There are several ways to hide a secret in terraform. Here is a blog that talks about them:
https://www.linode.com/docs/applications/configuration-management/secrets-management-with-terraform/
However if you are only concerned about encrypting the secrets file while checking in and checking out from git, you can use something like git-crypt
You would have to create a couple of files:
variables.tf -> Define your variables here
variable "client_secret" {
description = "Client Secret"
}
terraform.tfvars -> Give the value of the variable here
client_secret = 'your-secret-value'
Now use git-crypt to encrypt terraform.tfvars while checking into git
For your requirements, I think there are two secure ways for you in comparison.
One is that stored the credential as environment variables so that you do not expose the secret in the tf files. Here's the example.
The other one is that you can log in with the credential for Azure CLI, then just need to set the subscription without exposing the secret in the tf file. Here's the example.
The above two ways are that what I think is secure and possible for you. Hope it helps you.
Terraform doesn't have this feature but by using third party integration it can be achieved.
Storing Secret in Terraform:
Terraform has an external data resource that can be used to run an external program and use the return value further. I have used Ansible vault feature to encrypt and decrypt the secrets and store it encrypted in repository rather as plaintext.
data "external" "mysecret" {
program = ["bash", "-c", "${path.module}/get_ansible_secret.sh"]
query = {
var = "${var.secret_value}"
vault_password_file = "${path.module}/vault-password.sh"
# The file containing the secret we want to decrypt
file = "${var.encrypted_file}"
}
}
Refer the working example: github example
Going to create an ADO pipeline to handle this instead where the code just does not have to be available.
In our case we are doing the following:
1. Infra Agent
a. We create a KV
b. We create a SQL Database in the tf script, including assigning an admin username and password (randomly generated value).
c. We store the username and password as secrets in the newly created KV
2. Data Agent
a. We want to deploy the DDL from the repos onto the SQL Database we created in Infra Agent. We need to use the SQL database username and password stored in the KV to do so
b. In order to read the secrets from the KV our current thinking is to insert the username and password to pipeline parameters in step 1 (i.e. setting them at runtime) so we can reuse the values across other Agents.
A couple of questions:
- Is that the right approach? Should KV be created in the Infra Agent tf scripts? Should we randomly generate passwords (as secrets)?
- What is best practice to access the Database username and password in other Agents, given that:
o We can’t use variable groups because the KV and values won’t be known until runtime
o We can’t use the Key Vault Task (https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-key-vault?view=azure-devops) to read from the KV because the KV name is only known at runtime (via the tf vars file)
b. We create a SQL Database in the tf script, including assigning an admin username and password (randomly generated value).
If you're using Key Vault, then I assume you're talking about Azure SQL Databases. However at the moment Terraform only supports assigning a administrator username and password for the SQL Server instance, not SQL databases.
In this case, I recommend using random_password resources to assign values to azurerm_key_vault_secret which can then be assigned as the azurerm_sql_server administrator password.
With this setup you know for certain that the password in Key Vault is always in sync, and can be treated as the source of truth for your SQL server passwords (unless someone goes and resets the administrator password manually of course).
Now if you ever want to reset an SQL server password, simply taint the random_password, forcing it to be recreated with a new value, which in turn updates the azurerm_key_vault_secret value and then the azurerm_sql_server password.
Here's some quick HCL as an example
resource "random_password" "password" {
length = 16
special = false
}
resource "azurerm_key_vault_secret" "password_secret" {
depends_on = [<the Key Vault access policy for your infra agent which runs terraform apply>]
...
value = random_password.password.result
...
}
resource "azurerm_sql_server" "sql_server" {
...
administrator_login_password = azurerm_key_vault_secret.password_secret.value
...
}
Is that the right approach? Should KV be created in the Infra Agent tf scripts? Should we randomly generate passwords (as secrets)?
This is a sensible approach, but remember that billing is per secret, key or cert and Key Vaults themselves are free. It's recommended to create a Key Vault for each application because access policies can only be applied per Key Vault and not per secret/key/cert.
We can’t use the Key Vault Task (https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-key-vault?view=azure-devops) to read from the KV because the KV name is only known at runtime (via the tf vars file)
Why is this only known at runtime? This sounds like a limitation of your own process since Terraform allows you to specify a name for each Key Vault when you create it. Reconsider if this is really a requirement and why you are doign this. If it definitely is a requirement and your Key Vault names are dynamically generated, then you can use terraform output to get the Key Vault name during the pipeline and set it as a variable during the build.
To fetch the Key Vault name as an output just use the following HCL
output "key_vault_name" {
value = "${azurerm_key_vault.demo_key_vault.name}"
}
and run `terraform output key_vault_name" to write the value to stdout.