Context: I have a Release pipeline that does the following:
- Delete my runbooks from the automation account- reason
- Copy my runbooks from a my repository into my blob storage
- Run my ARM template where I fetch the blob storage runbook and deploy this to my automation account.
For the third step I need my ARM template to retrieve the blob storage. In Azure devops this is easily done by using the output variable of the copy step en use this inside my deployment step and just override the parameters. But the ARM template I use is a linked template (main template) and the ARM that deploys the runbook doesn't have parameters but a parameter file so I can't just override the parameters there.
The solution for this is to put the SAS token inside a Keyvault secret so the ARM template that needs the SAS token just gets this from the KeyVault.
To do this I copied my Runbook to my blob storage account and exported the sas token in Azure devops. In the next step I want to set the keyvault secret to this variable. So that with every run a fresh SAS token will be in place for the ARM template to retrieve. The problem I am facing now is this.
I use the following lines of code
$Secret = ConvertTo-SecureString -String $(StorageToken) -AsPlainText -Force
Set-AzureKeyVaultSecret -VaultName 'keyvault' -Name 'supersecret-sas-token' -SecretValue $(StorageToken)
But the sas token has a value like this ?sv=2015-05-14&551qf54q5f4&qz5f4qz5f4&qz5f
Like you can see there are some ampersand(&) in the string. And Powershell gives me a hard time for this.
I get the following error message
the ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
I tried replacing the amersand value in the string with the value "&" with the quotation marks. ----> didn't work
I tried adding "' '" before and after the string ----> didn't work
Does anyone know a workaround to deploy the sas token to a KeyVault.
Definitely put the string in quotes $Secret = ConvertTo-SecureString -String "$(StorageToken)" -AsPlainText -Force
If it still doesn't work try escaping the ampersands ?sv=2015-05-14`&551qf54q5f4`&qz5f4qz5f4`&qz5f
According to my test, you can use the following script to store SAS token to Azure Key Vault
Connect-AzAccount
$context = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
$sas = New-AzStorageAccountSASToken -Service Blob,File,Table,Queue -ResourceType Service,Container,Object -Permission "racwdlup" -Context $context
$vaule =ConvertTo-SecureString -String $sas -AsPlainText -Force
$secret =Set-AzKeyVaultSecret -VaultName testkey08 -Name test02 -SecretValue $vaule
$secret.SecretValueText
Related
I have a storage account in Azure which hosts a data lake. I want to authorize a specific directory with SAS token and I was able to do configure it by clicking in the portal.
First of all, I created a stored access policy lets call it "external1". The policy does not define a permission or an expiry date, it is just used to be able to revoke the SAS token before the token expires.
After that, I navigated to the container "axexternal" to the directory "/external1/central" and generated a SAS token, defining the stored access policy, permissions and expiration date:
These steps worked as expected.
I need to re-create those SAS token automatically. I chose to use an Automation Account (Authorization by its Identity) and use Powershell script to execute the recreation. In detail, I recreate the Storage Access Key first, and then recreate the SAS token. Since the recreation of the Storage Access Key worked like a charm, I focus on the code of the SAS token recreation.
Unfortunatly the documenation for SAS token is technically available but poor. I am not sure which of the commandlets I have to use to get the same result as I had in the Azure Portal.
Is it New-AzStorageAccountKey, New-AzStorageContainerSASToken, New-AzStorageBlobSASToken?
None of the possible combination of parameters in the documentation seems to fit for my needs.
I need to pass these parameters to the appropriate commandllet:
Which storage access key to use for encryption
Stored Access Policy to use
Expiration Date
Permissions
Moreover, I am not able to understand the purpose of the -Context parameter in these commandlets. Is this context used to connect to the storage and execute the script for the creation of the SAS token or is it used to pass parameters to the commandlet?
I tried many variations to achieve the goal, but I failed. Is there anybody able to give me some hints please?
Here is some code I tried:
$ctx = New-AzStorageContext `
-StorageAccountName $storageAccount `
-StorageAccountKey $key.Value `
-Protocol "Https"
$uri = New-AzStorageBlobSASToken `
-Context $ctx `
-Container $container `
-Blob "/external1/central" `
-Policy $policy
-StartTime (Get-Date).AddDays(-1) `
-ExpiryTime (Get-Date).AddDays(370) `
-FullUri
Update
I recognized that my post was maybe too unspecific. I want to supplement it with a specific question.
Given
A data lake container named "axexternal"
A directory "/external1/central"
A stored access policy "external1" which does not define permissions or expiry date
Wanted
A Powershell script which creates a SAS for the specific directory "/external1/central" in the container "axexternal"
SAS signed by account key
SAS uses stored access policy "external1"
SAS defines permissions
SAS defines expiration date
Thanks for your help, I really appreciate!
I tried in my environment and got successfully created azure SAS token with policy:
Initially I created access policy in portal like below:
I executed below command, and it created SAS token with URL successfully.
Command:
$accountname="venkat123"
$accountkey="<your storage account key >"
$containername="docker"
$blob="directory1/demo1mbimage.jpg"
$policy="external"
$ctx = New-AzStorageContext -StorageAccountName $accountName -StorageAccountKey $accountKey
New-AzStorageBlobSASToken `
-Context $ctx `
-Container $containername `
-Blob $blob `
-Policy $policy `
-FullUri
Console:
I checked the file URL with browser it worked perfectly.
Browser:
Reference:
New-AzStorageBlobSASToken (Az.Storage) | Microsoft Learn
After investigation I am now sure that currently it is not possible to get the exact same outcome using Powershell as you are able by using the Azure Portal.
The correct cmdlet to be used for my question is New-AzDataLakeGen2SasToken. That's the only way to be able to grant permission to a whole directory of a datalake.
Unfortunatly for this cmdlet there is no parameter to accept a stored access policy. For a datalake both APIs, datalake and BLOB is available for datalakes. As a consequence, the BLOB storage cmdlet New-AzStorageBlobSASToken works, but I didn't find a way to make it work on a directory. This seems to be consequent, since for BLOB storage, directories are just a part of a BLOB name.
Even if the permission to a single BLOB would be sufficient, it would not possible to use a Stored Access Policy which does not define permissions.
So I ended up using New-AzStorageBlobSASToken without using a Stored Access Policy. My plan for escaping problems when a SAS token got exploited is to rename the base folder and create new SAS tokens. This will be sufficient for my use case.
I use this kind of code now:
$container = "ext"
$centralPath = "external1/central"
$storageContext = New-AzStorageContext `
-StorageAccountName $storageAccount `
-StorageAccountKey $key.Value
New-AzDataLakeGen2SasToken `
-Context $storageContext `
-FileSystem $container `
-Path $centralPath `
-Permission "rdl" `
-StartTime (Get-Date).AddDays(-1).Date `
-ExpiryTime (Get-Date).AddDays(3).Date `
-Protocol Https `
-FullUri
I am trying to automate the creation of certain azure resources via an Azure PowerShell script that is triggered from an Azure DevOps release pipeline. I want to create a function app, and automatically integrate reading right access to secrets in an already existing Key Vault. This Key Vault is in the same Azure subscription.
While I can create most resources following the documentation, there seems to be a lack of documentation regarding the creation of certain resources using Azure PowerShell (or I can't find it).
If I follow the sample from this link, I can accomplish it without a problem by using the UI in the Azure Portal, but I can't find any documentation on Microsoft Docs to do it using PowerShell.
Write-Host "Creating Function App..."
$fnApp = New-AzFunctionApp -Name $functionAppName `
-ResourceGroupName $emailFunctionRg `
-Location "$(AzureRegion)" `
-StorageAccount $storageName `
-Runtime dotnet `
-FunctionsVersion '3' `
-IdentityType SystemAssigned
Write-Host "Function App created!"
Write-Host "Assigning Key Vault access..."
$appId = Get-AzADServicePrincipal -DisplayName $functionAppName
Set-AzKeyVaultAccessPolicy -VaultName EmailSettings -ServicePrincipalName $appId -PermissionsToSecrets Get,List
Write-Host "Key Vault access granted!"
Running Set-AzKeyVaultAccessPolicy fails with "Insufficient privileges to complete the operation.". But I am not sure if this is the right path to follow, it was just a guess, based on the available functions in the documentation.
Any ideas?
Two potential issues to check out here:
your app creation assigns the result to $fnApp. perhaps $fnApp or as commented above, $fnApp.ApplicationId is what you should be using for the -ServicePrincipalName parameter on the access policy grant.
you don't have privileges to assign RBAC roles. Go to the Key Vault, choose Access Control, then click the Role Assignments tab and verify that your user appears in the list as an Administrator, User Access Administrator, or Owner.
Edit: With respect to the RBAC privilege, since this is running in Azure Powershell from Azure DevOps, you need to check the role assignment for the Service Connection's service principal - under Azure Active Directory in the Azure Portal, look up the principal used to create the service connection, and make sure THAT gets the correct Role on the key vault.
After a little of trial and error I just came to the conclusion I was not using the right parameter for the Set-AzKeyVaultAccessPolicy cmdlet.
The following script will work (if the service principle running it has the appropriate role, like WaitingForGuacamole mentioned in his/her answer):
Write-Host "Creating Function App..."
$fnApp = New-AzFunctionApp -Name <FnAppName> `
-ResourceGroupName <ResourceGroupName> `
-Location <AzureRegion> `
-StorageAccount <StorageAccount> `
-Runtime dotnet `
-FunctionsVersion '3' `
-IdentityType SystemAssigned
Write-Host "Function App created!"
Write-Host "Assigning Key Vault access..."
Set-AzKeyVaultAccessPolicy -VaultName <NameOfTheKeyVault> -ObjectId (Get-AzADServicePrincipal -DisplayName <FnAppName>).Id -PermissionsToSecrets <Get, List, etc...>
Write-Host "Key Vault access granted!"
I need to upload files to an storage account without using SAS.
I create an "app registration" and give contributor access to the storage account.
If I want to upload files from powershell? How can I do it?
First az login? and then azcopy? Because I tried this way but ask me for a token
The Azure Powershell, Azure CLI and AzCopy are three different things, you should not mix them together.
If you want to use powershell to upload file with the service principal, after you create the App Registration, please get values for signing in, then create a new application secret.
In your storage account, the Contributor role is enough, but you should note, actually the Contributor does not have the permission to access the blob directly, it just let you get the context of the storage account, then use the context to access the blob, to access the blob directly, we need the Storage Blob Data Owner/Contributor as mentioned in the comment.
Then use the script below(the Get-Credential in another reply is an interactive way, here is a non-interactive way, usually, we use the service principal in a non-interactive way for the automation)
$azureAplicationId ="<Application-ID>"
$azureTenantId= "<Tenant-ID>"
$azurePassword = ConvertTo-SecureString "<Application-secret>" -AsPlainText -Force
$psCred = New-Object System.Management.Automation.PSCredential($azureAplicationId , $azurePassword)
Connect-AzAccount -Credential $psCred -TenantId $azureTenantId -ServicePrincipal
$context = (Get-AzStorageAccount -ResourceGroupName <group-name> -Name <storageaccount-name>).Context
Set-AzStorageBlobContent -Container <container-name> -File <localfile-path> -Blob <blob-name> -Context $context
You can use a Service Principal or a certificate to login using azcopy, and then copy your files to the storage account. Please reference this article for further information.
There are lots of ways to do this. Since you mentioned PowerShell, I'll use it in this example.
# The following assumes you have already created the App registration.
# Either through the portal, PS, whatever.
$creds = Get-Credential
# Uername = Application ID
# password = The service principle secret. You need to create this.
Connect-AzAccount `
-Credential $creds `
-Tenant $tenantId `
-ServicePrincipal
# you will need to get the storage accounts context.
# There are a few ways to do this I usually just get the storage account
$context = (Get-AzStorageAccount -Name $saName -ResourceGroupName $rgName).Context
# You will need to give the App Registration permissions to upload the blob.
# If you don't assign permissions this cmdlet will fail.
Set-AzStorageBlobContent `
-Container $containerName `
-File $filePath `
-Blob $blobName `
-Context $context
I am trying to set the secrets inside my Azure Keyvault using the Azure Powershell Task in Azure DevOps.
I use the following code:
Set-AzureKeyVaultSecret -VaultName $KeyvaultName -Name $SecretName -SecretValue $secretvalue
With the names and the value all setup inside variables and tried to use this without also variables.
The value is saved as a secure string with the following code.ConvertTo-SecureString
But when I run this powershell code inside my Azure DevOps Release pipeline I keep getting following Error message:
Cannot retrieve access token for resource 'AzureKeyVaultServiceEndpointResourceId'. Please ensure that you have provided the appropriate access tokens when using access token login.
So I've made sure that the service principal and the build server are having the right access on the keyvault by adding them both to the access policies with the get,list,set secrets permission.
I've also added following lines of code to make sure that the profile is loaded correctly
########################################################################################
$azureRmProfile = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile
$profileClient = New-Object -TypeName Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient -ArgumentList ($azureRmProfile)
$context = Get-AzureRmContext
$AzureToken = $profileClient.AcquireAccessToken($context.Tenant.Id)
Add-AzureRmAccount -AccessToken $AzureToken.AccessToken -AccountId $AzureToken.UserId
########################################################################################
By adding this code in the beginning of the inline script and using the profile with the commando as variable to the -DefaultProfile.
I also enabled the option to enable the script to access the Oauth token.
Is there someone who also tried to set the secret from the powershell task in Azure DevOps. Or know why the powershell script can't get access on the keyvault.
The azurermContext commando provided me with the right output, even tried the Get-AzureRmKeyvault command to figure out if the connection to the environment was already setup right. And that also didn't gave any problems.
below working for sure (using this reguarly)
Set-AzContext -SubscriptionId $SubscriptionId
## $SubscriptionId is a subscription ID where is the target KV
$Secretvalue = ConvertTo-SecureString $SecretValuePlainText -AsPlainText -Force
## $SecretValuePlainText is the secret to store
Set-AzKeyVaultSecret -VaultName $KeyVaultName -Name $SecretName -SecretValue $Secretvalue -ErrorVariable setSecretError -Expires $ExpirationDate -NotBefore $ActivationDate
## $SecretName, $ExpirationDate, $ActivationDate - obvious :)
of course if your refer to variable not from script or inline, but from release the use $(variable_name)
Service Principal/Service Connection we use for this is temporary an Owner of target subscription (or key vault, up to you).
I had the exact same issue.
Found that the problem was a missing access token.
Namely -KeyVaultAccessToken when you call Add-AzureRmAccount.
Found the solution here: https://github.com/Azure/azure-powershell/issues/4818#issuecomment-376155173
I fixed my question with the following.
I used a service connection that was based on a managed identity. And this needed some workaround to access the key vault like #john mentioned. But this was unnecessary. by creating a new service connection based on a service principal. This workaround was not necessary and fixed the issue.
Is there a way in Azure to create keyvault without any user? I am trying to follow the documentation but dont see any command that will achieve this in one line?
New-AzureRmKeyVault -VaultName my-test -ResourceGroupName abc -Location "Brazil South"
Any powershell command I can add above to not create any access policy or principal user?
You can use a template to create a vault with no access policy - see: https://github.com/Azure/azure-quickstart-templates/blob/master/101-key-vault-create/azuredeploy.json and just remove this: https://github.com/Azure/azure-quickstart-templates/blob/master/101-key-vault-create/azuredeploy.json#L105-L114
You can deploy that with PowerShell using New-AzureRMResourceGroupDeployment.
The New-AzureRmKeyVault create keyvault with access policy by default, if you want to achieve this in one line, you could use powershell pipeline, try the command below, it works fine on my side.
New-AzureRmKeyVault -VaultName '<VaultName>' -ResourceGroupName '<ResourceGroupName>' -Location 'East US' | Remove-AzureRmKeyVaultAccessPolicy -ObjectId "<ObjectId of your signin user or service principal>"
Update:
I suppose you create the keyvault via a user account, you could get the ObjectId via Get-AzureRmADUser,-UserPrincipalName is your account.
Complete command:
$ObjectId = (Get-AzureRmADUser -UserPrincipalName "xxxx#xxxx.com").Id
New-AzureRmKeyVault -VaultName '<VaultName>' -ResourceGroupName '<ResourceGroupName>' -Location 'East US' | Remove-AzureRmKeyVaultAccessPolicy -ObjectId $ObjectId