I am using the below powershell script to connect to azure datalake storage gen2 using my credentials for development and after deployment use managed identity credentials.
Code:
try {
$AzStorageContext = (Connect-AzAccount -Identity).context;
}
catch{
$AzStorageContext = (Connect-AzAccount).context;
}
$AzStorageContext = Set-AzContext -SubscriptionName $AzStorageContext.Subscription -DefaultProfile $AzStorageContext
Get-AzStorageContainer -Name 'MyContainer' -Context $AzStorageContext
The error I run into is
Error:
Get-AzStorageContainer : Cannot bind parameter 'Context'. Cannot convert the "Microsoft.Azure.Commands.Profile.Models.Core.PSAzureContext" value of type
"Microsoft.Azure.Commands.Profile.Models.Core.PSAzureContext" to type "Microsoft.Azure.Commands.Common.Authentication.Abstractions.IStorageContext".
Connecting Azure storage with system-assigned managed identity using Power Shell
$AzureContext = (Connect-AzAccount -Identity).context
$AzureContext = Set-AzContext -SubscriptionName $AzureContext.Subscription -DefaultProfile $AzureContext
For more information refer this SO thread by VenkateshDodda-MSFT
Related
I'm trying to use an Azure Automation account + accompanying powershell runbook to automate the process of creating an Azure Active Directory user.
When I run the following command, I'm presented with the error, am I trying to achieve the impossible here or is there an easy fix to this problem:
System.Management.Automation.ParameterBindingException: A parameter cannot be found that matches parameter name 'Surname'.
at System.Management.Automation.CmdletParameterBinderController.VerifyArgumentsProcessed(ParameterBindingException originalBindingException)
at System.Management.Automation.CmdletParameterBinderController.BindCommandLineParametersNoValidation(Collection`1 arguments)
at System.Management.Automation.CmdletParameterBinderController.BindCommandLineParameters(Collection`1 arguments)
at System.Management.Automation.CommandProcessor.BindCommandLineParameters()
at System.Management.Automation.CommandProcessorBase.DoPrepare(IDictionary psDefaultParameterValues)
at System.Management.Automation.Internal.PipelineProcessor.Start(Boolean incomingStream)
at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object input)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object input)
at System.Management.Automation.PipelineOps.InvokePipeline(Object input, Boolean ignoreInput, CommandParameterInternal[][] pipeElements, CommandBaseAst[] pipeElementAsts, CommandRedirection[][] commandRedirections, FunctionContext funcContext)
at System.Management.Automation.Interpreter.ActionCallInstruction`6.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
My runbook has the following script:
Param
(
[parameter(Mandatory=$true)]
[string] $firstname,
[parameter(Mandatory=$true)]
[string] $lastname,
[parameter(Mandatory=$true)]
[string] $city,
[parameter(Mandatory=$true)]
[string] $phone,
[parameter(Mandatory=$true)]
[string] $pw,
[string]$method,
[string]$UAMI
)
$displayname = $firstname + " " + $lastname
$upn = "$firstname.$lastname" + "#aguafriawindowslive.onmicrosoft.com"
#Secret Password
$secureStrPassword = ConvertTo-SecureString -String $pw -AsPlainText -Force
$automationAccount = "automationaccount01"
# Ensures you do not inherit an AzContext in your runbook
Disable-AzContextAutosave -Scope Process | Out-Null
# Connect using a Managed Service Identity
try {
$AzureContext = (Connect-AzAccount -Identity).context
}
catch{
Write-Output "There is no system-assigned user identity. Aborting.";
exit
}
# set and store context
$AzureContext = Set-AzContext -SubscriptionName $AzureContext.Subscription `
-DefaultProfile $AzureContext
if ($method -eq "SA")
{
Write-Output "Using system-assigned managed identity"
}
elseif ($method -eq "UA")
{
Write-Output "Using user-assigned managed identity"
# Connects using the Managed Service Identity of the named user-assigned managed identity
$identity = Get-AzUserAssignedIdentity -ResourceGroupName $resourceGroup `
-Name $UAMI -DefaultProfile $AzureContext
# validates assignment only, not perms
if ((Get-AzAutomationAccount -ResourceGroupName $resourceGroup `
-Name $automationAccount `
-DefaultProfile $AzureContext).Identity.UserAssignedIdentities.Values.PrincipalId.Contains($identity.PrincipalId))
{
$AzureContext = (Connect-AzAccount -Identity -AccountId $identity.ClientId).context
# set and store context
$AzureContext = Set-AzContext -SubscriptionName $AzureContext.Subscription -DefaultProfile $AzureContext
}
else {
Write-Output "Invalid or unassigned user-assigned managed identity"
exit
}
}
else {
Write-Output "Invalid method. Choose UA or SA."
exit
}
#Create User
New-AzADUser -DisplayName $displayname -UserPrincipalName $upn -Surname $lastname -City $city -Password $secureStrPassword -MailNickname $firstname -ForceChangePasswordNextLogin
Insufficient privileges error occurs when you have missed giving role required to do operation on the resources.Ensure that your runbook account has permissions to access any resources used in your script.
Try this:
Go to Azure portal --> Azure AD --> roles and Administrator-->Directory Readers role --> assign this role to the your runbook account name.
or
Try to add the application permissions > Directory.Read.All in for the azure ad App of your automation run as account and also Directory.ReadWrite.All if required and grant admin consent to it.
I am using below powershell command to download TSV file from ADL to local system and it works absolutely fine
#this appid has access to ADL
[string] $AppID = "bbb88818-aaaa-44fb-q2345678901y"
[string] $TenantId = "ttt88888-xxxx-yyyy-q2345678901y"
[string] $SubscriptionName = "Sub Sample"
[string] $AzureDataLakeAccountName = "sample.blob.core.windows.net"
[string] $AzureDataLakeSrcFilePath = "/accounts/sample/test.tsv"
[string] $LocalTargetFilePathName = "D:\MoveToModern"
Write-Host "AppID = " $AppID
Write-Host "TenantId = " $TenantId
Write-Host "SubscriptionName = " $SubscriptionName
Write-Host "AzureDataLakeAccountName = " AzureDataLakeAccountName
Write-Host "AzureDataLakeSrcFilePath = " $AzureDataLakeSrcFilePath
Write-Host "LocalTargetFilePathName = " $LocalTargetFilePathName
#this is the access key of the appid
$AccessKeyValue = "1234567=u-r.testabcdefaORYsw5AN5"
$azurePassword = ConvertTo-SecureString $AccessKeyValue -AsPlainText -Force
$psCred = New-Object System.Management.Automation.PSCredential($AppID, $azurePassword)
Login-AzureRmAccount -Credential $psCred -ServicePrincipal -Tenant $TenantId
Get-AzureRmSubscription
Get-AzureRmSubscription -SubscriptionName $SubscriptionName | Set-AzureRmContext
Export-AzureRmDataLakeStoreItem -AccountName $AzureDataLakeAccountName -Path $AzureDataLakeSrcFilePath -Destination $LocalTargetFilePathName -Force
But when I replace sourcefile path with value /accounts/sample/test-V4.parquet, I get below error:
ADLSException: Error in getting metadata for path /accounts/Partner/Non-PII/Account/Account-V4.parquet.
Operation: GETFILESTATUS failed with HttpStatus:Forbidden Error: Unexpected error in JSON parsing of the error stream. Content-Type of error response: application/xml.
ExceptionType: Newtonsoft.Json.JsonReaderException ExceptionMessage: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
AuthenticationFailedServer failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:de2e4b16-001e-002e-625f-c92785000000
Can you please let me know how can I download parquet file from ADL Gen2 using powershell.
According to the information you provide, you do not have enough permissions to do that. So you get Forbidden error. If you want to download a file from Azure data lake store with a service principal, we need to grant the security principal read access to the file and give the security principal Execute permissions to the container and each folder in the hierarchy of folders that lead to the file.
For example
Please check it and set the right ACL. Regarding how to do that, please refer to here
If you want to download files from Azure Data Lake Gen2, I suggest you use PowerShell module Az.Storage. For more details, please refer to here
For example
Configure ACL for the service principal.
If you want to download a file from Azure data lake Gen2 with a service principal, we need to grant the security principal read access to the file and give the security principal Execute permissions to the container and each folder in the hierarchy of folders that lead to the file. Regarding how to configure it, please refer to here.
Script
$AppID = ""
$AccessKeyValue = ""
$TenantId=""
$azurePassword = ConvertTo-SecureString $AccessKeyValue -AsPlainText -Force
$psCred = New-Object System.Management.Automation.PSCredential($AppID, $azurePassword)
Connect-AzAccount -Credential $psCred -ServicePrincipal -Tenant $TenantId
$AzureDataLakeAccountName = ""
$ctx =New-AzStorageContext -StorageAccountName $AzureDataLakeAccountName -UseConnectedAccount
$filesystemName="<your container name>"
$path="<your blob name>"
$LocalTargetFilePathName = ""
Get-AzDataLakeGen2ItemContent -Context $ctx -FileSystem $filesystemName -Path $path -Destination $LocalTargetFilePathName
I am figure out the commands in Azure PowerShell to add an the User.Read Ape Permission to my App Registration in Azure.
I can find some examples using *Azure, but would prefer one that uses the *Az commands, e.g. https://learn.microsoft.com/en-us/powershell/azure/?view=azps-2.8.0.
Wonder if anybody knows how to do this? Thanks!
This can currently only be achieved using the Azure AD PowerShell. Please note that there is a difference between Azure AD PowerShell and Azure PowerShell. The Azure AD PowerShell is not simply the old Azure PowerShell module.
Azure AD PowerShell is a separate module. There is no "AZ*" for Azure AD yet. Only couple of most commonly used commands, that have Azure Resource Provider implementation.
Azure PowerShell has a limited set of features for working with Azure AD. If you need more features, like the one you mention, you must use Azure AD PowerShell. Azure AD PowerShell is not depricated and is the officially supported PowerShell module for working with Azure AD.
You can manage these required permissions by the Set-AzureAdApplication cmdlet and passing proper -RequiredResourceAccess object.
In order to construct this object, you must first get a reference to "exposed" permissions. Because permissions are exposed by other service principals.
as I cannot upload whole file, here is a PowerShell script that creates a sample application with required permission to some MS Graph and some Power BI permissions.
Function GetToken
{
param(
[String] $authority = "https://login.microsoftonline.com/dayzure.com/oauth2/token",
[String] $clientId,
[String] $clientSecret,
[String] $resourceId = "https://graph.windows.net"
)
$scope = [System.Web.HttpUtility]::UrlEncode($resourceId)
$encSecret = [System.Web.HttpUtility]::UrlEncode($clientSecret)
$body = "grant_type=client_credentials&resource=$($scope)&client_id=$($clientId)&client_secret=$($encSecret)"
$res = Invoke-WebRequest -Uri $authority -Body $body -Method Post
$authResult = $res.Content | ConvertFrom-Json
return $authResult.access_token
}
#`
# -RequiredResourceAccess #($requiredResourceAccess)
#
Function CreateChildApp
{
param (
[string] $displayName,
[string] $tenantName
)
# create your new application
Write-Output -InputObject ('Creating App Registration {0}' -f $displayName)
if (!(Get-AzureADApplication -SearchString $displayName)) {
$app = New-AzureADApplication -DisplayName $displayName `
-Homepage "https://localhost" `
-ReplyUrls "https://localhost" `
-IdentifierUris ('https://{0}/{1}' -f $tenantName, $displayName)
# create SPN for App Registration
Write-Output -InputObject ('Creating SPN for App Registration {0}' -f $displayName)
# create a password (spn key)
$appPwd = New-AzureADApplicationPasswordCredential -ObjectId $app.ObjectId
$appPwd
# create a service principal for your application
# you need this to be able to grant your application the required permission
$spForApp = New-AzureADServicePrincipal -AppId $app.AppId -PasswordCredentials #($appPwd)
}
else {
Write-Output -InputObject ('App Registration {0} already exists' -f $displayName)
$app = Get-AzureADApplication -SearchString $displayName
}
#endregion
return $app
}
Function GrantAllThePermissionsWeWant
{
param
(
[string] $targetServicePrincipalName,
$appPermissionsRequired,
$childApp,
$spForApp
)
$targetSp = Get-AzureADServicePrincipal -Filter "DisplayName eq '$($targetServicePrincipalName)'"
# Iterate Permissions array
Write-Output -InputObject ('Retrieve Role Assignments objects')
$RoleAssignments = #()
Foreach ($AppPermission in $appPermissionsRequired) {
$RoleAssignment = $targetSp.AppRoles | Where-Object { $_.Value -eq $AppPermission}
$RoleAssignments += $RoleAssignment
}
$ResourceAccessObjects = New-Object 'System.Collections.Generic.List[Microsoft.Open.AzureAD.Model.ResourceAccess]'
foreach ($RoleAssignment in $RoleAssignments) {
$resourceAccess = New-Object -TypeName "Microsoft.Open.AzureAD.Model.ResourceAccess"
$resourceAccess.Id = $RoleAssignment.Id
$resourceAccess.Type = 'Role'
$ResourceAccessObjects.Add($resourceAccess)
}
$requiredResourceAccess = New-Object -TypeName "Microsoft.Open.AzureAD.Model.RequiredResourceAccess"
$requiredResourceAccess.ResourceAppId = $targetSp.AppId
$requiredResourceAccess.ResourceAccess = $ResourceAccessObjects
# set the required resource access
Set-AzureADApplication -ObjectId $childApp.ObjectId -RequiredResourceAccess $requiredResourceAccess
Start-Sleep -s 1
# grant the required resource access
foreach ($RoleAssignment in $RoleAssignments) {
Write-Output -InputObject ('Granting admin consent for App Role: {0}' -f $($RoleAssignment.Value))
New-AzureADServiceAppRoleAssignment -ObjectId $spForApp.ObjectId -Id $RoleAssignment.Id -PrincipalId $spForApp.ObjectId -ResourceId $targetSp.ObjectId
Start-Sleep -s 1
}
}
cls
#globaladminapp
$clientID = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
$key = "****"
$tenantId = "aaaaaaaa-bbbb-xxxx-yyyy-aaaaaaaaaaaa";
$TenantName = "customdomain.com";
$AppRegName = "globaladminChild-0003";
$token = GetToken -clientId $clientID -clientSecret $key
Disconnect-AzureAD
Connect-AzureAD -AadAccessToken $token -AccountId $clientID -TenantId $tenantId
$appPermissionsRequired = #('Application.ReadWrite.OwnedBy', 'Device.ReadWrite.All', 'Domain.ReadWrite.All')
$targetServicePrincipalName = 'Windows Azure Active Directory'
#$appPermissionsRequired = #('Files.ReadWrite.All','Sites.FullControl.All','Notes.ReadWrite.All')
#$targetServicePrincipalName = 'Microsoft Graph'
$app = CreateChildApp -displayName $AppRegName -tenantName $TenantName
$spForApp = Get-AzureADServicePrincipal -Filter "DisplayName eq '$($AppRegName)'"
$appPermissionsRequired = #('Tenant.ReadWrite.All')
$targetServicePrincipalName = 'Power BI Service'
GrantAllThePermissionsWeWant -targetServicePrincipalName $targetServicePrincipalName -appPermissionsRequired $appPermissionsRequired -childApp $app -spForApp $spForApp
$appPermissionsRequired = #('Files.ReadWrite.All','Sites.FullControl.All','Notes.ReadWrite.All')
$targetServicePrincipalName = 'Microsoft Graph'
GrantAllThePermissionsWeWant -targetServicePrincipalName $targetServicePrincipalName -appPermissionsRequired $appPermissionsRequired -childApp $app -spForApp $spForApp
The interesting parts are around "apppermissionrequired" and "targetserviceprincipalname" variables.
I can't reply to Rolfo's comment directly as I don't have enough clout yet. While it's true it's not dead simple, it's possible to use both in the same session as of July 2021. Not sure this was always the case, or something was updated to allow it.
#Import modules if needed
$mList = #("AzureAD","Az.Resources","Az.Accounts")
foreach($m in $mList){if ((gmo -l $m).Count -eq 0){Install-Module -Name $m -AllowClobber -Scope CurrentUser -Force}}
#Authentication Popup
Connect-AzAccount
#Use authentication context cached from above to authenticate to AAD graph
$IDObject = Get-AzAccessToken -Resource "https://graph.windows.net"
Connect-AzureAD -AadAccessToken $IDObject.token -AccountId $IDObject.UserId
UPDATE
With the new Graph API we can use the following command to add API permissions to an App Registration/Service Principal using PowerShell. It's much simpler than the old process.
Add-AzADAppPermission -ApplicationId "$spId" -ApiId "00000009-0000-0000-c000-000000000000" -PermissionId "7504609f-c495-4c64-8542-686125a5a36f"
(This is the case for the PowerBI API)
If deploying via an Azure Devops Pipeline I often recommend using the following script to authenticate into AAD:
echo "Install Azure AD module..."
Install-Module -Name "AzureAD" -Force
Import-Module AzureAD -Force
echo "Connect Azure AD..."
$context = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile.DefaultContext
echo $context
$graphToken = [Microsoft.Azure.Commands.Common.Authentication.AzureSession]::Instance.AuthenticationFactory.Authenticate($context.Account, $context.Environment, $context.Tenant.Id.ToString(), $null, [Microsoft.Azure.Commands.Common.Authentication.ShowDialog]::Never, $null, "https://graph.microsoft.com").AccessToken
echo $graphToken
$aadToken = [Microsoft.Azure.Commands.Common.Authentication.AzureSession]::Instance.AuthenticationFactory.Authenticate($context.Account, $context.Environment, $context.Tenant.Id.ToString(), $null, [Microsoft.Azure.Commands.Common.Authentication.ShowDialog]::Never, $null, "https://graph.windows.net").AccessToken
Write-Output "Hi I'm $($context.Account.Id)"
Connect-AzureAD -AadAccessToken $aadToken -AccountId $context.Account.Id -TenantId $context.tenant.id -MsAccessToken $graphToken
echo "Connection ends"
I am trying hands on on azure automation runbook, below is the small script i am using to create a storage account.
Param
(
[Parameter(Mandatory=$true)]
[String]
$AzureResourceGroup,
[Parameter(Mandatory=$true)]
[String]
$StorageAC,
[Parameter(Mandatory=$true)]
[String]
$Loc,
[Parameter(Mandatory=$true)]
[String]
$sku
)
$CredentialAssetName = "AutomationAccount";
$Cred = Get-AutomationPSCredential -Name $CredentialAssetName
if(!$Cred) {
Throw "Could not find an Automation Credential Asset named
'${CredentialAssetName}'. Make sure you have created one in this Automation
Account."
}
Add-AzureRmAccount -Credential $Cred
Add-AzureAccount -Credential $Cred
$storeac = Get-AzureRmStorageAccount -ResourceGroupName $AzureResourceGroup
if ($storeac.StorageAccountName -eq "testdd" ){
write-output " AC found !";
} else {
New-AzureRmStorageAccount -ResourceGroupName $AzureResourceGroup -Name
$StorageAC -Location $Loc -SkuName $sku
}
However, when ever i run it after publishing, the job is completed with an error
(New-AzureRmStorageAccount : A parameter cannot be found that matches parameter name 'SkuName')
Can someone tell me what am i doing wrong ??
To resolve the error :
New-AzureRmStorageAccount : A parameter cannot be found that matches parameter name 'SkuName'
Please update the Azure modules in you automation account and by clicking : "Update Azure Modules" under Automation Accounts => Modules and retry the runbook
Update Azure Modules
I've successfully created a self-signed certificate with application & service principle using the New-AzureRmADApplication and New-AzureRmADServicePrincipal cmdlets.
I can execute the login using this command after retrieving the certificate:
Login-AzureRmAccount -ServicePrincipal -CertificateThumbprint $cert.Thumbprint -TenantId $tenantID -ApplicationId $applicationID
However, the SubscriptionId/SubscriptionName attributes of this authentication display as blank:
Environment : AzureCloud
Account : ********************
TenantId : ********************
SubscriptionId :
SubscriptionName :
CurrentStorageAccount :
Subsquently, this command works!
$secret = Get-AzureKeyVaultSecret -VaultName $vaultName -Name $keyName
What is confusing to me is that I am able to retrieve a AzureKeyVaultSecret in my DEV subscription, but I do not understand how this cmdlet knows which of my subscriptions to use??? I intend to create the same vault in my PROD subscription, but first need to understand how this ServicePrincipal/Certificate authentication knows which subscription to pull from and/or how to manipulate it?
I can say that when I created the App/ServicePrincipal, I logged in specifying the "DEV" subscription like so:
$subscriptionName = "DEV"
$user = "user#company.com"
$password = "*****"
$securePassword = ConvertTo-SecureString $password -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential ($user, $securePassword)
Login-AzureRmAccount -Credential $credential -SubscriptionName $subscriptionName