I am using Azure Devops pipeline to automate azure alert creation using ARM deployment tasks. In order to automate the threshold values for the alerts I created runtime parameters inside the pipeline, which need to be replaced by a replace token task and will update the corresponding runtime parameter values to the "parameters.json" file.
Here I am facing challenge like, in order to replace the values of runtime parameters to parameters.json, first I need to convert the parameter values to variable either by defining those variables under variable section or by using a bash task separately to output the variable value. I am looking for solutions for below requirements
Is there any any method to directly update the runtime parameter values to the "parameters.json"
Can we add sub values to parameter values? For example, only need to have certain run time parameter values if the alert category is logquery type-metricmeasurement and other set of values required if the type is "count type".
If you are trying to do it with a parameters.json it is a bit inconvenient. Why ? Because you have to parse the json and then update the desired nodes in json using PS/Bash script at runtime.
I will propose another solution to do this. Use template parameter object instead of parameters.json. Just call the ARM template using PS/Bash
example creating a VM-
$a = "abc"
$b = "efg"
$c = "hij"
$d = "klm"
$e = "nop"
$paramList =
#{
"Params1" = #{ customVmName = $a ; userImageStorageAccountName = $b ; adminUsername = $c ; adminPassword = $d ; osDiskVhdUri = $e }
}
foreach ($keys in $paramList.Keys)
{
$paramvalues = $paramList.$keys
New-AzureRmResourceGroupDeployment -ResourceGroupName "deploymentRG" -TemplateFile "YourARMTemplate.json" -TemplateParameterObject $paramValues
}
Coming to part to of your question :
Overriding the values at runtime in Azure DevOps
Parameterize the script and override the values while you call the PS file.
If you are using PS task in Azure DevOps just override the param values in the argument section-
If you are a PS/Bash guy you can do the manipulation in the part of script lets say setting $a="abc" based on a condition or env and for another env a different value, mean to say the overriding of a param value can be done as a code at runtime (Just another option)
Related
I am new to Powershell. I am actually getting the details of the azure data factory linked services but after get I need to use contains to check if the element exists. In python I would just check if string in a list but powershell not quite sure. Please check the code below.
$output = Get-AzDataFactoryV2LinkedService -ResourceGroupName $ResourceGroupName -DataFactoryName "xxxxxxxx" | Format-List
The output of the below is :
sample output given below
LinkedServiceName : abcdef
ResourceGroupName : ghijk
DataFactoryName : lmnopq
Properties : Microsoft.Azure.Management.DataFactory.Models.AzureDatabricksLinkedService
So now I try to do this:
if ($output.Properties -contains "Microsoft.Azure.Management.DataFactory.Models.AzureDatabricksLinkedService") {
Write-Output "test output"
}
But $output.Properties gives us the properties of that json.
I need to check if "Microsoft.Azure.Management.DataFactory.Models.AzureDatabricksLinkedService" exists in output variable and perform the required operations. Please help me on this.
The -contains operator requires a collection and an element. Here's a basic example of its proper use:
$collection = #(1,2,3,4)
$element1 = 5
$element2 = 3
if ($collection -contains $element1) {'yes'} else {'no'}
if ($collection -contains $element2) {'yes'} else {'no'}
What you've done is ask PowerShell to look in an object that isn't a collection for an element of type [string] and value equal to the name of that same object.
What you need to do is inspect this object:
$output.Properties | format-list *
Then once you figure out what needs to be present inside of it, create a new condition.
$output.Properties.something -eq 'some string value'
...assuming that your value is a string, for example.
I would recommend watching some beginner tutorials.
Original reference - Quit condition on Terraform blueprint
Is it still possible to make conditional check like in the above question
resource "null_resource" "condition_checker" {
count = "${var.variable == 1 ? 0 : 1}"
"Insert your custom error message" = true
}
Similar format does not work in terraform 0.12 and 0.13 and I could not find any reference to removal of this feature. Is it possible to make a check like this 0.12 or 0.13?
Currently it is still not possible to validate inputs that require access to more than a variable. (The validation block only allows access to the validated variable.)
A hacky validation is still possible using the external data source:
data "external" "check_valid" {
count = var.to_test == true && some_other_condition ? 1 : 0
program = ["sh", "-c", ">&2 echo Condition must be satisfied when to_test is true; exit 1"]
}
This condition is checked before terraform asks for approval of a plan.
On the output it looks like this:
Error: failed to execute "sh": Condition must be satisfied when to_test is true
on variables.tf line 1, in data "external" "check_valid":
1: data "external" "check_valid" {
What you're referring to here was never an actual Terraform feature, but rather an example of exploiting a bug in an earlier version of Terraform to get a result that Terraform had no explicit support for.
With that said, modern versions of Terraform have support for custom variable validation rules which allow you to write out variable validation checks directly inside the corresponding variable block. For example:
variable "variable" {
type = number
validation {
condition = var.variable == 1
error_message = "Variable value must always be 1."
}
}
With that said, I just copied your contrived example from the question here, so this would require some adaptation for a real example. Note also that variable validation rules can only depend on the variable value and other constants, so you can't use this for more complicated checks such as those which involve two different variables. For that sort of situation, I'd recommend refactoring so that the values that are related arrive in a single variable of a object type, and then the validation can be for whether that object is valid.
I'm trying to do a deployment using powershell. Both parameter and template file are stored in blob storage, but I get the error below before it even tries to download the blobs.
New-AzResourceGroupDeployment : A positional parameter cannot be found that accepts argument 'newdeployment-878059'. At C:\Temp\New-Deployment\deploy-core.ps1:86 char:1
The code I use is below
$vnetRG = "rg-vnet"
$vpnRG = "rg-vpn"
$fwRG = "rg-fw"
$btnRG = "rg-bastion"
$loc = "west europe"
New-AzResourceGroupDeployment -Name = "newdeployment-$random"-ResourceGroupName "rg-vnet" `
-TemplateParameterFile "$PSScriptRoot\core.parameters.json" -TemplateUri = $templateFileUri `
-vpnResourceGroupName $rgRG -vpnResourceGroupName $vpnRG -fwResourceGroupName $fwRG -btnResourceGroupName $btnRG
I'm trying to deploy multiple resources to various resource groups in one subscription.
Thanks in advance :)
Here my comment as answer
In your code, you have added an = character between some of the parameter names and the actual value to use for that parameter.
In PowerShell you assign a value to a parameter with a space character between the two.
Also, there are parameters used that (according to the docs) don't exist for that cmdlet, like vpnResourceGroupName, fwResourceGroupName and btnResourceGroupName. To me, they sound like variables mistakenly added as parameter with the leading $ stripped off?
For cmdlets that can potentially use a large number of parameters, I'd recommend using Splatting, to keep the code clean and easy to maintain.
Something like:
$splatParams = #{
Name = "newdeployment-$random"
ResourceGroupName = "rg-vnet"
TemplateParameterFile = "$PSScriptRoot\core.parameters.json"
TemplateUri = $templateFileUri
}
New-AzResourceGroupDeployment #splatParams
I have written below code powershell script to pass the get the value from the logic apps and pass it to Azure data factory.
#I am sharing a part of the code which is should pass the parameter to the data factory
# get the clientName value from the logic apps.
Param (
Param (
[Parameter (Mandatory = $true)]
[string]
$clientName
)
$parameters = #{
"clientName" = $clientName
}
#
$Pipeline1 = Invoke-AzureRmDataFactoryV2Pipeline -ResourceGroupName $rg1 -DataFactoryName $adf1 -PipelineName $pn1 -Parameter $parameters
But do not know how does my datafactory would get this value.
Created a pipeline parameter with the name as same you have used in the declared in automation runbook
clientName
then use this variable in notebook activities at base parameter area
I am experimenting with Powershell runspaces and have noticed a difference in how output is written to the console depending on where I create my custom object. If I create the custom object directly in my script block, the output is written to the console in a table format. However, the table appears to be held open while the runspace pool still has open threads, i.e. it creates a table but I can see the results from finished jobs being appended dynamically to the table. This is the desired behavior. I'll refer to this as behavior 1.
The discrepancy occurs when I add a custom module to the runspace pool and then call a function contained in that module, which then creates a custom object. This object is printed to the screen in a list format for each returned object. This is not the desired behavior. I'll call this behavior 2
I have tried piping the output from behavior 2 to Format-Table but this just creates a new table for each returned object. I can achieve the desired effect somewhat by using Write-Host to print a line of the object values but I don't think this is appropriate considering it seems there is a built in behavior that can achieve my desired result if I can understand it.
My thoughts on the matter are that it has something to do with the asynchronous behavior of the runspace. I'm new to powershell but perhaps when the custom object comes directly from the script block there is a hidden method or type declaration telling powershell to hold the table open and wait for result? This would be overridden when using the second technique because its coming from my custom function?
I would like to understand why this is occurring and how I can achieve behavior 1 while being able to use the custom module, which will eventually be very large. I'm open to a different method technique as well, so long as its possible to essentially see the table of outputs grow as jobs finish. The code used is below.
$ISS = [InitialSessionState]::CreateDefault()
[void]$ISS.ImportPSModule(".\Modules\Test-Item.psm1")
$Pool = [RunspaceFactory]::CreateRunspacePool(1, 5, $ISS, $Host)
$Pool.Open()
$Runspaces = #()
# Script block to run code in
$ScriptBlock = {
Param ( [string]$Server, [int]$Count )
Test-Server -Server $Server -Count $Count
# Uncomment the three lines below and comment out the two
# lines above to test behavior 1.
#[int] $SleepTime = Get-Random -Maximum 4 -Minimum 1
#Start-Sleep -Seconds $SleepTime
#[pscustomobject]#{Server=$Server; Count=$Count;}
}
# Create runspaces and assign to runspace pool
1..10 | ForEach-Object {
$ParamList = #{ Server = "Server A" Count = $_ }
$Runspace = [PowerShell]::Create()
[void]$Runspace.AddScript($ScriptBlock)
[void]$Runspace.AddParameters($ParamList)
$Runspace.RunspacePool = $Pool
$Runspaces += [PSCustomObject]#{
Id = $_
Pipe = $Runspace
Handle = $Runspace.BeginInvoke()
Object = $Object
}
}
# Check for things to be finished
while ($Runspaces.Handle -ne $null)
{
$Completed = $Runspaces | Where-Object { $_.Handle.IsCompleted -eq $true }
foreach ($Runspace in $Completed)
{
$Runspace.Pipe.EndInvoke($Runspace.Handle)
$Runspace.Handle = $null
}
Start-Sleep -Milliseconds 100
}
$Pool.Close()
$Pool.Dispose()
The custom module I'm using is as follows.
function Test-Server {
Param ([string]$Server, [int]$Count )
[int] $SleepTime = Get-Random -Maximum 4 -Minimum 1
Start-Sleep -Seconds $SleepTime
[pscustomobject]#{Server = $Server;Item = $Count}
}
What you have mentioned sounds completely normal to me. That is how powershell is designed because it shares the burden of display. If the user has not specified how to display, PowerShell decides how to.
I couldn't reproduce your issue with the code provided but I think this will solve your problem.
$FinalTable = foreach ($Runspace in $Completed)
{
$Runspace.Pipe.EndInvoke($Runspace.Handle)
$Runspace.Handle = $null
}
$FinalResult will now have the table format you expect.
It appears that my primary issue, aside from errors in my code, was a lack of understanding related to powershell's default object handling. Powershell displays the output of objects as a table when there are less than four key-value pairs and as a list when there are more.
The custom object returned in my test module had more than for key-value pairs while the custom object I returned directly only had two. This resulted in what I thought was odd behavior. I compounded the issue by removing some key-value pairs in my posted code to shorten it and then didn't test it (sorry).
This stackoverflow post has a lengthy answer explaining the behavior some and providing examples for changing the default output.