I'm trying to get metric from Azure to Zabbix.
The issue is that Metric for VM consists of 2 words:Percentage CPU, and Zabbix doesn't allow item keys to consists of 2 words. I also tried Percentage%20CPU but getting errors in Zabbix, and I created Zabbix key percentage_cpu.
So I decided prior sending data from Zabbix to Azure to "translate" percentage_cpu to Percentage%20CPU. This works great if only that key is present, but issue starts when I add another key (in this example SQL metric).
For SQL metric all values are in one word - no need to change anything, but then metric for VM is also assigned to SQL. I'm trying to avoid writing separate file for every service
$host_items = Get-ZabbixHostItems -url $zabbix_url -auth $zabbix_auth -
zabbix_host $host_name
foreach ($host_item in $host_items)
{
#$host_item_details = select-string -InputObject $host_item.key_ -Pattern '^(azure\.sql)\.(.*)\.(.*)\[\"(.*)\"\]$';
$host_item_details = select-string -InputObject $host_item.key_ -Pattern '^(azure\.\w{2,})\.(.*)\.(.*)\[\"(.*)\"\,(.*)]$';
#$host_item_details = select-string -InputObject $host_item.key_ -Pattern '^(azure)\.(.*)\.(.*)\.(.*)\[\"(.*)\"\,(.*)]$';
$host_item_provider = $host_item_details.Matches.groups[1];
$host_item_metric = $host_item_details.Matches.groups[2];
$host_item_timegrain = $host_item_details.Matches.groups[3];
$host_item_resource = $host_item_details.Matches.groups[4];
$host_item_suffix = $host_item_details.Matches.groups[5];
if ($host_item_metric='percentage_cpu')
{$host_item_metric='Percentage%20CPU'}
else
{ $host_item_metric = $host_item_details.Matches.groups[2];}
#}
$uri = "https://management.azure.com{0}/providers/microsoft.insights/metrics?api-version={1}&interval={2}×pan={3}&metric={4}" -f `
$host_item_resource, `
"2017-05-01-preview", `
$host_item_timegrain.ToString().ToUpper(), `
$($(get-date).ToUniversalTime().addminutes(-15).tostring("yyyy-MM-ddTHH:mm:ssZ") + "/" + $(get-date).ToUniversalTime().addminutes(-2).tostring("yyyy-MM-ddTHH:mm:ssZ")), `
$host_item_metric;
write-host $uri;
}
output of hostitems_
azure.sql.dtu_consumption_percent.pt1m["/subscriptions/111-222/resourceGroups/rg/providers/Microsoft.Sql/servers/mojsql/databases/test",common]
azure.vm.percentage_cpu.pt1m["/subscriptions/111-222/resourceGroups/rg/providers/Microsoft.Compute/virtualMachines/test",common]
When I ran code above I'm getting these URI's
https://management.azure.com/subscriptions/111-222/resourceGroups/rg/providers/Microsoft.Sql/servers/mojsql/databases/test/providers/microsoft.insights/m
etrics?api-version=2017-05-01-preview&interval=PT1M×pan=2018-08-11T07:38:05Z/2018-08-11T07:51:05Z&metric=Percentage%20CPU
https://management.azure.com/subscriptions/111-222/resourceGroups/rg/providers/Microsoft.Compute/virtualMachines/test/providers/microsoft.insights/metric
s?api-version=2017-05-01-preview&interval=PT1M×pan=2018-08-11T07:38:05Z/2018-08-11T07:51:05Z&metric=Percentage%20CPU
For first link (SQL) metric should be dtu_consumption but I'm getting same metric for both links
Second attempt:
if ($host_item_metric -eq 'percentage_cpu')
{$host_item_metric='Percentage%20CPU';}
else
{ $host_item_metric = $host_item_details.Matches.groups[2];}
write-host $host_item_metric
}
output: (original values)
dtu_consumption_percent
percentage_cpu
had to use -like
if ($host_item_metric -like 'percentage_cpu')
{$host_item_metric='Percentage%20CPU';}
else
{ $host_item_metric = $host_item_details.Matches.groups[2]}
Related
I'm trying to populate an employee ID column in a CSV file by querying Active Directory against another column in the CSV file called "OwnerEmail" (Which is the user principal name). The problem is users in the owneremail column do not all belong to the same domain. How can I query 2 domains at one time?
Table for reference
Employee ID
OwnerEmail
DeptNumber
Costs
test#mail.com
0894
4654.45
test2#mail.com
3453
4994.15
This is what I have tried so far. The script isn't working and there are no error messages. Any Ideas
$Domains='us.domain.corp', 'uk.domain.corp'
$CSVImport |Select-Object #{
Name = "employeeID"
Expression = {
foreach($user in $CSVImport)
{
foreach($Domain in $Domains){
$user= (Get-ADUser -Filter "UserPrincipalName -eq '$($_.OwnerEmail)'" -Server $Domain -Properties 'SamAccountName').SamAccountName
}
}
}}, * |Select-Object employeeID, DepartmentNumber, OwnerEmail, #{Name="Costs"; Expression={"$ $($_.Cost)"}} | Export-Csv "$Env:temp/$OutputFile" -NoTypeInformation
How can I query 2 domains at one time?
There is no need to do this, you can query both at once with multithreading but seems like an overkill. What I would recommend is to query all users at once per Domain, the code below may seem awfully complicated but should be pretty efficient. See the inline comments for details.
# Import the Csv
$CSVImport = Import-Csv path\to\thecsv.csv
# Create a LDAP Filter to query all users at once
# This filter would look like this for example:
# (|(userPrincipalName=test#mail.com)(userPrincipalName=test2#mail.com))
$filter = "(|"
foreach($email in $CSVImport.OwnerEmail) {
if(-not [string]::IsNullOrWhiteSpace($email)) {
$filter += "(userPrincipalName=$email)"
}
}
$filter += ")"
# For each Domain, use the same filter and get all existing users
'us.domain.corp', 'uk.domain.corp' | ForEach-Object { $map = #{} } {
foreach($user in Get-ADUser -LDAPFilter $filter -Server $_) {
# and store them in a hashtable where
# the Keys are their `UserPrincipalName`
# and the Values are the attribute of interest (`SamAccountName`)
$map[$user.UserPrincipalName] = $user.SamAccountName
}
}
# Now we can simply use a calculated property with `Select-Object`
$CSVImport | Select-Object #{N='EmployeeID'; E={ $map[$_.OwnerEmail] }}, * |
Export-Csv "$Env:temp/$OutputFile" -NoTypeInformation
I am new to Powershell. I am actually getting the details of the azure data factory linked services but after get I need to use contains to check if the element exists. In python I would just check if string in a list but powershell not quite sure. Please check the code below.
$output = Get-AzDataFactoryV2LinkedService -ResourceGroupName $ResourceGroupName -DataFactoryName "xxxxxxxx" | Format-List
The output of the below is :
sample output given below
LinkedServiceName : abcdef
ResourceGroupName : ghijk
DataFactoryName : lmnopq
Properties : Microsoft.Azure.Management.DataFactory.Models.AzureDatabricksLinkedService
So now I try to do this:
if ($output.Properties -contains "Microsoft.Azure.Management.DataFactory.Models.AzureDatabricksLinkedService") {
Write-Output "test output"
}
But $output.Properties gives us the properties of that json.
I need to check if "Microsoft.Azure.Management.DataFactory.Models.AzureDatabricksLinkedService" exists in output variable and perform the required operations. Please help me on this.
The -contains operator requires a collection and an element. Here's a basic example of its proper use:
$collection = #(1,2,3,4)
$element1 = 5
$element2 = 3
if ($collection -contains $element1) {'yes'} else {'no'}
if ($collection -contains $element2) {'yes'} else {'no'}
What you've done is ask PowerShell to look in an object that isn't a collection for an element of type [string] and value equal to the name of that same object.
What you need to do is inspect this object:
$output.Properties | format-list *
Then once you figure out what needs to be present inside of it, create a new condition.
$output.Properties.something -eq 'some string value'
...assuming that your value is a string, for example.
I would recommend watching some beginner tutorials.
Team, I had created new CosmosDB account in Azure portal with a container contains list of collection items. I am able to access Container details in power shell script.
How to list collection items or show specific collection item using partition key using power shell script
Power shell Script :
Get-AzResource -ResourceType "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers" -ApiVersion "2020-03-01" -ResourceGroupName "testRG" -Name "cosmosaccount1/database1/containercollection1"
You would need to use something like a thirdparty module for this. Azure Resource Manager doesnt support that, hence you need to talk to Cosmos DB directly.
https://github.com/PlagueHO/CosmosDB
The Cosmos DB repo has a set of examples to use Powershell: https://github.com/Azure/azure-cosmos-dotnet-v3/tree/master/Microsoft.Azure.Cosmos.Samples/Usage/PowerShellRestApi
Particularly to read Items: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos.Samples/Usage/PowerShellRestApi/PowerShellScripts/ReadItem.ps1
They are all using the REST API to do REST request, in this case, it is an authenticated GET to https://{databaseaccount}.documents.azure.com/dbs/{db-id}/colls/{coll-id}/docs/{id} (where databaseaccount is your account name, db-id is the id of your database, coll-id is the id of your collection/container, and id is your document id). It is also setting the x-ms-documentdb-partitionkey header for the partition key.
Like #4c74356b41 has indicated, you can use the CosmosDB module, which is now part of the official Az module.
Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force
You can see the available commands with Get-Commands:
Import-Module Az
Import-Module -Name CosmosDB
Get-Command -Module CosmosDB
Get all items in a collection
In order to get all entries inside a container, we use the Get-CosmosDbDocument command:
$subscription = "SubscriptionName"
$resourceGroupName = "ResourceGroupName"
$accountName = "AzureCosmosDBAccount"
$databaseName = "DatabaseName"
$cosmosContainer = "TargetCosmosDBContainer"
Set-AzContext $subscription
$backOffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 5 -Method Additive -Delay 1000
$cosmosDbContext = New-CosmosDbContext -Account $accountName -Database
$databaseName -ResourceGroup $resourceGroupName -BackoffPolicy $backOffPolicy
$documentsPerRequest = 100
$continuationToken = $null
$documents = $null
do {
$responseHeader = $null
$getCosmosDbDocumentParameters = #{
Context = $cosmosDbContext
CollectionId = $cosmosContainer
MaxItemCount = $documentsPerRequest
ResponseHeader = ([ref] $responseHeader)
}
if ($continuationToken) {
$getCosmosDbDocumentParameters.ContinuationToken = $continuationToken
}
$documents += Get-CosmosDbDocument #getCosmosDbDocumentParameters
$continuationToken = Get-CosmosDbContinuationToken -ResponseHeader $responseHeader
} while (-not [System.String]::IsNullOrEmpty($continuationToken))
Note: There is no apparent limitation on the number of documents that can be retrieved with this command, but it stands to reason that the command will have the API limitation, and this is 4 MB (as documented here). The value here ($documentsPerRequest = 100) could prove to be either too big or too small, depending on the size of each document. I usually don't use this parameter, but I've mentioned it here in case someone needs it.
List specific collection item
To get a specific entry or group of entries from a container, we use the same Get-CosmosDbDocument command, in a slightly different way:
$query = "SELECT * FROM c WHERE c.property = 'propertyValue'"
$documents = Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId $cosmosContainer -Query $query -QueryEnableCrossPartition $true
Note: For brevity I haven't went to the process of getting a continuation token, but if the query will return a result that is larger than 4 MB, then we will only receive the first part of the response. To make sure this does not happen we should add "Query" and "QueryEnableCrossPartition" in the $getCosmosDbDocumentParameters dictionary.
I am experimenting with Powershell runspaces and have noticed a difference in how output is written to the console depending on where I create my custom object. If I create the custom object directly in my script block, the output is written to the console in a table format. However, the table appears to be held open while the runspace pool still has open threads, i.e. it creates a table but I can see the results from finished jobs being appended dynamically to the table. This is the desired behavior. I'll refer to this as behavior 1.
The discrepancy occurs when I add a custom module to the runspace pool and then call a function contained in that module, which then creates a custom object. This object is printed to the screen in a list format for each returned object. This is not the desired behavior. I'll call this behavior 2
I have tried piping the output from behavior 2 to Format-Table but this just creates a new table for each returned object. I can achieve the desired effect somewhat by using Write-Host to print a line of the object values but I don't think this is appropriate considering it seems there is a built in behavior that can achieve my desired result if I can understand it.
My thoughts on the matter are that it has something to do with the asynchronous behavior of the runspace. I'm new to powershell but perhaps when the custom object comes directly from the script block there is a hidden method or type declaration telling powershell to hold the table open and wait for result? This would be overridden when using the second technique because its coming from my custom function?
I would like to understand why this is occurring and how I can achieve behavior 1 while being able to use the custom module, which will eventually be very large. I'm open to a different method technique as well, so long as its possible to essentially see the table of outputs grow as jobs finish. The code used is below.
$ISS = [InitialSessionState]::CreateDefault()
[void]$ISS.ImportPSModule(".\Modules\Test-Item.psm1")
$Pool = [RunspaceFactory]::CreateRunspacePool(1, 5, $ISS, $Host)
$Pool.Open()
$Runspaces = #()
# Script block to run code in
$ScriptBlock = {
Param ( [string]$Server, [int]$Count )
Test-Server -Server $Server -Count $Count
# Uncomment the three lines below and comment out the two
# lines above to test behavior 1.
#[int] $SleepTime = Get-Random -Maximum 4 -Minimum 1
#Start-Sleep -Seconds $SleepTime
#[pscustomobject]#{Server=$Server; Count=$Count;}
}
# Create runspaces and assign to runspace pool
1..10 | ForEach-Object {
$ParamList = #{ Server = "Server A" Count = $_ }
$Runspace = [PowerShell]::Create()
[void]$Runspace.AddScript($ScriptBlock)
[void]$Runspace.AddParameters($ParamList)
$Runspace.RunspacePool = $Pool
$Runspaces += [PSCustomObject]#{
Id = $_
Pipe = $Runspace
Handle = $Runspace.BeginInvoke()
Object = $Object
}
}
# Check for things to be finished
while ($Runspaces.Handle -ne $null)
{
$Completed = $Runspaces | Where-Object { $_.Handle.IsCompleted -eq $true }
foreach ($Runspace in $Completed)
{
$Runspace.Pipe.EndInvoke($Runspace.Handle)
$Runspace.Handle = $null
}
Start-Sleep -Milliseconds 100
}
$Pool.Close()
$Pool.Dispose()
The custom module I'm using is as follows.
function Test-Server {
Param ([string]$Server, [int]$Count )
[int] $SleepTime = Get-Random -Maximum 4 -Minimum 1
Start-Sleep -Seconds $SleepTime
[pscustomobject]#{Server = $Server;Item = $Count}
}
What you have mentioned sounds completely normal to me. That is how powershell is designed because it shares the burden of display. If the user has not specified how to display, PowerShell decides how to.
I couldn't reproduce your issue with the code provided but I think this will solve your problem.
$FinalTable = foreach ($Runspace in $Completed)
{
$Runspace.Pipe.EndInvoke($Runspace.Handle)
$Runspace.Handle = $null
}
$FinalResult will now have the table format you expect.
It appears that my primary issue, aside from errors in my code, was a lack of understanding related to powershell's default object handling. Powershell displays the output of objects as a table when there are less than four key-value pairs and as a list when there are more.
The custom object returned in my test module had more than for key-value pairs while the custom object I returned directly only had two. This resulted in what I thought was odd behavior. I compounded the issue by removing some key-value pairs in my posted code to shorten it and then didn't test it (sorry).
This stackoverflow post has a lengthy answer explaining the behavior some and providing examples for changing the default output.
Previously I have always had separate cloud projects for each environment, like this:
This poses some problems:
Maintaining multiple ServiceDefinition.csdef files
When building to a common output path, which ServiceDefinition.csdef is copied?
I am proposing using a single Cloud Project with multiple ServiceConfiguration files for each environment, and multiple profiles for publishing:
Pros:
Less maintenance issues (1 project and 1 ServiceDefinition.csdef)
A single ServiceDefinition.csdef is copied to the output folder
The problem I have now is that all environments need to have the same instance size as this is defined in the ServiceDefinition.csdef.
Is there any way I can get around this problem?
Yes, we create multiple packages (one for each environment). We have different powershell scripts to patch things like the vm-size (one example):
param(
[parameter(Mandatory=$true)]
[string]$fileToPatch,
[parameter(Mandatory=$true)]
[string]$roleName,
[parameter(Mandatory=$true)]
[validateSet("ExtraSmall", "Small", "Medium", "Large", "ExtraLarge", "A5", "A6", "A7")]
[string]$vmsize = 'Small'
)
# MAIN
$xml = New-Object System.Xml.XmlDocument
$xml.Load($fileToPatch)
$namespaceMgr = New-Object System.Xml.XmlNamespaceManager $xml.NameTable
$namespace = $xml.DocumentElement.NamespaceURI
$namespaceMgr.AddNamespace("ns", $namespace)
$xpathWorkerRoles = "/ns:ServiceDefinition/ns:WorkerRole"
$xpathWebRoles = "/ns:ServiceDefinition/ns:WebRole"
$Roles = $xml.SelectNodes($xpathWebRoles, $namespaceMgr) + $xml.SelectNodes($xpathWorkerRoles, $namespaceMgr)
$Roles | Where-Object { $_.name -eq $RoleName} | % { $_.vmsize = $vmsize; Write-Host 'Patched vmsize to' $vmsize 'for' $_.name }
$xml.Save($fileToPatch)