Need to run app on N remotes hosts and handle them all - multithreading

Im using this code to run app on N remote hosts and need to wait until all of them are finished and get their results. But the execution just passes throuhg, all jobs are being marked Completed and code exits.
How to make it wait till apps finished their execution?
$procArray = [System.Collections.ArrayList]#()
foreach ($key in $simulatorServers.keys) {
$unitHost = $simulatorServers[$key][0]
$EXE="C:\app.exe"
Wr-DebugReg "Running $EXE on $unitHost "
$ScriptString="Start-Process -FilePath $EXE "
$ScriptBlock=[System.Management.Automation.ScriptBlock]::Create($ScriptString)
$Session=New-PSSession -ComputerName $unitHost -EnableNetworkAccess -Name "session$counter" -Credential $crNonProd
$rez2 = Invoke-Command -Session $Session -ScriptBlock $ScriptBlock -AsJob
$rez00=$procArray.Add($rez2)
Wr-DebugReg "Running process id=$($rez2.id) name=$($proc.Name)on $unitHost"
Wr-DebugReg ""
}
$procArray | Wait-Job
$procArray | Receive-Job
these jobs gone to status Completed even if launched processes still running

let invoke-command handle the the amount of jobs/sessions to open in parallel - you will receive 1 job with childs:
$scriptBlock = {
start-process -FilePath 'C:\app.exe'
}
$sessions = #(
foreach ($key in $simulatorServers.keys) {
$unitHost = $simulatorServers[$key][0]
New-PSSession -ComputerName $unitHost -EnableNetworkAccess -Name "session$counter" -Credential $crNonProd
}
)
$job = Invoke-Command -Session $Sessions -ScriptBlock $ScriptBlock -AsJob
$result = receive-job -Wait -Job $job
btw. I do not see, based on this sample, what you want to receive. you want to execute "start-process -FilePath 'C:\app.exe' " on the target machines but this won't give you anything back.
to get the information back modify the scriptblock like this:
$scriptBlock = {
$result = start-process -FilePath 'C:\app.exe' -wait -PassThru
return $result
}

This code is working. -Wait is the key to make it wait until all jobs are finished.
$procArray = [System.Collections.ArrayList]#()
foreach ($key in $hosts.keys) {
$unitHost = $hosts[$key][0]
$EXE="c:\app.exe"
$Session=New-PSSession -ComputerName $unitHost -EnableNetworkAccess -Name "session$counter" -Credential $crNonProd
$rez2 = Invoke-Command -Session $Session -ScriptBlock {Start-Process -FilePath $args[0] -Wait} -ArgumentList $EXE -AsJob
$rez00=$procArray.Add($rez2)
Wr-DebugReg "Starting process id=$($rez2.id) name=$($proc.Name)on $unitHost"
Wr-DebugReg ""
}
while ($procArray.State -contains 'Running')
{
Start-Sleep -Seconds 1
}

Related

How to use a variable ForegroundColor with write-host onto Start-job

I'm stuck into a little problem that I don't understand, here is a simplified version of my code :
[hashtable] $result_color = #{
OPENED = [ConsoleColor]::green;
FERME = [ConsoleColor]::red
}
$job = Start-Job -Name "test" -Scriptblock {
Write-Host "Test text" -ForegroundColor $result_color.OPENED
} -ArgumentList $result_color
Wait-Job $job | Out-Null
Receive-Job $job
I get the following error :
Unable to bind "ForegroundColor" parameter to target. Exception when defining "ForegroundColor": "Unable to convert Null value to "System.ConsoleColor" type due to invalid enumeration values."
I also tried this :
[hashtable] $result_color= #{
OPENED = [System.ConsoleColor] "green";
CLOSED = [System.ConsoleColor] "red"
}
$job = Start-Job -Name "test" -Scriptblock {
Write-Host "Test text" -ForegroundColor $result_color.OPENED
} -ArgumentList $result_color
Wait-Job $job | Out-Null
Receive-Job $job
But same error... Thanks in advance for your help !
Either do what Mathias commented, OR add a param block into your scriptblock and use the -ArgumentList parameter to feed it the correct value like
$job = Start-Job -Name "test" -Scriptblock {
param($color)
Write-Host "Test text" -ForegroundColor $color
} -ArgumentList $result_color.OPENED

Issues having a retry count work in powershell

I have a Powershell script that checks on a set of VM status before starting them. If the VM'S are in deallocating mode there should be a sleep and retry on 30 seconds. The code does not do a a retry. The code does a vm start in batches on 2 for vm's with wilcards as mentioned below in an order.
Need help if possible
$ResName= "resvmtest"
$action="start"
if($action -eq "start"){
$vnames=#('*dom*','*DBs*','*')
foreach($vname in $vnames) {
Write-Host "Starting VM with "$vname
$vmList = Get-AzVM -ResourceGroupName $ResName -Name $vname -Status | Select-Object Name, PowerState, ResourceGroupName
do{
$batch = #{
Skip = 0
First = 2
}
do{
foreach($vm in ($vmList | Select-Object #batch)){
$Stoploop = $false
[int]$Retrycount = "0"
do {
try {
if($vm.PowerState -eq "VM Deallocated"){
Write-Host "Job completed"
$Stoploop = $true
}
}
catch {
if ($vm.PowerState -eq "VM Deallocatting") {
Write-Host "VM Still not Deallocated"
Start-Sleep -Seconds 10
$Retrycount = $Retrycount + 1
}
}
}
While ($Stoploop -eq $false)
$params = #($vm.Name, $vm.ResourceGroupName,$vm.PowerState)
$job = Start-Job -ScriptBlock {
param($ComputerName,$serviceName,$statuses)
Start-AzVM -Name $ComputerName -ResourceGroupName $serviceName
} -ArgumentList $params
}
Wait-Job -Job $job
Get-Job | Receive-Job
$batch.Skip += 2
}
until($batch.skip -ge $vmList.count)
}
while($job.state -ne "Completed")
}
}
If you just want to wait for a VM's PowerState turns to VM Deallocated from VM Deallocatting, try the code below:
$vms = Get-AzVm -Status| Select-Object Name, PowerState,ResourceGroupName
#pick certain vm for demo here.
$vm = $vms[1]
$retryCount = 0
while($vm.PowerState -eq 'VM deallocating'){
Write-Host "waiting for VM deallocated..."
Start-Sleep -Seconds 5
$retryCount +=1
Write-Host "check count:$retryCount"
#get latest vm status
$vm.PowerState = (Get-AzVM -Name $vm.Name -ResourceGroupName $vm.ResourceGroupName -Status).Statuses[1].DisplayStatus
Write-Host "vm current state:"$vm.PowerState
}
Write-Host "vm new state:" + $vm.PowerState
Result:

PowerShell Mutex & Jobs

I'm trying to run a PowerShell file delete operation that deletes files based on specific parameters. So, the code goes like this:
$sb = {
Get-ChildItem -Path $SubFolder | Remove-ChildItem
if ($Error[0] -eq $null) {
$results = New-Object PSObject -Property #{
$Foldername = $Subfolder.Name
$TotalFiles = $SubFolder.(Count)
}
} else {
$errorresult = "Error deleting files from $($Subfolder.Name)"
}
#Error Mutex
$ErrorLogmutex = New-Object System.Threading.Mutex($false, "ErrorLogFileHandler")
$ErrorLogmutex.WaitOne()
Out-File -Append -InputObject $errorresult -FilePath $Errorlog
$ErrorLogmutex.ReleaseMutex()
#Success Mutex
$Successmutex = New-Object System.Threading.Mutex($false, "SuccessFileHandler")
$SuccessLogmutex.WaitOne()
Out-File -Append -InputObject $results -FilePath $successlog
$Successmutex.ReleaseMutex()
}
#Calling scriptblock in multi-thread count of 5
{
foreach ($Subfolder in $Folder) {
while ((Get-Job -State Running).Count -ge 5) {
Start-Sleep -Seconds 5
}
Start-Job -ScriptBlock $sb -ArgumentList $arguments | Out-Null
}
The script runs, able to see the results if I explicitly call out Receive-Job -Name JobID, but the output does not produce any log files as I would've thought it would.

Wait until all threads complete before running next task

I would wrap everything inside foreach($computer in $computers) in a Start-Job to make them run simultaneously. The only problem is, I need to wait for all the jobs to complete before I do the ConvertTo-Json at the bottom.
$sb = "OU=some,OU=ou,DC=some,DC=domain"
$computers = Get-ADComputer -Filter {(Enabled -eq $true)} -SearchBase "$sb" -Properties *
$hasmanufacturer = New-Object System.Collections.Generic.List[System.Object]
foreach($computer in $computers)
{
$drives = try{#(Get-WMIObject -Class Win32_CDROMDrive -Property * -ComputerName $computer.Name -ErrorAction Stop)} catch {$null}
foreach($drive in $drives)
{
if($drive.Manufacturer)
{
$hasmanufacturer.Add($computer)
continue
}
} # inner foreach
}
ConvertTo-Json $hasmanufacturer
Use a Get-Job | Wait-Job before executing the ConvertTo-Json
How about using the array of computer names as a parameter to Invoke-Command. It will run, by default, 32 concurrent remote sessions. The number can be changed with the -Throttle parameter.
$computers = Get-ADComputer -Filter {(Enabled -eq $true)} -SearchBase "OU=Servers,DC=xxx,DC=com" -Properties Name |
Where-Object { $_.Name -match 'LAX_*' } |
ForEach-Object { $_.Name }
$computers
$j = Invoke-Command `
-ComputerName $computers `
-ScriptBlock { Get-WMIObject -Class Win32_CDROMDrive -Property * -ErrorAction Stop } `
-AsJob
while ( (Get-Job -Id $j.Id).Status -eq 'Running') {}
Get-Job -Id $j.Id | Wait-Job
$results = Receive-Job -Id $j.Id
$results

Need to Multithread this script and log the results of each [duplicate]

I'm trying to run a virus scan on a list of servers in our environment. There are hundreds of machines, so we'd like to run the scan (using a command line prompt that we already have) around 10 at a time. We're totally new to PowerShell so any help would be really appreciated. We have a general idea of what commands we need to use -- here's how we think it might work for now:
$server = Get-Content "serverlist.txt"
$server | % {
$VirusScan = { Scan32.exe }
Invoke-Command -ScriptBlock { $VirusScan } -computerName $server -ThrottleLimit 10 -Authentication domain/admin
}
Does anyone have any suggestions on how we might orchestrate this?
I'm using something like this for running tasks in parallel on remote hosts:
$maxSlots = 10
$hosts = "foo", "bar", "baz", ...
$job = {
Invoke-Command -ScriptBlock { Scan32.exe } -Computer $ARGV[0] -ThrottleLimit 10 -Authentication domain/admin
}
$queue = [System.Collections.Queue]::Synchronized((New-Object System.Collections.Queue))
$hosts | ForEach-Object { $queue.Enqueue($_) }
while ( $queue.Count -gt 0 -or #(Get-Job -State Running).Count -gt 0 ) {
$freeSlots = $maxSlots - #(Get-Job -State Running).Count
for ( $i = $freeSlots; $i -gt 0 -and $queue.Count -gt 0; $i-- ) {
Start-Job -ScriptBlock $job -ArgumentList $queue.Dequeue() | Out-Null
}
Get-Job -State Completed | ForEach-Object {
Receive-Job -Id $_.Id
Remove-Job -Id $_.Id
}
Sleep -Milliseconds 100
}
# Remove all remaining jobs.
Get-Job | ForEach-Object {
Receive-Job -Id $_.Id
Remove-Job -Id $_.Id
}

Resources