calling a user defined function inside the scriptblock in Powershell [duplicate] - multithreading

How do I Start a job of a function i just defined?
function FOO { write-host "HEY" } Start-Job -ScriptBlock { FOO } |
Receive-Job
Receive-Job: The term 'FOO' is not recognized as the name of cmdlet,
function ,script file or operable program.
What do I do?
Thanks.

As #Shay points out, FOO needs to be defined for the job. Another way to do this is to use the -InitializationScript parameter to prepare the session.
For your example:
$functions = {
function FOO { write-host "HEY" }
}
Start-Job -InitializationScript $functions -ScriptBlock {FOO}|
Wait-Job| Receive-Job
This can be useful if you want to use the same functions for different jobs.

#Rynant's suggestion of InitializationScript is great
I thought the purpose of (script) blocks is so that you can pass them around. So depending on how you are doing it, I would say go for:
$FOO = {write-host "HEY"}
Start-Job -ScriptBlock $FOO | wait-job |Receive-Job
Of course you can parameterize script blocks as well:
$foo = {param($bar) write-host $bar}
Start-Job -ScriptBlock $foo -ArgumentList "HEY" | wait-job | receive-job

It worked for me as:
Start-Job -ScriptBlock ${Function:FOO}

An improvement to #Rynant's answer:
You can define the function as normal in the main body of your script:
Function FOO
{
Write-Host "HEY"
}
and then recycle this definition within a scriptblock:
$export_functions = [scriptblock]::Create(#"
Function Foo { $function:FOO }
"#)
(makes more sense if you have a substantial function body) and then pass them to Start-Job as above:
Start-Job -ScriptBlock {FOO} -InitializationScript $export_functions| Wait-Job | Receive-Job
I like this way, as it is easier to debug jobs by running them locally under the debugger.

The function needs to be inside the scriptblock:
Start-Job -ScriptBlock { function FOO { write-host "HEY" } ; FOO } | Wait-Job | Receive-Job

A slightly different take. A function is just a scriptblock assigned to a variable. Oh, it has to be a threadjob. It can't be foreach-object -parallel.
$func = { 'hi' } # or
function hi { 'hi' }; $func = $function:hi
start-threadjob { & $using:func } | receive-job -auto -wait
hi

#Ben Power's comment under the accepted answer was my concern also, so I googled how to get function definitions, and I found Get-Command - though this gets only the function body. But it can be used also if the function is coming from elsewhere, like a dot-sourced file. So I came up with the following (hold my naming convention :)), the idea is to re-build the function definitions delimited by newlines:
Filter Greeting {param ([string]$Greeting) return $Greeting}
Filter FullName {param ([string]$FirstName, [string]$LastName) return $FirstName + " " + $LastName}
$ScriptText = ""
$ScriptText += "Filter Greeting {" + (Get-Command Greeting).Definition + "}`n"
$ScriptText += "Filter FullName {" + (Get-Command FullName).Definition + "}`n"
$Job = Start-Job `
-InitializationScript $([ScriptBlock]::Create($ScriptText)) `
-ScriptBlock {(Greeting -Greeting "Hello") + " " + (FullName -FirstName "PowerShell" -LastName "Programmer")}
$Result = $Job | Wait-Job | Receive-Job
$Result
$Job | Remove-Job

As long as the function passed to the InitializationScript param on Start-Job isn't large Rynant's answer will work, but if the function is large you may run into the below error.
[localhost] There is an error launching the background process. Error
reported: The filename or extension is too long"
Capturing the function's definition and then using Invoke-Expression on it in the ScriptBlock is a better alternative.
function Get-Foo {
param
(
[string]$output
)
Write-Output $output
}
$getFooFunc = $(Get-Command Get-Foo).Definition
Start-Job -ScriptBlock {
Invoke-Expression "function Get-Foo {$using:getFooFunc}"
Get-Foo -output "bar"
}
Get-Job | Receive-Job
PS C:\Users\rohopkin> Get-Job | Receive-Job
bar

Related

Access local function from a start-job scriptblock [duplicate]

This question already has answers here:
Reuse 2 functions in Start-ThreadJob
(1 answer)
How to pass a custom function inside a ForEach-Object -Parallel
(3 answers)
Closed last month.
I have some Powershell functions I want to modify to use multi-threading.
They follow a pattern where local functions are defined in the Begin block and consumed in the Process block.
When I use Start-Job to multi-thread I can no-longer access the local functions.
Is there a way of scoping the local functions to make them available?
or is there a better pattern to use for multi-threading advanced functions?
Note: I am using Powershell 5 so can't use foreach parallel.
EXAMPLE
function Test-Jobs {
[CmdletBinding()]
param (
[Parameter(ValueFromPipeline=$true)]
[string[]]$ComputerName
)
Begin {
Write-host "Begin Block"
function DoThing {
param($ComputerName)
# may be a long running function
start-sleep -Seconds 5
"Did a thing on $ComputerName"
}
$Jobs = #()
}
Process {
Write-host "Process Block"
foreach ($Name in $ComputerName) {
$Jobs += Start-Job -ScriptBlock {
DoThing $Using:Name
}
}
}
End {
Write-host "End Block"
Wait-Job $jobs
Receive-Job $jobs
}
}
OUTPUT
The term 'DoThing' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is
correct and try again.

How to use a variable ForegroundColor with write-host onto Start-job

I'm stuck into a little problem that I don't understand, here is a simplified version of my code :
[hashtable] $result_color = #{
OPENED = [ConsoleColor]::green;
FERME = [ConsoleColor]::red
}
$job = Start-Job -Name "test" -Scriptblock {
Write-Host "Test text" -ForegroundColor $result_color.OPENED
} -ArgumentList $result_color
Wait-Job $job | Out-Null
Receive-Job $job
I get the following error :
Unable to bind "ForegroundColor" parameter to target. Exception when defining "ForegroundColor": "Unable to convert Null value to "System.ConsoleColor" type due to invalid enumeration values."
I also tried this :
[hashtable] $result_color= #{
OPENED = [System.ConsoleColor] "green";
CLOSED = [System.ConsoleColor] "red"
}
$job = Start-Job -Name "test" -Scriptblock {
Write-Host "Test text" -ForegroundColor $result_color.OPENED
} -ArgumentList $result_color
Wait-Job $job | Out-Null
Receive-Job $job
But same error... Thanks in advance for your help !
Either do what Mathias commented, OR add a param block into your scriptblock and use the -ArgumentList parameter to feed it the correct value like
$job = Start-Job -Name "test" -Scriptblock {
param($color)
Write-Host "Test text" -ForegroundColor $color
} -ArgumentList $result_color.OPENED

Register trackingevent for all background jobs?

Good afternoon,
I've been working with trying to register an event based on when all jobs are completed. Im able to successfully register one, but id like to get a message pop-up once all background jobs are completed. Anyone familiar with how to do so?
I attempted the following, but it errors out saying jobs is null:
1..10 | ForEach-Object -Process {
Start-Job { Start-Sleep $_ } -Name "$_" | Out-Null} -OutVariable $jobs
Register-ObjectEvent $jobs StateChanged -Action {
[System.Windows.MessageBox]::Show('Done')
$eventSubscriber | Unregister-Event
$eventSubscriber.Action | Remove-Job
} | Out-Null
I feel like a Do{}Until() loop can do it but, im not sure how to register that to check until the job has completed. Also tried to follow along with some ways other people have done it using different languages, but, I cant pick it up.
I don't want to post everything ive tried so this post doesn't bore anyone. Searched on google as well but, I couldn't find much on registering an object for multiple jobs.
EDIT
Heres what does work:
$job = Start-Job -Name GetLogFiles { Start-Sleep 10 }
Register-ObjectEvent $job StateChanged -Action {
[System.Windows.MessageBox]::Show('Done')
$eventSubscriber | Unregister-Event
$eventSubscriber.Action | Remove-Job
} | Out-Null
Which is what id like to happened, but to evaluate all jobs, not just one.
This is what a personally use when monitoring running jobs:
$jobs= 1..10 | ForEach-Object -Process {
Start-Job { Start-Sleep $using:_ ; "job {0} done" -f $using:_ } -Name "$_"
}
do{
$i = (Get-Job -State Completed).count
$progress = #{
Activity = 'Jobs completed'
Status = "$i of {0}" -f $jobs.Count
PercentComplete = $i / $jobs.count * 100
}
Write-Progress #progress
Start-Sleep -Milliseconds 10
}
until($i -eq $jobs.Count)
$result = Get-Job | Receive-Job
$jobs | Remove-Job
Of course, under certain scenarios where I know some jobs might fail I change the until(...) condition for something different and the do {...} contains the logic for restarting failing jobs.
Edit 1:
It's worth mentioning that Start-Job is not worth your time if you're interested in multithreading, it has been proven to be slower than a linear loop in many scenarios. You should be looking at the ThreadJob Module
Edit 2:
After some testing, this worked for me:
# Clear the Event queue
Get-EventSubscriber|Unregister-Event
# Clear the Job queue
Get-Job|Remove-Job
1..10 | ForEach-Object -Process {
$job = Start-Job { Sleep -Seconds (1..20|Get-Random) } -Name "$_"
Register-ObjectEvent -InputObject $job -EventName StateChanged -Action {
$eventSubscriber | Unregister-Event
$eventSubscriber.Action | Remove-Job
if(-not (Get-EventSubscriber))
{
[System.Windows.MessageBox]::Show('Done')
}
} | Out-Null
}
At first I didn't even know this was possible so thanks for pointing this out. Great question :)

Passing relative paths of scripts to powershell jobs

I have functions in separate files I need to run as jobs in one main file.
I need to be able to pass these functions arguments.
Right now my problem is figuring out how to pass the path of the function files to the jobs in a way that is not completely awful.
I need to have the functions defined at the top of the file for readability (just having a static comment that says "script uses somefunc.ps1" is not adequate)
I also need to refer to the scripts relative path (they will all be in the same folder).
Right now I am using env: to store the path of the scripts, but doing this I need to refer to the script in like 5 places!
This is what I have:
testJobsMain.ps1:
#Store path of functions in env so jobs can find them
$env:func1 = "$PSScriptRoot\func1.ps1"
$env:func2 = "$PSScriptRoot\func2.ps1"
$arrOutput = #()
$Jobs = #()
foreach($i in ('aaa','bbb','ccc') ) {
$Import = {. $env:func1}
$Execute = {func1 -myArg $Using:i}
$Jobs += Start-Job -InitializationScript $Import -ScriptBlock $Execute
}
$JobsOutput = $Jobs | Wait-Job | Receive-Job
$JobsOutput
$Jobs | Remove-Job
#Clean up env
Remove-Item env:\func1
$arrOutput
func1.ps1
function func1( $myArg ) { write-output $myArg }
func2.ps1
function func2( $blah ) { write-output $blah }
You can simply make array of paths, and then pass one of paths/all of them in -ArgumentList param from Start-Job:
#func1.ps1
function add($inp) {
return $inp + 1
}
#func2.ps1
function add($inp) {
return $inp + 2
}
$paths = "$PSScriptRoot\func1.ps1", "$PSScriptRoot\func2.ps1"
$i = 0
ForEach($singlePath in $paths) {
$Execute = {
Param(
[Parameter(Mandatory=$True, Position=1)]
[String]$path
)
Import-Module $path
return add 1
}
Start-Job -Name "Job$i" -ScriptBlock $Execute -ArgumentList $singlePath
$i++
}
for ($i = 0; $i -lt 2; $i++) {
Wait-Job "Job$i"
[int]$result = Receive-Job "Job$i"
}
You can skip all those $i iterators with names, Powershell will name jobs automatically, and easly predictable: Job1, Job2.. So it would make code a lot prettier.

What is the best way to collect and transform output from multiple PowerShell threads?

I am new to PowerShell scripting and would like to do the following:
Given a list of config names and servers, return the values for the configs from each server.
Transform them in such a way to group them by config name, and not server.
Currently, I have a script that spawns one job per server and calls a script remotely on the server to return the list of configs for that server.
However, I do not know how to aggregate and transform the output from these jobs so that instead of getting config names by server, I would like to sort them by config name first, then server.
Current output:
Server1:
Config1 = 'abc'
Config2 = 'def'
Server2:
Config1 = 'xyz'
Config2 = '123'
Desired output:
Config1:
Server1 : 'abc'
Server2 : 'xyz'
Config2:
Server1 : 'def'
Server2 : '123'
I don't want to iterate over the config names because that would waste time in connecting to the server for every call. Therefore I'd like to iterate over the servers and do some kind of transformation.
I'm wondering if this is a matter of having each job return some kind of dictionary, then iterate over them after all the threads finish to transform?
Here is the code that calls the jobs:
$all_servers = #('server1', 'server2')
$config_names = #('config1', 'config2')
foreach($servername in $all_servers) {
Start-Job -FilePath C:\scripts\get_config_from_servers.ps1
-ArgumentList $servername,$config_names
}
Get-Job | Wait-Job
Get-Job | Receive-Job | Out-GridView
Here is the job script:
Param($servername,$config_names)
$session = Get-Session -computername $servername
-username $$$$
-pwd ####
try {
$sb = {
param($servername,$config_names)
$output = #{}
foreach ($cfg in $config_names) {
$config_value = Get-Config -configname $cfg
$output.Add("$servername : $cfg", "($config_value)")
}
write-host $output | Out-String
return $output | Out-String
}
$out = Invoke-Command -session $session
-ScriptBlock $sb
-ArgumentList $servername,$config_names
write-host $out
return $out
}
finally {
Remove-PSSession $session
}
Instead of making a hash table and converting to a string you could create some custom object in you job script just like this SO Question
Instead of this:
$output = #{}
foreach ($cfg in $config_names) {
$config_value = Get-Config -configname $cfg
$output.Add("$servername : $cfg", "($config_value)")
}
write-host $output | Out-String
return $output | Out-String
You could try something like this:
$output = New-Object System.Object
Add-Member -MemberType NoteProperty -Name Server -Value $servername -InputObject $output
foreach ($cfg in $config_names) {
$config_value = Get-Config -configname $cfg
Add-Member -MemberType NoteProperty -Name "Config$cfg" -Value $config_value -InputObject $output
}
write-host $output
return $output
I can't test this accurately as i'm not sure what Get-Config is but hopefully it should be enough to get you thinking.

Resources