PoshRsJob Performance Issue - multithreading

Question

Why is using Multithreading in PowerShell so unbelievable slow. Am I doing anything wrong? I am using the PoshRsJob Module.
RSJobs:
(Measure-Command {
$output = Start-RSJob -InputObject $shortDump -ScriptBlock {
Param($out, $shortDump)
$retObj = [pscustomobject]#{
UserMail = $_.Mail
Type = $_.Type
}
# return $retObj
$retObj
} | Wait-RSJob
$out.Add( $( Get-RSJob | Receive-RSJob) )
# $out += $( Get-RSJob | Receive-RSJob )
}).TotalSeconds
and
Standard foreach:
(Measure-Command {
foreach ($obj in $shortDump) {
$retObj = [pscustomobject]#{
UserMail =$obj.Mail
Type = $obj.Type
}
# $out+= $retObj
$out.Add($retObj)
}
}).TotalSeconds
My goal is to build objects faster, because i have ~ 300.000 objects to build.
edit: Here is a another example. It's totally slow!
fast
$out = New-Object System.Collections.ArrayList
"default"
(Measure-Command {
for ($x = 0; $x -lt 100000; $x++)
{
$retObj = [pscustomobject]#{
UserMail = 'test'
Type = 'test2'
Test = 'default'
}
$out.Add($retObj)
}
}).TotalSeconds
$out2 = $out
horribly slow
$out = New-Object System.Collections.ArrayList
$Test = `"RSJobs"`
"RSJobs"
$ScriptBlock = {
[pscustomobject]#{
UserMail = 'test'
Type = 'test2'
Test = $Using:Test
}
}
(Measure-Command {
1..100000 | Start-RSJob -Name {$_} -ScriptBlock $ScriptBlock
$out.Add( $( Get-RSJob | Receive-RSJob) )
}).TotalSeconds

Answer 1

Creating a new runspace has overhead. So with many small jobs then you are adding the overhead every single time.
(measure-command {[pscustomobject]#{'a'='b'}}).totalmilliseconds
0.1773
{start-rsjob -scriptblock {[pscustomobject]#{'a'='b'}}}).totalmilliseconds
93.0173
Then you are adding even more overhead retrieving all of the returned data from the individual jobs into one object, which was basically your goal in the first place.
Basically, build 1 object from 100,000 objects vs create a runspace 100,000 times each creating 1 object then return all of these objects to build 1 object from 100,000 objects.
I don't see how you are going to get any gain in efficiency using runspaces in this application. If there was an expensive calculation to determine each object, and then you made just a few runspaces and ran a subset of your array in each, maybe.