Is it possible to share module between runspaces or assign a module to a runspace pool in Powershell? - multithreading

Question

I'm writing a multithreaded script which was leveraging the ability to multithread with runspaces and saving time until I had to use the Exchange module. It takes so long to load the Exchange module, it's almost as slow or slower than synchronous processing.
Below is the code I am working with, to repro you'll need to populate the servers array and the Exchange uri. At the bottom in quotes you can see the errors I'm receiving indicating the Exchange module is not being loaded and the Exchange cmdlets are not accessible.
Is there a way to copy the module to each new runspace and only initialize it one time or another way that I can significantly save time and reap the benefits of multiple runspaces with Exchange Powershell?
Get-Module | Remove-Module;
#Region Initial sessionstate
$servers = #("XXXXX", "XXXXX", "XXXXX");
$uri = 'https://XXXXX/powershell?ExchClientVer=15.1';
$session = new-pssession -ConfigurationName Microsoft.Exchange -ConnectionUri $uri -Authentication Negotiate;
$sessionImported = Import-PSSession $session -AllowClobber;
$sessionState = [System.Management.Automation.Runspaces.InitialSessionState]::Create($sessionImported);
$modules = Get-Module ;
foreach ($module in $modules)
{
if ($module.ExportedCommands.Keys -contains "get-mailbox")
{
$exchangeModule = $module;
}
}
$initialSessionState = [initialsessionstate]::CreateDefault()
$initialSessionState.ImportPSModule($exchangeModule)
#EndRegion Initial sessionstate
$RunspacePool = [runspacefactory]::CreateRunspacePool($initialSessionState);
$RunspacePool.Open();
foreach ($server in $servers)
{
$threads = New-Object System.Collections.ArrayList;
$PowerShell = [powershell]::Create($sessionState);
$PowerShell.RunspacePool = $RunspacePool;
[void]$PowerShell.AddScript({
Param ($server)
[pscustomobject]#{
server = $server
} | Out-Null
Set-ADServerSettings -ViewEntireForest:$true;
$mbs = Get-MailboxDatabase -Server $server -Status;
return $mbs;
}) # end of add script
$PowerShell.AddParameter('server', $server) | Out-Null;
$returnVal = $PowerShell.BeginInvoke();
$temp = "" | Select PowerShell,returnVal;
$temp.PowerShell = $PowerShell;
$temp.returnVal = $returnVal;
$threads.Add($Temp) | Out-Null;
} #foreach server
$completed = $false;
$threadsCompleted = New-Object System.Collections.ArrayList;
while ($completed -eq $false)
{
$completed = $true;
$threadsCompleted.Clear();
foreach ($thread in $threads)
{
$endInvoke = $thread.PowerShell.EndInvoke($thread.returnVal);
#$endInvoke;
$threadHandle = $thread.returnVal.AsyncWaitHandle.Handle;
$threadIsCompleted = $thread.returnVal.IsCompleted;
if ($threadIsCompleted -eq $false)
{
$completed = $false;
}
else
{
$threadsCompleted.Add($threadHandle) | Out-Null;
}
}
Write-Host "Threads completed count: " $threadsCompleted.Count;
foreach ($threadCompleted in $threadsCompleted)
{
Write-Host "Completed thread handle -" $threadCompleted;
}
#Write-Host "";
sleep -Milliseconds 1000;
} # while end
Write-Host "Seconds elapsed:" $stopwatch.Elapsed.TotalSeconds;
Write-Host "";
$temp.PowerShell.Streams.Error;
return;
Set-ADServerSettings : The term 'Set-ADServerSettings' is not
recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included,
verify that the path is correct and try again. At line:9 char:3
Set-ADServerSettings -ViewEntireForest:$true;
~~~~~~~~~~~~~~~~~~~~
CategoryInfo : ObjectNotFound: (Set-ADServerSettings:String) [], CommandNotFoundException
FullyQualifiedErrorId : CommandNotFoundException Get-MailboxDatabase : The term 'Get-MailboxDatabase' is not recognized
as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that
the path is correct and try again. At line:10 char:10
$mbs = Get-MailboxDatabase -Server $server -Status;
~~~~~~~~~~~~~~~~~~~
CategoryInfo : ObjectNotFound: (Get-MailboxDatabase:String) [], CommandNotFoundException
FullyQualifiedErrorId : CommandNotFoundException

Answer 1

So the answer is yes, I found out that you can use a property from initialsessionstate called ImportPSModule (which can accept a known module name from a static module or a file path to a dynamic or custom module), the tricky thing with Exchange is that that the module is created dynamically. There is hope though, if you capture the module the way I do below, there is a property to it called "Path" which gives you a local file path to where the module is temporarily stored on the machine after it is created the first time during that particular session.
I also discovered that you can specify a list of commands you want to import (commented out below) which saves an enormous amount of time with importing the Exchange module if you only need a few commands. Literally the progress bar for the Exchange command import only flashes when you filter on only a few commands and if you were to blink you'd miss it. If you use the specific command code make sure to import the get-mailbox command as that's how I filter on which module is for Exchange on prem, and configure the uri to your environment.
The benchmark for the single threaded version of this was 106 seconds, the fastest I've seen my code process with runspaces and multithreading is 9.8 seconds, so the potential for processing time savings is significant.
$WarningPreference = "SilentlyContinue";
Get-Module | Remove-Module;
#Region Initial sessionstate
$uri = 'https://xxxxx/powershell?ExchClientVer=15.1';
$session = new-pssession -ConfigurationName Microsoft.Exchange -ConnectionUri $uri -Authentication Negotiate;
Import-PSSession $session -AllowClobber > $null;
#Import-PSSession $session -AllowClobber -CommandName Get-Mailbox, Get-MailboxDatabaseCopyStatus, Get-MailboxDatabase, Get-DatabaseAvailabilityGroup > $null;
$modules = Get-Module;
foreach ($module in $modules)
{
if ($module.ExportedCommands.Keys -contains "get-mailbox")
{
$exchangeModule = $module;
}
}
$maxThreads = 17;
$iss = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
[void]$iss.ImportPSModule($exchangeModule.Path)
$runspacePool = [runspacefactory]::CreateRunspacePool(1, $maxThreads, $iss, $host)
$runspacePool.Open()
#Region Get dag members
$PowerShell = [powershell]::Create($iss); # may not need a runspace here
$PowerShell.RunspacePool = $RunspacePool;
$PowerShell.AddCommand("get-databaseavailabilitygroup") | Out-Null;
$PowerShell.AddParameter("identity", $DagName) | Out-Null;
$returnVal = $PowerShell.Invoke(); # writes to screen
$servers = New-Object System.Collections.ArrayList;
$serversUnsorted = $returnVal.Servers | sort;
foreach ($server in $serversUnsorted)
{
$servers.Add($server) | Out-Null;
}
#EndRegion Get dag members
#EndRegion Initial sessionstate
foreach ($server in $servers)
{
$PowerShell = [powershell]::Create($iss);
$PowerShell.RunspacePool = $RunspacePool;
[void]$PowerShell.AddScript({
Param ($server)
[pscustomobject]#{
server = $server
} | Out-Null
Set-ADServerSettings -ViewEntireForest:$true;
$copyStatus = Get-MailboxDatabaseCopyStatus -Server $server;
return $copyStatus;
}) # end of add script
$PowerShell.AddParameter('server', $server) | Out-Null;
$returnVal = $PowerShell.BeginInvoke();
$temp = "" | Select PowerShell,returnVal;
$temp.PowerShell = $PowerShell;
$temp.returnVal = $returnVal;
$threads.Add($Temp) | Out-Null;
} #foreach server