How to search for a string in multiple text files in an active running log - multithreading

Question

I am trying to search for a string in multiple text files to trigger an event. The log file is being actively added to by a program. The following script successfully achieves that goal, but it only works for one text file at a time:
$PSDefaultParameterValues = #{"Get-Date:format"="yyyy-MM-dd HH:mm:ss"}
Get-Content -path "C:\Log 0.txt" -Tail 1 -Wait | ForEach-Object { If ($_ -match 'keyword') {
Write-Host "Down : $_" -ForegroundColor Green
Add-Content "C:\log.txt" "$(get-date) down"
Unfortunately it means I have to run 3 instances of this script to search the 3 log files (C:\log 0.txt, C:\log 1.txt and C:'log 2.txt).
What I want to do is run one powershell script to search for that string across all three text files and not three.
I tried using a wildcard in the path ("C:\log*.txt)
I also tried adding a foreach loop:
$PSDefaultParameterValues = #{"Get-Date:format"="yyyy-MM-dd HH:mm:ss"}
$LogGroup = ('C:\log 0.txt', 'C:\Log 1.txt', 'C:\Log 2.txt')
ForEach ($log in $LogGroup) {
Get-Content $log -Tail 1 -Wait | ForEach-Object { If ($_ -match 'keyword') {
Write-Host "Down: $_" -ForegroundColor Green
Add-Content -path "C:\log.txt" "$(get-date) down"
Add-Content -path "C:\log.txt" "$(get-date) down"
}
}
}
This got me no errors but it also didn't work.
I saw others use Get-ChildItem instead of Get-Content but since this worked with one file... shouldn't it work with multiple? I assume it's my lack of scripting ability. Any help would be appreciated. Thanks.

Answer 1

This is how you can apply the same logic you already have for one file but for multiple logs at the same time, the concept is to spawn as many PowerShell instances as log paths there are in the $LogGroup array. Each instance is assigned and will be monitoring 1 log path and when the keyword is matched it will append to the main log file.
The instances are assigned the same RunspacePool, this help us initialize all with a SemaphoreSlim instance which help us ensure thread safety (only 1 thread can write to the main log at a time).
using namespace System.Management.Automation.Runspaces
using namespace System.Threading
# get the log files here
$LogGroup = ('C:\log 0.txt', 'C:\Log 1.txt', 'C:\Log 2.txt')
# this help us write to the main log file in a thread safe manner
$lock = [SemaphoreSlim]::new(1, 1)
# define the logic used for each thread, this is very similar to the
# initial script except for the use of the SemaphoreSlim
$action = {
param($path)
$PSDefaultParameterValues = #{ "Get-Date:format" = "yyyy-MM-dd HH:mm:ss" }
Get-Content $path -Tail 1 -Wait | ForEach-Object {
if($_ -match 'down') {
# can I write to this file?
$lock.Wait()
try {
Write-Host "Down: $_ - $path" -ForegroundColor Green
Add-Content "path\to\mainLog.txt" -Value "$(Get-Date) Down: $_ - $path"
}
finally {
# release the lock so other threads can write to the file
$null = $lock.Release()
}
}
}
}
try {
$iss = [initialsessionstate]::CreateDefault2()
$iss.Variables.Add([SessionStateVariableEntry]::new('lock', $lock, $null))
$rspool = [runspacefactory]::CreateRunspacePool(1, $LogGroup.Count, $iss, $Host)
$rspool.ApartmentState = [ApartmentState]::STA
$rspool.ThreadOptions = [PSThreadOptions]::UseNewThread
$rspool.Open()
$res = foreach($path in $LogGroup) {
$ps = [powershell]::Create($iss).AddScript($action).AddArgument($path)
$ps.RunspacePool = $rspool
#{
Instance = $ps
AsyncResult = $ps.BeginInvoke()
}
}
# block the main thread
do {
$id = [WaitHandle]::WaitAny($res.AsyncResult.AsyncWaitHandle, 200)
}
while($id -eq [WaitHandle]::WaitTimeout)
}
finally {
# clean all the runspaces
$res.Instance.ForEach('Dispose')
$rspool.ForEach('Dispose')
}