Remove matching collection object from text file - string

I have a list of users that I am storing in a text file. I am trying to update the text file so it removes any user that match $NotExpiring users variable, which is a collection. I just can't figure out how I would update the text file properly if more than one user needs to be removed from text file.
Below is the full function. You can ignore most of it Just look under #Stuck Here to get to the point.
function Get-NotExpiring{
$NotExpiring=New-Object System.Collections.Generic.List[System.Object]
$MatchedUser=New-Object System.Collections.Generic.List[System.Object]
$textfiles = Get-ChildItem $email_dir
#Day of Span
$Days="20"
#Settings
$Date=Get-Date ((Get-Date).adddays($Days))
$Users=Get-ADUser -filter {(Enabled -eq $True) -and (PasswordNeverExpires -eq $False)} -Properties SamAccountName, DisplayName, msDS-UserPasswordExpiryTimeComputed, Mail | Where-Object { $_.DisplayName -ne $nul -and ($_."msDS-UserPasswordExpiryTimeComputed" -gt ($NotExpDate.ToFileTime()))} | Select SamAccountName, Mail, DisplayName,#{Name="ExpiryDate";Expression={([datetime]::fromfiletime($_."msDS-UserPasswordExpiryTimeComputed")).DateTime}}
#Magic
foreach ($Entry in $Users) {
$EntryDate = Get-date($Entry.ExpiryDate)
if ($EntryDate -gt $Date){
$Account = $Entry.SamAccountName
$ExpDate = $Entry.ExpiryDate
$NotExpiring.add($Account)
}
}
#STUCK HERE
foreach($file in $textfiles){
foreach ($user in $NotExpiring){
if((Get-Content "$email_dir\$file") -contains $user){
$temp_get = Get-Content $email_dir\$file | where {$_ -notmatch $user}
}}}
$temp_get}
I tried below but it doesn't seem to work if more than one user are $NotExpiring that are also in the existing textfile. Any help would be appreciated. I know this is a simple fix but I can't seem to figure it out.
Get-Content $email_dir\$file | where {$_ -notmatch $user} | Set-Content <path>.txt
I was able to achieve exactly what I needed using the following solution.
foreach($file in $textfiles){ foreach ($user in $NotExpiring){
if((Get-Content "$email_dir\$file") -contains $user){
$MatchedUser.add($user)
}}
Get-Content "$email_dir\$file" | Where {$MatchedUser -NotContains $_ } | Set Content "$temp_dir\$file"
Copy-Item -path "$temp_dir\$file" -Destination "$email_dir\$file" -ErrorAction SilentlyContinue }

Basicly you are trying to match two arrays.
With where you do it foreach object. Now you have to match the single object $_ with the array $user.
Use:
...| where {$_ -notin $user}
or
...| where {$user -notcontains $_}

Related

Display Data on Two Columns Within Excel

I am trying to display data within an Excel document where Column A displays the server name and column B displays the .NET version. I'm running into an issue exporting to a .csv because it says that the file path does not exist. I would like some guidance on how I can resolve that issue and how I can display data on the two columns within Excel.
$Servers =
(
"test"
)
foreach ($Server in $Servers)
{
Invoke-Command -ComputerName $Server -ScriptBlock {
Write-Output "$(hostname)"
Get-ChildItem 'HKLM:\SOFTWARE\Microsoft\NET Framework Setup\NDP' -Recurse | Get-ItemProperty -Name Version,Release -EA 0 | where { $_.PSChildName -match '^(?!S)\p{L}'} | select PSChildName, Version, Release | Select -ExpandProperty Version | Sort-Object Version | Export-Csv -Path C:\Users\User\Desktop\example.csv
}
The main issue is that you're using Export-Csv on the remote hosts since it is inside the Invoke-Command script block, and the likeable error is because the path you are using as export doesn't exist on those hosts.
It's also worth noting that Invoke-Command can run in parallel, -ComputerName as well as -Session can take an array, this removes the need for the foreach loop as well as it is much faster / efficient.
Invoke-Command -ComputerName $servers -ScriptBlock {
Write-Host "Working on $($env:COMPUTERNAME)..."
Get-ChildItem 'HKLM:\SOFTWARE\Microsoft\NET Framework Setup\NDP' -Recurse |
Get-ItemProperty -Name Version, Release -EA 0 |
ForEach-Object {
if($_.PSChildName -notmatch '^(?!S)\p{L}') {
return # skip this
}
[pscustomobject]#{
HostName = $env:COMPUTERNAME
Version = $_.Version
}
} | Sort-Object Version
} -HideComputerName | Select-Object * -ExcludeProperty RunspaceID |
Export-Csv -Path C:\Users\User\Desktop\example.csv -NoTypeInformation

LOCATING FILES USING MULTIPLE STRING AND DATE CRITERIA

I am trying to use a code as follows, to locate files with specific strings, extension and lastwritetime :
get-childitem C:\users\nila9\Downloads -filter *.mkv -recurse
| where-object { $_.Name -match ("*bluray*" -and "*1080*") -and $_.lastwritetime -match "11/20/2020" }
The code is meant to first filter all files in the downloads folder with the extension *.mkv and then further shortlist for filenames containing the string "bluray" and "1080" and modified after the specified date.
While this code does not return any error, it does however not execute and releases the control to the prompt.
Is there someplace I am getting it wrong?
Thanks
As Lee_Dailey pointed out, -match uses regex and needs a different syntax. Your code needs a different operator that handles wildcards (*) and for that, there is -like
Also, you should not try to compare a DateTime object with a string, so -match is no good for that either.
If a filename needs BOTH bluray AND 1080, you can change the code to
$refDate = (Get-Date -Year 2020 -Month 11 -Day 20).Date # set time part to all 0 (--> midnight)
Get-ChildItem -Path 'C:\users\nila9\Downloads' -Filter '*.mkv' -Recurse |
Where-Object { $_.Name -like "*bluray*" -and $_.Name -like "*1080*" -and $_.LastWriteTime -ge $refDate }
If however the file needs to have bluray OR 1080 in its name, you can use -match on that part:
$refDate = (Get-Date -Year 2020 -Month 11 -Day 20).Date # set time part to all 0 (--> midnight)
Get-ChildItem -Path 'C:\users\nila9\Downloads' -Filter '*.mkv' -Recurse |
Where-Object { $_.Name -match "bluray|1080" -and $_.LastWriteTime -ge $refDate }
The pipe symbol | in regex is the OR operator

Code to grab file data into PSOObject and sort by LastWriteTime

I am looking to recursively grab a list of recently modified files under two network drives, sort them in descending date order, and make some edits to the CSV file to tidy the list for Excel
I have cobbled the code below from a number of sources (I am a powershell beginner) and it is now doing what I need (i.e. producing a list).
I need help in going a step further, I cannot sort the resultant CSV file by file last write time date, is this because my array is expecting text rather than a numeric field?
I also am returning the domain name as well as the file owner with ((Get-ACL $_.FullName).Owner). I tried using Replace to cut down the string, but had no luck with this approach.
$arr = #()
$days_to_check=$(Get-Date).AddDays(-28)
$items = #(Get-ChildItem '\\ND\dir 1\*.*' -Recurse -ErrorAction SilentlyContinue | where { $_.LastWriteTime -gt $days_to_check})
$items += #(Get-ChildItem '\\ND\dir 1\*.*' -Recurse -ErrorAction SilentlyContinue |
where { $_.LastWriteTime -gt $days_to_check})
$items | Foreach {
$obj = New-Object PSObject -prop $hash
$obj | Add-Member NoteProperty FullName $_.FullName
$obj | Add-Member NoteProperty Directory $_.Directory
$obj | Add-Member NoteProperty Name $_.Name
$obj | Add-Member NoteProperty LastTime $_.LastWriteTime
$obj | Add-Member NoteProperty Owner ((Get-ACL $_.FullName).Owner)
$arr += $obj
}
$arr | Format-List
$arr | Sort-Object -Property LastTime -Descending
$arr | Export-CSV -notypeinformation C:\temp\filenamesFO.csv
CSV file sorted by date field
You did sort your array in the output but that's all you did.
If you want to actually export it that way, you have to assign the sort to $arr
Replace
$arr | Sort-Object -Property LastTime -Descending
with
$arr = $arr | Sort-Object -Property LastTime -Descending
You can remove the Owner domain using the following Replace -replace '(.*\\)(.*)','$2'
Here's a complete example implementing the changes mentionned above.
$arr = new-object -TypeName 'System.Collections.Generic.List[PSObject]'
$days_to_check=$(Get-Date).AddDays(-28)
$items = #(Get-ChildItem '\\ND\dir 1\*.*' -Recurse -ErrorAction SilentlyContinue | where { $_.LastWriteTime -gt $days_to_check})
$items += #(Get-ChildItem '\\ND\dir 1\*.*' -Recurse -ErrorAction SilentlyContinue |
where { $_.LastWriteTime -gt $days_to_check})
Foreach ($item in $items) {
$obj = [PSCustomObject]#{
FullName = $item.FullName
Directory = $item.Directory
Name = $item.Name
LastTime = $item.LastWriteTime
Owner = (Get-ACL $item.FullName).Owner -replace '(.*\\)(.*)','$2'
}
$arr.add($obj)
}
$arr = $arr | Sort-Object -Property LastTime -Descending
#$arr | Format-List
$arr | Export-CSV -notypeinformation C:\temp\filenamesFO.csv
I made some additional changes:
Instead of using an array, I used a List of PSObject. If you have a lot of files, the processing time will be improved in comparison with an array.
I used the PSCustomObject declaration just to show an alternative to all those Add-member. I find it cleaner but it is up to you in the end.

Decreased output with PowerShell multithreading than with singlethread script

I am using PowerShell 2.0 on a Windows 7 desktop. I am attempting to search the enterprise CIFS shares for keywords/regex. I already have a simple single threaded script that will do this but a single keyword takes 19-22 hours. I have created a multithreaded script, first effort at multithreading, based on the article by Surly Admin.
Can Powershell Run Commands in Parallel?
Powershell Throttle Multi thread jobs via job completion
and the links related to those posts.
I decided to use runspaces rather than background jobs as the prevailing wisdom says this is more efficient. Problem is, is I am only getting partial resultant output with the multithreaded script I have. Not sure if it is an I/O thing or a memory thing, or something else. Hopefully someone here can help. Here is the code.
cls
Get-Date
Remove-Item C:\Users\user\Desktop\results.txt
$Throttle = 5 #threads
$ScriptBlock = {
Param (
$File
)
$KeywordInfo = Select-String -pattern KEYWORD -AllMatches -InputObject $File
$KeywordOut = New-Object PSObject -Property #{
Matches = $KeywordInfo.Matches
Path = $KeywordInfo.Path
}
Return $KeywordOut
}
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, $Throttle)
$RunspacePool.Open()
$Jobs = #()
$Files = Get-ChildItem -recurse -erroraction silentlycontinue
ForEach ($File in $Files) {
$Job = [powershell]::Create().AddScript($ScriptBlock).AddArgument($File)
$Job.RunspacePool = $RunspacePool
$Jobs += New-Object PSObject -Property #{
File = $File
Pipe = $Job
Result = $Job.BeginInvoke()
}
}
Write-Host "Waiting.." -NoNewline
Do {
Write-Host "." -NoNewline
Start-Sleep -Seconds 1
} While ( $Jobs.Result.IsCompleted -contains $false)
Write-Host "All jobs completed!"
$Results = #()
ForEach ($Job in $Jobs) {
$Results += $Job.Pipe.EndInvoke($Job.Result)
$Job.Pipe.EndInvoke($Job.Result) | Where {$_.Path} | Format-List | Out-File -FilePath C:\Users\user\Desktop\results.txt -Append -Encoding UTF8 -Width 512
}
Invoke-Item C:\Users\user\Desktop\results.txt
Get-Date
This is the single threaded version I am using that works, including the regex I am using for socials.
cls
Get-Date
Remove-Item C:\Users\user\Desktop\results.txt
$files = Get-ChildItem -recurse -erroraction silentlycontinue
ForEach ($file in $files) {
Select-String -pattern '[sS][sS][nN]:*\s*\d{3}-*\d{2}-*\d{4}' -AllMatches -InputObject $file | Select-Object matches, path |
Format-List | Out-File -FilePath C:\Users\user\Desktop\results.tx -Append -Encoding UTF8 -Width 512
}
Get-Date
Invoke-Item C:\Users\user\Desktop\results.txt
I am hoping to build this answer over time as I dont want to over comment. I dont know yet why you are losing data from the multithreading but i think we can increase performace with an updated regex. For starters you have many greedy quantifiers that i think we can shrink down.
[sS][sS][nN]:*\s*\d{3}-*\d{2}-*\d{4}
Select-String is case insensitive by default so you dont need the portion in the beginning. Do you have to check for multiple colons? Since you looking for 0 or many :. Same goes for the hyphens. Perhaps these would be better with ? which matches 0 or 1.
ssn:?\s*\d{3}-?\d{2}-?\d{4}
This is assuming you are looking for mostly proper formatted SSN's. If people are hiding them in text maybe you need to look for other delimiters as well.
I would also suggest adding the text to separate files and maybe combining them after execution. If nothing else just to test.
Hoping this will be the start of a proper solution.
It turns out that for some reason the Select-String cmdlet was having problems with the multithreading. I don't have enough of a developer background to be able to tell what is happening under the hood. However I did discover that by using the -quiet option in Select-String, which turns it into a boolean output, I was able to get the results I wanted.
The first pattern match in each document gives a true value. When I get a true then I return the Path of the document to an array. When that is finished I run the pattern match against the paths that were output from the scriptblock. This is not quite as effective performance wise as I had hoped for but still a pretty dramatic improvement over singlethread.
The other issue I ran into was the read/writes to disk by trying to output results to a document at each stage. I have changed that to arrays. While still memory intensive, it is much quicker.
Here is the resulting code. Any additional tips on performance improvement are appreciated:
cls
Remove-Item C:\Users\user\Desktop\output.txt
$Throttle = 5 #threads
$ScriptBlock = {
Param (
$File
)
$Match = Select-String -pattern 'ssn:?\s*\d{3}-?\d{2}-?\d{4}' -Quiet -InputObject $File
if ( $Match -eq $true ) {
$MatchObjects = Select-Object -InputObject $File
$MatchOut = New-Object PSObject -Property #{
Path = $MatchObjects.FullName
}
}
Return $MatchOut
}
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, $Throttle)
$RunspacePool.Open()
$Jobs = #()
$Files = Get-ChildItem -Path I:\ -recurse -erroraction silentlycontinue
ForEach ($File in $Files) {
$Job = [powershell]::Create().AddScript($ScriptBlock).AddArgument($File)
$Job.RunspacePool = $RunspacePool
$Jobs += New-Object PSObject -Property #{
File = $File
Pipe = $Job
Result = $Job.BeginInvoke()
}
}
$Results = #()
ForEach ($Job in $Jobs) {
$Results += $Job.Pipe.EndInvoke($Job.Result)
}
$PathValue = #()
ForEach ($Line in $Results) {
$PathValue += $Line.psobject.properties | % {$_.Value}
}
$UniqValues = $PathValue | sort | Get-Unique
$Output = ForEach ( $Path in $UniqValues ) {
Select-String -Pattern '\d{3}-?\d{2}-?\d{4}' -AllMatches -Path $Path | Select-Object -Property Matches, Path
}
$Output | Out-File -FilePath C:\Users\user\Desktop\output.txt -Append -Encoding UTF8 -Width 512
Invoke-Item C:\Users\user\Desktop\output.txt

Powershell filter a List by Name and Date

I need a bit of help... I'm new to powershell and i want to Filter a List (csv). I would love to remove all lines with certain names in it. and cut the list down to the last month. In the script you can see how far i got till now.
param(
[Parameter(ValueFromPipeline=$true,HelpMessage="Enter CSV path(s)")]
[String[]]$Path = $null
)
if($Path -eq $null) {
Add-Type -AssemblyName System.Windows.Forms
$Dialog = New-Object System.Windows.Forms.OpenFileDialog
$Dialog.InitialDirectory = "$InitialDirectory"
$Dialog.Title = "Select CSV File(s)"
$Dialog.Filter = "CSV File(s)|*.csv"
$Dialog.Multiselect=$true
$Result = $Dialog.ShowDialog()
if($Result -eq 'OK') {
Try {
$Path = $Dialog.FileNames
}
Catch {
$Path = $null
Break
}
}
else {
Write-Host -ForegroundColor Yellow "Notice: No file(s) selected."
Break
}
}
$info=Import-Csv "$path" -Delimiter ';'
$info | Get-Member
$info | Format-Table
as you can see i tryed to link the path to a filebrowser.
For the purposes of discussion, I will assume that the full pathname of the CSV is in the variable $InputPath, and that you want to write the result to a CSV file whose full pathname is in the variable $OutputPath. I will also assume that the CSV file contains a column named 'Name', and that the value from the Name column that you want to exclude is in the variable $ExcludedName. Given that, you can simply do
Import-CSV -Path $InputPath | Where-Object {$_.Name -ne $ExcludedName} | Export-CSV -Path $OutputPath -NoTypeInformation
You can do this by my code,but dont forget that first row must contains names of column and delimiter must be ';' and $nameslist is array of names that you need delete:
$info=Import-Csv "D:\testdir\file2.csv" -Delimiter ';'
$nameslist=#('James','John','andrew')
foreach($i in $info){
if($nameslist -contains $i.Name){
$i.Name=""
}
$i|Export-Csv -Path "D:\testdir\file1.csv" -Delimiter ';' -NoTypeInformation -Force -Encoding UTF8 -Append
}
Try this:
$data = Import-Csv "Path" | Select-Object * -ExcludeProperty Names
$data | export-csv "Path" -Notype
This will cut the column names.
Try it first without using a function:
Import-Csv <Filename> | Where-Object {$_.<FieldName> -notlike "*<Value>*"}
Also, you might consider something like this:
[CmdletBinding()]
param (
[Parameter(ValueFromPipeline = $true, HelpMessage = "Enter CSV path(s)")]
[String[]]$Path = $(
Add-Type -AssemblyName System.Windows.Forms
$DialogProperties = #{
Title = 'Select CSV File(s)'
Filter = 'CSV File(s)|*.csv'
Multiselect = $True
}
$Dialog = New-Object System.Windows.Forms.OpenFileDialog -Property $DialogProperties
$Dialog.ShowDialog()
If ($Result -eq 'OK') {
$Path = $Dialog.FileNames
} Else {
Write-Error 'Notice: No file(s) selected.'
}
)
)
Process {
ForEach ($PathItem in $Path) {
Import-Csv $PathItem | Where-Object { $_.Name -notlike "*NotThisOne*" }
}
}

Resources