PowerShell set each string part as a variable for reuse - string

I have a list of files in a folder each are in this format: custID_invID_prodID or custID_invID_prodID_Boolvalue. For every file I need to break it into sections based on '_'. Currently I have this code:
$files = Get-ChildItem test *.txt
foreach($f in $files){
$file = #()
$file += ([String]$f).Split("_")
$total = ([String]$f).Split("_") | Measure-Object | select count
Write-Host "${total}"
if($total -eq 2) {
for($i = 2; $i -lt $file.length; $i+=3) {
$file[$i] = $file[$i].trimend(".txt")
Write-Host "${file}"
}
}
}
The problem is that Write-Host "${total}" equals #{Count=#} where # is real number of times "_" is found in file. How can I use $total inside my if statement to do different operations based upon the number of "_" found?

Would it not be simpler just to assign the parts you want directly to named variables rather than working with an array?
foreach($f in (Get-ChildItem test *.txt)) {
$custId, $invID, $prodID, $Boolvalue = $f.BaseName -split "_"
Write-Host $custId, $invID, $prodID, $Boolvalue
}
If the name only has 3 parts this will simply set $Boolvalue to an empty string.
Also note that you don't have to trim the extension off the last element after splitting, just use the BaseName property to get the name without extension.

You need to get the count-property value, like $total.count in your if test. You could also clean it up like this.
$files = Get-ChildItem test *.txt
foreach($f in $files){
$file = #(([String]$f).Split("_"))
Write-Host "$($file.Count)"
if($file.Count -eq 2) {
for($i = 2; $i -lt $file.length; $i+=3) {
$file[$i] = $file[$i].trimend(".txt")
Write-Host "${file}"
}
}
}
If you had included more information about what you were trying to do, we could clean it up alot more. Ex. It's seems like you want to do something like this:
Get-ChildItem test *.txt | ForEach-Object {
$file = #($_.BaseName.Split("_"))
Write-Host "$($file.Count)"
if($file.Count -eq 2) {
Write-Host $file
}
}

Seems to me that you're doing it the hard way. Why not:
$x = "aaa_bbb_ccc"
$cnt = $x.Split("_").count

Related

Efficiently search a string in large files

How can I check if a string exists in:
1 text file;
size up until 10GB;
taking into account that the file is only one line;
the file only contains random numbers 1 to 9;
using powershell (because I think it will be more efficient, although I don't know how to program in this language);
I have tried this in batch:
FINDSTR "897516" decimal_output.txt
pause
But as I said I need the faster and more efficient way to do this.
I also tried this code that I have found in stackoverflow:
$SEL = Select-String -Path C:\Users\fabio\Desktop\CONVERTIDOS\dec_output.txt -Pattern "123456"
if ($SEL -ne $null)
{
echo Contains String
}
else
{
echo Not Contains String
}
But I get the error below, and I don't know if this code is the most solid or adequate. The error:
Select-String : Tipo de excepção 'System.OutOfMemoryException' accionado.
At C:\Users\fabio\Desktop\1.ps1:1 char:8
+ $SEL = Select-String -Path C:\Users\fabio\Desktop\CONVERTIDOS\dec_out ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Select-String], OutOfMemoryException
+ FullyQualifiedErrorId : System.OutOfMemoryException,Microsoft.PowerShell.Commands.SelectStringCommand
This should do the job:
#################################################################################################################
#
# Searches for a user defined string in the $input_file and counts matches. Works with files of any size.
#
# Adjust source directory and input file name.
#
$source = "C:\adjust\path"
$input_file = "file_name.extension"
#
#
# Define the string you want to search for. Keep quotation marks even if you only search for numbers (otherwise
# $pattern.Length will be 1 and this script will no longer work with files larger than the $split_size)!
#
$pattern = "Enter the string to search for in here"
#
#
# Using Get-Content on an input file with a size of 1GB or more will cause System.OutOfMemoryExceptions,
# therefore a large file gets temporarily split up.
#
$split_size = 100MB
#
#
# Thanks #Bob (https://superuser.com/a/1295082/868077)
#################################################################################################################
Set-Location $source
if (test-path ".\_split") {
while ($overwrite -ne "true" -and $overwrite -ne "false") {
"`n"
$overwrite = Read-Host ' Splitted files already/still exist! Delete and overwrite?'
if ($overwrite -match "y") {
$overwrite = "true"
Remove-Item .\_split -force -recurse
$a = "`n Deleted existing splitted files!"
} elseif ($overwrite -match "n") {
$overwrite = "false"
$a = "`n Continuing with existing splitted files!"
} elseif ($overwrite -match "c") {
exit
} else {
Write-Host "`n Error: Invalid input!`n Type 'y' for 'yes'. Type 'n' for 'no'. Type 'c' for 'cancel'. `n`n`n"
}
}
}
Clear-Host
if ((Get-Item $input_file).Length -gt $split_size) {
while ($delete -ne "true" -and $delete -ne "false") {
"`n"
$delete = Read-Host ' Delete splitted files afterwards?'
if ($delete -match "y") {
$delete = "true"
$b = "`n Splitted files will be deleted afterwards!"
} elseif ($delete -match "n") {
$delete = "false"
$b = "`n Splitted files will not be deleted afterwards!"
} elseif ($delete -match "c") {
exit
} else {
Write-Host "`n Error: Invalid input!`n Type 'y' for 'yes'. Type 'n' for 'no'. Type 'c' for 'cancel'. `n`n`n"
}
}
Clear-Host
$a
$b
Write-Host `n This may take some time!
if ($overwrite -ne "false") {
New-Item -ItemType directory -Path ".\_split" >$null 2>&1
[Environment]::CurrentDirectory = Get-Location
$bytes = New-Object byte[] 4096
$in_file = [System.IO.File]::OpenRead($input_file)
$file_count = 0
$finished = $false
while (!$finished) {
$file_count++
$bytes_to_read = $split_size
$out_file = New-Object System.IO.FileStream ".\_split\_split_$file_count.splt",CreateNew,Write,None
while ($bytes_to_read) {
$bytes_read = $in_file.Read($bytes, 0, [Math]::Min($bytes.Length, $bytes_to_read))
if (!$bytes_read) {
$finished = $true
break
}
$bytes_to_read -= $bytes_read
$out_file.Write($bytes, 0, $bytes_read)
}
$out_file.Dispose()
}
$in_file.Dispose()
}
$i++
while (Test-Path ".\_split\_split_$i.splt") {
$cur_file = (Get-Content ".\_split\_split_$i.splt")
$temp_count = ([regex]::Matches($cur_file, "$pattern")).Count
$match_count += $temp_count
$n = $i - 1
if (Test-Path ".\_split\_split_$n.splt") {
if ($cur_file.Length -ge $pattern.Length) {
$file_transition = $prev_file.Substring($prev_file.Length - ($pattern.Length - 1)) + $cur_file.Substring(0,($pattern.Length - 1))
} else {
$file_transition = $prev_file.Substring($prev_file.Length - ($pattern.Length - 1)) + $cur_file
}
$temp_count = ([regex]::Matches($file_transition, "$pattern")).Count
$match_count += $temp_count
}
$prev_file = $cur_file
$i++
}
} else {
$a
$match_count = ([regex]::Matches($input_file, "$pattern")).Count
}
if ($delete -eq "true") {
Remove-Item ".\_split" -Force -Recurse
}
if ($match_count -ge 1) {
Write-Host "`n`n String '$pattern' found:`n`n $match_count matches!"
} else {
Write-Host "`n`n String '$pattern' not found!"
}
Write-Host `n`n`n`n`n
Pause
This will split a large file into mutliple smaller files, search them for $pattern and count the matches (taking file transitions into account).
It also offers you to delete or keep the splitted files afterwards so you can reuse them and don't have to split the large file every time you run this script.

Powershell comparing data in a CSV against files in a folder

I'm fairly new to powershell.
I'm trying to compare data in a CSV File against random files in a specific folder.
I want to see if and what has changed and then log that in another column called "Changed".
Here's what I've done below, it seems to create a new column called 'Changed' but doesn't input the changes in it.
$Spreadsheet = 'C:\Powershell\CSV\inv.csv'
$SpreadSheetPath = "C:\Powershell\CSV"
Import-Csv $Spreadsheet -Delimiter "|" -Encoding Default | ForEach-Object -
{
$Path += $_.Path
$Filename += $_.Filename
$DateModified += $_.DateModified
$FileSize += $_.FileSize
$MD5Hash += $_.MD5Hash
}
{
$Msg1 = "Path changed"
$Msg2 = "File Name changed"
$Msg3 = "Date Modified changed"
$Msg4 = "File Size changed"
$Msg5 = "MD5 changed"
$Msg6 = "Files are the same"
$psdata = "D:\ps-test\data\*.*"
}
If (($Path -eq $psdata))
{
Import-Csv C:\Powershell\CSV\inv.csv |
Select-Object *,#{Name='Changed';Expression={$Msg6}} |
Export-Csv C:\Powershell\CSV\NewSpreadsheet4.csv
}
Else
{
Import-Csv C:\Powershell\CSV\inv.csv |
Select-Object *,#{Name='Changed';Expression={$Msg1}} |
Export-Csv C:\Powershell\CSV\NewSpreadsheet4.csv
}
Here is an example of what the CSV looks like:
Path Filename Date Modified File Size MD5 Hash
D:\ps-test\data adminmodeinfo.htm 03/11/2010 22:42 1079 BD1C9468D71FD33BB35716630C4EC6AC
E:\ps-test\data admintoolinfo.htm 03/11/2010 22:42 868 24B99B6316F0C49C23F27FEA6FF1C6AC
E:\ps-test\data admin_ban.bmp 03/11/2010 22:42 63480 C856F1F3C58962B456E749F2EA9C933A
E:\ps-test\data baseline.dat 03/20/2010 03:18:33 173818 F13183D88AABD1A725437802F8551A06
E:\ps-test\data blueRule.gif 03/11/2010 22:42 815 D1AEFE884935095DAB42DAFD072AA46F
E:\ps-test\data deffactory.dat 03/20/2010 03:18:33 706 862D4DFD2F49021BB7C145BDAFE62F6F
E:\ps-test\data dividerArt.jpg 03/11/2010 22:42 367 F7050C596C097C0B01A443058CD15E35
There are many issues with your code.I will try to highlight a few of the issues, link to documentation and point you in the right direction so that you can resolve your issues. A proper solution would require getting many more requirements, or writing code (off-topic for StackOverflow)
Change
| ForEach-Object -
{
to
| ForEach-Object {
In the Foreach-Object, you are concatenating values from each line because you are using +=.
On the first run, $Path contains D:\ps-test\data.
After the second run, it contains D:\ps-test\dataE:\ps-test\data.
At the end of your test data, it contains D:\ps-test\dataE:\ps-test\dataE:\ps-test\dataE:\ps-test\dataE:\ps-test\dataE:\ps-test\dataE:\ps-test\data
The messages are contained in a script block, but it does not look like this is intentional as this is never executed. So after the scriptblock, the variable $Msg1 has not been created; it's blank.
If (($Path -eq $psdata))
double brackets not required.
will always be false because the variable $psdata does not exist as it was stated inside a script block.
will always be false because you are attempting to equate the strings; your input does not literally contain "D:\ps-test\data\*.*". You probably want -like instead of -eq.
will always be inaccurate because even if the paths are compared, there is no check that the file actually exists on the system.
Useful links
Test-Path to check if file exists.
Get-FileHash to get MD5 hash and compare to file.
Get-ChildItem to get a list of directories/files in a directory.
Write-Output so that you can print variables and make sure they contain what you expect.
about_comparison_operators - -in and -contains will help you.
This is a suggestion to help you get started. It's not complete and not tested! Let me know if it works as expected and if you have any questions.
Import-Csv 'C:\Powershell\CSV\inv.csv' -Delimiter "|" -Encoding Default | foreach {
$Path += $_.Path
$Filename += $_.Filename
$DateModified += $_.DateModified
$FileSize += $_.FileSize
$MD5Hash += $_.MD5Hash
$file = [System.IO.FileInfo](Join-Path $Path $Filename)
if (-not $file.Exists) {
$message = "File does not exist"
}
elseif ($file.LastWriteTime -ne [DateTime]$DateModified) {
$message = "Dates differ"
}
elseif ($file.Length -ne [int]$FileSize) {
$message = "Sizes differ"
}
# and so on...
# (You cannot really compared a changed file name btw)
New-Object -Type PSObject -Prop #{
Path = $Path
Filename = $Filename
DateModified = $DateModified
FileSize = $FileSize
MD5Hash = $MD5Hash
Message = $message
}
} | Export-CSV 'C:\Powershell\CSV\NewSpreadsheet4.csv'

PowerShell does not replace string although you can see it in cmd

I normally find the answer to my problem by going through the site, but this time I have read every question yet still I am in despair and really need an experienced eye.
What I have is basically a structural health monitoring system. I measure strains and receive raw data. This raw data is processed by a MATLAB executable that I wrote myself and then uploaded to an ftp-server. We had a student that automated this with a PowerShell script which was working perfectly until I changed literally one small line in MATLAB and recompiled the code.
I do not understand much about PowerShell, so please be patient with me. The error I receive is you cannot call a method on a null-valued expression. This occurs when I try to replace a set of strings (just called xxx_xxx) with a date that exists as a variable in PowerShell. I can see xxx_xxx in the command window (see attached image), I can print out the date that I want to use as replacement, but somehow it does not work.
I cannot provide a working code snippet because you would need the DAQ to generate data, and as I said, I don't understand the language much. But below is the code. For easier reading, the line that I am receiving the error is the following:
$outData = $cmdOutput.Replace("xxx_xxx",$snaps[$i].Substring(6,4)+"-"+$snaps[$i].Substring(3,2)+"-"+$snaps[$i].Substring(0,2)+" "+$snaps[$i].Substring(11,8)+";")
If anyone could help me with this, I would be eternally grateful!
$retry=3
while(1){
#$dir = "C:\Users\Petar\Documents\Zoo\PetarData\INPUT DATA\New folder\"
$dir = "C:\Users\Yunus\Documents\Micron Optics\ENLIGHT\Data\" + $(get-date -f yyyy) + "\" + $(get-date -f MM) + "\"
#$outdir = "C:\Users\Petar\Documents\Zoo\PetarData\OUTPUT DATA\New folder\"
$archivedirin = "C:\Users\Yunus\Documents\Elefantenhaus\Archive\IN\"
$archivedirout = "C:\Users\Yunus\Documents\Elefantenhaus\Archive\OUT\"
$tempdir = "C:\Users\Yunus\Documents\Elefantenhaus\Archive\TEMP\"
$prefix = "EHZZ";
$filecount=(Get-ChildItem $dir).Count
$latest = Get-ChildItem -Path $dir | Sort-Object LastAccessTime -Descending | Select-Object -First 1
if($filecount -gt 1){
$exclude = $latest.name
$Files = GCI -path $dir | Where-object {$_.name -ne $exclude}
$dest = $archivedirin + "batch_"+$(get-date -f MM-dd-yyyy_HH_mm_ss)+"\"
new-item -type directory $dest
foreach ($file in $Files){move-item -path ($dir+$file) -destination $dest}
$latest = Get-ChildItem -Path $dest | Sort-Object LastAccessTime -Descending | Select-Object -First 1
$filename = $dest + $latest.name
$s=Get-Content $filename
while($s -eq $null){
if($retry -lt 0){break}
write-host "could not read file"
$retry = $retry -1
$s=Get-Content $filename
}
#read content of input file
$snaps = $s
#loop through the lines in the file until the first occurence of a timestamp, that is our desired line
for ($i = 0; $i -lt $snaps.length; $i++)
{
$ismatch =[regex]::Matches($snaps[$i], '^(\d\d.\d\d.\d\d\d\d\s\d\d+)')
if ( $ismatch -ne $null -and $ismatch[0].Groups[1].Value)
{
$temp=Get-Content $filename | select -skip $i
$filenametemp = $tempdir+"\temp.txt" #temp file path, don't change the filename "temp.txt"
#$filename3 = $tempdir+"\test.txt"
Add-Content $filenametemp $temp
$filename = $archivedirout+$prefix+"_"+$snaps[$i].Substring(8,2)+$snaps[$i].Substring(3,2)+$snaps[$i].Substring(0,2)+"_"+$snaps[$i].Substring(11,2)+$snaps[$i].Substring(14,2)+$snaps[$i].Substring(17,2)+".txt"
$cmdOutput = (cmd /c new_modified.exe $tempdir) | Out-String
write-output $cmdOutput #"$cmdOutput is:"
#IF ([string]::IsNullOrWhitespace($cmdOutput)){
# break
#}
$outData = $cmdOutput.Replace("xxx_xxx",$snaps[$i].Substring(6,4)+"-"+$snaps[$i].Substring(3,2)+"-"+$snaps[$i].Substring(0,2)+" "+$snaps[$i].Substring(11,8)+";")
Add-Content $filename $outData
remove-item -path $filenametemp
break
}
}
#break
}
else
{
write-host "waiting for file"
}
Start-Sleep -s 30
}
I think what is happening is that the output of the external program isn't being piped into a variable correctly. I haven't had a chance to test this but Tee-Object looks like the appropriate method for you.
I would suggest you try replacing...
$cmdOutput = (cmd /c new_modified.exe $tempdir) | Out-String
with...
cmd /c new_modified.exe $tempdir | Tee-Object -variable $cmdOutput

PowerShell: retrieve number of applications in AppPool

How to retrieve the number of applications associated with a specific IIS AppPool via PowerShell command?
We can see the associated applications manually using:
Get-Item IIS:\AppPools\AppPoolName
However, if we manually want to select the Applications column, it is not possible. Also, the Applications column is not listed within | Get-Member *.
Why is the column not listed?
How to find the number of applications associated with a specific IIS AppPool using PowerShell?
The trick is: PowerShell established so-called "view definition files" which tell PowerShell how to format objects (e.g. whether the object is formatted as a a list or a table, which columns are displayed, etc.). Those files can be found at C:\Windows\System32\WindowsPowerShell\v1.0 and are all ending in .format.ps1xml.
To answer the original question: The file C:\Windows\System32\WindowsPowerShell\v1.0\Modules\WebAdministration\iisprovider.format.ps1xml contains the view definition for the AppPool type which defines a calculated column looking like this:
<TableColumnItem>
<ScriptBlock>
$pn = $_.Name
$sites = get-webconfigurationproperty "/system.applicationHost/sites/site/application[#applicationPool=`'$pn`'and #path='/']/parent::*" machine/webroot/apphost -name name
$apps = get-webconfigurationproperty "/system.applicationHost/sites/site/application[#applicationPool=`'$pn`'and #path!='/']" machine/webroot/apphost -name path
$arr = #()
if ($sites -ne $null) {$arr += $sites}
if ($apps -ne $null) {$arr += $apps}
if ($arr.Length -gt 0) {
$out = ""
foreach ($s in $arr) {$out += $s.Value + "`n"}
$out.Substring(0, $out.Length - 1)
}
</ScriptBlock>
</TableColumnItem>
This answers why the column itself is not a member of the AppPool type. The second question can be easily answered now extracting the necessary code from the "scriptlet" above:
$applicationsInAppPoolCount = #(Get-WebConfigurationProperty `"/system.applicationHost/sites/site/application[#applicationPool=`'$appPool`'and #path!='/']"` "machine/webroot/apphost" -name path).Count
I dealt with this same issue for many hours until finally arriving at the solution. The answer from D.R. was very helpful but it was not working for me. After some tweaks, I came up with the code below which retrieves the number of applications in an app pool.
I noticed that this part of the code nd #path!='/' threw off the count.
$appPool = "REPLACE ME with a value from your app pool"
#(Get-WebConfigurationProperty "/system.applicationHost/sites/site/application[#applicationPool=`'$appPool`']" "machine/webroot/apphost" -name path).Count
I ended up with the following Code (basically the same as above, but differently formatted)
$appPools = Get-ChildItem –Path IIS:\AppPools
foreach ($apppool in $apppools) {
$appoolName = $apppool.Name
[string] $NumberOfApplications = (Get-WebConfigurationProperty "/system.applicationHost/sites/site/application[#applicationPool='$appoolName']" "machine/webroot/apphost" -name path).Count
Write-Output "AppPool name: $appoolName has $NumberOfApplications applications"
}
I recently came across this post searching for ways to get the active Application Pools. The information provided above was great, but I kept digging to see if there was another way get this information. I was able to find a way to do this through Get-IISSite, which I used the following:
Get-IISSite | Select-Object -ExpandProperty Applications | Select-Object Path,ApplicationPoolName
I tested this on a server that only had one website, but if there are multiple sites on the server, you could also add VirtualDirectories for the Select.
I also had a need to just get a unique list of the Application Pools being used, so I did the following:
$appPoolInfo = Get-IISSite | Select-Object -ExpandProperty Applications | Select-Object Path,ApplicationPoolName
$appPoolInfo | Select-Object -Unique ApplicationPoolName
This gives what you are looking in an array.
Import-Module WebAdministration;
Get-ChildItem IIS:\AppPools >> AppPoolDetails.txt;
$appPoolDetails = Get-Content .\AppPoolDetails.txt;
$w = ($appPoolDetails |Select-String 'State').ToString().IndexOf("State");
$w = $w -1;
$res1 = $appPoolDetails | Foreach {
$i=0;
$c=0; `
while($i+$w -lt $_.length -and $c++ -lt 1) {
$_.Substring($i,$w);$i=$i+$w-1}}
Write-Host "First Column---";
$res1.Trim();
$j = $w + 1;
$w = ($appPoolDetails |Select-String 'Applications').ToString().IndexOf("Applications");
$w = $w -$j;
$res2 = $appPoolDetails | Foreach {
$i=$j;
$c=0; `
while($i+$w -lt $_.length -and $c++ -lt 1) {
$_.Substring($i,$w);$i=$i+$w-1}}
Write-Host "Second Column---";
$res2.Trim();
$lineLength=0
$appPoolDetails | Foreach {
if($lineLength -lt $_.TrimEnd().Length )
{
$lineLength = $_.TrimEnd().Length;
#Write-Host $lineLength;
}
}
$j = ($appPoolDetails | Select-String 'Applications').ToString().IndexOf("Applications");
$w = $lineLength;
$w = $w -$j;
#Write-Host $j $w;
$res3 = $appPoolDetails | Foreach {
$i=$j;
$c=0; `
while($i+$w -lt $_.length -and $c++ -lt 1) {
$_.Substring($i,$w);$i=$i+$w-1}}
Write-Host "Third Column---";
$res3;

PowerShell: What's wrong with "... ${_.Name} ..."?

Why am i not able to use $_ inside of text as one can use other variables?
Get-ChildItem -Path $path -filter *.mp3 | foreach {
$count++;
write-host "File${count}=${_.Name}";
}
I know I can write it this way:
Get-ChildItem -Path $path -filter *.mp3 | foreach {
$count++;
write-host "File${count}=$($_.Name)";
}
When you write ${_.Name} you're actually asking for the variable named _.Name, not the Name property of the $_ variable.
PS > ${_.Name} = "test"
PS > Get-Variable _*
Name Value
---- -----
_.Name test
The reason $($_.Name) works is because $() means "process this first", so you can specify whatever you want inside. In this case you just specified a variable name and the property you wanted, but you could also make it more complex like:
PS > $a = 1
PS > "A's value is 1(true or false?): $(if($a -eq 1) { "This is TRUE!" } else { "This is FALSE!" })"
A's value is 1(true or false?): This is TRUE!
PS > $a = 2
PS > "A's value is 1(true or false?): $(if($a -eq 1) { "This is TRUE!" } else { "This is FALSE!" })"
A's value is 1(true or false?): This is FALSE!

Resources