Return duplicate names (including partial matches) - excel

Excel guy here that occasionally turns to automating powershell via vba.
I tried to solve https://stackoverflow.com/q/36538022/641067 (now closed) and couldn't get there with my basic powershell knowledge and googlefu alone.
In essence the problem the OP presented is:
There are a list of names in a text file.
Aim is to capture only those names that occurr at least once (so discard unique names, see point (3)).
Names occurring at least once include partial matches, ie Will and William can be considered duplicates and should be retained. Whereas Bill is not a duplicate of William.
I tried various approaches including
Group
Compare-Object see example below
But I was stymied by part (3). I suspect that a loop is required to do this but am curious whether there is a direct Powershellapproach,
Looking forward to hearing from the experts.
what I tried
$a = Get-Content "c:\temp\in.txt"
$b = $a | select -unique
[regex] $a_regex = ‘(?i)(‘ + (($a |foreach {[regex]::escape($_)}) –join “|”) + ‘)’
$c = $b -match $a_regex
Compare-object –referenceobject $c -IncludeEqual $a

Following testscript using a loop would work for the rules you outlined and looks foolproof to me
$t = ('first', 'will', 'william', 'williamlong', 'unique', 'lieve', 'lieven')
$s = $t | sort-object
[String[]]$r = #()
$i = 0;
while ($i -lt $s.Count - 1) {
if ($s[$i+1].StartsWith($s[$i])) {
$r += $s[$i]
$r += $s[$i+1]
}
$i++
}
$r | Sort-Object -Unique
and following testscript using a regex might get you started.
$content = "nomatch`nevenmatch1`nevenmatch12`nunevenmatch1`nunevenmatch12`nunevenmatch123"
$string = (($content.Split("`n") | Sort-Object -Unique) -join "`n")
$regex = [regex] '(?im)^(\w+)(\n\1\w+)+'
$matchdetails = $regex.Match($string)
while ($matchdetails.Success) {
$matchdetails.Value
$matchdetails = $matchdetails.NextMatch()
}

Related

How to get a hashtable in PowerShell from a multiline string in which keys and values are on different lines?

I have a string of the following format (the number of lines in this string may vary):
$content = #"
key1
value1
key2
value2
key3
value3
"#
I want to put this data in a hashtable.
(In my case, the data in the $content variable is received in the body of the HTTP response from the Invoke-WebRequest cmdlet by Client/Server Protocol in the 'LiveJournal'. But I am interested in the answer to my question for the general case as well.)
I tried to use the cmdlet ConvertFrom-StringData, but it doesn't work for this case:
PS C:\> ConvertFrom-StringData -StringData $content -Delimiter "`n"
ConvertFrom-StringData: Data line 'key1' is not in 'name=value' format.
I wrote the following function:
function toHash($str) {
$arr = $str -split '\r?\n'
$hash = #{}
for ($i = 0; $i -le ($arr.Length - 1); $i += 2) {
$hash[$arr[$i]] = $arr[$i + 1]
}
return $hash
}
This function works well:
PS C:\> toHash($content)
Name Value
---- -----
key3 value3
key2 value2
key1 value1
My question is: is it possible to do the same thing, but shorter or more elegant? Preferably in one-liner (see the definition of this term in the book 'PowerShell 101'). Maybe there is a convenient regular expression for this case?
As commented by #Santiago Squarzon;
The code you already have looks elegant to me, shorter -ne more elegant
For the "Preferably in one-liner", what exactly is definition of a one line:
A single line, meaning a text string with no linefeeds?
Or a single statement meaning a text string with no linefeeds and no semicolons?
Knowing that there are several ways to cheat on this, like assigning a variable in a condition (which is hard to read).
Anyways, a few side notes:
The snippet you show might have a pitfall if you have an odd number of lines and Set-StrictMode -Version Lastest enabled:
Set-StrictMode -Version Latest
toHash "key1`nvalue1`nkey2"
OperationStopped:
Line |
5 | $hash[$arr[$i]] = $arr[$i + 1]
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Index was outside the bounds of the array.
Name Value
---- -----
key1 value1
The variable neme $content, suggests that you reading the data from a file, possibly with Get-Connent. If that is indeed the case you might consider to stream the input (which conserves memory):
$Content -split '\r?\n' | # Get-Content .\Data.txt
Foreach-Object -Begin {
$Hash = #{}
$Key = $Null
} -Process {
if (!$Key) {
$Key = $_
}
else {
$Hash[$Key] = $_
$Key = $Null
}
} -End {
$Hash
}
And if you create use an [ordered] dictionary, you might even put this is a single statement like:
$Content -split '\r?\n' |Foreach-Object { $h = [Ordered]#{} } { if (!$h.count -or $h[-1]) { $h[$_] = $Null } else { $h[$h.Count - 1] = $_ } } { $h }
(Note that -as with the propose in the question- I do not take into account that there might be empty lines in the input data)
See also PowerShell issue: #13817 Enhance hash table syntax

How to modify excel data and export to text file using PowerShell script?

First time poster here. Apologies if I am not following best practices for posting this question.
I am very new to scripting and PowerShell.
Problem:
I have data in an excel sheet in this format.
Excel Data Image Link
I want to modify and export this data into a text file. In this format.
Required Output Image Link
Till now I have tried to modify the excel data by accessing each cell. To access each cell I am using a similar code mentioned below.
for (($i = 1); $i -lt 4; $i++)
{
$column=$ExcelWorkSheet.Columns.Item(1).Rows.Item($i).Text
$dataType=$ExcelWorkSheet.Columns.Item(2).Rows.Item($i).Text
$c1=("`"" + "$column" + "`""+":")
$c2=("`"" + "$dataType" + "`"" + ",")
$ExcelWorkSheet.Columns.Item(1).Rows.Item($i).Value=$c1
$ExcelWorkSheet.Columns.Item(2).Rows.Item($i).Value=$c2
}
I am still not sure if this is the correct way to go.
what would be the best way to solve this?
Just want to understand what I should do to solve this problem. I am not looking for the exact code.
Step by step instructions or some resources would be helpful.
Thanks!
This might help... maybe...
# Import Stuff
$Data = Import-Csv -Path .\Desktop\data.csv
# New Array
$Output = #()
# Run through Unique Owners
foreach ($Owner in ($Data | Select-Object OWNER -Unique)) {
$Lines = $Data | Where-Object {$_.OWNER -eq $Owner.OWNER}
# Lazy way to do a bit of checking, if same then use it or Break
if ($Lines[0].TABLE_NAME -eq $Lines[1].TABLE_NAME) {
$Out_TableName = $Lines[0].TABLE_NAME
# ID and NAME data
$Out_ID = $Lines | Where-Object {$_.COLUMN_NAME -eq "ID"} | Select-Object COLUMN_NAME, DATA_TYPE, DATA_LENGTH
$Out_NAME = $Lines | Where-Object {$_.COLUMN_NAME -eq "NAME"} | Select-Object COLUMN_NAME, DATA_TYPE, DATA_LENGTH
} else {
# Show the user that something
Write-Host "Problem with Owner ""$($Owner.OWNER)"" Data?!" -ForegroundColor Red
Break
}
# Output into the array in format
$Output += #"
"$($Owner.OWNER).$($Out_TableName)":{
"$($Out_ID.COLUMN_NAME)": "$($Out_ID.DATA_TYPE) ($($Out_ID.DATA_LENGTH))",
"$($Out_NAME.COLUMN_NAME)": "$($Out_NAME.DATA_TYPE) ($($Out_NAME.DATA_LENGTH))"
}
"#
}
# Put Output in a text file
$Output | Set-Content .\Desktop\output.txt -Force
I should add, that I had your data in a CSV like this...
OWNER,TABLE_NAME,COLUMN_NAME,DATA_TYPE,DATA_LENGTH
A,Employee,ID,NUMBER,22
A,Employee,NAME,VARCHAR2,22
B,Department,ID,NUMBER,23
B,Department,NAME,VARCHAR2,24

How do I check a string exist in a file using PowerShell?

I have a 1st text file looks like this : 12AB34.US. The second text file is CD 34 EF.
I want to find my 2nd text file exist or not in the 1st text file.
I tried to cut 3 characters last in the first text file (.US). Then I split to each 2 characters (because the 2nd text file consist of 2 characters). Then, I tried this code, and it always return "Not Found".
$String = Get-Content "C:\Users\te2.txt"
$Data = Get-Content "C:\Users\Fixed.txt"
$Split = $Data -split '(..)'
$Cut = $String.Substring(0,6)
$String_Split = $Cut -split '(..)'
$String_Split
$Check= $String_Split | %{$_ -match $Split}
if ($Check-contains $true) {
Write-Host "0"
} else {
Write-Host "1"
}
There are a number of problems with your current approach.
The 2-char groups don't align:
# strings split into groups of two
'12' 'AB' '34' # first string
'CD' ' 3' '4 ' # second string
When you test multiple strings with -match, you need to
escape the input string to avoid matchings on meta characters (like .), and
place the collection on the left-hand side of the operator, the pattern on the right:
$Compare = $FBString_Split | % {$Data_Split -match [regex]::Escape($_)}
if ($Compare -contains $true) {
Write-Host "Found"
} else {
Write-Host "Not Found"
}
For a more general solution to find out if any substring of N chars of one string is also a substring of another, you could probably do something like this instead:
$a = '12AB34.US'
$b = 'CD 34 EF'
# we want to test all substrings of length 2
$n = 2
$possibleSubstrings = 0..($n - 1) | ForEach-Object {
# grab substrings of length $n at every offset from 0 to $n
$a.Substring($_) -split "($('.'*$n))" | Where-Object Length -eq $n |ForEach-Object {
# escape the substring for later use with `-match`
[regex]::Escape($_)
}
} |Sort-Object -Unique
# We can construct a single regex pattern for all possible substrings:
$pattern = $possibleSubstrings -join '|'
# And finally we test if it matches
if($b -match $pattern){
Write-Host "Found!"
}
else {
Write-Host "Not found!"
}
This approach will give you the correct answer, but it'll become extremely slow on large inputs, at which point you may want to look at non-regex based strategies like Boyer-Moore

PowerShell: retrieve number of applications in AppPool

How to retrieve the number of applications associated with a specific IIS AppPool via PowerShell command?
We can see the associated applications manually using:
Get-Item IIS:\AppPools\AppPoolName
However, if we manually want to select the Applications column, it is not possible. Also, the Applications column is not listed within | Get-Member *.
Why is the column not listed?
How to find the number of applications associated with a specific IIS AppPool using PowerShell?
The trick is: PowerShell established so-called "view definition files" which tell PowerShell how to format objects (e.g. whether the object is formatted as a a list or a table, which columns are displayed, etc.). Those files can be found at C:\Windows\System32\WindowsPowerShell\v1.0 and are all ending in .format.ps1xml.
To answer the original question: The file C:\Windows\System32\WindowsPowerShell\v1.0\Modules\WebAdministration\iisprovider.format.ps1xml contains the view definition for the AppPool type which defines a calculated column looking like this:
<TableColumnItem>
<ScriptBlock>
$pn = $_.Name
$sites = get-webconfigurationproperty "/system.applicationHost/sites/site/application[#applicationPool=`'$pn`'and #path='/']/parent::*" machine/webroot/apphost -name name
$apps = get-webconfigurationproperty "/system.applicationHost/sites/site/application[#applicationPool=`'$pn`'and #path!='/']" machine/webroot/apphost -name path
$arr = #()
if ($sites -ne $null) {$arr += $sites}
if ($apps -ne $null) {$arr += $apps}
if ($arr.Length -gt 0) {
$out = ""
foreach ($s in $arr) {$out += $s.Value + "`n"}
$out.Substring(0, $out.Length - 1)
}
</ScriptBlock>
</TableColumnItem>
This answers why the column itself is not a member of the AppPool type. The second question can be easily answered now extracting the necessary code from the "scriptlet" above:
$applicationsInAppPoolCount = #(Get-WebConfigurationProperty `"/system.applicationHost/sites/site/application[#applicationPool=`'$appPool`'and #path!='/']"` "machine/webroot/apphost" -name path).Count
I dealt with this same issue for many hours until finally arriving at the solution. The answer from D.R. was very helpful but it was not working for me. After some tweaks, I came up with the code below which retrieves the number of applications in an app pool.
I noticed that this part of the code nd #path!='/' threw off the count.
$appPool = "REPLACE ME with a value from your app pool"
#(Get-WebConfigurationProperty "/system.applicationHost/sites/site/application[#applicationPool=`'$appPool`']" "machine/webroot/apphost" -name path).Count
I ended up with the following Code (basically the same as above, but differently formatted)
$appPools = Get-ChildItem –Path IIS:\AppPools
foreach ($apppool in $apppools) {
$appoolName = $apppool.Name
[string] $NumberOfApplications = (Get-WebConfigurationProperty "/system.applicationHost/sites/site/application[#applicationPool='$appoolName']" "machine/webroot/apphost" -name path).Count
Write-Output "AppPool name: $appoolName has $NumberOfApplications applications"
}
I recently came across this post searching for ways to get the active Application Pools. The information provided above was great, but I kept digging to see if there was another way get this information. I was able to find a way to do this through Get-IISSite, which I used the following:
Get-IISSite | Select-Object -ExpandProperty Applications | Select-Object Path,ApplicationPoolName
I tested this on a server that only had one website, but if there are multiple sites on the server, you could also add VirtualDirectories for the Select.
I also had a need to just get a unique list of the Application Pools being used, so I did the following:
$appPoolInfo = Get-IISSite | Select-Object -ExpandProperty Applications | Select-Object Path,ApplicationPoolName
$appPoolInfo | Select-Object -Unique ApplicationPoolName
This gives what you are looking in an array.
Import-Module WebAdministration;
Get-ChildItem IIS:\AppPools >> AppPoolDetails.txt;
$appPoolDetails = Get-Content .\AppPoolDetails.txt;
$w = ($appPoolDetails |Select-String 'State').ToString().IndexOf("State");
$w = $w -1;
$res1 = $appPoolDetails | Foreach {
$i=0;
$c=0; `
while($i+$w -lt $_.length -and $c++ -lt 1) {
$_.Substring($i,$w);$i=$i+$w-1}}
Write-Host "First Column---";
$res1.Trim();
$j = $w + 1;
$w = ($appPoolDetails |Select-String 'Applications').ToString().IndexOf("Applications");
$w = $w -$j;
$res2 = $appPoolDetails | Foreach {
$i=$j;
$c=0; `
while($i+$w -lt $_.length -and $c++ -lt 1) {
$_.Substring($i,$w);$i=$i+$w-1}}
Write-Host "Second Column---";
$res2.Trim();
$lineLength=0
$appPoolDetails | Foreach {
if($lineLength -lt $_.TrimEnd().Length )
{
$lineLength = $_.TrimEnd().Length;
#Write-Host $lineLength;
}
}
$j = ($appPoolDetails | Select-String 'Applications').ToString().IndexOf("Applications");
$w = $lineLength;
$w = $w -$j;
#Write-Host $j $w;
$res3 = $appPoolDetails | Foreach {
$i=$j;
$c=0; `
while($i+$w -lt $_.length -and $c++ -lt 1) {
$_.Substring($i,$w);$i=$i+$w-1}}
Write-Host "Third Column---";
$res3;

PowerShell - paste data into Excel

Today I have just thrown together this PowerShell script which
takes a tab-delimited text file,
reads it into memory,
makes a variable number of filter queries based on distinct values of a certain column
creates a new empty Excel workbook
adds each of the subsets of filtered data to
a new Excel worksheet
The last step is where I am stuck. Currently my code puts a few lines of data into a range in the worksheet, in the form of unrolled/transposed "key: value" entries, resulting in a horizontal data layout. The same range of data is always overwritten.
I want data in the form of a vertical layout, i.e., data in columns, just the same way as if the CSV file was imported with the import-file-wizard of MS Excel.
Is there a simpler way to do it than below?
I admit, some of the PowerShell features are pasted in here in a cargo-cult mode of programming. Please note that I have no PowerShell experience whatsoever. I did some batchfile, VBScript, and VBA coding a few years back. So, other criticisms are also welcome.
PARAM (
[Parameter(ValueFromPipeline = $true)]
$infile = ".\04-2011\110404-13.txt"
)
PROCESS {
echo " $infile"
Write-Host "Num Args:" $args.Length;
$xl = New-Object -comobject Excel.Application;
$xl.Visible = $true;
$Workbook = $xl.Workbooks.Add();
$content = Import-Csv -delimiter "`t" $infile;
$ports = $content | Select-Object Port# | Sort-Object Port# -Unique -Descending;
$ports | ForEach-Object {
$p = $_;
Write-Host $p.{Port#};
$Worksheet = $Workbook.Worksheets.Add();
$workSheet.Name = [string]::Format("{0} {1}", "PortNo", $p.{Port#});
$filtered = $content | Where-Object {$_.{Port#} -eq $p.{Port#} };
$filtered | ForEach-Object {
Write-Host $_.{ObsDateTime}, $_.{Port#}
}
$filtered | clip.exe;
$range = $Workbook.ActiveSheet.Range("a2", "a$($filtered.count)");
$Workbook.ActiveSheet.Paste($range, $false);
}
$xl.Quit()
}
Data Output Example
Wrong
Port# : 1
Obs# : 1
Exp_Flux : 0,99
IV Cdry : 406.96
IV Tcham : 16.19
IV Pressure : 100.7
IV H2O : 9.748
IV V3 : 11.395
IV V4 : 0.759
IV RH : 53.12
Right
Port# Obs# Exp_Flux IV Cdry IV Tcham IV Pressure IV H2O IV V3 IV V4 IV RH
1 1 0,99 406.96 16.19 100.7 9.748 11.395 0.759 53.12
Try Export-Xls, it looks very nice. Never had the chance to use it, but (virtually) knowing the person who worked on it, I'm sure you will be very happy to use it. If you'll go with it, please provide a feedback here will be appreciated.
POSSIBLE WORKAROUND FOR UNORDERED PROPERTIES IN Export-Xls
The function Add-Array2Clipboard could be changed so that it accepts a new input parameter: an array providing the name of the properties ordered as required.
Then the you can change the section where get-member is used. Silly example:
"z", "a", "c" | %{ get-member -name $_ -inputobject $thecurrentobject }
This is just an example on how you can achieve ordered properties from get-member.
I've used the $Workbook.ActiveSheet.Cells.Item($row, $col).Value2 function to more be able to pinpoint more precisely where to put the data when exporting to Excel.
Something like
$row = 1
Get-Content $file | Foreach-Object {
$cols = $_.split("`t")
for ($i = 0; $i < $cols.count; $i++)
{
$Workbook.ActiveSheet.Cells.Item($row, $i+1).Value2 = $cols[$i]
}
$row++
}
Warning: dry-coded! You'll probably need some try..catch as well.
I used a modified Export-Xls function, a bit different as User empo suggested.
This is my call to it
Export-Xls $filtered -Path $outfile -WorksheetName "$wn" -SheetPosition "end" | Out-Null # -SheetPosition "end";
However, the current release of Export-Xls re-orders the columns of the in-memory representation of the csv-text -file. I want the data columns of the text file in their original order, so I had to hack and simplify the original code as follows:
function Add-Array2Clipboard {
param (
[PSObject[]]$ConvertObject,
[switch]$Header
)
process{
$array = #();
$line =""
if ($Header) {
$line = #()
$row = $ConvertObject | Select -First 1
$row.psobject.properties | Foreach {$line += "$($_.Name)" }
$array += [String]::Join("`t", $line)
}
else {
foreach($row in $ConvertObject){
$line =""
$vals = #()
$row.psobject.properties | Foreach {$vals += $_.Value}
$array += [String]::Join("`t", $vals)
}
}
$array | clip.exe;
}
}

Resources