powershell - remove string containing line breaks and spaces - string

I have a script running in powershell (v2), that removes strings from a file.
The basic process is:
(Get-Content $Local_Dir1\$filename1) -replace 'longString', 'shortString' | `
Set-Content $cfg_Local_Dir\$filename1
Get-Content $Local_Dir1\$filename1 | `
Where-Object {$_ -notmatch 'stringToMatch'} | `
Where-Object {$_ -notmatch 'secondStringToMatch'} | `
Set-Content $Local_Dir1\$filename
This works fine. However, I have an annoying string that I can't get rid of.
It basically consists of: a line break and carriage return, 4 spaces, and then a line break and carriage return. In HEX it is 0D 0A 20 20 20 20 0D 0A
How can I remove this?
I tried simply:
Where-Object {$_ -notmatch ' '} #4 x spaces
But that removed all content after that line (and this is on the second line).
I looked at:
Where-Object {$_ -notmatch '$([char]0x0D)'}
(I would have expanded it if it had removed all the Carriage Returns) which I saw in another post somewhere, but that did nothing.
What is the correct way of dealing with this problem?
Additional: 2015-11-24 13:49
Example Data:
<?xml version="1.0" encoding="UTF-8"?>
<start_of_data>
<job>123456</job>
<name>ABC123</name>
<start></start>
</start_of_data>
<start_of_data>
<job>789012</job>
<name>DEF345</name>
<start></start>
</start_of_data>
Initially there is a string on line 2 which is removed by 'stringToMatch', and the spaces are on line3.

Couple of things worth pointing out here. When you use -match/-notmatch you are using regex. We can consolidate your strings and space issue into one string.
Get-Content $Local_Dir1\$filename1 |
Where-Object {$_ -notmatch 'stringToMatch|secondStringToMatch|\s{4,}'} |
Set-Content $Local_Dir1\$filename
That works using alternation to match either element separated by pipes. This is by no means perfect as we don't have sample data to work with but if you have lines with either of those two string or at least 4 consecutive spaces they will be omitted.
From talking in the comments and looking at the example file you are just trying to omit lines that are blank. Using another string class or regex could fix that. These lines function differently but would both ignore lines that are just white-space.
![string]::IsNullOrWhiteSpace($_)
-notmatch ^\s+$
I will op'd for the former as it is more intuitive.
Where-Object {![string]::IsNullOrWhiteSpace($_) -and $_ -notmatch 'stringToMatch|secondStringToMatch'}
Like I said in comments if you are picky on this requirement that you could filter out lines with exactly 4 white-space characters with -notmatch ^\s{4}$
Also like sodawillow says you should have used double quotes to allow variable expansion. Since you are using regex \r would have worked just as well.
Where-Object {$_ -notmatch "$([char]0x0D)"}
However I don't think you would have seen that character anyway in order to exclude it. Get-Content would scrub that out to make a string array. That might depend on encoding.

Try .Net String class:
Where-Object {-not[string]::IsNullOrEmpty(([string]$_).trim())}
Trim will remove spaces and IsNullOrEmpty will check the rest.

Related

Manipulate strings in a txt file with Powershell - Search for words in each line containing "." and save them in new txt file

I have a text file with different entries. I want to manipulate it to filter out always the word, containing a dot (using Powershell)
$file = "C:\Users\test_folder\test.txt"
Get-Content $file
Output:
Compass Zype.Compass 1.1.0 thisisaword
Pomodoro Logger zxch3n.PomodoroLogger 0.6.3 thisisaword
......
......
......
Bla Word Program.Name 1.1.1 this is another entry
As you can see, in all lines, the "second" "word" contains a dot, like "Program.Name".
I want to create a new file, which contains just those words, each line one word.
So my file should look something like:
Zype.Compass
zxch3n.PomodoroLogger
Program.Name
What I have tried so far:
Clear-Host
$folder = "C:\Users\test_folder"
$file = "C:\Users\test_folder\test.txt"
$content_txtfile = Get-Content $file
foreach ($line in $content_textfile)
{
if ($line -like "*.*"){
$line | Out-File "$folder\test_filtered.txt"
}
}
But my output is not what I want.
I hope you get what my problem is.
Thanks in advance! :)
Here is a solution using Select-String to find sub strings by RegEx pattern:
(Select-String -Path $file -Pattern '\w+\.\w+').Matches.Value |
Set-Content "$folder\test_filtered.txt"
You can find an explanation and the ability to experiment with the RegEx pattern at RegEx101.
Note that while the RegEx101 demo also shows matches for the version numbers, Select-String gives you only the first match per line (unless argument -AllMatches is passed).
This looks like fixed-width fields, and if so you can reduce it to this:
Get-Content $file | # Read the file
%{ $_.Substring(29,36).Trim()} | # Extract the column
?{ $_.Contains(".") } | # Filter for values with "."
Set-Content "$folder\test_filtered.txt" # Write result
Get-content is slow and -like is sometimes slower than -match. I prefer -match but some prefer -like.
$filename = "c:\path\to\file.txt"
$output = "c:\path\to\output.txt"
foreach ($line in [System.IO.File]::ReadLines($filename)) {
if ($line -match "\.") {
$line | out-file $output -append
}
}
Otherwise for a shorter option, maybe
$filename = "c:\path\to\file.txt"
$output = "c:\path\to\output.txt"
Get-content "c:\path\to\file.txt" | where {$_ -match "\.") | Out-file $output
For other match options that are for the first column, either name the column (not what you do here) or use a different search criteria
\. Means a period anywhere seein the whole line
If it's all periods and at the beginning you can use begining of line so..
"^\." Which means first character is a period.
If it's always a period before the tab maybe do an anything except tab period anything except tab or...
"^[^\t]*\.[^\t]*" this means at the start of the line anything except tab any quantity then a period then anything except a tab any number of times.

Extract substrings where match is found

I have a text file with a number of lines. I would like to search each line individually for a particular pattern and, if that pattern is found output a substring at a particular position relative to where the pattern was found.
i.e. if a line contains the pattern at position 20, I would like to output the substring that begins at position 25 on the same line and lasts for five characters.
The following code will output every line that contains the pattern:
select-string -path C:\Scripts\trimatrima\DEBUG.txt -pattern $PATTERN
Where do I go from here?
You can use the $Matches automatic variable:
Last match is stored in $Matches[0], but you can also use named capture groups, like this:
"test","fest","blah" |ForEach-Object {
if($_ -match "^[bf](?<groupName>es|la).$"){
$Matches["groupName"]
}
}
returns es (from "fest") and la (from "blah")
Couple of options.
Keeping Select-String, you'll want to use the .line property to get your substrings:
select-string -path C:\Scripts\trimatrima\DEBUG.txt -pattern $PATTERN |
foreach { $_.line.Substring(19,5) }
For large files, Get-Content with -ReadCount and -match may be faster:
Get-Content C:\Scripts\trimatrima\DEBUG.txt-ReadCount 1000 |
foreach {
$_ -match $pattern |
foreach { $_.substring(19,5) }
}

Replacing a set of strings containing any character

Is there a way to replace a set of string no matter what the string contains?
I am trying to replace one string containing: quotes(""), brackets([]), #, e.
gci C:\test *.txt -recurse | ForEach {(Get-Content $_ | ForEach {$_ -replace '"my"', "money"}) | Set-Content $_ }
but what if a string I want to replace has EVERYTHING in <>:
PowerPlayReport Product_version="10.2.6100.36" xmlns="http://www.cognos.com/powerplay/report[1234#1]" Author="PPWIN" Version="4.0"
So you want to replace everything in [] in the sample text you included in your question. If you were not aware ( although I think you are now ) -replace supports regular expressions. A simple regex can find the text you are looking for. I am also going remove some of the redundancy in your code.
Get-ChildItem C:\test -Filter *.txt -Recurse | ForEach-Object{
$file = $_.FullName
(Get-Content $file) -replace "\[.*?]","[bagel]" | Set-Content $file
}
Explanation borrowed from regex101.com
\[ matches the character [ literally
.*? matches any character (except newline). Quantifier: *? Between zero and unlimited times, as few times as possible, expanding as needed [lazy]
] matches the character ] literally
So that line would then appear as the following inside the source file.
PowerPlayReport Product_version="10.2.6100.36" xmlns="http://www.cognos.com/powerplay/report[bagel]" Author="PPWIN" Version="4.0"

Specifying certain characters for powershell script to run on

This may be a complete shot in the dark, I have done some research and can't seem to find anything on this. But anythings possible with powershell I guess!
I asked a question earlier here! on how to change certain characters in a script.
$infopath = Get-ChilItem "C:\Users\X\Desktop\Info\*.txt" -Recurse
$infopath | %{
(gc $_) -replace "bs", "\" -replace "fs", "/" -replace "co", ":" -replace ".name", "" | Set-Content $_.fullname
However there are some parts of the text file, that may contain bs, fs or co, that I don't want changing. Therefore what I would like to do, is add some sort of parameter into this existing script that only changes the text in the last 70 characters, or after the 4th # or after the 78th character.
Like I said, this could well be a ridiculous idea, but I would like to see other peoples views.
Any help greatly appreciated!
Using your suggestion of getting the substring after the 78th character, I've created the following using 78 as the starting point. This leaves the first 78 characters unedited, and replaces the given strings after the 78th character. The whole file is replaced with the output.
$infopath = Get-ChilItem "C:\Users\X\Desktop\Info\*.txt" -Recurse
$startpos = 78
$infopath | %{
$content = (gc $_);
$content.Substring(0,$startpos)+($content.Substring($startpos,$content.Length-$startpos) -replace "bs", "\" -replace "fs", "/" -replace "co", ":" -replace ".name", "")|Set-Content $_.fullname
}
I hope this helps.
Thanks, Chris.

Passing string included : signs to -Replace Variable in powershell script

$FilePath = 'Z:\next\ResourcesConfiguration.config'
$oldString = 'Z:\next\Core\Resources\'
$NewString = 'G:\PublishDir\next\Core\Resources\'
Any Idea how can you replace a string having : sign in it. I want to change the path in a config file. Simple code is not working for this. tried following
(Get-Content $original_file) | Foreach-Object {
$_ -replace $oldString, $NewString
} | Set-Content $destination_file
The Replace operator takes a regular expression pattern and '\' is has a special meaning in regex, it's the escape character. You need to double each backslash, or better , use the escape method:
$_ -replace [regex]::escape($oldString), $NewString
Alterntively, you can use the string.replace method which takes a string and doesn't need a special care:
$_.Replace($oldString,$NewString)
Try this,
$oldString = [REGEX]::ESCAPE('Z:\next\Core\Resources\')
You need escape the pattern to search for.
This works:
$Source = 'Z:\Next\ResourceConfiguration.config'
$Dest = 'G:\PublishDir\next\ResourceConfiguration.config'
$RegEx = "Z:\\next\\Core\\Resources"
$Replace = 'G:\PublishDir\next\Core\Resources'
(Get-Content $FilePath) | Foreach-Object { $_ -replace $RegEx,$Replace } | Set-Content $Dest
The reason that your attempt wasn't working is that -replace expects it's first parameter to be a Regular Expression. Simply put, you needed to escape the backslashes in the directory path, which is done by adding an additional backspace (\\). It expects the second parameter to be a string, so no changes need to be done there.

Resources