Use Powershell to extract remainder of line after certain text - string

Question

I have a text file with headlines from a LexisNexis Search. I would like to extract the headline from each entry, which comes after the string "HEADLINE: " in the file, and append it to another text file using PowerShell. I am using this line:
select-string -path "C:\Users\WGA\Documents\Personal\ANTH_5330\Content_Analysis\Newspaper_Stories,_Combined_Papers2016-04-18_17-59.txt" -Pattern "HEADLINE: " | select line | out-file C:\Users\WGA\Documents\Personal\ANTH_5330\Content_Analysis\Headlines.txt -append
It is sort of working and I am looking to improve the output. I am linking to the two files below (One is the file to be searched, the other is the output):
https://drive.google.com/folderview?id=0Byxg512qAqFgU0JrRTNUbVlkeGs&usp=sharing
I am open to suggestions to improve this output as, ideally, I would like one line per headline only in the output file.

Answer 1

Let use the regex a little more to get exactly what we want and nothing more. Select-String returns match info objects that contain much of the information you are looking for, including capture groups. Knowing the object properties certain helps. I am assuming you have PowerShell 2.0 for this so it is a little more verbose but works just as well.
$path = "D:\Downloads\Newspaper_Stories,_Combined_Papers2016-04-18_17-59.TXT"
Get-Content $path | Out-String | Select-String -Pattern "(?smi)HeadLine: (.*?)`r`n`r`n" -AllMatches |
Select-Object -ExpandProperty Matches |
ForEach-Object{$_.Groups[1]} |
ForEach-Object{$_.Value -replace "`r`n"," "} |
Set-Content $outputFile
We read in the file as one large string. That is what Out-String is for. We do that since some of your headlines take up multiple lines. Find every line that has "headline" and then grab everything after the colon space up until the first set of newlines. The text we are looking for is inside the capture group (.*?). Next we have to expand the matches objects to get into the groups. Using for each we get the second group which contains our captured group text. A second for each replaces all the newlines with spaces so that the headlines appear as one line in the output.
I noticed that your output file had extra spaces. That is because the default encoding of Out-File is Unicode. Using Set-Content means you won't have to worry about that.
Another thing. If I am wrong and you prefer what you have you can at least skip the header of your output file by changing the select statement to use -ExpandProperty
Sample Output
Charter Schools Fall Short In Public Schools Matchup
State's charter schools buck trend Students at the 108 charters in Colorado have scored higher on state assessment tests than their peers in traditional public schools.
Bills would bypass districts to create charter schools
EDITORIAL The reality of charter schools
EDITORIAL Learning more about charters As Colorado and the nation gain more experience with charter schools, we're discovering that results are mixed-- not unlike public schools.
SPEAK OUT;2 studies, 2 views of charter schools
... output truncated.

Answer 2

try this
Get-Content c:\temp\stories.txt | ? {$_.startswith('HEADLINE: ')} | % {$_.substring(10)} | Out-File c:\temp\headlines.txt -enc ascii