Tuning PowerShell script for Logs - windows-server-2008-r2

Question

I have written the following script for generating a report regarding SSRS, Excel and performance point by analyzing IIS logs:
Write-Host "`r"
if(!(Test-Path E:\BI_ToolUsage)){New-Item E:\BI_ToolUsage -type directory -force}
$today=(get-date).ToString("dd_MM_yyyy")
Write-Host "Content Farm Selected" -f green
$ConPaths="\\Server_logs\IC1-PS502\IIS-exports\*.log"
"date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken" |Out-file E:\BI_ToolUsage\Content_SSRS_$today.csv
"date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken" |Out-file E:\BI_ToolUsage\Content_ExcelService_$today.csv
"date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken" |Out-file E:\BI_ToolUsage\Content_PerformancePoint_$today.csv
Foreach($path in $ConPaths)
{
$path
gc $path |?{($_ | Select-String "RSViewerPage.aspx" | Out-file E:\BI_ToolUsage\Content_SSRS_$today.csv -Width 30000 -append);($_ | Select-String "xlviewer.aspx" | Out-file E:\BI_ToolUsage\Content_ExcelService_$today.csv -append -Width 30000);($_ | Select-String "PPSWebParts/ppsDashboard.css" |Out-file E:\BI_ToolUsage\Content_PerformancePoint_$today.csv -Width 30000 -append)}
}
$files=Get-ChildItem E:\BI_ToolUsage\Content*
foreach($file in $files){(Get-Content $file.FullName)| Where-Object {$_ -match '\S'}|%{$_ -replace " " , ","}|Out-File $file.FullName}
Write-Host "Completed !"
The thing here is the size of the folder is 19.3 GB and there are 671 files in the location : \Server_logs\IC1-PS502\IIS-exports
When I run this script, it usually takes around 3-4 days for generating the reports. Is it somehow possible to optimize the above script which can reduce the time it takes to complete? As per my knowledge modification in the below piece of code can help a lot:
gc $path |?{($_ | Select-String "RSViewerPage.aspx" | Out-file E:\BI_ToolUsage\Content_SSRS_$today.csv -Width 30000 -append);($_ | Select-String "xlviewer.aspx" | Out-file E:\BI_ToolUsage\Content_ExcelService_$today.csv -append -Width 30000);($_ | Select-String "PPSWebParts/ppsDashboard.css" |Out-file E:\BI_ToolUsage\Content_PerformancePoint_$today.csv -Width 30000 -append)}
Kindly look into this and share your views. Thanks

Answer 1

If you're interested in LogParser, there's any number of sites, but honestly I found the command line help, LogParser.exe -h, and the LogParser.chm file that installs with the program pretty helpful.
Start with something like:
LogParser.exe -i:IISW3C -o:csv "SELECT date, time, s-ip, cs-method, cs-uri-stem,
cs-uri-query, s-port, cs-username, c-ip, cs(User-Agent), sc-status, sc-substatus,
sc-win32-status, time-taken INTO C:\Output\Content_SSRS_2015-08-01.csv
FROM C:\Logs\ex150801.log WHERE cs-uri-query LIKE '%RSViewerPage.aspx%'"`
Note that I may have field names wrong in the query here. You might want to start with SELECT * just to see how it all works. There's also additional fields that LogParser adds (i.e., line number, file name, et al) which you'll see if you run SELECT *. I'm also guessing that the string you're looking for is in cs-uri-query. Without seeing the file it's hard to tell, and it's been just long enough since I've worked with IIS logs to forget.
It's not the most straightforward tool to learn, but the command line help and help file are very good, especially if you're familiar with command line and are somewhat familiar with SQL. If you put a day into learning how it works, it really pays off.
An alternative is the Log Parser Studio program which has a GUI, but I've never really cared for it. I don't like the interface, but I'm very comfortable with command line.