I'm using Windows 7. I have a bunch of text files, each containing one email message. Each starts this way:
FROM: Person
TO: Another Person
DATE: 01-Jan-11 at 18:12:00
SUBJECT: Whatever
I want to rename these files so that their names look like this:
2011-01-01 18.12 Email from Person to Another Person re Whatever.txt
Batch programming is all I know, and I don't know it very well. For purposes of restraining this to a project that I can understand quickly, I think my best solution will be to extract the essential data into a text file that I can then massage into a batch renaming file.
In that case, what I'm looking for is a batch file that will extract the data into single lines in a text file that I can then massage into shape with global edits. In other words, I think I'm looking for text lines in this format:
[current filename] [extracted date and time string] [from] [to] [subject]
Example:
file01.txt 01-Jan-11 at 18:12:00 from Person to Another Person re Whatever
If I've got lines like that, I can parse them into renaming commands pretty quickly in Excel.
Thanks!
Given that your using Windows 7, I thought I'd suggest an alternative. Windows Powershell is a a very useful command tool that can be used for a ton of stuff. I think I solved your complete problem:
$folder = "C:\..."
$regex = "FROM: (.*) TO: (.*) DATE: (.*) at (.*) SUBJECT: (.*)"
$files = Get-ChildItem $folder *.txt
ForEach ($file in $files) {
$line = (Get-Content $file.FullName -TotalCount 1)
$match = ([regex]$regex).matches($line)[0]
$date = [DateTime]($match.Groups[3]).Value + [TimeSpan]($match.Groups[4]).Value
$from = ($match.Groups[1])
$to = ($match.Groups[2])
$subject = ($match.Groups[5])
# You can change the naming format in the brackets below
Rename-Item $file.FullName -NewName ( $date.ToString("yyyy-MM-dd_HH-mm-ss") + " Email From " + $from + " to " + $to + " RE " + $subject)
}
It makes a few assumptions (like a match will always be found). You can easily adjust naming format and other things. Save this code as a script (.ps1) and run it in the Powershell prompt (powershell.exe)
Related
I can't seem to solve this, been struggling with it for a while (maybe it's simple; I just can't see it as I've been looking at it for so long).
I can get the 7z.exe syntax to work, but when I try and put it together into a simple script, it fails.
e.g., if I run .\zip.ps1 "C:\test\test.zip" "C:\test\test1.txt" "C:\test\test2.txt*
Instead of zipping up the 2 files required, it zips up everything in the C:\test folder, completely ignoring my arguments.
How can I adjust the below string syntax so that 7z.exe will correctly respect the input arguments and compress them from within a PowerShell script?
"Received $($args.Count) files:"
$parent = Split-Path (Split-Path $args[0]) -Leaf
$sevenzip = "C:\Program Files\7-Zip\7z.exe"
$zipname = "$($parent) $(Get-Date -Format 'yyyy_MM_dd HH_mm_ss').zip"
$args_line = $args | foreach-object { "`"$_`"" }
# $args_line = '"' + $($args -join """ """) + '"' # Want to use """ here so that it can capture not only files with spaces, but files with ' in the filename
''
"Zip Name : $zipname"
''
"Arguments : $args_line"
''
if (Test-Path $sevenzip) {
if (Test-Path "C:\0\$zipname") { rm "C:\0\$zipname" -Force }
''
'String output of the line to run:'
"& ""$sevenzip"" a -r -tzip ""C:\0\$zipname"" $args_line" # Taking this output and pasting onto console works.
''
& "$sevenzip" a -r -tzip "C:\0\$zipname" "$args_line" # This does not work
} else {
"7z.exe was not found at '$sevenzip', please check and try again"
}
The error that I get is:
Files read from disk: 0
Archive size: 22 bytes (1 KiB)
Scan WARNINGS for files and folders:
1 : The system cannot find the file specified.
----------------
Pass $args directly to your & "$sevenzip" call:
& "$sevenzip" a -r -tzip "C:\0\$zipname" $args
This makes PowerShell pass the array elements as individual arguments, automatically enclosing them in "..." if needed (based on whether they contain spaces).
Using arrays as arguments for external programs is in effect an implicit form of array-based splatting; thus, you could alternatively pass #args.
Generally, note that in direct invocation[1] you cannot pass multiple arguments to an external program via a single string; that is, something like "$args_line" cannot be expected to work, because it is passed as a single argument to the target program.
If you want to emulate the resulting part of the command line, for display purposes:
($argListForDisplay = $args.ForEach({ ($_, "`"$_`"")[$_ -match ' '] })) -join ' '
Note:
Each argument is conditionally enclosed in "..." - namely based on whether it contains at least one space - and the resulting tokens are joined to form a single, space-separated list (string).
The assumption is that no argument has embedded " chars.
A simplified example:
& { # an ad-hoc script block that functions like a script or function
# Construct the argument list *for display*
$argListForDisplay = $args.ForEach({ ($_, "`"$_`"")[$_ -match ' '] }) -join ' '
#"
The following is the equivalent of:
Write-Output $argListForDisplay
"#
# Pass $args *directly*
# Simply prints the argument received one by one, each on its own line.
# Note: With PowerShell-native commands, generally use #args instead (see below).
Write-Output $args
} firstArg 'another Arg' lastArg # sample pass-through arguments
Output:
The following is the equivalent of:
Write-Output firstArg "another Arg" lastArg
firstArg
another Arg
lastArg
Note: Write-Output is used in lieu of an external program for convenience. Technically, you'd have to use #args instead of $args in order to pass the array elements as individual, positional arguments, but, as stated, this is the default behavior with externals programs. Write-Output, as a PowerShell-native command, receives the array as a whole, as a single argument when $args is used; it just so happens to process that array the same way as if its elements had been passed as individual arguments.
[1] You can use a single string as an -ArgumentList value for Start-Process. However, Start-Process is usually the wrong tool for invoking console applications such as 7z.exe - see this answer.
I have a single file that contains 1GB worth of data. This data is actually 10's of thousands of individual mini files.
I need to extract each individual file and place them in their own separate Distinct file.
So essentially, I need to go from a single file to 30K+ separate files.
Here is a sample of what My file looks like.
FILENAM1 VER 1 32 D
10/15/87 09/29/87
PREPARED BY ?????
REVISED BY ?????
DESCRIPTION USER DOMAIN
RECORD FILENAM1 VER 1 D SUFFIX -4541
100 05 ST-CTY-CDE-FMHA-4541 DISPLAY
200 10 ST-CDE-FMHA-4541 9(2) DISPLAY
300 10 CTY-CDE-FMHA-4541 9(3) DISPLAY
400 05 NME-CTY-4541 X(20) DISPLAY
500 05 LST-UPDTE-DTE-4541 9(06) DISPLAY
600 05 FILLER X DISPLAY 1REPORT NO. 08
DATA DICTIONARY REPORTER REL 17.0 09/23/21
PAGE 2 DREPORT 008
RECORD REPORT
-************************************************************************************************************************************
RECORD RECORD ---- D A T E ----
RECORD NAME LENGTH BUILDER TYPE
OCCURRENCES UPDATED CREATED
************************************************************************************************************************************ 0
FILENAM2 VER 1 176 D
03/09/98 02/21/84
PREPARED BY ??????
REVISED BY ??????
DEFINITION
I Need split the files out based upon a match of VER in position 68, 69 and 70. I also need to name each file uniquely. That information is stored on the same line in position 2-9. In the example above that string is "FILENAM1" and FILENAM2".
So just using the example above I would create two output files and they would be named FILENAM1.txt and FILENAM2.txt.
Since I have 30K+ files I need to split, doing this manually is impossible.
I do have a script that will split a file into multiple files but it will not search for strings by position.
Would anyone be able to assist me with this?
Here is script that DOES NOT Work. Hopefully I can butcher it and get some valid results....
$InputFile = "C:\COPIES.txt"
$Reader = New-Object System.IO.StreamReader($InputFile)
$OPName = #()
While (($Line = $Reader.ReadLine()) -ne $null) {
If ($Line -match "VER"(67,3)) {
$OPName = $Line.(2,8)
$FileName = $OPName[1].Trim()
Write-Host "Found ... $FileName" -foregroundcolor green
$OutputFile = "$FileName.txt"
}
Add-Content $OutputFile $Line
}
Thank you in advance,
-Ron
I suggest using a switch statement, which offers both convenient and fast line-by-line reading of files via -File and regex-matching via -Regex:
$streamWriter = $null
switch -CaseSensitive -Regex -File "C:\COPIES.txt" {
'^.(.{8}).{58}VER' { # Start of a new embedded file.
if ($streamWriter) { $streamWriter.Close() } # Close previous output file.
# Create a new output file.
$fileName = $Matches[1].Trim() + '.txt'
$streamWriter = [System.IO.StreamWriter] (Join-Path $PWD.ProviderPath $fileName)
$streamWriter.WriteLine($_)
}
default { # Write subsequent lines to the same file.
if ($streamWriter) { $streamWriter.WriteLine($_) }
}
}
$streamWriter.Close()
Note: A solution using the .Substring() method of the [string] type is possible too, but would be more verbose.
The ^.(.{8}).{58} portion of the regex matches the first 67 characters on each line, while capturing those in (1-based) columns 2 through 9 (the file name) via capture group (.{8}), which makes the captured text available in index [1] of the automatic $Matches variable. The VER portion of the regex then ensures that the line only matches if VER is found at column position 68.
For efficient output-file creation, [System.IO.StreamWriter] instances are used, which is much faster than line-by-line Add-Content calls. Additionally, with Add-Content you'd have to ensure that a target file doesn't already exist, as the existing content would then be appended to.
I have a log file like this :
[2021/04/13 18:21:57.577+02:00][VERBOSE] Finished: 0 file(s), 5.23 GB; Average Speed:17.26 MB/s.
I just want to remove all string between the "," and "/s." I tried many times I can't do it correctly.
Can someone help me to do this on Powershell ?
If you do not only need to get the interesting part from the log line, but also need to be able to do math on the number ('5.23 GB') in your example, you need to do some more splitting:
foreach ($line in (Get-Content -Path 'TheFile.log')) {
$interestingPart = ($line -split ',')[-1].Trim()
$logSize, $logAverage = $interestingPart -split ';'
$size, $unit = $logSize -split '\s+'
# calculate the size from both the number ('5.23') and the unit ('GB')
$size = [double]::Parse($size) * "1$unit"
# now you have the number to do further math on
}
I have a text file which is comprised of only one line. I have had much trouble with splitting the file into a specific number of characters, then adding a string in front of each chunk of characters.
With a multi-line file, I can add characters to each line very easily using
Get-Content -Path $path | foreach-object {$string + $_} | out-file $output but it is much more complicated with a file with only one line.
For example, if I had a file with these random characters,
(******************************************) and i wanted to add a string to the start of every 10 chars, then it would look like this, (examplestring**********examplestring**********examplestring**********) and so on. I have researched everywhere but I have just managed to add the chars to the end of each chunk of characters.
Does anyone have a way of doing this? Preferably using streamreader and writer as get-content may not work for very large files. Thanks.
Hmm, there are some dynamic parameters applicable to file-system get-content and set-content commands that are close to what you are asking for. For example, if test.txt contains a number of * characters, you might interleave every four * with two + characters with something like this:
get-content .\test.txt -Delimiter "****" | % { "++$_" } | Set-Content test2.txt -NoNewline
I don't know how close that is to a match for what you want, but it's probably useful to know that some of these provider-specific parameters, like '-Delimiter' aren't obvious. See https://technet.microsoft.com/en-us/library/hh847764.aspx under the heading 'splitting large files'.
Alternatively, here's a quick function that reads length-delimited strings from a file.
Set-StrictMode -Version latest
function read-characters( $path, [int]$charCount ) {
begin {
$buffer = [char[]]::new($charCount)
$path = Join-Path $pwd $path
[System.IO.StreamReader]$stream = [System.IO.File]::OpenText($path)
try {
while (!$stream.EndOfStream) {
$len = $stream.ReadBlock($buffer,0,$charCount);
if ($len) {Write-Output ([string]::new($buffer,0,$len))}
}
} catch {
Write-Error -Exception $error[0]
} finally {
[void]$stream.Close()
}
}
}
read-characters .\test.txt -charCount 10 |
% {"+$_"} |
write-host -NoNewline
It could use some parameter checking, but should get you started...
With a manageable file size, you might want to try something like this:
$directory = "C:\\"
$inputFile = "test.txt"
$reader = new-object System.IO.StreamReader("{0}{1}" -f ($directory, $inputFile))
# prefix string of each line
$startString = "examplestring"
# how many chars to put on each line
$range = 10
$outputLine = ""
$line = $reader.ReadLine()
$i = 0
while ($i -lt $line.length) {
$outputLine += $($startString + $line.Substring($i, [math]::min($range, ($line.length - $i))))
$i += $range
}
$reader.Close()
write-output $outputLine
Basically it's using substring to cut out each chunk, prefixing the chumk with given string, and appending to the result variable.
Sample input:
==========================
Sample output:
examplestring==========examplestring==========examplestring======
I'd like to write a script/batch that will bunch up my daily IIS logs and zip them up by month.
ex080801.log which is in the format of exyymmdd.log
ex080801.log - ex080831.log gets zipped up and the log files deleted.
The reason we do this is because on a heavy site a log file for one day could be 500mb to 1gb so we zip them up which compresses them by 98% and dump the real log file. We use webtrend to analyze the log files and it is capable of reading into a zip file.
Does anyone have any ideas on how to script this or would be willing to share some code?
You'll need a command line tool to zip up the files. I recommend 7-Zip which is free and easy to use. The self-contained command line version (7za.exe) is the most portable choice.
Here's a two-line batch file that would zip the log files and delete them afterwards:
7za.exe a -tzip ex%1-logs.zip %2\ex%1*.log
del %2\ex%1*.log
The first parameter is the 4 digit year-and-month, and the second parameter is the path to the directory containing your logs. For example: ziplogs.bat 0808 c:\logs
It's possible to get more elaborate (i.e. searching the filenames to determine which months to archive). You might want to check out the Windows FINDSTR command for searching input text with regular expressions.
Here's my script which basically adapts David's, and zips up last month's logs, moves them and deletes the original log files. this can be adapted for Apache logs too.
The only problem with this is you may need to edit the replace commands, if your DOS date function outputs date of the week.
You'll also need to install 7-zip.
You can also download IISlogslite but it compresses each day's file into a single zip file which I didn't find useful. There is a vbscript floating about the web that does the same thing.
-------------------------------------------------------------------------------------
#echo on
:: Name - iislogzip.bat
:: Description - Server Log File Manager
::
:: History
:: Date Authory Change
:: 27-Aug-2008 David Crow Original (found on stack overflow)
:: 15-Oct-2008 AIMackenzie Slimmed down commands
:: ========================================================
:: setup variables and parameters
:: ========================================================
:: generate date and time variables
set month=%DATE:~3,2%
set year=%DATE:~8,2%
::Get last month and check edge conditions
set /a lastmonth=%month%-1
if %lastmonth% equ 0 set /a year=%year%-1
if %lastmonth% equ 0 set lastmonth=12
if %lastmonth% lss 10 set lastmonth=0%lastmonth%
set yymm=%year%%lastmonth%
set logpath="C:\WINDOWS\system32\LogFiles"
set zippath="C:\Program Files\7-Zip\7z.exe"
set arcpath="C:\WINDOWS\system32\LogFiles\WUDF"
:: ========================================================
:: Change to log file path
:: ========================================================
cd /D %logpath%
:: ========================================================
:: zip last months IIS log files, move zipped file to archive
:: then delete old logs
:: ========================================================
%zippath% a -tzip ex%yymm%-logs.zip %logpath%\ex%yymm%*.log
move "%logpath%\*.zip" "%arcpath%"
del %logpath%\ex%yymm%*.log
We use a script like the following. Gzip is from the cygwin project. I'm sure you could modify the syntax to use a zip tool instead. The "skip" argument is the number of files to not archive off -- we keep 11 days in the 'current' directory.
#echo off
setlocal
For /f "skip=11 delims=/" %%a in ('Dir D:\logs\W3SVC1\*.log /B /O:-N /T:C')do move "D:\logs\W3SVC1\%%a" "D:\logs\W3SVC1\old\%%a"
d:
cd "\logs\W3SVC1\old"
gzip -n *.log
Endlocal
exit
You can grab the command-line utilities package from DotNetZip to get tools to create zips from scripts. There's a nice little tool called Zipit.exe that runs on the command line, adds files or directories to zip files. It is fast, efficient.
A better option might be to just do the zipping from within PowerShell.
function ZipUp-Files ( $directory )
{
$children = get-childitem -path $directory
foreach ($o in $children)
{
if ($o.Name -ne "TestResults" -and
$o.Name -ne "obj" -and
$o.Name -ne "bin" -and
$o.Name -ne "tfs" -and
$o.Name -ne "notused" -and
$o.Name -ne "Release")
{
if ($o.PSIsContainer)
{
ZipUp-Files ( $o.FullName )
}
else
{
if ($o.Name -ne ".tfs-ignore" -and
!$o.Name.EndsWith(".cache") -and
!$o.Name.EndsWith(".zip") )
{
Write-output $o.FullName
$e= $zipfile.AddFile($o.FullName)
}
}
}
}
}
[System.Reflection.Assembly]::LoadFrom("c:\\\bin\\Ionic.Zip.dll");
$zipfile = new-object Ionic.Zip.ZipFile("zipsrc.zip");
ZipUp-Files "DotNetZip"
$zipfile.Save()
Borrowed zip function from http://blogs.msdn.com/daiken/archive/2007/02/12/compress-files-with-windows-powershell-then-package-a-windows-vista-sidebar-gadget.aspx
Here is powershell answer that works wonders:
param([string]$Path = $(read-host "Enter the path"))
function New-Zip
{
param([string]$zipfilename)
set-content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $zipfilename).IsReadOnly = $false
}
function Add-Zip
{
param([string]$zipfilename)
if(-not (test-path($zipfilename)))
{
set-content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $zipfilename).IsReadOnly = $false
}
$shellApplication = new-object -com shell.application
$zipPackage = $shellApplication.NameSpace($zipfilename)
foreach($file in $input)
{
$zipPackage.CopyHere($file.FullName)
Start-sleep -milliseconds 500
}
}
$FilesToZip = dir $Path -recurse -include *.log
foreach ($file in $FilesToZip) {
New-Zip $file.BaseName
dir $($file.directoryname+"\"+$file.name) | Add-zip $($file.directoryname+"\$($file.basename).zip")
del $($file.directoryname+"\"+$file.name)
}
We use this powershell script: http://gallery.technet.microsoft.com/scriptcenter/31db73b4-746c-4d33-a0aa-7a79006317e6
It uses 7-zip and verifys the files before deleting them
Regex will do the trick... create a perl/python/php script to do the job for you..
I'm pretty sure windows batch file can't do regex.