Im currently working on a programm that needs a .xml file, reads it into a Oracle Database and afterwards exports a new .xml file. But the problem is that the new file has to have the exact same name as the original file.
I saved the original filenames into a .txt file and i'm now trying to search for a keyword inside the lines to rename the right files with the correct names inside the .txt file. Here an example:
My 4 files (exported from the Database):
PM.Data_information.xml
PM.Data_location.xml
PM.Cover_quality.xml
PM.Cover_adress.xml
Content of Namefile.txt (original names):
PM.Data_information_provide_SE-R-SO_V0220_657400509_3_210.xml
PM.Data_location_provide_SE-R-SO_V0220_9191200509_3_209.xml
PM.Cover_quality_provide_SE-R-SO_V0220_354123509_3_211.xml
PM.Cover_adress_provide_SE-R-SO_V0220_521400509_3_212.xml
I only worked out how to get a line by selecting the linenumber:
$content = Get-Content C:\Namefile.txt
$informationanme = $content[0]
Rename-Item PM.Data_information.xml -NewName $informationname
Isn't there a way to select that line by searching for the keyword inside the string?
$content = Get-Content C:\temp\ps\NewFile.txt
$files = Get-ChildItem c:\temp\ps\
$content |
%{
$currentLine = $_
$file = $files | Where-Object { $currentLine.StartsWith($_.Name.Replace(".xml", "")) }
Rename-Item $file.Name $currentLine
}
This code should do the trick. Note you will need to have all of your files that need renaming in one folder. Set the folder path to the $files variable (currently set to c:\temp\ps). Set the path where your NewFile.txt is to the $content path.
The code works by looping around each line in the NewFile.txt and finding any file where the name matches the start of the line (if there are any files that do not follow this pattern you will obviously need to update the code but hopefully gives you a good starting point).
other solution ;)
gci -Path "c:\temp" -File -Filter "*.xml" | % { rni $_.fullname (sls "C:\temp\Namefile.txt" -Pattern ([System.IO.Path]::GetFileNameWithoutExtension($_.fullname))).Line }
Related
I'm not understanding where the recursion is occurring nor how it's used in the below tree function (which is meant to emulate some of the linux tree command results).
From the tree function, how are files (or file names and their path) passed to, here, a SearchString function?
for context, here's a REPL session demonstrating the end-goal on a single file: getting the PSPath property for a file, and using that property for a simple regex.
Session transcript:
posh> $dir = "/home/nicholas/Calibre Library/Microsoft Office User/549 (1476)"
posh> $files = Get-ChildItem -Path $dir –File
posh> $files.Length
3
posh> $files[0].Extension
.txt
posh> $files[0].PSPath
Microsoft.PowerShell.Core\FileSystem::/home/nicholas/Calibre Library/Microsoft Office User/549 (1476)/549 - Microsoft Office User.txt
posh> $pattern = '(?=.*?foo)(?=.*?bar)'
posh> $string = Get-Content $files[0]
posh> $string | Select-String $pattern
This file doesn't have any "foo" and "bar" matches. The goal is to search the entire Calibre library using PowerShell as above.
large output from a tree of the Calibre library trimmed to a single result:
Directory: /home/nicholas/Calibre Library/Microsoft Office User/548 (1474)
Mode LastWriteTime Length Name
---- ------------- ------ ----
----- 2/20/2021 3:22 AM 159883 548 - Microsoft Office User.txt
----- 2/20/2021 2:13 AM 351719 cover.jpg
----- 2/20/2021 2:39 AM 1126 metadata.opf
posh> ./worker.ps1
How is the above file and path passed to the SearchString function?
the goal being to iterate through the entire library and search all plain-text file. (Assumption being that plain-text files have a ".txt" extension.)
library code:
function SearchFile($dir,$file)
{
$path = [string]::Concat($dir,"/",$file)
$pattern='(?=.*?foo)(?=.*?bar)'
$string = Get-Content $path
$result = $string | Select-String $pattern
$result
}
function tree($dir)
{
"$dir"
$tree = Get-ChildItem -Recurse
$tree = Get-ChildItem -Path $dir -Recurse
# get any files and invoke SearchFile here ?
$tree
}
worker code:
. /home/nicholas/powershell/functions/library.ps1
$dir = "/home/nicholas/Calibre Library"
tree $dir
The execution of the SearchFile function should be triggered when a ".txt" file is found. That logic is missing. But the larger missing piece is how to invoke SearchFile from the tree function so that every file gets searched.
How is that done? Leaving aside the file-type or file extension. Not seeing where the recursion occurs.
You are really overcomplicating things. You can do this very easily by using Get-ChildItem to find your txt files recursively in $dir path and then piping these FileInfo objects directly to Select-String cmdlet which accepts pipeline input and will grab the PSPath from the FileInfo object being passed to it and do its thing. Select-String will do this for every object that Get-ChildItem sends to it which are FileInfo objects for all txt files found recursively in your $dir path.
$dir = '/home/nicholas/Calibre Library/Microsoft Office User/549 (1476)'
Get-ChildItem -Recurse -Path $dir -Filter *.txt |
Select-String -Pattern '(?=.*?foo)(?=.*?bar)'
Get-ChildItem already does the recursion for you when you specify the -Recurse argument. For your code it doesn't make any difference. You get a linear list of all file informations that you can process using ForEach-Object in the same way as if you didn't specify -Recurse.
The SearchFile function should be executed when a ".txt" file is found.
Use the -Filter parameter to specify *.txt. Also when you want to get files only, always pass -File. This allows the filesystem provider to already skip directories, which is faster and also more correct (in theory there could be directories named e. g. foo.txt which would let SearchFile run into an error).
function tree($dir)
{
"$dir"
Get-ChildItem -Path $dir -Recurse -File -Filter *.txt | ForEach-Object {
SearchFile -dir $_.Directory.PSPath -file $_.Name
}
}
I don't know why your function SearchFile has separate parameters for directory and file name. The Get-ChildItem already outputs the full path in $_.PSPath. It doesn't make much sense to split the path apart and join it together again in SearchFile. I suggest you replace them by a single Path parameter.
I have an application where I am getting a list of new\modified files from git status, then I take the incomplete strings from that file, concatenate them with the root dir file path, then move those files to an archive. I have it half working, but the nature of how I am using powershell does not provide error reports and the process is obviously erroring out. Here is the code I am trying to use. (It has gone through several iterations, please excuse the commented out portions) Basically I am trying to Get-Content from the txt file, then replace ? with \ (for some reason the process that creates the txt love forward slashes...), then split that string at the spaces. The only part of the string I am interested in is the last part, which I am trying to concatenate with the known working root directory, then I am attempting to move those to an archive location. Before you ask, this is something we are not willing to track in git, due to the nature of the files (they are test outputs that are time stamped, we want to save them on a per test run basis, not in git) I am still fairly new to powershell and have been banging my head against this rock for far too long.
Get-Content $outfile | Foreach-Object
{
#$_.Replace("/","\")
#$lineSplit = $_.Split(' ')
$_.Split(" ")
$filePath = "$repo_dir\$_[-1]"
$filePath.Replace('/','\')
"File Path Created: $filePath"
$untrackedLegacyTestFiles += $filePath
}
Get-Content $untrackedLegacyTestFiles | Foreach-Object
{
Copy-Item $_ $target_root -force
"Copying File: $_ to $target_root"
}
}
the $outfile is a text file where each line has a partial file path leading to a txt file generated by a test application we use. This info is provided by git, so it looks like this in the $outfile txt file:
!! Some/File/Path/Doc.txt
The "!!" mean git sees it as a new file, however it could be several characters from a " M" to "??". Which is why I am trying to split it on the spaces and take only the last element.
My desired output would be to take the the last element of the split string from the $outfile (Some/File/Path/Doc.txt) and concatenate it with the $repo_dir to form a complete file path, then move the Doc.txt to an archive location ($target_root).
To combine a path in PowerShell, you should use the Join-Path cmdlet. To extract the path from your string, you can use a regex:
$extractedPath = [regex]::Match('!! Some/File/Path/Doc.txt', '.*\s(.+)$').Groups[1].Value
$filePath = Join-Path $repo_dir $extractedPath
The Join-Path cmldet will also convert all forward slashes to backslashes so no need to replace them :-).
Your whole script could look like this:
Get-Content $outfile | Foreach-Object {
$path = Join-Path $repo_dir ([regex]::Match($_, '.*\s(.+)$').Groups[1].Value)
Copy-Item $path $target_root -force
}
If you don't like to use regexin your code, you can also extract the path using:
$extractedPath = '!! Some/File/Path/Doc.txt' -split ' ' | select -Last 1
or
$extractedPath = ('!! Some/File/Path/Doc.txt' -split ' ')[-1]
I'm creating in Excel a sub-folder in a directory and save there multiple CSV-files from a Excel Workbook
My problem is that I need to do this on a system where the list separator is a ','. The CSV files are getting read from a system where the default list separator is a ';'. I cannot change this
So I need to change the ',' in the CSV files into a ';'. My idea is to achieve this using PowerShell.
My first attempt was to change the delimiter of the CSV immediately after creating it in excel by passing to a script the file-name. I manage to change the delimiter for a certain file but I struggle to pass the pathname to the script (no error but also no change in the file):
Script Code:
param([string]$path)
$content = [IO.File]::ReadAllText($path) #readParameter
Import-CSV -Path $content -Delimiter ','|Export-CSV -Path C:\Users\Desktop\temp.csv -Delimiter ';' -NoTypeInformation #Export a CSV-File with ;
(Get-Content C:\Users\Desktop\temp.csv) | % {$_ -replace '"', ""} | out-file -FilePath C:\Users\Desktop\temp.csv -Force -Encoding ascii #remove " from file
Remove-Item -Path $content #remove old CSV-file
Rename-Item -Path C:\Users\Desktop\temp.csv -NewName $content #change file name
Excel Call:
Call Shell("powershell.exe -ExecutionPolicy Unrestricted -File C:\Users\Desktop\delimiterChange.ps1 -path """ & location & """", 1)
Thank You
If you want to use PS, this is the easy quick and dirty. Works like a charm.
$csv = Import-csv "C:\initial.csv"
$csv | Export-Csv "C:\converted.csv" -NoClobber -NoTypeInformation -Delimiter ";"
param([string]$path)
$content = [IO.File]::ReadAllText($path) #readParameter
Import-CSV -Path $content -Delimiter ','|Export-CSV -Path C:\Users\Desktop\temp.csv -Delimiter ';' -NoTypeInformation
Your code reads the content of the CSV (assuming that the path to the CSV is passed via the parameter -Path) and tries to pass that as the path to Import-Csv. Change the above to this:
param([string]$path)
Import-CSV -Path $path |
Export-CSV -Path C:\Users\Desktop\temp.csv -Delimiter ';' -NoType
You can even replace the content of the file if you run Import-Csv in an expression:
(Import-Csv -Path $path) |
Export-Csv -Path $path -Delimiter ';' -NoType
I'd recommend keeping the double quotes, but if you must remove them you can do that in the pipeline like this:
(Import-Csv -Path $path) |
ConvertTo-Csv -Delimiter ';' -NoType |
% { $_ -replace '"', '' } |
Set-Content -Path $path
In Control Panel > Regional Settings > Additional Settings
Set the List Separator to the semi-colon:
Then, in Excel SaveAs CSV
I would take a different approach.
In a Macro enabled Excel workbook:
1) Create a routine which will import a semi-colon-delimited file. It should take a filename as a parameter and return a workbook.
2) Create a routine which will export a workbook as a CSV. It should take a workbook as a parameter, a file name as a parameter, and export/close the workbook
3) Create a routine which reads a file list from the directory and then runs 1) and 2) on each file.
Additionally, I would not name the semi-colon delimited files CSV if you have any control over the original file names. By definition, CSV means Comma Separated Values. Name them something else. Then your routine only has to find the semi-colon files and can skip the CSVs because those have already been converted to comma separated.
I'm using PowerShell and it is two days that I'm struggling on this issue.
In the directory C:\dir_1 I have many subfolders (sub_1, sub_2, ..., sub_n). Each of them contains several text files. For each subfolder i=1,2,...,n, I want to move the text files that contain the string "My-String" to the directory C:\dir_2\sub_i.
For example, if the file X in the path C:\dir1\sub_5 contains the string "My-String", I want to move it to the location C:\dir_2\sub_5. The destination folder is already existing.
I tried several modifications of the following code, but it does not work:
Get-ChildItem "C:\dir_1" | Where-Object {$_.PSIsContainer -eq $True} | Foreach-Object {Get-ChildItem "C:\dir_1\$_" | Select-String -pattern "My-String" | group path | select name | %{Move-Item $_.name "C:\dir_2\$_"}}
So, basically, what I tried to do is: foreach subfolder in dir_1, take the files that contain the string and move them to the subfolder in dir_2 with the same name. I tried several small modifications of that code, but I cannot get around my mistakes. The main error is "move-item: The given path format is not supported"... any help?
I feel like I could do better but this is my first approach
$dir1 = "C:\temp\data\folder1"
$dir2 = "C:\temp\data\folder2"
$results = Get-ChildItem $dir1 -recurse | Select-String -Pattern "asdf"
$results | ForEach-Object{
$parentFolder = ($_.Path -split "\\")[-2]
Move-Item -Path $_.Path -Destination ([io.path]::combine($dir2,$parentFolder))
}
Select-String can take file paths for its pipeline input. We feed it all the files that are under $dir1 using -recurse to get all of its children in sub folders. $results would contain an array of match objects. One of the properties is the path of the matched file.
With all of those $results we then go though each and extract the parent folder from the path. Then combine that folder with the path $dir2 in order to move it to it destination.
There are several assumptions that we are taking here. Some we could account for if need be. I will mention the one I know could be an issue first.
Your folders should not have any other subfolders under "sub_1, sub_2, ..., sub_n" else they will attempt to move incorrectly. This can be addressed with a little more string manipulation. In an attempt to make the code terse using -Recurse created this caveat.
Here is a one liner that does what you want too:
Get-ChildItem "C:\dir_1" | Where-Object {$_.PSIsContainer -eq $True} | ForEach-Object {$SubDirName = $_.Name;ForEach ($File in $(Get-ChildItem $_.FullName)){If ($File.Name -like "*My-String*"){Move-Item $File.FullName "C:\dir_2\$SubDirName"}}}
And if you'd like to see it broken out like Matt's answer:
$ParentDir = Get-ChildItem "C:\dir_1" | Where-Object {$_.PSIsContainer -eq $True}
ForEach ($SubDir in $ParentDir){
$SubDirName = $SubDir.Name
ForEach ($File in $(Get-ChildItem $SubDir.FullName)){
If ($File.Name -like "*My-String*"){
Move-Item $File.FullName "C:\dir_2\$SubDirName"
}
}
}
I have a directory with ~ 3000 text files in it, and I'm doing periodic search and replaces on those text files as I transition a program to a new server.
Each text file may have an average of ~3000 lines, and I need to search the files for maybe 300 - 1000 terms at a time.
I'm replacing the server prefix which is related to the string I'm searching for. So for every one of the csv entries, I'm looking for Search_String, \\Old_Server\"Search_String" and making sure that after the program completes, the result is "\\New_Server\Search_String".
I cobbled together a powershell program, and it works. But it's so slow I've never seen it complete.
Any suggestions for making it faster?
EDIT 1:
I changed get-content as suggested, but it still took 3 minutes to search two files (~8000 lines) for 9 separate search terms. I must still be screwing up; a notepad++ search and replace would still be way faster if done manually 9 times.
I'm not sure how to get rid of the first (Get-Content) because I want to make a copy of the file for backup before I make any changes to it.
EDIT 2:
So this is an order of magnitude faster; it's searching a file in maybe 10 seconds. But now it doesn't write changes to files, and it only searches the first file in the directory! I didn't change that code, so I don't know why it broke.
EDIT 3:
Success! I adapted a solution posted below to make it much, much faster. It's searching each file in a couple of seconds now. I may reverse the loop order, so that it loads the file into the array and then searches and replaces each entry in the CSV rather than the other way around. I'll post that if I get it to work.
Final script is below for reference.
#get input from the user
$old = Read-Host 'Enter the old cimplicity qualifier (F24, IRF3 etc'
$new = Read-Host 'Enter the new cimplicity qualifier (CB3, F24_2 etc)'
$DirName = Get-Date -format "yyyy_MM_dd_hh_mm"
New-Item -ItemType directory -Path $DirName -force
New-Item "$DirName\log.txt" -ItemType file -force -Value "`nMatched CTX files on $dirname`n"
$logfile = "$DirName\log.txt"
$VerbosePreference = "SilentlyContinue"
$points = import-csv SearchAndReplace.csv -header find #Import CSV File
#$ctxfiles = Get-ChildItem . -include *.ctx | select -expand fullname #Import local directory of CTX Files
$points | foreach-object { #For each row of points in the CSV file
$findvar = $_.find #Store column 1 as string to search for
$OldQualifiedPoint = "\\\\"+$old+"\\" + $findvar #Use escape slashes to escape each invidual bs so it's not read as regex
$NewQualifiedPoint = "\\"+$new+"\" + $findvar #escape slashes are NOT required on the new string
$DuplicateNew = "\\\\" + $new + "\\" + "\\\\" + $new + "\\"
$QualifiedNew = "\\" + $new + "\"
dir . *.ctx | #Grab all CTX Files
select -expand fullname | #grab all of those file names and...
foreach {#iterate through each file
$DateTime = Get-Date -Format "hh:mm:ss"
$FileName = $_
Write-Host "$DateTime - $FindVar - Checking $FileName"
$FileCopied = 0
#Check file contents, and copy matching files to newly created directory
If (Select-String -Path $_ -Pattern $findvar -Quiet ) {
If (!($FileCopied)) {
Copy $FileName -Destination $DirName
$FileCopied = 1
Add-Content $logfile "`n$DateTime - Found $Findvar in $filename"
Write-Host "$DateTime - Found $Findvar in $filename"
}
$FileContent = Get-Content $Filename -ReadCount 0
$FileContent =
$FileContent -replace $OldQualifiedPoint,$NewQualifiedPoint -replace $findvar,$NewQualifiedPoint -replace $DuplicateNew,$QualifiedNew
$FileContent | Set-Content $FileName
}
}
$File.Dispose()
}
If I'm reading this correctly, you should be able to read a 3000 line file into memory, and do those replaces as an array operation, eliminating the need to iterate through each line. You can also chain those replace operations into a single command.
dir . *.ctx | #Grab all CTX Files
select -expand fullname | #grab all of those file names and...
foreach {#iterate through each file
$DateTime = Get-Date -Format "hh:mm:ss"
$FileName = $_
Write-Host "$DateTime - $FindVar - Checking $FileName"
#Check file contents, and copy matching files to newly created directory
If (Select-String -Path $_ -Pattern $findvar -Quiet ) {
Copy $FileName -Destination $DirName
Add-Content $logfile "`n$DateTime - Found $Findvar in $filename"
Write-Host "$DateTime - Found $Findvar in $filename"
$FileContent = Get-Content $Filename -ReadCount 0
$FileContent =
$FileContent -replace $OldQualifiedPoint,$NewQualifiedPoint -replace $findvar,$NewQualifiedPoint -replace $DuplicateNew,$QualifiedNew
$FileContent | Set-Content $FileName
}
}
On another note, Select-String will take the filepath as an argument, so you don't have to do a Get-Content and then pipe that to Select-String.
Yes, you can make it much faster by not using Get-Content... Use Stream Reader instead.
$file = New-Object System.IO.StreamReader -Arg "test.txt"
while (($line = $file.ReadLine()) -ne $null) {
# $line has your line
}
$file.dispose()
i wanted to use PowerShell for this and created a script like the one below:
$filepath = "input.csv"
$newfilepath = "input_fixed.csv"
filter num2x { $_ -replace "aaa","bbb" }
measure-command {
Get-Content -ReadCount 1000 $filepath | num2x | add-content $newfilepath
}
It took 19 minutes on my laptop to process 6.5Gb file. The code below is reading file in a batch (using ReadCount) and uses filter that should optimize performance.
But then I tried FART and it did the same thing in 3 minutes! quite a difference!