I have a CSV file which looks all fine until it is imported into PowerShell, when its imported each character is followed by a space like C : \ instead of C:\.
It would be easy enough to format the cells to text in Excel (which works) but this CSV file is created on multiple servers by in an automation policy so going through each of these files and formatting them will take a while as you can imagine.
I was wondering if there was a way in which I can format the cells first in PowerShell then import the CSV.
PowerShell code I am using:
$data = import-csv -Path $path -UseCulture -Header #("Path", "Folder", "Size")
CSV Snippet:
C:\,C:\,14.0GB
C:\Program Files,Program Files,4.5GB
C:\Program Files\Microsoft Office,Microsoft Office,2.8GB
It sounds like the file might be Unicode, but without the proper byte order marks, which would cause PowerShell to use the default ASCII encoding. If that is the case, you'll need to specify the encoding:
$data = import-csv -Encoding Unicode -Path $path ...
Another option is to convert the file to ASCII prior to the import [credit to OP for the command]:
Get-content C:\path\TestXml.csv | Set-Content -Encoding Ascii TestXml.csv
this might present a different problem, but it may work for removing all the spaces
$data = import-csv -Path $path -UseCulture -Header #("Path", "Folder", "Size")
$data | % {
$_.path = $_.path -replace '\s'
$_.folder = $_.folder -replace '\s'
$_.size = $_.size -replace '\s'
}
$data
It would be beneficial to get a snippet of what the CSV looks like. Can you provide the header and 1 or 2 rows?
Is the header specified in the CSV file?
When using the Header parameter, delete the original header row from the CSV file. Otherwise, Import-Csv creates an extra object from the items in the header row.
https://technet.microsoft.com/en-us/library/hh849891.aspx
You are specifying the UseCulture switch which will use the default delimiter specified by the environment. You can run the following command to find your culture's delimiter:
(Get-Culture).TextInfo.ListSeparator
https://technet.microsoft.com/en-us/library/hh849891.aspx
#Tony Hinkle sent me in the right direction so I have marked his answer as correct, here is the code that I have used:
Set the content of the CSV to ascii encoding
Get-content C:\Users\sam\Desktop\TestXml.csv | Set-Content -Encoding Ascii TestXml.csv
Then import the Csv
$data = Import-Csv C:\Users\sam\Desktop\TestXml.csv -Header #("Path", "Folder", "Size")
Related
Not sure what this is an cant find it on the internet
$myString = #"
string
"#
$myRegularString = "string"
write-output $myString.getType() # outputs System.String
write-output $myRegularString.getType() # outputs System.String
I encountered the latter when someone converted the text of a string to a .pbk file. They originally said there was a problem converting it to utf8 but when I imported the .pbk properties from a text using get-content $myString -encoding utf8 file it was fine
So whats the difference? Is one fancier?
Those are here-strings, they allow to preserve formatting, including line breaks:
$string = #"
this is
a test
string
"#
Also please be aware that when you use Get-Content -Path test.txt the result is an array of strings (each line is an array item). If you want to get the file content as a single string object you will need to use Get-Content -Path test.txt -Raw.
Im currently working on a programm that needs a .xml file, reads it into a Oracle Database and afterwards exports a new .xml file. But the problem is that the new file has to have the exact same name as the original file.
I saved the original filenames into a .txt file and i'm now trying to search for a keyword inside the lines to rename the right files with the correct names inside the .txt file. Here an example:
My 4 files (exported from the Database):
PM.Data_information.xml
PM.Data_location.xml
PM.Cover_quality.xml
PM.Cover_adress.xml
Content of Namefile.txt (original names):
PM.Data_information_provide_SE-R-SO_V0220_657400509_3_210.xml
PM.Data_location_provide_SE-R-SO_V0220_9191200509_3_209.xml
PM.Cover_quality_provide_SE-R-SO_V0220_354123509_3_211.xml
PM.Cover_adress_provide_SE-R-SO_V0220_521400509_3_212.xml
I only worked out how to get a line by selecting the linenumber:
$content = Get-Content C:\Namefile.txt
$informationanme = $content[0]
Rename-Item PM.Data_information.xml -NewName $informationname
Isn't there a way to select that line by searching for the keyword inside the string?
$content = Get-Content C:\temp\ps\NewFile.txt
$files = Get-ChildItem c:\temp\ps\
$content |
%{
$currentLine = $_
$file = $files | Where-Object { $currentLine.StartsWith($_.Name.Replace(".xml", "")) }
Rename-Item $file.Name $currentLine
}
This code should do the trick. Note you will need to have all of your files that need renaming in one folder. Set the folder path to the $files variable (currently set to c:\temp\ps). Set the path where your NewFile.txt is to the $content path.
The code works by looping around each line in the NewFile.txt and finding any file where the name matches the start of the line (if there are any files that do not follow this pattern you will obviously need to update the code but hopefully gives you a good starting point).
other solution ;)
gci -Path "c:\temp" -File -Filter "*.xml" | % { rni $_.fullname (sls "C:\temp\Namefile.txt" -Pattern ([System.IO.Path]::GetFileNameWithoutExtension($_.fullname))).Line }
I have an application where I am getting a list of new\modified files from git status, then I take the incomplete strings from that file, concatenate them with the root dir file path, then move those files to an archive. I have it half working, but the nature of how I am using powershell does not provide error reports and the process is obviously erroring out. Here is the code I am trying to use. (It has gone through several iterations, please excuse the commented out portions) Basically I am trying to Get-Content from the txt file, then replace ? with \ (for some reason the process that creates the txt love forward slashes...), then split that string at the spaces. The only part of the string I am interested in is the last part, which I am trying to concatenate with the known working root directory, then I am attempting to move those to an archive location. Before you ask, this is something we are not willing to track in git, due to the nature of the files (they are test outputs that are time stamped, we want to save them on a per test run basis, not in git) I am still fairly new to powershell and have been banging my head against this rock for far too long.
Get-Content $outfile | Foreach-Object
{
#$_.Replace("/","\")
#$lineSplit = $_.Split(' ')
$_.Split(" ")
$filePath = "$repo_dir\$_[-1]"
$filePath.Replace('/','\')
"File Path Created: $filePath"
$untrackedLegacyTestFiles += $filePath
}
Get-Content $untrackedLegacyTestFiles | Foreach-Object
{
Copy-Item $_ $target_root -force
"Copying File: $_ to $target_root"
}
}
the $outfile is a text file where each line has a partial file path leading to a txt file generated by a test application we use. This info is provided by git, so it looks like this in the $outfile txt file:
!! Some/File/Path/Doc.txt
The "!!" mean git sees it as a new file, however it could be several characters from a " M" to "??". Which is why I am trying to split it on the spaces and take only the last element.
My desired output would be to take the the last element of the split string from the $outfile (Some/File/Path/Doc.txt) and concatenate it with the $repo_dir to form a complete file path, then move the Doc.txt to an archive location ($target_root).
To combine a path in PowerShell, you should use the Join-Path cmdlet. To extract the path from your string, you can use a regex:
$extractedPath = [regex]::Match('!! Some/File/Path/Doc.txt', '.*\s(.+)$').Groups[1].Value
$filePath = Join-Path $repo_dir $extractedPath
The Join-Path cmldet will also convert all forward slashes to backslashes so no need to replace them :-).
Your whole script could look like this:
Get-Content $outfile | Foreach-Object {
$path = Join-Path $repo_dir ([regex]::Match($_, '.*\s(.+)$').Groups[1].Value)
Copy-Item $path $target_root -force
}
If you don't like to use regexin your code, you can also extract the path using:
$extractedPath = '!! Some/File/Path/Doc.txt' -split ' ' | select -Last 1
or
$extractedPath = ('!! Some/File/Path/Doc.txt' -split ' ')[-1]
I'm creating in Excel a sub-folder in a directory and save there multiple CSV-files from a Excel Workbook
My problem is that I need to do this on a system where the list separator is a ','. The CSV files are getting read from a system where the default list separator is a ';'. I cannot change this
So I need to change the ',' in the CSV files into a ';'. My idea is to achieve this using PowerShell.
My first attempt was to change the delimiter of the CSV immediately after creating it in excel by passing to a script the file-name. I manage to change the delimiter for a certain file but I struggle to pass the pathname to the script (no error but also no change in the file):
Script Code:
param([string]$path)
$content = [IO.File]::ReadAllText($path) #readParameter
Import-CSV -Path $content -Delimiter ','|Export-CSV -Path C:\Users\Desktop\temp.csv -Delimiter ';' -NoTypeInformation #Export a CSV-File with ;
(Get-Content C:\Users\Desktop\temp.csv) | % {$_ -replace '"', ""} | out-file -FilePath C:\Users\Desktop\temp.csv -Force -Encoding ascii #remove " from file
Remove-Item -Path $content #remove old CSV-file
Rename-Item -Path C:\Users\Desktop\temp.csv -NewName $content #change file name
Excel Call:
Call Shell("powershell.exe -ExecutionPolicy Unrestricted -File C:\Users\Desktop\delimiterChange.ps1 -path """ & location & """", 1)
Thank You
If you want to use PS, this is the easy quick and dirty. Works like a charm.
$csv = Import-csv "C:\initial.csv"
$csv | Export-Csv "C:\converted.csv" -NoClobber -NoTypeInformation -Delimiter ";"
param([string]$path)
$content = [IO.File]::ReadAllText($path) #readParameter
Import-CSV -Path $content -Delimiter ','|Export-CSV -Path C:\Users\Desktop\temp.csv -Delimiter ';' -NoTypeInformation
Your code reads the content of the CSV (assuming that the path to the CSV is passed via the parameter -Path) and tries to pass that as the path to Import-Csv. Change the above to this:
param([string]$path)
Import-CSV -Path $path |
Export-CSV -Path C:\Users\Desktop\temp.csv -Delimiter ';' -NoType
You can even replace the content of the file if you run Import-Csv in an expression:
(Import-Csv -Path $path) |
Export-Csv -Path $path -Delimiter ';' -NoType
I'd recommend keeping the double quotes, but if you must remove them you can do that in the pipeline like this:
(Import-Csv -Path $path) |
ConvertTo-Csv -Delimiter ';' -NoType |
% { $_ -replace '"', '' } |
Set-Content -Path $path
In Control Panel > Regional Settings > Additional Settings
Set the List Separator to the semi-colon:
Then, in Excel SaveAs CSV
I would take a different approach.
In a Macro enabled Excel workbook:
1) Create a routine which will import a semi-colon-delimited file. It should take a filename as a parameter and return a workbook.
2) Create a routine which will export a workbook as a CSV. It should take a workbook as a parameter, a file name as a parameter, and export/close the workbook
3) Create a routine which reads a file list from the directory and then runs 1) and 2) on each file.
Additionally, I would not name the semi-colon delimited files CSV if you have any control over the original file names. By definition, CSV means Comma Separated Values. Name them something else. Then your routine only has to find the semi-colon files and can skip the CSVs because those have already been converted to comma separated.
I am downloading a CSV from a SharePoint site. It comes with a .csv file extension.
When I inspect the file's contents by opening it in Notepad, I see data that looks like this sample row:
"TITLE",OFFICE CODE,="","CUSTOMER'S NAME",ACCOUNT
I want the data look like this:
TITLE,OFFICE CODE,,"CUSTOMER'S NAME",ACCOUNT
One way to solve this problem is manually. When I open the file in Excel and save it (without altering anything), it prompts me with the following: fileOrig.csv may contain features that are not compatible with CSV (Comma delimited). Do you want to keep the workbook in this format? When I save it, and then inspect it in Notepad, the data is formatted according to how I want it do look.
Is there a quick way to resave the original CSV with PowerShell?
If there is no quick way to resave the file with PowerShell, I would like to use PowerShell to parse it.
These are the parsing rules I want to introduce:
Remove encapsulating doublequote from cells that do not contain a , char
Remove the = char
I tried writing a test script that just looks at the column that potentially contains , chars. It is supposed to find the cells that do not contain a , char, and remove the doublequotes that encapsulate the text. It does not work, because I think it tosses the doublequote upon Import-Csv
$source = 'I:\dir\fileOrig.csv'
$dest = 'I:\dir\fileStaging.csv'
$dest2 = 'I:\dir\fileFinal.csv'
get-content $source |
select -Skip 1 |
set-content "$file-temp"
move "$file-temp" $dest -Force
$testcsv = Import-Csv $dest
foreach($test in $testcsv)
{
#Write-Host $test."CUSTOMER NAME"
if($test."CUSTOMER NAME" -NotLike "*,*") {
$test."CUSTOMER NAME" -replace '"', ''
}
}
$testcsv | Export-Csv -path $dest2 -Force
Can someone please help me either with implementing the logic above, or if you know of a better way to save the file as a proper CSV, can you please let me know?
Since Excel can handle the problem, why not use a vbs script to automate it? Use notepad to create "Fix.vbs" with the following lines:
Set objExcel = CreateObject("Excel.Application")
Set objWorkbook = objExcel.Workbooks.Open("C:\test\test.csv")
objworkbook.Application.DisplayAlerts = False
objworkbook.Save
objexcel.quit
run it from a command prompt and it should do the trick.
I see that there's already an approved answer, I'm just offering an alternative.
If you want to keep it in PowerShell you could do this:
$File = 'I:\dir\fileOrig.csv'
$dest = 'I:\dir\fileStaging.csv'
$Output = 'I:\dir\fileFinal.csv'
$CSV = Import-Csv $file
$Members = $test|gm -MemberType Properties|select -ExpandProperty name
$test|%{$row=$_;$Members|%{if(!($row.$_ -match "\w+")){$row.$_=$null}};$_=$row}|export-csv $dest -NoTypeInformation -Force
gc $file|%{($_.split(",") -replace "^`"(.*)`"$","`$1") -join ","}|Out-File $Output
That imports the CSV, makes sure that there are words (letters, numbers, and/or underscores... don't ask my why underscores are considered words, RegEx demands that it be so!) in each property for each entry, exports the CSV, then runs through the file again as just text splitting at commas and if it shows up enclosed in double quotes it strips those, re-joins the line, and then outputs it to a file. The only thing that I don't think shows up like your "preferred output" in the OP is that instead of "CUSTOMER'S NAME" you get CUSTOMER'S NAME.