Actually I have this loop :
foreach($line in Get-Content .\script2.csv)
{ $firstname = $line.split(';')[0]
$lastname = $line.split(';')[1]
$email = $line.split(';')[2]
$newLine = "$firstname,""$lastname"",""$email"""
$newLine >> newCSV.csv }
I use it to extract data and paste it in a correct format.
I would like to know what is the correct syntax to start it from the row 2 and not taking all my sheet ?
Thanks !
Use Select -Skip $N to skip the first $N items of a collection:
foreach($line in Get-Content .\script2.csv |Select -Skip 1)
{
$firstname = $line.split(';')[0]
$lastname = $line.split(';')[1]
$email = $line.split(';')[2]
$newLine = "$firstname,""$lastname"",""$email"""
$newLine >> newCSV.csv
}
If what you want to do is to convert a CSV file that uses the semi-colon ; as delimiter to a new Csv file that uses the comma , as delimiter, and in the process remove the header line from it, you can do:
Import-Csv -Path 'D:\Test\blah.csv' -Delimiter ';' | ConvertTo-Csv -NoTypeInformation |
Out-String -Stream | Select-Object -Skip 1 | Set-Content -Path 'D:\Test\newCSV.csv'
Related
1 .. $Count | ForEach-Object {
$i = $_.ToString($Length)
$Offset = $BatchSize * ($_ - 1)
$outputFile = $ParentDirectory + "\" + $strBaseName + "-" + $i + $strExtension
If($_ -eq 1) {
$objFile | Select-Object -First $BatchSize | Export-Csv $outputFile -NoTypeInformation -Encoding UTF8
} Else {
$objFile | Select-Object -First $BatchSize -Skip $Offset | Export-Csv $outputFile -NoTypeInformation -Encoding UTF8
}
}
I have a .txt with a comma on row 3 below. My code is stripping anything after the comma as seen below. how do I fix it? My file is pipe delimited.
Original file contains
|Header1|Header2|Header3|Header4|
|320|A1| |0900|
|320|A2|This, comma is needed|0900|
|320|A3| |0700|
|320|A4|f2|0900|
|320|A5| |0700|
|320|L2|c6|0900|
After splitting into 2 files -notice the missing text after "this,"
file1
|Header1|Header2|Header3|Header4|
|320|A1| |0900|
|320|A2|This,
|320|A3| |0700|
file2
|Header1|Header2|Header3|Header4|
|320|A4|f2|0900|
|320|A5| |0700|
|320|L2|c6|0900|
Please advise. Thanks
I tried to use delimiter and replace commands. Didn't work
It looks like when you imported your delimited file into $objFile, you forgot to pass -Delimiter '|' to the Import-Csv call, which would not interpret your |-separated file properly, given that Import-Csv - as well as Export-Csv - default to , as the separator.
Thus, the solution is to use -Delimiter '|' in both your Import-Csv and Export-Csv calls.
As for what you tried:
Here's a minimal example that demonstrates the problem with omitting -Delimiter '|', using the in-memory CSV processing cmdlets, ConvertFrom-Csv and ConvertTo-Csv:
#'
Field1|Field2
Value1|Value2, and more
'# |
ConvertFrom-Csv |
ConvertTo-Csv
Output (note the missing , and more part, and how the output lines as a whole are double-quoted):
"Field1|Field2"
"Value1|Value2"
Since header row Field1|Field2 contains no , it became a single property in the resulting objects, literally named Field1|Field2.
Since the data row happened to contain , it was parsed as two fields, and since there is only one column, the extra field was simply discarded.
Below is the code I am having trouble with. It pulls from a config.txt file and sets information to childofRoot.properties file. for some reason the string when I set it is coming with extra added on.
$date = (Get-Date -Format "MM-dd-yyyy'_'HH-mm")
$backup = 'backuplocation'
$temp = 'replace'
$p = ($PSScriptRoot + 'childofRoot.properties')
$cuidConf = Get-Content -Path ($PSScriptRoot + 'config.txt') | Select-String "cuids_rootfolder=" | Out-String
$cuidConf = $cuidConf.replace("cuids_rootfolder=", "").replace("`n", "")
$cuids = $cuidConf.Split(";")
$backuppath = ((Get-Content -Path ($PSScriptRoot + 'config.txt'))) | Select-String "backuplocation=" | Out-String
$backuppath = $backuppath.replace("backuplocation=", "").replace("`n", "")
$backuppath = $backuppath + $date + "\"
$foldername, $cuid = ($cuids[0].Split(","))
$backup1 = ($backuppath + "Folder a.lcmbiar")
((Get-content -Path $p).replace($temp, $cuid)) | Set-Content -Path $p
((Get-content -Path $p).replace($backup, $backup1)) | Set-Content -Path $p
Contents of config.txt:
backuplocation=C:\backuplcm\backupfiles\
Content of childofRoot.properties:
exportLocation=backuplocation
Expected output written to childofRoot.properties:
exportLocation=C:\backuplcm\backupfiles\08-18-2020_17-59\Folder\a.lcmbiar
Output I'm seeing written to childofRoot.properties:
exportLocation=
C:\backuplcm\backupfiles\
08-18-2020_18-02\Folder a.lcmbiar
Yes I already tried removing new line characters with .replace("`n","")
I had others look at this script too and the variable looks fine just when i set it it comes out different in the text file.
I was able to replicate this on Powershell 4 and Powershell 5 on different machines. If someone would try this on their machine and see if they can fix the problem.
The extra line breaks are from doing | Select-String "backuplocation=" | Out-String, and the reason .Replace("`n", "") doesn't solve the problem is that Windows uses the sequence "`r`n" for line breaks.
I suggest simplifying your script with Where-Object instead of Select-String | Out-String:
$backuppath = Get-Content -Path (Join-Path $PSScriptRoot config.txt) | Where-Object {$_ -match "backuplocation="}
$backuppath = $backuppath -replace 'backuplocation='
I'm looking for a translation of my Excel formula in a form of a script in Powershell, vbscript or Excel VBA. I'm trying to get the list of column headers and the max length of string under it.
Normally, what I do is manually open the .txt file in Excel, from there I can get the header names.. next, I create an array formula =MAX(LEN(A1:A100,000)) for example. This will get the max length of string in the column. I'll do the same formula to other columns.
Right now I can't do this since files have increased to 1GB in size and i can't open them anymore, my desktop crashes. It is also maybe because theyre more than 1 million rows which Excel cant handle. My friend suggested Powershell but I have limited knowledge there.. don't know if it can be done in vbscript or Excel VBA.
Thanks in advance for your help.
Below code works for .csv files but does not with .txt delimited files -
$fileName = "C:\Desktop\EFile.csv"
<#
Sample format of c:\temp\data.csv
"id","name","grade","address"
"1","John","Grade-9","test1"
"2","Ben","Grade-9","test12222"
"3","Cathy","Grade-9","test134343"
#>
$colCount = (Import-Csv $fileName | Get-Member | Where-Object {$_.MemberType -eq 'NoteProperty'} | Measure-Object).Count
$csv = Import-Csv $fileName
$csvHeaders = ($csv | Get-Member -MemberType NoteProperty).name
$dict = #{}
foreach($header in $csvHeaders) {
$dict.Add($header,0)
}
foreach($row in $csv)
{
foreach($header in $csvHeaders)
{
if($dict[$header] -le ($row.$header).Length)
{
$dict[$header] =($row.$header).Length
}
}
}
$dict.Keys | % { "key = $_ , Column Length = " + $dict.Item($_) }
This is how I get my data.
$data = #"
"id","name","grade","address"
"1","John","Grade-9","test1"
"2","Ben","Grade-9","test12222"
"3","Cathy","Grade-9","test134343"
"#
$csv = ConvertFrom-Csv -Delimiter ',' $data
But you should get your data like this
$fileName = "C:\Desktop\EFile.csv"
$csv = Import-Csv -Path $fileName
And then
# Extract the header names
$headers = $csv | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty Name
# Capture output in $result variable
$result = foreach($header in $headers) {
# Select all items in $header column, find the longest, and select the item for output
$maximum = $csv | Select-Object -ExpandProperty $header | Measure-Object -Maximum | Select-Object -ExpandProperty Maximum
# Generate new object holding the information.
# This will end up in $results
[pscustomobject]#{
Header = $header
Max = $maximum.Length
String = $maximum
}
}
# Simple output
$result | Format-Table
This is what I get:
Header Max String
------ --- ------
address 10 test134343
grade 7 Grade-9
id 1 3
name 4 John
Alternatively, if you have memory issues dealing with large files, you may have to get a bit more dirty with the .NET framework. This snippet processes one csv line at a time, instead of reading the entire file into memory.
$fileName = "$env:TEMP\test.csv"
$delimiter = ','
# Open a StreamReader
$reader = [System.IO.File]::OpenText($fileName)
# Read the headers and turn it into an array, and trim away any quotes
$headers = $reader.ReadLine() -split $delimiter | % { $_.Trim('"''') }
# Prepare a hashtable for the results
$result = #{}
# So long as there's more data, keep running
while(-not $reader.EndOfStream) {
# Read a single line and process it as csv
$csv = $reader.ReadLine() | ConvertFrom-Csv -Header $headers -Delimiter $delimiter
# Determine if the item in the result hashtable is smaller than the current, using the header as a key
foreach($header in $headers) {
$item = $csv | Select-Object -ExpandProperty $header
if($result[$header].Maximum -lt $item.Length) {
$result[$header] = [pscustomobject]#{
Header = $header
Maximum = $item.Length
String = $item
}
}
}
}
# Clean up our spent resource
$reader.Close()
# Simple output
$result.Values | Format-Table
I use Export-Excel module for Powershell. I try to get results of "foreach" from 2 links in a table. But finally i have only one link in the table(only second one). "Out-File" has "=Append",that appends to a file. Maybe there is something like this for "Export-Excel"?
[Reflection.Assembly]::LoadFrom("C:\selenium\net40\WebDriver.dll")
$url = Get-Content D:\dww.txt
$ie = New-Object OpenQA.Selenium.PhantomJS.PhantomJSDriver
foreach ($u in $url) {
$ie.Navigate().GoToUrl($u)
$title = $ie.FindElementByXPath("//h1[contains(#class, 'fullstory')]")
$fullstory = $ie.FindElementByClassName("comment")
$tracklist = $ie.FindElementByXPath("//*[contains(#id,'news-id')]")
$links = $ie.FindElementByXPath("//*[contains(#class, 'link')]")
$img = $ie.FindElementsByclassname("stars").findelementbytagname("img").getattribute("src")
$h = Write-Output Стиль,Формат,"Год выпуска",Размер,"Название альбома",Исполнитель,Треклист,Ссылка,Обложка
$data = ($fullstory.text).split("`n") -replace ": ","=" | Where {$_.Trim()} | Out-String | ConvertFrom-StringData | Foreach {[pscustomobject]$_} | Select $h
$data.Ссылка = $links.text
$data.Треклист = $tracklist.text
$data.Обложка = $img
$data | Export-Excel d:\filedark2.xlsx -AutoSize
Write-Host $u " - выгружено"
}
$ie.Quit()
if you're only using export-excel to save to a new excel document, you might try just exporting to csv using -append and converting to excel afterward with this
https://github.com/gangstanthony/PowerShell/blob/master/Save-CSVasExcel.ps1
...
$data | Export-Csv d:\filedark2.csv -Append -NoTypeInformation
...
}
Save-CSVasExcel d:\filedark2.csv
...
I want to append several strings in a data set with custom strings.
Example
Content of Dataset:
Test1
Test2
Test3
Result after appending:
Test1.com
Test2.com
Test3.com
Would I have to use regex to parse to the end of each Test[n] to be able to append it with a custom string (.com)? Has anyone got an example that describes exactly how to do it?
I am reading from a SQL-Table and writing values into a DataSet which is exported to CSV the following way:
$DataSet.Tables[0] | ConvertTO-Csv -Delimiter ',' -NotypeInformation |`% { $_ -replace '"','' } | out-file $outfile -Encoding "unicode"
The DataSet contains of Strings such as:
Banana01
Banana02
Apple01
Cherry01
Cherry02
Cherry03
The thing I want to do is append .com to only Cherry01, Cherry02, and Cherry03, and after appending .com, export it as a CSV file.
There are many ways. Here are a few:
# Using string concatenation
'Test1','Test2','Test3' | Foreach-Object{ $_ + '.com' }
# Using string expansion
'Test1','Test2','Test3' | Foreach-Object{ "$_.com" }
# Using string format
'Test1','Test2','Test3' | Foreach-Object{ "{0}{1}" -f $_,'.com' }
You could use something like this:
Example 1
$t = "test"
$t = $t + ".com"
Example 2
$test = #("test1","test2")
$test | ForEach-Object {
$t = $_ + ".com"
Write-Host $t}
With your added code I did this. I don't have a database to test it on, so I made the data set manually, so you might just have to change the $DataSet[0] in my code to $DataSet.Tables[0].
$DataSet[0] | ConvertTO-Csv -Delimiter ',' -NotypeInformation | Foreach-Object{$T=$_
IF($T -match "(Cherry\d\d)"){$T = $T -replace "(Cherry\d\d)(.+)",'$1.com$2'};$T } | out-file $outfile -Encoding "unicode"
$array | %{if($_ -match "^Cherry\d\d"){$_ += ".com"};$_}