I have a PowerShell script which pulls data from DB and pushes it to a excel sheet. I am facing slowness (45 mins approx) while copying the records in the dataset to the excel sheet as the number of records exceed 200K. And I am trying to loop them one by one using the below snippet, which takes more time. Is there a way in which I can transfer the data from dataset to excel more efficiently?
$cells=$Worksheet.Cells
$row=1
foreach ($rec in $dataset.Tables[0].Rows)
{
$row++
$col=1
$cells.item($Row,$col)=$USR.ID
$col++
$cells.item($Row,$col)=$USR.Name
$col++
$cells.item($Row,$col)=$USR.Age
$col++
}
You shoud try PSExcel module. There's no need to create COM object and even have Excel installed. Your example would look like this and be lightning fast:
$dataset.Tables[0] |
Select-Object ID,Name,Age |
Export-XLSX -Path $FullName -AutoFit -WorksheetName 'MyData'
A nice little workaround I saw sometime ago was to format the rows as a CSV string and simply paste them in. For the environment I was using, this proved to be more efficient than creating a file using Export-CSV, then loading it in Excel.
#Row data joined with tabs
$data = #("[A1]", "[A2]", "[A3]", "[A4]", "[A5]", "[A6]") -join "`t"
#Multiple rows joined with new lines
$dataToPaste = "{0}`n{1}`n{2}" -f $data, $data.replace("A", "B"), $data.replace("A", "C")
$excel = New-Object -ComObject Excel.Application
$book = $excel.Workbooks.Add()
$sheet = $book.Worksheets.Add()
#Activate where to put data
$sheet.Range("B2").Activate() | Out-Null
#Copy data to clipboard and paste into sheet.
$dataToPaste | Clip
$sheet.Paste()
$excel.Visible = $true
#Cleanup
[Runtime.InteropServices.Marshal]::ReleaseComObject($excel) | Out-Null
$excel = $null
I did find that, very rarely, the Paste method throws an error, which was fixed by retrying a second time if it failed:
try{
$sheet.Paste()
}catch{
$sheet.Paste()
}
This may not be a preferred option if you are running something on a PC being used by someone, as the user could copy something to the clipboard after the script does (but before $sheet.Paste()) and invalidate your data.
Related
I am trying to automate the following manual task, and am struggling with part of it:
1) Open a text file that contains multiple lines containing data.
2) Copy the contents of this file to the clipboard.
3) Open and Excel spreadsheet.
4) Rename the spreadsheet to Test.
5) Paste the contents of the clipboard.
When this is done manually the content is pasted and each line in the text file is inserted as a new row in column A.
Originally the customer wanted all of the file content to be injected into cell A1. I was able to achieve this with the below PowerShell code.
However they have since changed this back to wanting each line of text to go into a separate row in column A.
I cannot figure out how to do this gracefully via the Get-Content method of copying out the text data. I have seen workarounds to this issue whereby Excel opens the text file and copies the text into an intermediate workbook and then into the final workbook.
Could someone please let me know if it's possible to amend my already working code below so that it adds the text to rows in column A rather than to cell A1?
# Clear the screen of any previous text.
cls
$ExcelFile="C:\Users\User\Desktop\Test\Test.xlsx"
$TextFile="C:\Users\User\Desktop\Test\TestText.txt"
$Content = Get-Content $TextFile -Raw
# Perform operations in Excel based on content of the downloaded file.
$Excel = New-Object -ComObject Excel.Application
# For troubleshooting enable the below to view Excel as file is manipulated:
#$Excel.Visible=$true
# Disable Excel alerts. Hash this line out for troubleshooting.
$Excel.DisplayAlerts = $false
# Set up workbook...
$Workbook = $Excel.Workbooks.Add()
$Data = $Workbook.Worksheets.Item(1)
$Data.Name = 'Test'
# Insert Data
$Data.Cells.Item(1,1) = "$Content"
# Format, save and quit excel
$UsedRange = $Data.UsedRange
$UsedRange.EntireColumn.AutoFit() | Out-Null
$Workbook.SaveAs("$ExcelFile")
$Excel.Quit()
I know that the part I would need to change is as follows, but I'm not sure what to change it to:
# Insert Data
$Data.Cells.Item(1,1) = "$Content"
Many thanks in advance.
To do this, you need to find the last used row in the sheet and write each line from there:
$ExcelFile = "C:\Users\User\Desktop\Test\Test.xlsx"
$TextFile = "C:\Users\User\Desktop\Test\TestText.txt"
# Perform operations in Excel based on content of the downloaded file.
$Excel = New-Object -ComObject Excel.Application
$Excel.Visible = $false
$Excel.DisplayAlerts = $false
# open the file and select the first worksheet
$WorkBook = $Excel.Workbooks.Open($ExcelFile)
$WorkSheet = $Workbook.Worksheets.Item(1)
# get the first unused row
$row = ($WorkSheet.UsedRange.Rows).Count + 1
# fill in the data
Get-Content -Path $TextFile | ForEach-Object {
$WorkSheet.Cells.Item($row++, 1) = $_
}
# format column A and save the file
$UsedRange = $WorkSheet.UsedRange
$UsedRange.EntireColumn.AutoFit() | Out-Null
$WorkBook.Save()
# quit excel and clean up the used COM objects
$Excel.Quit()
$null = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($WorkSheet)
$null = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($WorkBook)
$null = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($Excel)
[System.GC]::Collect()
[System.GC]::WaitForPendingFinalizers()
I think the solution would be to read each line in content by for or foreach loop
in loop, write the line's content into the last row of column A in the excel file.
It's will be something like this
foreach($line in $Content){
$Data.Cells.Item($LastRow,1) = $line
}
I am trying replicate an end users experience by monitoring the time it takes to copy a large file off of a network directory.
I'm using Measure-Command to find the total time it takes to copy an item from a network directory onto the computer that this script is scheduled on (using Windows Scheduler) and output the time in an xlsx.
The issue I'm running into is that every time this script runs off of the scheduler (daily), it overwrites the previous day's data instead of posting the result in the next cell. When I run it manually multiple times, it works just fine and posts separate results under each other. I think the issue is that it's going by the instance (so running the code a handful of times in the same PowerShell instance sees that the $previousRow works, but in the daily schedule it opens a new instance every time and writes over the old data in cell (1,1) and (1,2).
Any suggestions on how to keep historical data?
$seconds = Measure-Command {
Copy-Item -Path X:\shareddrive\test.pdf -Destination C:\Users\Me\Desktop
} | select TotalSeconds
$erroractionpreference = "SilentlyContinue"
$a = New-Object -ComObject Excel.Application
$dt = Get-Date -Format "MM/dd/yyyy hh:mm:ss"
$a.Workbooks.Open("X:\shareddrive\output\timesample.xlsx")
$a.Visible = $true
$a.Worksheets.Item(1)
$previousRow += 1
$a.Cells.Item($previousRow,1) = "OfficeLocation - " + $dt
$a.Cells.Item($previousRow,2) = $seconds.TotalSeconds
$a.ActiveWorkbook.Save()
$a.Workbooks.Close()
$a.Quit()
Remove-Item -Path C:\Users\Me\Desktop\test.pdf
Not sure if you haven't copied all your code in? But it looks like you aren't actually defining $previousRow anywhere? So your existing code runs $previousRow += 1 and that sets $previousRow to 1... which means each time the code runs it will hit row 1 first
To be able to find the row that has the last information in it (ie the value to set $previousValue to + 1) you can use this code:
$filepath = "C:\Folder\ExcelFile.xlsx"
$objExcel = New-Object -ComObject Excel.Application
$objExcel.Visible = $False
$WorkBook = $objExcel.Workbooks.Open($filepath)
$WorkSheet = $objExcel.WorkSheets.item(1)
$WorkSheet.activate()
[int]$lastRowvalue = ($WorkSheet.UsedRange.rows.count + 1) - 1
$lastrow = $WorkSheet.Cells.Item($lastRowvalue, 1).Value2
write-host $previousValue
write-host $lastRowvalue
Copy/paste from here (with a few slight modifications): To get the value of last cell used in Excel
That tells you the last row with data... so you would need to add 1 to that first before you set it as the value for $previousRow:
$previousRow = $previousRow + 1
I am doing data output to csv file via powershell. Generally things goes well.
I have exported the data to csv file. It contains about 10 columns. When I open it with MS Excel it's all contained in first column. I want to split it by several columns programmatically via powershell(same GUI version offers). I could make looping and stuff to split the every row and then put values to appropriate cell but then it would take way too much time.
I believe there should be an elegant solution to make one column split to multiple. Is there a way to make it in one simple step without looping?
This is what I came up with so far:
PS, The CSV file is 100% FINE. The delimiter is ','
Get-Service | Export-Csv -NoTypeInformation c:\1.csv -Encoding UTF8
$xl = New-Object -comobject Excel.Application
$xl.Visible = $true
$xl.DisplayAlerts = $False
$wb = $xl.Workbooks.Open('c:\1.csv')
$ws = $wb.Sheets|?{$_.name -eq '1'}
$ws.Activate()
$col = $ws.Cells.Item(1,1).EntireColumn
This will get you the desired functionality; add to your code. Check out the MSDN page for more information on TextToColumns.
# Select column
$columnA = $ws.Range("A1").EntireColumn
# Enumerations
$xlDelimited = 1
$xlTextQualifier = 1
# Convert Text To Columns
$columnA.texttocolumns($ws.Range("A1"),$xlDelimited,$xlTextQualifier,$true,$false,$false,$true,$false)
$ws.columns.autofit()
I had to create a CSV which had "","" as delimiter to test this out. The file with "," was fine in excel.
# Opens with all fields in column A, used to test TextToColumns works
"Name,""field1"",""field2"",""field3"""
"Test,""field1"",""field.2[]"",""field3"""
# Opens fine in Excel
Name,"field1","field2","field3"
Test,"field1","field.2[]","field3"
Disclaimer: Tested with $ws = $wb.Worksheets.item(1)
I'm an end user who hosts a public Excel 2010 Workbook (without macros, so other users can feel safe) which contains all of the individual player stats for the Madden NFL Mobile game by EA Sports.
I've also created a non-public Macro-Enabled 2010 Workbook which I use to automate the extraction all of the relevant data from a 3rd party website and reparse all of that data into a spreadsheet layout I desire.
My first column of the Macro-Enabled Workbook contains the player's name with a hyperlink to that player's webpage on that 3rd party website, and the macro creates that hyperlink for me just fine.
When I use a Data Connection to automate the syncing of the data from the Macro Book to the Non-Macro Book, everything copies fine, except for the player's name which is only in plain text (no hyperlink).
I also tried to make that first column an excel hyperlink formula (instead of VBA's hyperlink function), but the formula won't transfer via the data connection either.
Is there something I can edit maybe inside of the Data Connection file (.odc) to accomplish my goal?
My only other workaround so far is to add 2 extra hidden columns containing the text link and player name, and then mucking around with the destination Table to make the first column a pre-defined excel HYPERLINK formula to convert the two columns back into a hyperlink.
Any ideas?
Nevermind, I decided to just use a powershell script and batch file to do all of my copy/pasting, and skipping the whole Data Connection thing all together.
My "TransferMaddenData.ps1" PowerShell Code:
$path = "C:\Users\Grego\Desktop\MaddenData.xlsm"
$Excel = New-Object -ComObject excel.application
$Excel.visible = $false
$Workbook = $Excel.Workbooks.open($path)
$Worksheet = $Workbook.WorkSheets.item("DataOutput")
$Rows = $Worksheet.UsedRange.Rows.Count
$Rows += 1
$Cells = "A3:BD" + $Rows
$Worksheet.activate()
$range = $WorkSheet.Range($Cells)
$range.Copy() | out-null
$path = "C:\Users\Grego\Desktop\Madden1.xlsm"
$Workbook2 = $Excel.Workbooks.open($path)
$Worksheet = $Workbook2.Worksheets.item("Raw Data")
$range = $Worksheet.Range("A3")
$Worksheet.Paste($range)
$Workbook2.Save()
$Workbook.Close()
$Excel.Quit()
Remove-Variable -Name excel
[gc]::collect()
[gc]::WaitForPendingFinalizers()
This is my "FixMadden.bat" batch file:
#echo off
color 1F
echo.
C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File "C:\Users\Grego\Desktop\TransferMaddenData.ps1"
:EOF
echo Waiting seconds
timeout /t 10 /nobreak > NUL
Hope this maybe helps someone else someday!
My powershell script inputs data into EXCEL worksheet
and I am trying to create a line chart that will resemble
However, this is the code I have so far:
$xlConditionValues=[Microsoft.Office.Interop.Excel.XLConditionValueTypes]
$xlTheme=[Microsoft.Office.Interop.Excel.XLThemeColor]
$xlChart=[Microsoft.Office.Interop.Excel.XLChartType]
$xlIconSet=[Microsoft.Office.Interop.Excel.XLIconSet]
$xlDirection=[Microsoft.Office.Interop.Excel.XLDirection]
...
$chart=$ws.Shapes.AddChart().Chart
$chart.chartType=$xlChart::xlBarClustered
$start=$ws.range("A1")
$Y=$ws.Range($start,$start.End($xlDirection::xlDown))
$start=$ws.range("B2")
$X=$ws.Range($start,$start.End($xlDirection::xlDown))
$chartdata=$ws.Range("A$($Y.item(1).Row):A$($Y.item($Y.count).Row),B$($X.item(1).Row):B$($X.item($X.count).Row)")
$chart.SetSourceData($chartdata)
#add labels
$chart.seriesCollection(1).Select() | Out-Null
$chart.SeriesCollection(1).ApplyDataLabels() | out-Null
#modify the chart title
$chart.ChartTitle.Text = "Number of Computer"
$ws.shapes.item("Chart 1").top=40
And this is the graph it generates
How do I even begin to fix this? Any helpful tutorials?
Degustaf is 100% correct, his is the correct answer, but you have a lot of extra stuff in there that isn't needed. I can replicate your desired outcome including populating the spreadsheet with your test data, and do it in fewer lines than what you have. Here, check this out, you may come away with a few pointers for your future endeavors.
#Test Data
$Data=("8/15/2014",3091),("8/14/2014",240),("8/13/2014",519),("8/12/2014",622),("8/11/2014",2132),("8/10/2014",1255),("8/9/2014",3240)|ForEach{[PSCustomObject][Ordered]#{'Date_to_Display'=$_[0];'Number_of_Computers'=$_[1]}}
$xlConditionValues=[Microsoft.Office.Interop.Excel.XLConditionValueTypes]
$xlTheme=[Microsoft.Office.Interop.Excel.XLThemeColor]
$xlChart=[Microsoft.Office.Interop.Excel.XLChartType]
$xlIconSet=[Microsoft.Office.Interop.Excel.XLIconSet]
$xlDirection=[Microsoft.Office.Interop.Excel.XLDirection]
$xl = new-object -ComObject Excel.Application
$wb = $xl.workbooks.add()
$ws = $wb.activesheet
$xl.Visible = $true
#Populate test data onto worksheet
$Data |ConvertTo-CSV -NoTypeInformation -Delimiter "`t"| c:\windows\system32\clip.exe
$ws.Range("A1").Select | Out-Null
$ws.paste()
$ws.UsedRange.Columns.item(1).numberformat = "dddd, mmm dd, yyyy"
$ws.UsedRange.Columns.AutoFit() |Out-Null
#Create Chart
$chart=$ws.Shapes.AddChart().Chart
$chart.chartType=$xlChart::xlLine
#modify the chart title
$chart.ChartTitle.Text = "Number of Computers"
$ws.shapes.item("Chart 1").top=40
If you work with Powershell and Excel much you'll probably find the line up there $Data|ConvertTo-CSV... extremely useful.
Based on my experience with VBA, it appears that the first thing to try is changing your chart type. You have $chart.chartType=$xlChart::xlBarClustered. Based on the similar VBA commands, I would try changing this to $chart.chartType=$xlChart::xlLine. That should make a big difference, and let you see what else needs to be tweaked.
Fancy stuff to avoid seeing unnecessary red lines (Exception setting "ChartType": "Unspecified error) all over:
$chart.chartType = 4
Detailed list of chart types and more information available here: http://learn-powershell.net/2012/12/24/powershell-and-excel-adding-a-chart-and-header-filter-to-a-report/