access database lock errors and script crashing (oleaut32.dll) - multithreading

Trying to use runspaces to do some web scraping and database inserts. when trying to multithread the job, any more than a single thread causes database lock errors and my script to crash. I'm using adodb connection to an access database.
Function InsertToTable1 {
param ($database,$query,$col2,$col3,$col4,$col5,$col6,$col7,$col8,$col9,$col10,$col11)
$OpenStatic = 3
$LockOptimistic = 3
$Connection = New-Object -ComObject ADODB.Connection
$Connection.Open("Provider = Microsoft.ACE.OLEDB.12.0;Data Source=$database")
$Recordset = new-object -ComObject ADODB.Recordset
$Recordset.open($query,$Connection,$OpenStatic,$LockOptimistic)
$Recordset.AddNew()
$Recordset.Fields.Item("col2") = $col2
$Recordset.Fields.Item("col3") = $col3
$Recordset.Fields.Item("col4") = $col4
$Recordset.Fields.Item("col5") = $col5
$Recordset.Fields.Item("col6") = $col6
$Recordset.Fields.Item("col7") = $col7
$Recordset.Fields.Item("col8") = $col8
$Recordset.Fields.Item("col9") = $col9
$Recordset.Fields.Item("col10") = $col10
$Recordset.Fields.Item("col11") = $col11
$Recordset.Update()
$Recordset.close()
$Connection.close()
}
$code = #"
Function InsertToTable1 {$(Get-Command InsertToTable1 | Select -expand Definition)}
"# #
$pool = [RunspaceFactory]::CreateRunspacePool(1,1) #[int]$env:NUMBER_OF_PROCESSORS+1
$pool.ApartmentState = "MTA"
$pool.Open()
$runspaces = #()
$scriptblock = {
......
InsertToTable1 $DB $Q1 $varState $varTown $varStreet $fullURL $varStyle $varModel $varDescription $varUseCode $varDate $varDate
......
}
foreach ($x in $xxxxx) {
$runspace = [PowerShell]::Create().AddScript($scriptblock).AddArgument($x).AddArgument($code)
$runspace.RunspacePool = $pool
# Add runspace to runspaces collection and "start" it
# Asynchronously runs the commands of the PowerShell object pipeline
$runspaces += [PSCustomObject]#{ Pipe = $runspace; Status = $runspace.BeginInvoke() }
}
while ($runspaces.Status.IsCompleted -notcontains $true) {}
$results = #()
foreach ($runspace in $runspaces ) {
$results = $runspace.Pipe.EndInvoke($runspace.Status)
$runspace.Pipe.Dispose()
}
$pool.Close()
$pool.Dispose()

Related

Powershell script and Excel

I had coded a powershell executable in a evaluation windows server, where it worked fine. That is because it had Office (Excel) installed, hence it was possible to run $ExcelObj = New-Object -comobject Excel.Application. However, when moving on to the production server, my systems admin stressed that Office installation is strictly prohibited in the production server, making it not possible to execute the above command, and hence the remaining commands. I am not sure how to continue/change my code from this point onwards. I did search up and found this. But Its structure and method of writing to excel file is too "example", and am not able to figure out how to modify it to my preference. I really need this to work out.
Script.ps1:
#Exception for error hiding during debug
$ErrorActionPreference = 'SilentlyContinue'
#Variables
$PathFileCSV = "C:\Users\Administrator\Desktop\FinalPrototype.csv" #for actual
$PathFileXLSX = "C:\Users\Administrator\Desktop\FinalPrototype.xlsx" #for actual
#Function for file deletion (xlsx)
if ((Test-Path $PathFileXLSX)) {
Remove-Item $PathFileXLSX -Force
}
#Code to stop any running instance of excel, to prevent additional files from being created
# Stop-Process -processname excel -Force
#Function from Github (src:https://github.com/gangstanthony/PowerShell)
function Save-CSVasExcel {
param (
[string]$CSVFile = $(Throw 'No file provided.')
)
BEGIN {
function Resolve-FullPath ([string]$Path) {
if ( -not ([System.IO.Path]::IsPathRooted($Path)) ) {
# $Path = Join-Path (Get-Location) $Path
$Path = "$PWD\$Path"
}
[IO.Path]::GetFullPath($Path)
}
function Release-Ref ($ref) {
([System.Runtime.InteropServices.Marshal]::ReleaseComObject([System.__ComObject]$ref) -gt 0)
[System.GC]::Collect()
[System.GC]::WaitForPendingFinalizers()
}
$CSVFile = Resolve-FullPath $CSVFile
$xl = New-Object -ComObject Excel.Application
}
PROCESS {
$wb = $xl.workbooks.open($CSVFile)
$xlOut = $CSVFile -replace '\.csv$', '.xlsx'
# can comment out this part if you don't care to have the columns autosized
# $ws = $wb.Worksheets.Item(1)
# $range = $ws.UsedRange
# [void]$range.EntireColumn.Autofit()
$num = 1
$dir = Split-Path $xlOut
$base = $(Split-Path $xlOut -Leaf) -replace '\.xlsx$'
$nextname = $xlOut
while (Test-Path $nextname) {
$nextname = Join-Path $dir $($base + "-$num" + '.xlsx')
$num++
}
$wb.SaveAs($nextname, 51)
}
END {
$xl.Quit()
$null = $ws, $wb, $xl | ForEach-Object { Release-Ref $_ }
del $CSVFile
}
}
#Function to create the CSV File
Get-ADUser -Filter * -Properties * -SearchBase "DC=devops,DC=company1" | select-object displayName, mail, #{n = 'OU'; e = { (($_.DistinguishedName -replace '^.*?,(?=[A-Z]{2}=)' -split ',')[0]).substring(3) } } | Export-CSV -Path $PathFileCSV -NoTypeInformation
#select-object #{name = "MemberOf"; expression = { ($_.memberof | ForEach-Object { (Get-ADObject $_).Name }) -join " | " } } --- for user roles
#select-object sAMAccountName --- for username
#Function to convert created CSV to XLS (With formatted column)
Save-CSVasExcel $PathFileCSV
#Function for file deletion (csv)
#if ((Test-Path $PathFileCSV)) {
# Remove-Item $PathFileCSV -Force
#}
#Beautification
$ExcelObj = New-Object -comobject Excel.Application
$ExcelWorkBook = $ExcelObj.Workbooks.Open($PathFileXLSX)
$ExcelWorkSheet = $ExcelWorkBook.Sheets.Item("FinalPrototype")
#$ExcelWorkSheet.Cells.Font.Name = "Comfortaa"
$ExcelWorkSheet.Cells.Font.Size = 11
#Function for finding user groups & more
$usersAMAccName = Get-ADUser -Filter * -Properties * -SearchBase "DC=devops,DC=company1" | select-object sAMAccountName
$hash.Clear()
$data = #('Name', 'Email', 'Organizational Unit')
$data += #(Get-ADGroup -filter * -searchbase "OU=DevOps User Groups,OU=C1,DC=devops,DC=company1" | Select-Object -expandproperty name )
foreach ($name in $usersAMAccName) {
$hash += [ordered]#{ $name.sAMAccountName = (Get-ADPrincipalGroupMembership $name.sAMAccountName | select-object name).name }
}
#Header-Row-only Beautification
$Rows = $data.Count
for ($i = 1; $i -le $Rows; $i++) {
$ExcelWorkSheet.Cells.Item(1, $i).Font.Size = 16
$ExcelWorkSheet.Cells.Item(1, $i) = $data[$i - 1]
$ExcelWorkSheet.Cells.Item(1, $i).EntireRow.Interior.ColorIndex = 6
}
#Insertion profile-related data
$range = $ExcelWorkSheet.UsedRange
$rangeROW = $ExcelWorkSheet.UsedRange.Rows.Count
$rangeCOL = $ExcelWorkSheet.UsedRange.Columns.Count
for ($i = 2; $i -le $rangeROW; $i++) {
for ($j = 4; $j -le $rangeCOL; $j++) {
if ($hash.($usersAMAccName[$i - 2] | select-object -expandproperty sAMAccountName) -contains $ExcelWorkSheet.Cells.Item(1, $j).value2()) {
$ExcelWorkSheet.Cells.Item($i, $j).Interior.ColorIndex = 4
$ExcelWorkSheet.Cells.Item($i, $j) = "Assigned"
}
else {
$ExcelWorkSheet.Cells.Item($i, $j).Interior.ColorIndex = 3
$ExcelWorkSheet.Cells.Item($i, $j) = "Not Assigned"
}
}
}
#Content-usedRange-only Beautification
$range.EntireColumn.Autofit()
#Non-Human user omission
for ($i = 1; $i -le $rangeROW; $i++) {
If ([string]::IsNullOrEmpty($ExcelWorkSheet.Cells.Item($i, 1).value2)) {
$ExcelWorkSheet.Cells.Item($i, 1).EntireRow.hidden = $true
}
}
#Beautification
$range.WrapText = "True"
$range.HorizontalAlignment = -4108
#Saving the final form of excel
$ExcelObj.DisplayAlerts = $FALSE
$ExcelWorkBook.Save()
$ExcelWorkBook.close($true)
#User Ref (Debug)
Clear-Host
Write-Host 'Debug: Reading' $PathFileXLSX 'now...'
$data = Import-Excel $PathFileXLSX
$data
Final .xlsx file when ran in evaluation server (with excel):

How can I collect the return value data when using a runspace pool and begininvoke?

I have this code working well synchronously with powershell.Invoke() however with powershell.BeginInvoke() I am not able to capture the output. To use the below code you'll need to populate the $servers variable and it should run otherwise.
I tried to capture the output by adding each thread to the $threads variable while using EndInvoke() and I am able to see the thread handle and the iscompleted value, however I can't figure out where the value I am returning with the return portion of each function is stored.
The first block is the output I see, showing false for one async being finished until it finishes and then all thread handles show true.
Thanks!
8804 is True
16420 is True
13352 is True
11184 is True
3872 is True
8288 is True
17296 is False
20816 is True
11628 is True
17688 is True
12856 is True
19400 is True
8804 is True
16420 is True
13352 is True
11184 is True
3872 is True
8288 is True
17296 is True
20816 is True
11628 is True
17688 is True
12856 is True
19400 is True
Thread count: 12
Time elapsed: 3
cls;
$stopwatch = [system.diagnostics.stopwatch]::StartNew();
#region Runspace Pool
[runspacefactory]::CreateRunspacePool() | Out-Null;
$SessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault();
$RunspacePool = [runspacefactory]::CreateRunspacePool(
1, #Min Runspaces
16 #Max Runspaces
);
$RunspacePool.Open();
#endregion Runspace pool
$threads = New-Object System.Collections.ArrayList;
$servers = #("goodServer1", "goodServer2", "badServer1", "goodServer3");
foreach ($server in $servers)
{
$PowerShell = [powershell]::Create();
$PowerShell.RunspacePool = $RunspacePool;
[void]$PowerShell.AddScript({
Param ($server, $portNumber)
[pscustomobject]#{
server = $server
portNumber = $portNumber
} | Out-Null
Function testPort ($server, $portNumber)
{
$testPort = New-Object System.Net.Sockets.TCPClient # -ArgumentList $server, 3389;
$testPort.SendTimeout = 3;
try
{
$testPort.Connect($server, $portNumber);
}
catch
{
#do nothing;
}
$result = $testPort.Connected;
$testPort.Close();
$dateTime = ([DateTime]::Now.ToString());
return "$server|testPort|$result|$dateTime"; # server | function | result | DateTime
}
testPort -server $server -portNumber $portNumber;
}) # end of add script
$portNumber = "3389";
$PowerShell.AddParameter('server', $server).AddParameter('portNumber', $portNumber) | Out-Null;
$returnVal = $PowerShell.BeginInvoke();
$temp = "" | Select PowerShell,returnVal;
$temp.PowerShell = $PowerShell;
$temp.returnVal = $returnVal;
$threads.Add($Temp) | Out-Null;
$PowerShell = [powershell]::Create();
$PowerShell.RunspacePool = $RunspacePool;
[void]$PowerShell.AddScript({
Param ($server, $shareName, $timeOutInMs)
[pscustomobject]#{
server = $server
shareName = $shareName
timeOutInMs = $timeOutInMs
} | Out-Null
Function testShare ($server, $shareName, $timeOutInMs)
{
$cSharp =
#'
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
namespace cSharp7
{
public class cSharpClass
{
public bool verifyDirectoryExists(string uri, int timeoutInMs)
{
var task = new Task<bool>(() =>
{
var dir = new DirectoryInfo(uri);
return dir.Exists;
});
task.Start();
return task.Wait(timeoutInMs) && task.Result;
}
public bool verifyFileExists(string uri, int timeoutInMs)
{
var task = new Task<bool>(() =>
{
var fi = new FileInfo(uri);
return fi.Exists;
});
task.Start();
return task.Wait(timeoutInMs) && task.Result;
}
}
}
'#
$assemblies = ("System", "System.Collections", "System.ComponentModel", "System.Data", "System.Drawing", "System.Linq", "System.Threading.Tasks", "System.Windows.Forms", "System.Management.Automation", "System.Security", "System.Threading", "System.Collections.Concurrent", "System.Security.Principal", "System.Management", "System.IO", "System.Collections");
Add-Type -TypeDefinition $cSharp -ReferencedAssemblies $assemblies -Language CSharp;
$directoryExists = New-Object CSharp7.cSharpClass;
$path = "\\" + $server + "\" + $shareName;
try
{
$result = $directoryExists.verifyDirectoryExists($path, $timeOutInMs); # has a 2 minute timeout period, needs an asynchronous thread with a timeout period
#Write-Host $result;
}
catch
{
# do nothing
}
$dateTime = ([DateTime]::Now.ToString());
return "$server|testShare|$result|$dateTime"; # server | function | result | DateTime
}
testShare -server $server -shareName $shareName -timeOutInMs $timeOutInMs;
}) # end of add script
$shareName = "c$";
$timeOutInMs = "3000";
$PowerShell.AddParameter('server', $server).AddParameter('shareName', $shareName).AddParameter('timeOutInMs', $timeOutInMs) | Out-Null;
$returnVal = $PowerShell.BeginInvoke();
$temp = "" | Select PowerShell,returnVal;
$temp.PowerShell = $PowerShell;
$temp.returnVal = $returnVal;
$threads.Add($Temp) | Out-Null;
$PowerShell = [powershell]::Create();
$PowerShell.RunspacePool = $RunspacePool;
[void]$PowerShell.AddScript({
Param ($server, $pingCount)
[pscustomobject]#{
server = $server
pingCount = $pingCount
} | Out-Null
Function testPing ($server, $pingCount)
{
try
{
$result = Test-Connection $server -Count $pingCount -Quiet;
}
catch
{
# do nothing
}
$dateTime = ([DateTime]::Now.ToString());
return "$server|testPing|$result|$dateTime"; # server | function | result | DateTime
}
testPing -server $server -pingCount $pingCount;
}) # end of add script
$pingCount = "1";
$PowerShell.AddParameter('server', $server).AddParameter('pingCount', $pingCount) | Out-Null;
$returnVal = $PowerShell.BeginInvoke();
$temp = "" | Select PowerShell,returnVal;
$temp.PowerShell = $PowerShell;
$temp.returnVal = $returnVal;
$threads.Add($Temp) | Out-Null;
}
$completed = $false;
while ($completed -eq $false)
{
$completed = $true;
foreach ($thread in $threads)
{
$endInvoke = $thread.PowerShell.EndInvoke($thread.returnVal);
$endInvoke;
$threadHandle = $thread.returnVal.AsyncWaitHandle.Handle;
$threadIsCompleted = $thread.returnVal.IsCompleted;
#Write-Host "$threadHandle is $threadIsCompleted";
if ($threadIsCompleted -eq $false)
{
$completed = $false;
}
}
Write-Host "";
sleep -Milliseconds 500;
}
foreach ($thread in $threads)
{
$thread.PowerShell.Dispose();
}
$stopwatch.Stop();
Write-Host "";
Write-Host "Thread count:" $threads.Count;
Write-Host "Time elapsed:" $stopwatch.Elapsed.Seconds;
Here is how you capture the return value data. You define the custom object $temp with 2 properties names, in thise case Powershell and returnVal. Then you add them to an array list. After the async BeginInvoke is completed you can call EndInvoke against the asyncResult by doing this $endInvoke = $thread.PowerShell.EndInvoke($thread.returnVal); and parse it however you want.
This has to be the most complicated script I've ever written with a runspace pool, runspaces, asynchronous returns, functions being passed and even some c# mixed in. Hopefully others can draw from parts or all of this.
$returnVal = $PowerShell.BeginInvoke();
$temp = "" | Select PowerShell,returnVal;
$temp.PowerShell = $PowerShell;
$temp.returnVal = $returnVal;
$threads.Add($Temp) | Out-Null;
$completed = $false;
while ($completed -eq $false)
{
$completed = $true;
foreach ($thread in $threads)
{
$endInvoke = $thread.PowerShell.EndInvoke($thread.returnVal);
$endInvoke;
$threadHandle = $thread.returnVal.AsyncWaitHandle.Handle;
$threadIsCompleted = $thread.returnVal.IsCompleted;
#Write-Host "$threadHandle is $threadIsCompleted";
if ($threadIsCompleted -eq $false)
{
$completed = $false;
}
}
Write-Host "";
sleep -Milliseconds 500;
}
foreach ($thread in $threads)
{
$thread.PowerShell.Dispose();
}

How to retrieve the user that did the last modification in the site

I must retrieve the name of the user that did the last modification in a given web, using a Power Shell script.
I know that retrieving the last item modification date is straightforward, but how to retrieve the user that did such modification?
Retrieving the last modified item in the web would be fine too, since I would then pick the value of the "Modified By" field.
So... just to confirm... You want to check all the items in all the lists and libs in some web and get the last modified item from every list and get the user that modified it, correct? :)
if that is the case this kind of PS script should do the trick
if ((Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue) -eq $null)
{
Add-PSSnapin "Microsoft.SharePoint.PowerShell"
}
try
{
$siteUrl="[URL]";
$web = Get-SPWeb $siteUrl;
$spQuery = New-Object Microsoft.SharePoint.SPQuery;
$spQuery.ViewAttributes = "Scope='Recursive'";
$spQuery.Query = ""
$spQuery.RowLimit = 1;
$lastDate = $null;
$lastUser = $null;
foreach($list in $web.Lists)
{
Write-Host 'checking list -' $list.Title;
$items = $list.GetItems($spQuery);
if($items.Count -gt 0)
{
if($lastDate -eq $null)
{
$lastDate = $items[0]['Modified'];
$lastUser = $items[0]['Editor'];
}
else
{
if((get-date $items[0]['Modified']) -gt (get-date $lastDate))
{
$lastDate = $items[0]['Modified'];
$lastUser = $items[0]['Editor'];
}
}
}
}
Write-Host 'last user that modfied some item in this web was - '$lastUser ' - ' $lastDate;
}
catch
{
Write-Host $_.Exception.Message;
}
I hope it helps :).
You could compare the web object's LastItemModifiedDate property to the property with the same name on each list on the site. There should only be one list with the identical LastItemModifiedDate.
$list = $web.lists | where-object {$_.LastItemModifiedDate -eq $web.LastItemModifiedDate }
You can then just query for the most recently modified item in that list and see who edited it.
$query = new-object Microsoft.SharePoint.SPQuery
$query.Query = "<OrderBy><FieldRef Name='Modified' Ascending='FALSE' /></OrderBy>"
$query.RowLimit = 1
$items = $list.GetItems($query)
$item = $items[0]
A complete script might look something like this (replacing $webUrl with the desired web URL):
$web = get-spweb $webUrl
$list = $web.lists | where-object {$_.LastItemModifiedDate -eq $web.LastItemModifiedDate }
$query = new-object Microsoft.SharePoint.SPQuery
$query.Query = "<OrderBy><FieldRef Name='Modified' Ascending='FALSE' /></OrderBy>"
$query.RowLimit = 1
$items = $list.GetItems($query)
$item = $items[0]
write-host "$($web.AllUsers.GetById([int32]$item["Editor"].split(";#")[0]).DisplayName) modified item with ID $($item.ID) in $($list.Title) on $($item["Modified"])"
$web.dispose()
Below is an improved and tested version:
$web = Get-SPWeb $url
$webLastItemModifiedDate = $web.LastItemModifiedDate
$list = $web.lists | where-object {$_.LastItemModifiedDate -eq $webLastItemModifiedDate }
$formattedLastItemModifiedDate = $webLastItemModifiedDate.toString("yyyy-MM-ddTHH:mm:ssZ")
$query = new-object Microsoft.SharePoint.SPQuery
$query.Query = "<Where><Eq><FieldRef Name='Modified' /><Value Type='DateTime'>" + $formattedLastItemModifiedDate + "</Value></Eq></Where>"
$query.RowLimit = 1
$items = $list.GetItems($query)
$item = $items[0]
$webLastModifier = $web.AllUsers.GetById([int32]$item["Editor"].split(";#")[0]).DisplayName

Can't Make Two Identical QR Code Srtings Equal Each Other

This is my first post on here. I am pretty new to coding and Powershell so I might be missing something obvious here. I made a QR code label maker for my work using Powershell and Smartsheet. Without getting into too much detail on that it pulls data from a line in a "Product Tracker" we made in Smartsheet and creates a label with a QR code from that data. In one of the columns on that line it mimics the data produced when you scan the QR code. It mimics it like this....
$SkuCell.Value = "BEGIN:VCARD
VERSION:3.0
KIND:individual
N:$($ptCO.Desc);$($ptCO.Po)
FN:$($ptCO.Po) $($ptCO.Desc)
ORG:All New Glass
EMAIL;TYPE=INTERNET:$($ptCO.Assign)
END:VCARD"
... and the output looks like this.....
BEGIN:VCARD
VERSION:3.0
KIND:individual
N:Test stuff here;12354
FN:12354 Test stuff here
ORG:All New Glass
EMAIL;TYPE=INTERNET:ericg#allnewglass.com
END:VCARD
Then on a different Smartsheet our drivers will scan the QR code on the label signifying that its leaving our hub. The scanned data in that cell looks like this.....
BEGIN:VCARD
VERSION:3.0
KIND:individual
N:Test stuff here;12354
FN:12354 Test stuff here
ORG:All New Glass
EMAIL;TYPE=INTERNET:ericg#allnewglass.com
END:VCARD
I then have a different script checking for matches between the two sheets so In the product tracker we know when something leaves the hub. The problem is that even though the scanned data and the mimicked data look absolutely identical and they both say they are strings, they never match. They are not equal to each other and I have tried EVERYTHING ([string], out-string, convert-string, compare-object, trim, etc) I can think of to get them to match and the only thing that works is if I pipeline Out-Host on the objects. However, these objects are contained in a foreach loop that checks through all the rows in both sheets which means that the script runs way to slow having to display all that data. It seems likely that there are some hidden characters in the scanned data causing them to not match. Can anyone think of a solution to make these two strings identical? Is there a way of removing all hidden characters? Or does anyone have a better idea? A different way to compare them maybe?
Here's the code that tries to compare....
# by GrEcHkO
#cd "C:\Users\Jacly\Desktop\ANG_System_Files"
cd "P:\ANG_System_Files"
function Load-Dll
{
param(
[string]$assembly
)
Write-Host "Loading $assembly"
$driver = $assembly
$fileStream = ([System.IO.FileInfo] (Get-Item $driver)).OpenRead();
$assemblyBytes = new-object byte[] $fileStream.Length
$fileStream.Read($assemblyBytes, 0, $fileStream.Length) | Out-Null;
$fileStream.Close();
$assemblyLoaded = [System.Reflection.Assembly]::Load($assemblyBytes);
}
function Get-ComparisonObjects
{
param([Smartsheet.Api.Models.Sheet]$sheet)
Write-Host "Getting Sheet $($sheet.Name) Comparison Objects"
$data = $sheet.Rows | foreach {
$checkVal = $false
$trackedCheckVal = $false
$finishedCheckVal = $false
if($_.Cells[3].Value -eq $true)
{
$checkVal = $true
}
if($_.Cells[16].Value -eq $true)
{
$trackedCheckVal = $true
}
if($_.Cells[12].Value -eq $true)
{
$finishedCheckVal = $true
}
[pscustomobject]#{
Attachments = $_.Attachments;
RowId = $_.Id;
RowNumber = $_.RowNumber;
Parent = $_.ParentId;
PoCol = $_.Cells[0].ColumnId;
Po = $_.Cells[0].Value;
JobsCol = $_.Cells[1].ColumnId;
Jobs = $_.Cells[1].Value;
DescCol = $_.Cells[2].ColumnId;
Desc = $_.Cells[2].Value;
CheckCol = $_.Cells[3].ColumnId;
Check = $checkVal;
SupplierCol = $_.Cells[4].ColumnId;
Supplier = $_.Cells[4].Value;
AssignCol = $_.Cells[5].ColumnId;
Assign = $_.Cells[5].Value;
DestinationCol = $_.Cells[11].ColumnId;
Destination = $_.Cells[11].Value;
FinishedCol = $_.Cells[12].ColumnId;
Finished = $finishedCheckVal;
DueCol = $_.Cells[14].ColumnId;
Due = $_.Cells[14].Value;
DeliveryCol = $_.Cells[15].ColumnId;
Delivery = $_.Cells[15].Value;
TrackedCol = $_.Cells[16].ColumnId;
Tracked = $trackedCheckVal;
SKUCol = $_.Cells[18].ColumnId;
SKU = $_.Cells[18].Value;
}
}| where {![string]::IsNullOrWhiteSpace($_.Po)}
Write-Host "$($data.Count) Returned"
return $data
}
function Get-DriverComparisonObjects
{
param([Smartsheet.Api.Models.Sheet]$sheet)
Write-Host "Getting Sheet $($sheet.Name) Comparison Objects"
$data = $sheet.Rows | foreach {
$checkVal = $false
$trackedCheckVal = $false
$finishedCheckVal = $false
$archiveCheckVal = $false
if($_.Cells[2].Value -eq $true)
{
$checkVal = $true
}
if($_.Cells[8].Value -eq $true)
{
$trackedCheckVal = $true
}
if($_.Cells[3].Value -eq $true)
{
$finishedCheckVal = $true
}
if($_.Cells[11].Value -eq $true)
{
$archiveCheckVal = $true
}
[pscustomobject]#{
Attachments = $_.Attachments;
RowId = $_.Id;
RowNumber = $_.RowNumber;
Parent = $_.ParentId;
DayCol = $_.Cells[0].ColumnId;
Day = $_.Cells[0].Value;
DueCol = $_.Cells[1].ColumnId;
Due = $_.Cells[1].Value;
CompletedCol = $_.Cells[2].ColumnId; #########hidden
Completed = $checkVal;
CheckCol = $_.Cells[3].ColumnId;
Check = $finishedCheckVal;
SupplierCol = $_.Cells[4].ColumnId;#########hidden
Supplier = $_.Cells[4].Value;
AssignCol = $_.Cells[5].ColumnId;#########hidden
Assign = $_.Cells[5].Value;
JobNameCol = $_.Cells[6].ColumnId;
JobName = $_.Cells[6].Value;
MainCol = $_.Cells[7].ColumnId;
Main = $_.Cells[7].Value;
TrackedCol = $_.Cells[8].ColumnId;
Tracked = $trackedCheckVal;
PoNumCol = $_.Cells[9].ColumnId;
PoNum = $_.Cells[9].Value;
ModifiedCol = $_.Cells[10].ColumnId;
Modified = $_.Cells[10].Value;
ArchiveCol = $_.Cells[11].ColumnId;
Archive = $archiveCheckVal;
}
} | where {![string]::IsNullOrWhiteSpace($_.PoNum)}
Write-Host "$($data.Count) Returned"
return $data
}
function Get-AttachmentFromSmartsheet
{
param (
[long]$attachmentId,
[long]$sheetId
)
Write-Host "Getting Attachement $attachmentId of Sheet $sheetId"
try
{
$attachment = $client.SheetResources.AttachmentResources.GetAttachment($sheetId,$attachmentId)
}
catch
{
Write-Error $_.Exception.Message
Write-Host ""
}
$downloads = New-Item -ItemType Directory ".\downloads" -Force
$filepath = "$($downloads.Fullname)\$($attachment.Name)"
Write-Host "Downloading $filepath"
Invoke-WebRequest -Uri $attachment.Url -OutFile $filepath
Get-Item $filepath
}
function Save-AttachmentToSheetRow
{
param(
[long]$sheetId,
[long]$rowId,
[System.IO.FileInfo]$file,
[string]$mimeType
)
Write-Host "Saving $($file.Fullname) to Sheet $sheetId"
$result = $client.SheetResources.RowResources.AttachmentResources.AttachFile($sheetId, $rowId, $file.FullName, $mimeType)
return $result
}
function Merge-DriverChecklistWithProductTracker
{
param(
[pscustomobject[]]$orbitalRecords,
[long]$orbitalId
)
foreach ($orbitalRecord in $orbitalRecords)
{
$descFound = $false
$skuFound = $false
if ($orbitalRecord.Archive -eq $false) #THIS IS WHERE I NEED HELP
{ #THIS IS WHERE I NEED HELP
foreach ($ptCO in $ptCOs) #THIS IS WHERE I NEED HELP
{ #THIS IS WHERE I NEED HELP
$orbitalQR = "$($orbitalRecord.PoNum)" #THIS IS WHERE I NEED HELP
$ptQR = "$($ptCO.SKU)" #THIS IS WHERE I NEED HELP
$noParent = $false #THIS IS WHERE I NEED HELP
#THIS IS WHERE I NEED HELP
if ($orbitalQR -eq $ptQR) #THIS IS WHERE I NEED HELP
{ #THIS IS WHERE I NEED HELP
$descFound = $true #THIS IS WHERE I NEED HELP
break #THIS IS WHERE I NEED HELP
} #THIS IS WHERE I NEED HELP
#THIS IS WHERE I NEED HELP
if ($orbitalQR -eq $ptQR) #THIS IS WHERE I NEED HELP
{ #THIS IS WHERE I NEED HELP
$skuFound = $true #THIS IS WHERE I NEED HELP
break #THIS IS WHERE I NEED HELP
} #THIS IS WHERE I NEED HELP
} #THIS IS WHERE I NEED HELP
#THIS IS WHERE I NEED HELP
if ($skuFound) #THIS IS WHERE I NEED HELP
{ #THIS IS WHERE I NEED HELP
if (![string]::IsNullOrWhiteSpace($orbitalRecord.PoNum) -and ![string]::IsNullOrWhiteSpace($ptCO.SKU)) #THIS IS WHERE I NEED HELP
{ #THIS IS WHERE I NEED HELP
Write-Host "SKU found. Updating data on PT." #THIS IS WHERE I NEED HELP
#THIS IS WHERE I NEED HELP
$destinationCell = [Smartsheet.Api.Models.Cell]::new() #THIS IS WHERE I NEED HELP
$destinationCell.ColumnId = $ptDestinationCol.Id #THIS IS WHERE I NEED HELP
$destinationCell.Value = "ITEM SCANNED TO ANG TRUCK" #THIS IS WHERE I NEED HELP
#THIS IS WHERE I NEED HELP
$shippedCell = [Smartsheet.Api.Models.Cell]::new() #THIS IS WHERE I NEED HELP
$shippedCell.ColumnId = $ptShippedCol.Id #THIS IS WHERE I NEED HELP
$shippedCell.Value = if ($orbitalRecord.Modified -ne $null){$orbitalRecord.Modified} else {[string]::Empty} #THIS IS WHERE I NEED HELP
#THIS IS WHERE I NEED HELP
$row = [Smartsheet.Api.Models.Row]::new() #THIS IS WHERE I NEED HELP
$row.Id = $ptCO.RowId #THIS IS WHERE I NEED HELP
$row.Cells = [Smartsheet.Api.Models.Cell[]]#($destinationCell, $shippedCell) #THIS IS WHERE I NEED HELP
#THIS IS WHERE I NEED HELP
$updateRow = $client.SheetResources.RowResources.UpdateRows($ptId, [Smartsheet.Api.Models.Row[]]#($row)) #THIS IS WHERE I NEED HELP
} #THIS IS WHERE I NEED HELP
} #THIS IS WHERE I NEED HELP
elseif ($orbitalRecord.PoNum -eq $ptCO.Po)
{
if ($descFound)
{
if ([string]::IsNullOrWhiteSpace($ptCO.Parent))
{
$noParent = $true
}
if ($noParent)
{
Write-Host "Updating Product Tracker with Driver Checklist $($orbitalRecord.PoNum) $($orbitalRecord.Main)"
$jobsCell = [Smartsheet.Api.Models.Cell]::new()
$jobsCell.ColumnId = $ptJobsCol.Id
$JobsCell.Value = if ($orbitalRecord.JobName -ne $null){$orbitalRecord.JobName} else {[string]::Empty}
$descCell = [Smartsheet.Api.Models.Cell]::new()
$descCell.ColumnId = $ptDescCol.Id
$descCell.Value = if ($orbitalRecord.Main -ne $null){$orbitalRecord.Main} else {[string]::Empty}
$checkCell = [Smartsheet.Api.Models.Cell]::new()
$checkCell.ColumnId = $ptCheckCol.Id
$checkCell.Value = $orbitalRecord.Completed
$assignCell = [Smartsheet.Api.Models.Cell]::new()
$assignCell.COlumnId = $ptAssignCol.Id
$assignCell.Value = if ($orbitalRecord.Assign -ne $null){$orbitalRecord.Assign} else {"ianz#allnewglass.com"}
$supCell = [Smartsheet.Api.Models.Cell]::new()
$supCell.COlumnId = $ptSupplierCol.Id
$supCell.Value = if (($orbitalRecord.Completed -eq $false) -and ($orbitalRecord.Supplier -eq "In Shop FAB")){$suppliers["$fabId"]} else {$suppliers["$DriveId"]}
$finishedCell = [Smartsheet.Api.Models.Cell]::new()
$finishedCell.ColumnId = $ptFinishedCol.Id
$finishedCell.Value = $orbitalRecord.Check
$row = [Smartsheet.Api.Models.Row]::new()
$row.Id = $ptCO.RowId
$row.Cells = [Smartsheet.Api.Models.Cell[]]#($JobsCell, $checkCell, $supCell, $assignCell, $finishedCell)
try
{
$updateRow = $client.SheetResources.RowResources.UpdateRows($ptId, [Smartsheet.Api.Models.Row[]]#($row))
$reference = if ($orbitalRecord.Attachments.Name -ne $null){$orbitalRecord.Attachments.Name} else {[string]::Empty}
$difference = if ($ptCO.Attachments.Name -ne $null){$ptCO.Attachments.Name} else {[string]::Empty}
$compareResults = Compare-Object -ReferenceObject $reference -DifferenceObject $difference -IncludeEqual -SyncWindow ([int]::MaxValue)
$missing = $compareResults | where SideIndicator -eq '<='
$missingAttachments = $missing.InputObject
$attachmentsToGet = $orbitalRecord.Attachments | where Name -in $missingAttachments
foreach($attachment in $attachmentsToGet)
{
Write-Host "Adding missing attachment $($attachment.name)"
$file = Get-AttachmentFromSmartsheet -attachmentId $attachment.Id -sheetId $orbitalId
$result = Save-AttachmentToSheetRow -sheetId $ptId -rowId $newRow.Id -file $file.FullName -mimeType $attachment.MimeType
}
}
catch
{
Write-Error $_.Exception.Message
Write-Host ""
}
if ($orbitalRecord.Check -eq $true)
{
$dateCell = [Smartsheet.Api.Models.Cell]::new()
$dateCell.ColumnId = $ptShippedCol.Id
$dateCell.Value = $orbitalRecord.Modified
$row = [Smartsheet.Api.Models.Row]::new()
$row.Id = $ptCO.RowId
$row.Cells = [Smartsheet.Api.Models.Cell[]]#($dateCell)
try
{
$updateRow = $client.SheetResources.RowResources.UpdateRows($ptId, [Smartsheet.Api.Models.Row[]]#($row))
}
catch
{
Write-Error $_.Exception.Message
Write-Host ""
}
}
if (![string]::IsNullOrWhiteSpace($orbitalRecord.Due))
{
$dateCell = [Smartsheet.Api.Models.Cell]::new()
$dateCell.ColumnId = $ptAnticipatedCol.Id
$dateCell.Value = $orbitalRecord.Due
$row = [Smartsheet.Api.Models.Row]::new()
$row.Id = $ptCO.RowId
$row.Cells = [Smartsheet.Api.Models.Cell[]]#($dateCell)
try
{
$updateRow = $client.SheetResources.RowResources.UpdateRows($ptId, [Smartsheet.Api.Models.Row[]]#($row))
}
catch
{
Write-Error $_.Exception.Message
Write-Host ""
}
}
}
}
}
else
{
if (!($descFound -or $skuFound))
{
if (![string]::IsNullOrWhiteSpace($orbitalRecord.PoNum))
{
if ($orbitalRecord.Tracked -eq "$true")
{
Write-Host "Adding to Product Tracker from Driver Checklist $($orbitalRecord.Po) $($orbitalRecord.Main)"
$poCell = [Smartsheet.Api.Models.Cell]::new()
$poCell.ColumnId = $ptPoCol.Id
$poCell.Value = if ($orbitalRecord.PoNum -ne $null){$orbitalRecord.PoNum} else {[string]::Empty}
$jobsCell = [Smartsheet.Api.Models.Cell]::new()
$jobsCell.ColumnId = $ptJobsCol.Id
$JobsCell.Value = if ($orbitalRecord.JobName -ne $null){$orbitalRecord.JobName} else {[string]::Empty}
$descCell = [Smartsheet.Api.Models.Cell]::new()
$descCell.ColumnId = $ptDescCol.Id
$descCell.Value = if ($orbitalRecord.Main -ne $null){$orbitalRecord.Main} else {[string]::Empty}
$checkCell = [Smartsheet.Api.Models.Cell]::new()
$checkCell.ColumnId = $ptCheckCol.Id
$checkCell.Value = $orbitalRecord.Completed
$supCell = [Smartsheet.Api.Models.Cell]::new()
$supCell.COlumnId = $ptSupplierCol.Id
$supCell.Value = if (($orbitalRecord.Completed -eq $false) -and ($orbitalRecord.Supplier -eq "In Shop FAB")){$suppliers["$fabId"]} else {$suppliers["$DriveId"]}
$assignCell = [Smartsheet.Api.Models.Cell]::new()
$assignCell.COlumnId = $ptAssignCol.Id
$assignCell.Value = if ($orbitalRecord.Assign -ne $null){$orbitalRecord.Assign} else {"ianz#allnewglass.com"}
$finishedCell = [Smartsheet.Api.Models.Cell]::new()
$finishedCell.ColumnId = $ptFinishedCol.Id
$finishedCell.Value = $orbitalRecord.Check
$row = [Smartsheet.Api.Models.Row]::new()
$row.ToBottom = $true
$row.Cells = [Smartsheet.Api.Models.Cell[]]#($poCell,$jobsCell,$descCell,$checkCell,$supCell, $assignCell, $finishedCell)
try
{
$newRow = $client.SheetResources.RowResources.AddRows($ptId, [Smartsheet.Api.Models.Row[]]#($row))
foreach($attachment in $orbitalRecord.Attachments)
{
$file = Get-AttachmentFromSmartsheet -attachmentId $attachment.Id -sheetId $orbitalId
$result = Save-AttachmentToSheetRow -sheetId $ptId -rowId $newRow.Id -file $file.FullName -mimeType $attachment.MimeType
}
}
catch
{
Write-Error $_.Exception.Message
Write-Host ""
}
}
}
}
}
}
}
}
Write-Host "Loading Dlls"
Load-Dll ".\smartsheet-csharp-sdk.dll"
Load-Dll ".\RestSharp.dll"
Load-Dll ".\Newtonsoft.Json.dll"
Load-Dll ".\NLog.dll"
while($true)
{
Write-Host "Fab Log to Driver List to Product Tracker system starting up."
$DriveId = ""
$fabId = ""
$ptId = ""
$suppliers = #{
$fabId = "In Shop FAB";
$DriveId = "DRIVER CHECKLIST";
}
$token = ""
$smartsheet = [Smartsheet.Api.SmartSheetBuilder]::new()
$builder = $smartsheet.SetAccessToken($token)
$client = $builder.Build()
$includes = #([Smartsheet.Api.Models.SheetLevelInclusion]::ATTACHMENTS)
$includes = [System.Collections.Generic.List[Smartsheet.Api.Models.SheetLevelInclusion]]$includes
Write-Host "Loading Sheets"
$Drive = $client.SheetResources.GetSheet($DriveId, $includes, $null, $null, $null, $null, $null, $null);
$fab = $client.SheetResources.GetSheet($fabId, $includes, $null, $null, $null, $null, $null, $null);
$pt = $client.SheetResources.GetSheet($ptId, $includes, $null, $null, $null, $null, $null, $null);
Write-Host "Comparing Objects"
$DriveCOs = Get-DriverComparisonObjects $Drive
$fabCOs = Get-ComparisonObjects $fab
$ptCOs = Get-ComparisonObjects $pt
Write-Host "Identifying Driver Checklist Columns"
#NO SPACE... BODY CHARACTER LIMIT FOR STACK OVERFLOW
Merge-DriverChecklistWithProductTracker -orbitalRecords $DriveCOs -orbitalId $DriveId
Write-Host "Driver Checklist to Product Tracker finished."
Start-Sleep -Seconds 10
}

PowerShell DataTable delete empty rows

I've stumbled upon some (DCOM) issues when using the Excel.Application object. So I've rewritten my function to read Excel files by using the OleDbConnectionand OleDbCommand objects available within AccessDatabaseEngine_x64. So there's no need to install MS Office on the server anymore and it's also more reliable I think.
It works great but the only thing missing is that I can't seem to figure out how to remove empty lines from the output of the function. I know it can be done by using $DataTable[0].Delete(), but I don't know how to determine if a complete row is empty. As it can vary in size depending on the input.
XLSX-File example:
Name | Sir name
Bob Lee | Swagger
|
Jake | Thornton
In the example above I would like to have the output only 2 (or 3 lines depending on the $Header switch), but I don't want to see the blank line in the output.
I found a solution in another language, but I can't translate it to PowerShell.
The code:
Function Import-Excel {
[CmdletBinding()]
Param (
[parameter(Mandatory=$true,Position=0)]
[ValidateScript({Test-Path $_ -PathType Leaf})]
[String]$FileName,
[parameter(Mandatory=$true,Position=1)]
[String]$WorksheetName,
[Switch]$Header
)
Begin {
# We can't read open XLSX-Files
Copy-Item -LiteralPath $FileName -Destination $env:TEMP
}
Process {
$OleDbConnection = New-Object 'System.Data.OleDb.OleDbConnection'
$OleDbCommand = New-Object 'System.Data.OleDb.OleDbCommand'
$ConnString = 'Provider=Microsoft.ACE.OLEDB.12.0;Data Source='
$ConnString += "$env:TEMP\$(Split-Path $FileName -Leaf)"
if ($header) {
$ConnString += ';Extended Properties="Excel 12.0;HDR=YES;IMEX=1";'
}
else {
$ConnString += ';Extended Properties="Excel 12.0;HDR=NO;IMEX=1";'
}
$OleDbConnection.ConnectionString = $ConnString
$OleDbConnection.Open()
$OleDbCommand.Connection = $OleDbConnection
$OleDbCommand.CommandText = "SELECT * FROM [$WorksheetName$]"
$OleDbAdapter = New-Object "System.Data.OleDb.OleDbDataAdapter"
$OleDbAdapter.SelectCommand = $OleDbCommand
$DataTable = New-Object "System.Data.DataTable"
$OleDbAdapter.Fill($DataTable)
$OleDbConnection.Close()
Write-Output $DataTable
}
End {
Remove-Item "$env:TEMP\$(Split-Path $FileName -Leaf)"
}
}
Thank you for your help.
Found the answer in the meantime, for anyone encountering the same issues:
Function Import-Excel {
[CmdletBinding()]
Param (
[parameter(Mandatory=$true,Position=0)]
[ValidateScript({Test-Path -LiteralPath $_ -PathType Leaf})]
[String]$FileName,
[parameter(Mandatory=$true,Position=1)]
[String]$WorksheetName,
[Switch]$Header
)
Begin {
# We can't read open XLSX-Files
Copy-Item -LiteralPath $FileName -Destination $env:TEMP
}
Process {
$OleDbConnection = New-Object 'System.Data.OleDb.OleDbConnection'
$OleDbCommand = New-Object 'System.Data.OleDb.OleDbCommand'
$OleDbAdapter = New-Object 'System.Data.OleDb.OleDbDataAdapter'
$DataTable = New-Object 'System.Data.DataTable'
$ConnString = 'Provider=Microsoft.ACE.OLEDB.12.0;Data Source='
$ConnString += "$env:TEMP\$(Split-Path $FileName -Leaf)"
if ($header) {
$ConnString += ';Extended Properties="Excel 12.0;HDR=YES;IMEX=1";'
}
else {
$ConnString += ';Extended Properties="Excel 12.0;HDR=NO;IMEX=1";'
}
$OleDbConnection.ConnectionString = $ConnString
$OleDbConnection.Open()
$OleDbCommand.Connection = $OleDbConnection
$OleDbCommand.CommandText = "SELECT * FROM [$WorksheetName$]"
$OleDbAdapter.SelectCommand = $OleDbCommand
$OleDbAdapter.Fill($DataTable)
$OleDbConnection.Close()
# Remove empty lines
$Columns = $DataTable.Columns.Count
$Rows = $DataTable.Rows.Count
for ($r = 0; $r -lt $Rows; $r++) {
$Empty = 0
for ($c = 0; $c -lt $Columns; $c++) {
if ($DataTable.Rows[$r].IsNull($c)) {
$Empty++
}
}
if ($Empty -eq $Columns) {
# Mark row for deletion:
$DataTable.Rows[$r].Delete()
}
}
# Delete marked rows:
$DataTable.AcceptChanges()
Write-Output $DataTable
}
End {
Remove-Item "$env:TEMP\$(Split-Path $FileName -Leaf)"
}
}

Resources