Powershell Datahandling - excel

Dear Stack Overflow Community,
I think you can guess it, Houston has got a problem.
Background: Our Company was attacked by Hackers and we lost a TON of data. Now, where everything is back up and running (more or less) it came to light that we have thousands and thousands of old .xls and .doc Documents on our servers which have to be converted to the newer formats before re-entering our brand new server structure.
Currently I am trying to write a script that converts XLS to XLSX. The Problem: I am good at Batch Programming but completly lost when it comes to Powershell. This is my third attempt after writing it from scratch. It does what it is supposed to do, but I don´t know how to do proper data handling in PS.
What it is supposed to do:
Ask the User where to look for XLS files.
Ask the User where to store the new XLSX files.
Take every .xls file from the directory (and subfolders), open it and save it as .xlsx to the given path.
What it does so far:
Ask the User where to look for XLS files.
Ask the User where to store the new XLSX files.
Screw up the paths completly + cant open file.
It is able to convert .xls to .xlsx when I modify the script to feed it the complete paths. So the Excel part seems to be correct. I know my data handling part is very wrong. I tried to rewrite it many times now. I left it in so you may see what I tried to do.
$LookXLS = Read-Host -Prompt 'Where do I look for XLS?'
$TargetpathXLSX = Read-Host -Prompt 'Where to save the XLSX?'
Write-Host $LookXLS
cd $LookXLS # Change Directory to given path
Get-ChildItem -Path .\ -Filter *.xls -Recurse -File -Name| ForEach-Object {
[System.IO.Path]::GetFileName($_)
$excel=New-Object -comobject Excel.Application
$excel.Visible =$TRUE
$excel.DisplayAlerts = $FALSE
$wb = $excel.Workbooks.Open($LookXLS)
$LookXLS=$TargetpathXLSX
$typ=".xlsx"
$TargetPathXLSX="$TargetpathXLSX\$_$typ"
echo $TargetPathXLSX
$excel.ActiveWorkbook.SaveAs("$TargetPathXLSX",51)
$excel.ActiveWorkbook.Close()
$excel.Quit()
}
echo Process finished.
pause
The Output looks like this:
(Comments by me because it is partly German)
PS C:\Users\USER\Desktop\Testordner\Userverzeichnis> C:\Users\USER\Desktop\new 1.ps1
Where do I look for XLS?: C:\Users\USER\Desktop\Testordner\Userverzeichnis
Where to save the XLSX?: C:\Users\USER\Desktop\Testordner\Ziel
C:\Users\USER\Desktop\Testordner\Userverzeichnis
Microsoft Excel-Arbeitsblatt (neu) (2).xls
Wir konnten 'Arbeit\Microsoft Excel-Arbeitsblatt (neu) (2).xls' nicht finden. Wurde das Objekt vielleicht verschoben, umbenannt oder gelöscht? **#Can´t find it**
In C:\Users\USER\Desktop\new 1.ps1:17 Zeichen:1
+ $wb = $excel.Workbooks.Open($_)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException
C:\Users\USER\Desktop\Testordner\Ziel\Arbeit\Microsoft Excel-Arbeitsblatt (neu) (2).xls.xlsx
Es ist nicht möglich, eine Methode für einen Ausdruck aufzurufen, der den NULL hat.
**#NULL (obviously, the path is screwed up)**
In C:\Users\USER\Desktop\new 1.ps1:22 Zeichen:1
+ $excel.ActiveWorkbook.SaveAs("$TargetPathXLSX",51)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
Es ist nicht möglich, eine Methode für einen Ausdruck aufzurufen, der den NULL hat.
In C:\Users\USER\Desktop\new 1.ps1:24 Zeichen:1
+ $excel.ActiveWorkbook.Close()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
Microsoft Excel-Arbeitsblatt (neu).xlsx
Wir konnten 'Arbeit\Microsoft Excel-Arbeitsblatt (neu).xlsx' nicht finden. Wurde das Objekt vielleicht verschoben, umbenannt oder gelöscht?
In C:\Users\USER\Desktop\new 1.ps1:17 Zeichen:1
+ $wb = $excel.Workbooks.Open($_)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException
C:\Users\USER\Desktop\Testordner\Ziel\Arbeit\Microsoft Excel-Arbeitsblatt (neu) (2).xls.xlsx\Arbeit\Microsoft Excel-Arbeitsblatt (neu).xlsx.xlsx
**#It gets worse every time it loops.**
Es ist nicht möglich, eine Methode für einen Ausdruck aufzurufen, der den NULL hat.
In C:\Users\USER\Desktop\new 1.ps1:22 Zeichen:1
+ $excel.ActiveWorkbook.SaveAs("$TargetPathXLSX",51)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
Es ist nicht möglich, eine Methode für einen Ausdruck aufzurufen, der den NULL hat.
In C:\Users\USER\Desktop\new 1.ps1:24 Zeichen:1
+ $excel.ActiveWorkbook.Close()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
I hope, somehow you can help me. I´m not a PS guy, but sadly Batch can´t work with Excel. I don´t think I am too far from a solution.

Try this -
$LookXLS = Read-Host -Prompt 'Where do I look for XLS?'
$TargetpathXLSX = Read-Host -Prompt 'Where to save the XLSX?'
Write-Host "$($LookXLS) is where we look" -ForegroundColor cyan
cd $LookXLS # Change Directory to given path
Get-ChildItem -Path .\ -Filter *.xls -Recurse | ForEach-Object {
$path = ($_.fullname).substring(0, ($_.FullName).lastindexOf("."))
$excel = New-Object -comobject Excel.Application
$excel.Visible = $TRUE
$excel.DisplayAlerts = $FALSE
$wb = $excel.Workbooks.Open($_.FullName)
$path += ".xlsx"
$wb.saveas($path, 51)
$wb.close()
$excel.Quit()
}
The above code changes the xls files to xlsx. You need to add the code for copying the xlsx files to the target folder.

Related

Error while adding tags to vm with powershell after reading from csv

I am trying to add tags to azure vm by reading from csv file with a powershell script. I want to read the value via loop and add to existing tags of vm, if any. Below is my code and the respective errors.
$data = Import-CSV C:\Documents\tags-vms.csv
foreach($info in $data){
$tags = (Get-AzResource -ResourceGroupName policyResourceGroup -Name $info.psobject.properties.value[0]).Tags
$scriptBlock = [scriptblock]::Create('#{$info.Tags}')
$newtags = (& $scriptBlock)
$tags += $newtags
Write-Host $tags
}
Now the error is
Exception calling "Create" with "1" argument(s): "At line:1 char:13
2021-12-20T16:01:09.2145247Z + #{$info.Tags}
2021-12-20T16:01:09.2147301Z + ~
2021-12-20T16:01:09.2149246Z Missing '=' operator after key in hash literal.
2021-12-20T16:01:09.2152282Z At line:1 char:13
2021-12-20T16:01:09.2154454Z + #{$info.Tags}
2021-12-20T16:01:09.2156590Z + ~
2021-12-20T16:01:09.2166796Z The hash literal was incomplete."
2021-12-20T16:01:09.2168854Z At C:\agent\_work\_temp\e8ee61c2-c3f4-4ea5-99ac-4feb671fff57.ps1:18 char:1
2021-12-20T16:01:09.2169842Z + $scriptBlock = [scriptblock]::Create('#{$info.Tags}')
2021-12-20T16:01:09.2170521Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-12-20T16:01:09.2171329Z + CategoryInfo : NotSpecified: (:) [], ParentContainsErrorRecordException
2021-12-20T16:01:09.2171826Z + FullyQualifiedErrorId : ParseException
Can someone please help.
**csv file contains**
VmName Tags
test-vm01 "loc"="us"
test-vm02 "Loc"="Us";"doseage"="Second"
I think your issue lies in this line and how you're retrieving the tag info from the CSV object from your foreach loop:
$scriptBlock = [scriptblock]::Create('#{$info.Tags}')
In the previous line in your script, to define the $tags variable, you retrieve the value of the name of the VM in the CSV object using ($info.psobject.properties.value[0]). I would attempt to use that same format in the next line, but modifying it to retrieve the tag part of the CSV object using the index of 1:
So instead of:
$scriptBlock = [scriptblock]::Create('#{$info.Tags}')
use:
$scriptBlock = [scriptblock]::Create('#{$info.psobject.properties.value[1]}')

Azure Powershell Command failure when using Invoke-AzsRegistrationValidation

I have been following the steps given here(https://learn.microsoft.com/en-us/azure-stack/operator/azure-stack-validate-registration?view=azs-2008&tabs=az) to validate a subscription but when i run the invoke command it fails with the below error:
> PS C:\WINDOWS\system32> Invoke-AzsRegistrationValidation
> -RegistrationSubscriptionID $subscriptionID Invoke-AzsRegistrationValidation : Parameter set cannot be resolved
> using the specified named parameters. At line:1 char:1
> + Invoke-AzsRegistrationValidation -RegistrationSubscriptionID $subscri ...
> + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> + CategoryInfo : InvalidArgument: (:) [Invoke-AzsRegistrationValidation], ParentContainsErrorRecordExcept
> ion
> + FullyQualifiedErrorId : AmbiguousParameterSet,Invoke-AzsRegistrationValidation
The obvious question is do you have the correct subscriptionID in $SubscriptionID
The doc you reference suggest some logs might be created, is there anything in them that help?
Log location (contains PII):
C:\Users\username\AppData\Local\Temp\AzsReadinessChecker\AzsReadinessChecker.log**
s
Report location (contains PII):
C:\Users\username\AppData\Local\Temp\AzsReadinessChecker\AzsReadinessCheckerReport.json

Convert the Powershell script to Python 3

I'm trying to perform an upload files from Linux to share point using Python. However I tried a lot by googling but nothing help. At last I got a power shell script that is working. So requesting for help to convert the below script to Python 3
Specify tenant admin and site URL
$User = "justin.jacob#spidersoft.in"
$SiteURL = "https://test-my.sharepoint.com/personal/justin_jacob_spidersoftin";
$Folder = "C:\Users\justin.jacob\Desktop\New folder"
$DocLibName = "Documents"
#Add references to SharePoint client assemblies and authenticate to Office 365 site – required for CSOM
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
$Password = ConvertTo-SecureString ‘123#123’ -AsPlainText -Force
#Bind to site collection
$Context = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$Creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($User,$Password)
$Context.Credentials = $Creds
#Retrieve list
$List = $Context.Web.Lists.GetByTitle("$DocLibName")
$Context.Load($List)
$Context.ExecuteQuery()
#Upload file
Foreach ($File in (dir $Folder -File))
{
$FileStream = New-Object IO.FileStream($File.FullName,[System.IO.FileMode]::Open)
$FileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
$FileCreationInfo.Overwrite = $true
$FileCreationInfo.ContentStream = $FileStream
$FileCreationInfo.URL = $File
$Upload = $List.RootFolder.Files.Add($FileCreationInfo)
$Context.Load($Upload)
$Context.ExecuteQuery()
}
I understand you want to upload files to sharepoint, you can take a reference of below code:
import os
from config import config
from shareplum import Site
from shareplum import Office365
from shareplum.site import Version
# get data from configuration
username = config['sp_user']
password = config['sp_password']
authcookie = Office365('https://xxx.sharepoint.com', username=username, password=password).GetCookies()
site = Site('https://xxx.sharepoint.com/sites/abc',version=Version.v365, authcookie=authcookie)
spfolder = site.Folder('Shared Documents/testfolder')
for root, dirs, files in os.walk(r"D:\mytestfolder"):
for file in files:
filepath = os.path.join(root, file)
print(filepath)
# perform the actual upload
with open(filepath, 'rb+') as file_input:
try:
spfolder.upload_file(file_input, file)
except Exception as err:
print("Some error occurred: " + str(err))
The code uses following python library:
https://shareplum.readthedocs.io/en/latest/files.html

Insufficient memory to continue the execution of the program. Creating Xlxs from CSV

I have a script that cycles through a folder and condenses multiple CSVs to one xlsx file with the names of the CSV as worksheets. However, when the script runs as part of a larger script it failes when it refreshes the query.
$Query.Refresh()
On its own the script runs fine, but when added to the larger one it fails. Can anyone advise why this is the case?
Below is the error I get:
Insufficient memory to continue the execution of the program.
At C:\Temp\Scripts\Shares_Complete.psm1:254 char:13
+ $Query.Refresh()
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], OutOfMemoryException
+ FullyQualifiedErrorId : System.OutOfMemoryException
I have tried single csv with the same code and still the same result.
$script:SP = "C:\Temp\Servers\"
$script:TP = "C:\Temp\Servers\Pc.txt"
$script:FSCSV = "C:\Temp\Server_Shares\Server Lists\"
$script:Message1 = "Unknown Hosts"
$script:Message2 = "Unable to connect"
$script:Message3 = "Unknown Errors Occurred"
$script:Txt = ".txt"
$script:OT = ".csv"
$script:FSERROR1 = $FSCSV+$Message1+$OT
$script:FSERROR2 = $FSCSV+$Message2+$OT
$script:FSERROR3 = $FSCSV+$Message2+$OT
$script:ERL3 = $E4 + "Shares_Errors_$Date.txt"
$script:ECL1 = $E4 + "Shares_Exceptions1_$Date.txt"
$script:ERL1 = $E4 + "Shares_Errors1_$Date.txt"
$script:ECL3 = $E4 + "Shares_Exceptions_$Date.txt"
function Excel-Write {
if ($V -eq "1") {
return
}
[System.GC]::Collect()
$RD = $FSCSV + "*.csv"
$CsvDir = $RD
$Ma4 = $FSCSV + "All Server Shares for Domain $CH4"
$csvs = dir -path $CsvDir # Collects all the .csv's from the driectory
$FSh = $csvs | Select-Object -First 1
$FSh = ($FSh -Split "\\")[4]
$FSh = $FSh -replace ".{5}$"
$FSh
$outputxls = "$Ma4.xlsx"
$script:Excel = New-Object -ComObject Excel.Application
$Excel.DisplayAlerts = $false
$workbook = $excel.Workbooks.Add()
# Loops through each CVS, pulling all the data from each one
foreach ($iCsv in $csvs) {
$script:iCsv
$WN = ($iCsv -Split "\\")[-1]
$WN = $WN -replace ".{4}$"
if ($WN.Length -gt 30) {
$WN = $WN.Substring(0, [Math]::Min($WN.Length, 20))
}
$Excel = New-Object -ComObject Excel.Application
$Excel.DisplayAlerts = $false
$Worksheet = $workbook.Worksheets.Add()
$Worksheet.Name = $WN
$TxtConnector = ("TEXT;" + $iCsv)
$Connector = $worksheet.Querytables.Add($txtconnector,$worksheet.Range("A1"))
$query = $Worksheet.QueryTables.Item($Connector.Name)
$query.TextfileOtherDelimiter = $Excel.Application.International(5)
$Query.TextfileParseType = 1
$Query.TextFileColumnDataTypes = ,2 * $worksheet.Cells.Column.Count
$query.AdjustColumnWidth = 1
$Query.Refresh()
$Query.Delete()
$Worksheet.Cells.EntireColumn.AutoFit()
$Worksheet.Rows.Item(1).Font.Bold = $true
$Worksheet.Rows.Item(1).HorizontalAlignment = -4108
$Worksheet.Rows.Item(1).Font.Underline = $true
$Workbook.Save()
}
$Empty = $workbook.Worksheets.Item("Sheet1")
$Empty.Delete()
$Workbook.SaveAs($outputxls,51)
$Workbook.Close()
$Excel.Quit()
$ObjForm.Close()
Delete
}
Should continue script and create the xlsx.
Looking at your script, it doesn't surprise me you eventually run out of memory, because you are continouisly creating Com objects and never release them from memory.
Whenever you have created Com objects and finished with them, use these lines to free up the memory:
$Excel.Quit()
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($workbook) | Out-Null
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($Excel) | Out-Null
[System.GC]::Collect()
[System.GC]::WaitForPendingFinalizers()
Also, take a good look at the code.
You are creating a $Script:Excel = New-Object -ComObject excel.application object before the foreach loop but you don't use that. Instead, you are creating new Excel and workbook objects inside the loop over and over again where there is absolutely no reason for it since you can re-use the one you created before the loop.
As an aside: The following characters are not allowed in excel worksheet names
\/?*[]:
Length limitation is 31 characters.
EDIT
I had a look at your project and especially the Shares_Complete.psm1 file.
Although I'm not willing of course to rewrite your entire project, I do have some remarks that may help you:
[System.Net.Dns]::GetHostByName() is obsolete. Use GetHostEntry()
when done with a Windows form, use $ObjForm.Dispose() to clear it from memory
you do a lot of [System.GC]::Collect(); [System.GC]::WaitForPendingFinalizers() for no reason
Why not use [System.Windows.MessageBox]::Show() instead of using a Com object $a = new-object -comobject wscript.shell. Again you leave that object lingering in memory..
use Join-Path cmdlet instead of $RD = $FSCSV + "*.csv" or $Cop = $FSCSV + "*.csv" constructs
remove invalid characters from Excel worksheet names (replace '[\\/?*:[\]]', '')
use Verb-Noun names for your functions so it becomes clear what they do. Now you have functions like Location, Delete and File that don't mean anything
you are calling functions before they are defined like in line 65 where you call function Shares. At that point it does not exist yet because the function itself is written in line 69
add [System.Runtime.Interopservices.Marshal]::ReleaseComObject($worksheet) | Out-Null in function Excel-Write
there is no need to use the variable $Excel in script scope ($Script:Excel = New-Object -ComObject excel.application) where it is used only locally to the function.
you may need to look at Excel specifications and limits
fix your indentation of code so it is clear when a loop or if starts and ends
I would recommend using variable names with more meaning. For an outsider or even yourself after a couple of months two-letter variable names become confusing

Calculate specific Date in powershell

I'm a beginner in powershell
$test = ("=7*" + $newWorkSheet.Cells.Item($indexDate,9).Text + "+DATE(" + $newWorkSheet.Cells.Item($indexDate,8).Text + ";1;3)-WEEKDAY(DATE(" + $newWorkSheet.Cells.Item($indexDate,8).Text + ";1;3))-2")
$newWorkSheet.Cells.Item($indexDate,1) = $test
Result to an incomprehensive error : Exception de HRESULT : 0x800A03EC
this is what I want to put in my cell :
echo $test
=7*40+DATE(2014;1;3)-WEEKDAY(DATE(2014;1;3))-2
How can i resolve it ? Thx ! and sorry for my poor english :'(
Edit
I did :
$test = '=7*' + $newWorkSheet.Cells.Item($indexDate,9).Text + '+DATE(' + $newWorkSheet.Cells.Item($indexDate,8).Text + ';1;3)-WEEKDAY(DATE(' + $newWorkSheet.Cells.Item($indexDate,8).Text + ';1;3))-2'
if I echo $test I got what I want :
=7*40+DATE(2014;1;3)-WEEKDAY(DATE(2014;1;3))-2
but when i put $test into my cell like this :
$newWorkSheet.Cells.Item($indexDate,1) = $test
Result an error :
Exception de HRESULT : 0x800A03EC
Au caractère D:\Users\sadm\Documents\salesforce_1.ps1:839 : 1
+ $newWorkSheet.Cells.Item($indexDate,1) = $tmp1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException
I search to display a specific date with a number of week and a year.
Number of week and year are containing in cells of an excel.
$newWorkSheet.Cells.Item($indexDate,9) = $idsem ##Here is my number of week
$newWorkSheet.Cells.Item($indexDate,8) = $tempDate3.ToString("yyyy")
$test = "=" + "7*" + $newWorkSheet.Cells.Item($indexDate,9).Text + '+DATE(' + $newWorkSheet.Cells.Item($indexDate,8).Text + ';1;3)-JOURSEM(DATE(' + $newWorkSheet.Cells.Item($indexDate,8).Text + ';1;3))-2'
$newWorkSheet.Cells.Item($indexDate,1) = $test
$newWorkSheet.Cells.Item($indexDate,1).NumberFormat = "jj/mm/aaaa"
$global:indexDate++
When I launch the script with powershell 5.0 I had this error :
Exception de HRESULT : 0x800A03EC
Au caractère D:\Users\sadm\Documents\salesforce_1.ps1:831 : 1
+ $newWorkSheet.Cells.Item($indexDate,1) = $test
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException
with 2.0 that works...

Resources