Reading and writing from a Postgres database in Powershell for Linux - linux

I'm on an Ubuntu server (20.04 LTS) using Powershell for Linux (7.2.1) and am trying to figure out how to read and write from a Postgres 13 database also installed on the same Ubuntu server.
Can anyone post an example script showing how this is done? Most scripts assume Windows is involved and it is not. I'm converting a Windows Powershell script that used to access MS SQL Server to a new home on Ubuntu with Postgres. I hope to reuse the same Powershell logic instead of rewriting the script in Bash.
Thanks in advance!

Here's an ODBC example I use on Rhel 7, pwsh v7.2.1, without any extra modules except for the postgresql-odbc driver:
$conn = New-Object System.Data.Odbc.OdbcConnection
$conn.ConnectionString= "Driver={PostgreSQL};Server=SERVERNAME;Port=1234;Database=DBNAME;Uid=USERNAME;Pwd=PASS;"
$conn.open()
$query = 'SELECT * FROM pg_catalog.pg_tables'
$command = New-Object System.Data.Odbc.OdbcCommand($query,$conn)
# For commands/queries that return rows
$reader = $command.ExecuteReader()
# Optionally, convert reader output to DataTable object for viewing
# I disable constraints on my result table for this query. Otherwise returns "Failed to enable constraints"
$DataSet = New-Object Data.Dataset -Property #{ EnforceConstraints = $false }
$DataTable = New-Object Data.DataTable
$DataSet.Tables.Add($DataTable)
$DataTable.Load($reader,[Data.LoadOption]::OverwriteChanges)
# For Write/Inserts:
$command.ExecuteNonQuery() ## for commands that don't return data e.g. Update
# Cleanup:
$command.Dispose() ## OdbcCommand objects cannot be re-used, so it helps to close them after execution
$conn.Close() ## Connections can be reused, but close it before ending your script.
# displaying results
$DataTable|ft
schemaname tablename tableowner tablespace hasindexes hasrules hastriggers rowsecurity
---------- --------- ---------- ---------- ---------- -------- ----------- -----------
public sys_db_changelog admin01 1 0 0 0
public sys_server admin01 1 0 0 0
public sys_config admin01 1 0 0 0

Related

Terraform windows vs linux issue

So this issue is a bit convoluted but I need this for a very specific case in azure. I'm trying to create an APIM subnet inside an azure k8s vnet, but I haven't been able to find a return value from the k8s terraform call that gives me the ID/name for the vnet. Instead I used a powershell command to query Azure and get the name of the new vnet. I was working on this code locally on my windows box and it works fine:
data "external" "cluster_vnet_name" {
program = [var.runPSVer6 == true ? "pwsh" : "powershell","Select-AzSubscription '${var.subscriptionName}' |out-null; Get-AzVirtualNetwork -ResourceGroupName '${module.kubernetes-service.k8ResourceGroup}' | Select Name | convertto-json}"]
depends_on = [module.kubernetes-service]
}
I have a toogle in my variables for runpsver6 so when I run on a linux machine it will change powershell to pwsh. Now, this is were is starts getting a little weird. When I run this on my windows machine, its not an issue, however when I run this from a linux box I get the following error:
can't find external program "pwsh"
I have tried a number of different work arounds (such as using the full powershell snapin path /snap/bin/powershell and putting the commands in a .ps1 file) to no avail. Every single time it throws the error that it can't find pwsh as an external program.
I use this same runPSVer6 toggle for local-exec terraform commands with no issue, but I need the name of the Vnet as a response.
Anyone have any ideas what I'm missing here?
ADDED AFTER SEPT 30th
So I tried the alternative way of firing commands:
variable "runPSVer6" {
type = bool
default = true
}
variable "subscriptionName" {
type = string
}
variable "ResourceGroup" {
type = string
}
data "external" "runpwsh" {
program = [var.runPSVer6 == true ? "pwsh" : "powershell", "test.ps1"]
query = {
subscriptionName = var.subscriptionName
ResourceGroup = var.ResourceGroup
}
}
output "vnet" {
value = data.external.runpwsh.result.name
}
and this appears to allow the command to execute, however its not pulling back the result of the json response (even when I confirmed that I do get a response).
This is what I'm using for my .ps1:
Param($subscriptionName,$ResourceGroup)
$subscription = Select-AzSubscription $subscriptionName
$name = (Get-AzVirtualNetwork -ResourceGroupName $ResourceGroup | Select Name).Name
Write-Output "{`n`t""name"":""$name""`n}"
When i don't use the .name in the out, this is what I get:
data.external.runpwsh: Refreshing state...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
vnet = { "name" = "" }
And this is the output from the .ps1:
{
"name":"vnettest"
}
Can you check if pwsh is working in the terminal. It should bring up the powershell prompt...
The path of pwsh must be added to the PATH.. /usr/bin is in my PATH as you can see below.
ubuntu#myhost:~$ whereis pwsh
pwsh: /usr/bin/pwsh
ubuntu#myhost:~$
ubuntu#myhost:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
ubuntu#myhost:~$ pwsh
PowerShell 7.0.3
Copyright (c) Microsoft Corporation. All rights reserved.
https://aka.ms/powershell
Type 'help' to get help.
PS /home/ubuntu>
PS /home/ubuntu> exit
ubuntu#myhost:~$
============
Added later after 29 Sep 2020.
I tried again in Ubuntu 20.
Terraform 13 was downloaded as Zip from main site
PowerShell 7.0.3 installed with snap install powershell --classic
I tried the below test code which worked.
varriable runPSVer6 {}
default = true
}
data "external" "testps" {
program = [var.runPSVer6 == true ? "pwsh" : "powershell","/tmp/testScript.ps1"]
}
output "ps_out" {
value = data.external.testps.result
}
The output was like...
Outputs:
ps_out = {
"name" = "My Resource Name"
"region" = "West Europe"
}
/tmp/testScript.ps1 code was simple output statement...
Write-Output '{ "name" : "My Resource Name", "region" : "West Europe" }'
I tried to null out the path variable just to see if i get the error message you mentioned. I did, as expected..
ubuntu#ip-172-31-53-128:~$ ./terraform apply
data.external.test_ps: Refreshing state...
Error: can't find external program "pwsh"
on main.tf line 5, in data "external" "test_ps":
5: data "external" "test_ps" {
but when i used the full path, it worked again. (even /snap/bin/powershell works)
program = [var.runPSVer6 == true ? "/snap/bin/pwsh" : "powershell","/tmp/testScript.ps1"]
I ealier wrognly blamed snap with my issue, but snap did work now.
This does not give any clue here or pin-point the issue you are having. But may be you try a couple of things just to be sure,
1.) issue "pwsh" in the current directory and see that Powershell prompt does come up.. not sure if you already checked this, but sometime some other characters in path could cause an issue.
2.) can you run tf once after exporting PATH=/snap/bin ... (do it inside a shell and exit later so that you back to old path. or. export the correct path later after test)
3.) If you used full path, the error message must have been different other "Error: can't find external program "pwsh" ... can you cross check if there was diff error msg
this is how the pwsh bin and the sym link looks like in my machine...
ubuntu#ip-172-31-53-128:~$ /usr/bin/ls -lt /snap/bin/pwsh
lrwxrwxrwx 1 root root 10 Sep 29 15:40 /snap/bin/pwsh -> powershell
ubuntu#ip-172-31-53-128:~$
ubuntu#ip-172-31-53-128:~$ /usr/bin/ls -lt /snap/bin/powershell
lrwxrwxrwx 1 root root 13 Sep 29 15:40 /snap/bin/powershell -> /usr/bin/snap
ubuntu#ip-172-31-53-128:~$

How to resolve null-valued expression error

I have a PowerShell script (copied from Net) that would be executed using SQL Server Agent Job. This script will read an excel and create a CSV file from it.
I do not have excel installed on the machine running this script hence using this method.
Code is :
$strFileName = "C:\xxxx\yyyy.xlsx"
$strSheetName = 'MasterCalendar$'
$strProvider = "Provider=Microsoft.ACE.OLEDB.12.0"
$strDataSource = "Data Source = $strFileName"
$strExtend = "Extended Properties='Excel 8.0;HDR=Yes;IMEX=1';"
$strQuery = "Select [Machine_ID],[1469] from [$strSheetName]"
$objConn = New-Object System.Data.OleDb.OleDbConnection("$strProvider;$strDataSource;$strExtend")
$sqlCommand = New-Object System.Data.OleDb.OleDbCommand($strQuery)
$sqlCommand.Connection = $objConn
$objConn.open()
$da = New-Object system.Data.OleDb.OleDbDataAdapter($sqlCommand)
$dt = New-Object system.Data.datatable
[void]$da.fill($dt)
$dataReader.close()
$objConn.close()
$dt
$dt | Export-Csv C:\xxxx\data.csv -NoTypeInformation
When executed through SQL server agent it is throwing below error :
Message Executed as user: Test\User1. A job step received
an error at line 17 in a PowerShell script. The corresponding line is
'$dataReader.close() '. Correct the script and reschedule the job.
The error information returned by PowerShell is: 'You cannot call a
method on a null-valued expression. '. Process Exit Code -1. The
step failed.
I am new to scripting and PowerShell, unable to understand the issue as this exact code is running fine on my machine but starts throwing error when executed through SQL server agent job.

Powershell - String Manipulation - getting dir from full file-path

Keep in mind I'm new to this and be gentle.
I have a full file path for a document "C:\folder1\folder2\01.03.2017 - FileName.csv" and I want to manipulate it to return the dir that the file is stored in (C:\folder1\folder2), minus the filename (01.03.2017 - FileName.csv).
I'm trying to make this modular so that it will work regardless of the amount of sub-folders a file sits in; we also won't know the FileName in advance, so again this needs to be modular and remove up to and including the last "\"
For background info on how this is currently built, I nicked a bit of code from a previous question I saw on StackOverflow:
Function Get-FileName($initialDirectory)
{
[System.Reflection.Assembly]::LoadWithPartialName(“System.windows.forms”) |
Out-Null
$OpenFileDialog = New-Object System.Windows.Forms.OpenFileDialog
$OpenFileDialog.initialDirectory = $initialDirectory
$OpenFileDialog.filter = “All files (*.*)| *.*”
$OpenFileDialog.ShowDialog() | Out-Null
$OpenFileDialog.filename
} #end function Get-FileName
# *** Entry Point to Script ***
$originalData = Get-FileName -initialDirectory “c:\” | Out-String
Write-Host $originalData
$originalDir = $originalData.Split('\')
$originalDir
Running this currently prompts for an "open dialog box" you would see in Windows. You select a folder and the output is currently:
C:\folder1\folder2\01.03.2017 - FileName.csv
C:
folder1
folder2
01.03.2017 - FileName.csv
I've tried a few different -join attempts but none successful.
We will have an input of C:\folder1\folder2\01.03.2017 - FileName.csv as a variable $originalData.
We want the output to be C:\folder1\folder2 as a variable $originalDir.
Function Get-FileName($initialDirectory)
{
[System.Reflection.Assembly]::LoadWithPartialName(“System.windows.forms”) |
Out-Null
$OpenFileDialog = New-Object System.Windows.Forms.OpenFileDialog
$OpenFileDialog.initialDirectory = $initialDirectory
$OpenFileDialog.filter = “All files (*.*)| *.*”
$OpenFileDialog.ShowDialog() | Out-Null
$OpenFileDialog.filename
} #end function Get-FileName
$originalData = Get-FileName -initialDirectory “c:\”
Write-Host $originalData
$originalDir = (Get-ChildItem $originalData).DirectoryName
you can use it like so, so take result or your function and use it with get-childitem.
edit: notice there's no | Out-String on the third to last line

Windows Command Line Equivalent to "time" in Linux? [duplicate]

This question already has answers here:
How do I measure execution time of a command on the Windows command line?
(32 answers)
Equivalent of Unix time command in PowerShell?
(4 answers)
Closed 8 years ago.
I've got what might be a dumb question but I can't seem to find the answer anywhere online. In linux based systems, in the terminal typing "time" before any command gives how long the command takes in terms of real, user, and system time. For example, typing
time ls
lists the files and folders in the current directory then gives the amount of real, user, and system time that it took to list the files and folders. Is there a windows equivalent to this? I am trying to compare the performance of different algorithms but don't have a linux machine to work on so I was hoping that there was a similar command in Windows.
The following is far from perfect. But it's the closest I could come up with to simulate UNIX time behavior. I'm sure it can be improved a lot.
Basically I'm creating a cmdlet that receives a script block, generates a process and uses GetProcessTimes to get Kernel, User and Elapsed times.
Once the cmdlet is loaded, just invoke it with
Measure-Time -Command {your-command} [-silent]
The -Silent switch means no output generated from the command (I.e you are interested only in the time measures)
So for example:
Measure-Time -Command {Get-Process;sleep -Seconds 5} -Silent
The output generated:
Kernel time : 0.6084039
User time : 0.6864044
Elapsed : 00:00:06.6144000
Here is the cmdlet:
Add-Type -TypeDefinition #"
using System;
using System.Runtime.InteropServices;
public class ProcessTime
{
[DllImport("kernel32.dll", CharSet = CharSet.Unicode)]
public static extern bool GetProcessTimes(IntPtr handle,
out IntPtr creation,
out IntPtr exit,
out IntPtr kernel,
out IntPtr user);
}
"#
function Measure-Time
{
[CmdletBinding()]
param ([scriptblock] $Command,
[switch] $Silent = $false
)
begin
{
$creation = 0
$exit = 0
$kernel = 0
$user = 0
$psi = new-object diagnostics.ProcessStartInfo
$psi.CreateNoWindow = $true
$psi.RedirectStandardOutput = $true
$psi.FileName = "powershell.exe"
$psi.Arguments = "-command $Command"
$psi.UseShellExecute = $false
}
process
{
$proc = [diagnostics.process]::start($psi)
$buffer = $proc.StandardOutput.ReadToEnd()
if (!$Silent)
{
Write-Output $buffer
}
$proc.WaitForExit()
}
end
{
$ret = [ProcessTime]::GetProcessTimes($proc.handle,
[ref]$creation,
[ref]$exit,
[ref]$kernel,
[ref]$user
)
$kernelTime = [long]$kernel/10000000.0
$userTime = [long]$user/10000000.0
$elapsed = [datetime]::FromFileTime($exit) - [datetime]::FromFileTime($creation)
Write-Output "Kernel time : $kernelTime"
Write-Output "User time : $userTime"
Write-Output "Elapsed : $elapsed"
}
}
I found a similar question on SuperUser which covers some alternatives. First and foremost being my suggestion to use Measure-Command in PowerShell.
Measure-Command {ls}
Got the syntax wrong in my comment.

Powershell Script loop output to txt file

I am using a Powershell script to write to a text file. A client showed me this Powershell script to use to replace a excel macro I used to use...
$computers= gc "C:\Users\Powershell\DeviceList.txt"
foreach ($computername in $computers)
{
write-output "<$computername>
active = yes
group =
interval = 5min
name = $computername
host = $computername
community =
version = 1
timeout = 0
retries = default
port = 161
qos_source = 1
</$computername>" | Out-File -filepath "C:\Users\Powershell\Cisco_Mon.txt" -append
}
It works great but now I wanted to build on it to add additional variables. In a perfect world I would like it to read from an excel spreadsheed grabbing each rowof data and each column being defined as a variable. For now using another text file is fine as well. Here is what I started with (it doesnt work) but you can see where I am going with it...
$computers= gc "C:\Users\Powershell\devicelist.txt"
$groups= gc "C:\Users\Powershell\grouplist.txt"
foreach ($computername in $computers) + ($groupname in $groups)
{
write-output "<$computername>
active = yes
group = $groupname
interval = 5min
name = $computername
host = $computername
community =
version = 1
timeout = 0
retries = default
port = 161
qos_source = 1
</$computername>" | Out-File -filepath "C:\Users\Powershell\Cisco_Mon.txt" -append
}
Of course it is not working. Essentially I would LOVE it if I could define each of the above options into a variable from an excel spreadsheet, such as $community, $interval, $active, etc.
Any help with this would be very much appreaciated. If someone could show me how to use an excel spreadsheet, have each column defined as a variable, and write the above text with the variables, that would be GREAT!!!.
Thanks,
smt1228#gmail.com
An Example of this would be the following...
Excel Data: (Colums seperated with "|"
IP | String | Group
10.1.2.3 | Public | Payless
Desired Output:
<10.1.2.3>
active = yes
group = Payless
interval = 5min
name = 10.1.2.3
host = 10.1.2.3
community =
version = 1
timeout = 0
retries = default
port = 161
qos_source = 1
</10.1.2.3>
Addition:
Pulling data from CSV for IP, String, Group where data is as follows in CSV...
10.1.2.3,public,group1
10.2.2.3,default,group2
10.3.2.3,public,group3
10.4.2.3,default,group4
to be writting into a .txt file as
IP = 10.1.2.3.
String = public
Group = Group1
and look for each line in the CSV
Ok, new answer now. The easiest way would be to save your Excel document as CSV so that it looks like this (i.e. very similar to how you presented your data above):
IP,String,Group
10.1.2.3,Public,Payless
You can still open that in Excel, too (and you avoid having to use the Office interop to try parsing out the values).
PowerShell can parse CSV just fine: with Import-Csv:
Import-Csv grouplist.csv | ForEach-Object {
"<{0}>
active = yes
group = {1}
interval = 5min
name = 10.1.2.3
host = 10.1.2.3
community =
version = 1
timeout = 0
retries = default
port = 161
qos_source = 1
</{0}>" -f $_.IP, $_.Group
}
I'm using a format string here where {0}, etc. are placeholders. -f then is the format operator which takes a format string on the left and arguments for the placeholders on the right. You can also see that you can access the individual columns by their name, thanks to Import-Csv.

Resources