PowerShell, getting string syntax correct in 7-zip script - string

I can't seem to solve this, been struggling with it for a while (maybe it's simple; I just can't see it as I've been looking at it for so long).
I can get the 7z.exe syntax to work, but when I try and put it together into a simple script, it fails.
e.g., if I run .\zip.ps1 "C:\test\test.zip" "C:\test\test1.txt" "C:\test\test2.txt*
Instead of zipping up the 2 files required, it zips up everything in the C:\test folder, completely ignoring my arguments.
How can I adjust the below string syntax so that 7z.exe will correctly respect the input arguments and compress them from within a PowerShell script?
"Received $($args.Count) files:"
$parent = Split-Path (Split-Path $args[0]) -Leaf
$sevenzip = "C:\Program Files\7-Zip\7z.exe"
$zipname = "$($parent) $(Get-Date -Format 'yyyy_MM_dd HH_mm_ss').zip"
$args_line = $args | foreach-object { "`"$_`"" }
# $args_line = '"' + $($args -join """ """) + '"' # Want to use """ here so that it can capture not only files with spaces, but files with ' in the filename
''
"Zip Name : $zipname"
''
"Arguments : $args_line"
''
if (Test-Path $sevenzip) {
if (Test-Path "C:\0\$zipname") { rm "C:\0\$zipname" -Force }
''
'String output of the line to run:'
"& ""$sevenzip"" a -r -tzip ""C:\0\$zipname"" $args_line" # Taking this output and pasting onto console works.
''
& "$sevenzip" a -r -tzip "C:\0\$zipname" "$args_line" # This does not work
} else {
"7z.exe was not found at '$sevenzip', please check and try again"
}
The error that I get is:
Files read from disk: 0
Archive size: 22 bytes (1 KiB)
Scan WARNINGS for files and folders:
1 : The system cannot find the file specified.
----------------

Pass $args directly to your & "$sevenzip" call:
& "$sevenzip" a -r -tzip "C:\0\$zipname" $args
This makes PowerShell pass the array elements as individual arguments, automatically enclosing them in "..." if needed (based on whether they contain spaces).
Using arrays as arguments for external programs is in effect an implicit form of array-based splatting; thus, you could alternatively pass #args.
Generally, note that in direct invocation[1] you cannot pass multiple arguments to an external program via a single string; that is, something like "$args_line" cannot be expected to work, because it is passed as a single argument to the target program.
If you want to emulate the resulting part of the command line, for display purposes:
($argListForDisplay = $args.ForEach({ ($_, "`"$_`"")[$_ -match ' '] })) -join ' '
Note:
Each argument is conditionally enclosed in "..." - namely based on whether it contains at least one space - and the resulting tokens are joined to form a single, space-separated list (string).
The assumption is that no argument has embedded " chars.
A simplified example:
& { # an ad-hoc script block that functions like a script or function
# Construct the argument list *for display*
$argListForDisplay = $args.ForEach({ ($_, "`"$_`"")[$_ -match ' '] }) -join ' '
#"
The following is the equivalent of:
Write-Output $argListForDisplay
"#
# Pass $args *directly*
# Simply prints the argument received one by one, each on its own line.
# Note: With PowerShell-native commands, generally use #args instead (see below).
Write-Output $args
} firstArg 'another Arg' lastArg # sample pass-through arguments
Output:
The following is the equivalent of:
Write-Output firstArg "another Arg" lastArg
firstArg
another Arg
lastArg
Note: Write-Output is used in lieu of an external program for convenience. Technically, you'd have to use #args instead of $args in order to pass the array elements as individual, positional arguments, but, as stated, this is the default behavior with externals programs. Write-Output, as a PowerShell-native command, receives the array as a whole, as a single argument when $args is used; it just so happens to process that array the same way as if its elements had been passed as individual arguments.
[1] You can use a single string as an -ArgumentList value for Start-Process. However, Start-Process is usually the wrong tool for invoking console applications such as 7z.exe - see this answer.

Related

How to extract key value pairs from a file when values span multiple lines?

I'm a few weeks into bash scripting and I haven't advanced enough yet to get my head wrapped around this problem. Any help would be appreciated!
I have a "script.conf" file that contains the following:
key1=value1
key2=${HOME}/Folder
key3=( "k3v1" "k3 v2" "k3v3")
key4=( "k4v1"
"k4 v2"
"k4v3"
)
key5=value5
#key6="Do Not Include Me"
In a bash script, I want to read the contents of this script.conf file into an array. I've learned how to handle the scenarios for keys 1, 2, 3, and 5, but the key4 scenario throws a wrench into it with it spanning across multiple lines.
I've been exploring the use of sed -n '/=\s*[(]/,/[)]/{/' which does capture key4 and its value, but I can't figure out how to mix this so that the other keys are also captured in the matches. The range syntax is also new to me, so I haven't figured out how to separate the key/value. I feel like there is an easy regex that would accomplish what I want... in plain-text: "find and group the pattern ^(.*)= (for the key), then group everything after the '=' char until another ^(.*)= match is found, rinse and repeat". I guess if I do this, I need to change the while read line to not handle the key/value separation for me (I'll be looking into this while I'm waiting for a response). BTW, I think a solution where the value of key4 is flattened (new lines removed) would be acceptable; I know for key3 I have to store the value as a string and then convert it to an array later when I want to iterate over it since an array element apparently can't contain a list.
Am I on the right path with sed or is this a job for awk or some other tool? (I haven't ventured into awk yet). Is there an easier approach that I'm missing because I'm too deep into the forest (like changing the while read line in the LoadConfigFile function)?
Here is the code that I have so far in script.sh for processing and capturing the other pairs into the $config array:
__AppDir=$(dirname $0)
__AppName=${__ScriptName%.*}
typeset -A config #init config array
config=( #Setting Default Config values
[key1]="defaultValue1"
[key2]="${HOME}/defaultFolder"
[QuietMode]=0
[Verbose]=0 #Ex. Usage: [[ "${config[Verbose]}" -gt 0 ]] && echo ">>>Debug print"
)
function LoadConfigFile() {
local cfgFile="${1}"
shopt -s extglob #Needed to remove trailing spaces
if [ -f ${cfgFile} ]; then
while IFS='=' read -r key value; do
if [[ "${key:0:1}" == "#" ]]; then
#echo "Skipping Comment line: ${key}"
elif [ "${key:-EMPTY}" != "EMPTY" ]; then
value="${value%%\#*}" # Delete in-line, right comments
value="${value%%*( )}" # Delete trailing spaces
value="${value%%( )*}" # Delete leading spaces
#value="${value%\"*}" # Delete opening string quotes
#value="${value#\"*}" # Delete closing string quotes
#Manipulate any variables included in the value so that they can be expanded correctly
# - value must be stored in the format: "${var1}". `backticks`, "$var2", and "doubleQuotes" are left as is
value="${value//\"/\\\"}" # Escape double quotes for eval
value="${value//\`/\\\`}" # Escape backticks for eval
value="${value//\$/\\\$}" # Escape ALL '$' for eval
value="${value//\\\${/\${}" # Undo the protection of '$' if it was followed by a '{'
value=$(eval "printf '%s\n' \"${value}\"")
config[${key}]=${value} #Store the value into the config array at the specified key
echo " >>>DBG: Key = ${key}, Value = ${value}"
#else
# echo "Skipped Empty Key"
fi
done < "${cfgFile}"
fi
}
CONFIG_FILE=${__AppDir}/${__AppName}.conf
echo "Config File # ${CONFIG_FILE}"
LoadConfigFile ${CONFIG_FILE}
#Print elements of $config
echo "Script Config Values:"
echo "----------------------------"
for key in "${!config[#]}"; do #The '!' char gets an array of the keys, without it, we would get an array of the values
printf " %-20s = %s\n" "${key}" "${config[${key}]}"
done
echo "------ End Script Config ------"
#To convert to an array...
declare -a valAsArray=${config[RequiredAppPackages]} #Convert the value from a string to an array
echo "Count = ${#valAsArray[#]}"
for itemCfg in "${valAsArray[#]}"; do
echo " item = ${itemCfg}"
done
As I mentioned before, I'm just starting to learn bash and Linux scripting in general, so if you see that I'm doing some taboo things in other areas of my code too, please feel free to provide feedback in the comments... I don't want to start bad habits early on :-).
*If it matters, the OS is Ubuntu 14.04.
EDIT:
As requested, after reading the script.conf file, I would like for the elements in $config[#] to be equivalent to the following:
typeset -A config #init config array
config=(
[key1]="value1"
[key2]="${HOME}/Folder"
[key3]="( \"k3v1\" \"k3 v2\" \"k3v3\" )"
[key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )"
[key5]="value5"
)
I want to be able to convert the values of elements 'key4' and 'key3' into an array and iterated over them the same way in the following code:
declare -a keyValAsArray=${config[keyN]} #Convert the value from a string to an array
echo "Count = ${#keyValAsArray[#]}"
for item in "${keyValAsArray[#]}"; do
echo " item = ${item}"
done
I don't think it matters if \n is preserved for key4's value or not... that depends on if declare has a problem with it.
A shell is an environment from which to call tools with a language to sequence those calls. It is NOT a tool to manipulate text. The standard UNIX tool to manipulate text is awk. Trying to manipulate text in shell IS a bad habit, see why-is-using-a-shell-loop-to-process-text-considered-bad-pr‌​actice for SOME of the reasons why
You still didn't post the expected result of populating the config array so I'm not sure but I think this is what you wanted:
$ cat tst.sh
declare -A config="( $(awk '
{ gsub(/^[[:space:]]+|([[:space:]]+|#.*)$/,"") }
!NF { next }
/^[^="]+=/ {
name = gensub(/=.*/,"",1)
value = gensub(/^[^=]+=/,"",1)
n2v[name] = value
next
}
{ n2v[name] = n2v[name] OFS $0 }
END {
for (name in n2v) {
value = gensub(/"/,"\\\\&","g",n2v[name])
printf "[%s]=\"%s\"\n", name, value
}
}
' script.conf
) )"
declare -p config
$ ./tst.sh
declare -A config='([key5]="value5" [key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )" [key3]="( \"k3v1\" \"k3 v2\" \"k3v3\")" [key2]="/home/Ed/Folder" [key1]="value1" )'
The above uses GNU awk for gensub(), with other awks you'd use [g]sub() instead.

Why file path now getting concatenate in variable

I am writing powershell script to find few string in a path. It returns file path correctly in variable. Now if I try to add it to some other variable it adds empty (adds nothing), but if I print variable separately it gets print correctly.
My code,
$final = ""
foreach($e in $stringIDColum)
{
$e
$var = (Get-ChildItem “C:\path” -recurse -exclude $excluded | Select-String -pattern $e | group path | select name)
$final += "," + $e + "," + $var
}
Here value of $e is getting added, $var is not getting added. But if I print $var separately it prints path correctly.
Secondly I have tried various ways, to print line number, but it does not print line number in front of path.
I think the issue is that $var is potentially an array. So you need to convert it a string. Check the type once it's created.

How do I delay expansion of variables in PowerShell strings?

Whatever you want to call it, I'm trying to figure out a way to take the contents of an existing string and evaluate them as a double-quoted string. For example, if I create the following strings:
$string = 'The $animal says "meow"'
$animal = 'cat'
Then, Write-Host $string would produce The $animal says "meow". How can I have $string re-evaluated, to output (or assign to a new variable) The cat says "meow"?
How annoying...the limitations on comments makes it very difficult (if it's even possible) to include code with backticks. Here's an unmangled version of the last two comments I made in response to zdan below:
----------
Actually, after thinking about it, I realized that it's not reasonable to expect The $animal says "meow" to be interpolated without escaping the double quotes, because if it were a double-quoted string to begin with, the evaluation would break if the double quotes weren't escaped. So I suppose the answer would be that it's a two step process:
$newstring = $string -replace '"', '`"'
iex "`"$string`""
One final comment for posterity: I experimented with ways of getting that all on one line, and almost anything that you'd think works breaks once you feed it to iex, but this one works:
iex ('"' + ($string -replace '"', '`"') + '"')
Probably the simplest way is
$ExecutionContext.InvokeCommand.ExpandString($var)
You could use Invoke-Expression to have your string reparsed - something like this:
$string = 'The $animal says `"meow`"'
$animal = 'cat'
Invoke-Expression "Write-Host `"$string`""
Note how you have to escape the double quotes (using a backtick) inside your string to avoid confusing the parser. This includes any double quotes in the original string.
Also note that the first command should be a command, if you need to use the resulting string, just pipe the output using write-output and assign that to a variable you can use later:
$result = Invoke-Expression "write-output `"$string`""
As noted in your comments, if you can't modify the creation of the string to escape the double quotes, you will have to do this yourself. You can also wrap this in a function to make it look a little clearer:
function Invoke-String($str) {
$escapedString = $str -replace '"', '`"'
Invoke-Expression "Write-Output `"$escapedString`""
}
So now it would look like this:
# ~> $string = 'The $animal says "meow"'
# ~> $animal = 'cat'
# ~> Invoke-String $string
The cat says "meow"
You can use the -f operator. This is the same as calling [String]::Format as far as I can determine.
PS C:\> $string = 'The {0} says "meow"'
PS C:\> $animal = 'cat'
PS C:\> Write-Host ($string -f $animal)
The cat says "meow"
This avoids the pitfalls associated with quote stripping (faced by ExpandString and Invoke-Expression) and arbitrary code execution (faced by Invoke-Expression).
I've tested that it is supported in version 2 and up; I am not completely certain it's present in PowerShell 1.
Edit: It turns out that string interpolation behavior is different depending on the version of PowerShell. I wrote a better version of the xs (Expand-String) cmdlet with unit tests to deal with that behavior over here on GitHub.
This solution is inspired by this answer about shortening calls to object methods while retaining context. You can put the following function in a utility module somewhere, and it still works when you call it from another module:
function xs
{
[CmdletBinding()]
param
(
# The string containing variables that will be expanded.
[parameter(ValueFromPipeline=$true,
Position=0,
Mandatory=$true)]
[string]
$String
)
process
{
$escapedString = $String -replace '"','`"'
$code = "`$ExecutionContext.InvokeCommand.ExpandString(`"$escapedString`")"
[scriptblock]::create($code)
}
}
Then when you need to do delayed variable expansion, you use it like this:
$MyString = 'The $animal says $sound.'
...
$animal = 'fox'
...
$sound = 'simper'
&($MyString | xs)
&(xs $MyString)
PS> The fox says simper.
PS> The fox says simper.
$animal and $sound aren't expanded until the last two lines. This allows you to set up a $MyString up front and delay expansion until the variables have the values you want.
Invoke-Expression "`"$string`""

Powershell replace function strange behavior

a simple example and i don't know how to get it to work...
function replace($rep, $by){
Process { $_ -replace $rep, $by }
}
when I do
"test" | replace("test", "foo")
the result is
test
When I do
function replace(){
Process { $_ -replace "test", "foo" }
}
"test" | replace()
the result is
foo
any idea ?
Functions in PowerShell follow the same argument rules as cmdlets and native commands, that is, arguments are separated by spaces (and yes, this also means you don't need to quote your arguments, as they are automatically interpreted as strings in that parsing mode):
'test' | replace test foo
So if you call a PowerShell function or cmdlet with arguments in parentheses you'll get a single argument that is an array within the function. Invocations of methods on objects follow other rules (that look roughly like in C#).
To elaborate a little: PowerShell has two different modes in which it parses a line: expression mode and command mode. In expression mode PowerShell behaves like a REPL. You can type 1+1 and get 2 back, or type 'foo' -replace 'o' and get f back. Command mode is for mimicking a shell's behaviour. That's when you want to run command, e.g. Get-ChildItem or & 'C:\Program Files\Foo\foo.exe' bar blah. Within parentheses mode determination starts anew which is why Write-Host (Get-ChildItem) is different from Write-Host Get-ChildItem.
Remove the () in your function call and remove comma.
"test" | replace "test" "foo"

Why does Powershell replace commas with whitespaces?

I'm doing some script in Powershell to automate a task. This script is going to get arguments, such as:
PS > myscript.ps1 par=a,1,2,0.1 par=b,3,4,0.1 par=c,5,6,0.1 bin=file.exe exeargs="fixed args for file.exe"
In short, file.exe is an executable which accept parameters (including a, b and c) and this ps1 script is going to execute file.exe passing args a, b and c within the specified range, varying 'em by the specified precision.
The question is, I first split each $arg in $args by the character "=", and then I should split them by "," to get the specified values.
The thing is, when I do:
foreach ($arg in $args)
{
$parts = ([string] $arg).split("=")
Write-Host $parts[1]
}
The output is
a 1 2 0.1
b 3 4 0.1
c 5 6 0.1
file.exe
fixed args for file.exe
I.e., it already substituted the "," character with a whitespace, so my second split should be with white space, not with comma.
Any guess on why does it happen?
Thanks in advance!
First of all why are you writing it like a C program or something? You don't have to pass arguments like that, use $args and split on = etc. when Powershell has a more powerful concept of parameters, whereby you can pass the named paramters and arguments rather than doing the parsing that you are doing. ( More on parameters here: http://technet.microsoft.com/en-us/library/dd315296.aspx)
With that said, let me answer your question:
What you are doing is when you pass in arguments like:
par=a,1,2,0.1 par=b,3,4,0.1 par=c,5,6,0.1 bin=file.exe exeargs="fixed args for file.exe"
you are passing in array of arrays. The first element is the array with elements:
par=a
1
2
0.1
Ok coming to the split:
When you do [string] $a, you are converting the array into a string. By default this means an array with elements 1,2,3 will become 1 2 3.
So your first argument there par=a,1,2,0.1, becomes par=a 1 2 0.1 and the split on = means parts[1] becomes a 1 2 0.1, which is what you see.
So how can you retain the comma?
Just make an array to be converted into a string with , inbetween than space, right?
Here are some ways to do that:
Using -join operator:
foreach ($arg in $args)
{
$parts = ($arg -join ",").split("=")
Write-Host $parts[1]
}
now you will see the output with the commas that you want.
Another way, using $ofs special variable:
foreach ($arg in $args)
{
$ofs =","
$parts = ([string] $arg).split("=")
Write-Host $parts[1]
}
(more on $ofs here: http://blogs.msdn.com/b/powershell/archive/2006/07/15/what-is-ofs.aspx )
Disclaimer - All this explanation to make you understand what is happening. Please do not continue this and use paramters.
This is happening in the parsing of the command line for your script and not during the split() method. To see this, try putting a "Write-Host $args" at the beginning, like so:
Write-Host $args
foreach ($arg in $args)
{
$parts = ([string] $arg).split("=")
Write-Host $parts[1]
}
This is because the ',' character is used to separate elements in an array. For example:
PS C:\temp> $a = 1,2,3
PS C:\temp> Write-Host $a
1 2 3
PS C:\temp> $a.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
Try this command line instead:
.\myscript.ps1 "par=a,1,2,0.1" "par=b,3,4,0.1" "par=c,5,6,0.1" "bin=file.exe" "exeargs=fixed args for file.exe"

Resources