Powershell, use (space-separated) string as arguments to a program - string

I've read this and it doesn't solve my problem.
I have a space-separated string, let's say $MyString = "arg1 arg2". Suppose I have a command line program called MyProgram, which accepts an arbitrary number of positional arguments, so it can be run like MyProgram arg1 arg2. However doing MyProgram $MyString doesn't work, and neither does MyProgram ($MyString -split ' ') nor MyProgram $($MyString -split ' '). I get the same error which basically says that it doesn't recognise the argument "arg1 arg2", which I guess is because it still thinks it's one argument containing a space rather than two arguments. In practice, $MyString may be quite huge and is read from a file. How do I make this work?

Oh I just found out how LOL. I should have thought of this sooner; basically, just use splatting The following worked for me:
$MyArray = $($MyString -split " ")
MyProgram #MyArray
Explanation: The first line converts the string into an array of strings split by space (" "); The $(...) notation around a command captures the output of the command, which I then assign to $MyArray. Then, instead of using $MyArray with a dollar sign $, I use it with # to splat the array of strings into arguments for MyProgram.

tl;dr
For calling PowerShell commands you indeed need splatting in order to pass the elements of an array as individual, positional arguments; this requires defining the array in an auxiliary variable that can then be passed with sigil # in lieu of $ to request splatting:
$myArray = -split $myString # See below for limitations, bottom section for fix
MyPowerShellCommand #myArray # Array elements are passed as indiv. arguments.
While this technique also works with external programs, it isn't strictly necessary there, and you can pass an array directly to achieve the same effect:
MyExternalProgram (-split $myString) # Array elements are passed as indiv. args.
Note that (...) rather than $(...) is used to pass the expression as an argument. (...) is usually sufficient and generally preferable, because $(...) can have side effects - see this answer for details.
Just to bring the post you link to in your question and your answer here together:
First, to be clear: neither answer, due to splitting by spaces only will deal properly with arguments inside the argument-list string that have embedded spaces (and therefore, of necessity use embedded quoting), e.g., $myString = "arg1 `"arg2 with spaces`" arg3" would not work as expected - see the bottom section for a solution.
Leaving that aside, the difference is:
When calling an external program, as in the linked post, passing an array causes each element to become its own argument.
That is, myExternalProgram (-split $MyString) would work.
Note that I'm using the unary form of the -split operator for more flexible tokenization, which splits by any non-empty run of whitespace while ignoring leading and trailing whitespace (same as awk's default behavior).
When calling a PowerShell command, as in your case, an array is by default passed as-is, as a whole, as a single argument.
To achieve the same effect as with external programs, i.e. to pass the array's elements as individual, positional arguments, you indeed have to use splatting, i.e. you have to:
save the array in a variable first: $myArray = -split $myString,
which you can then pass as as a splatted argument by using # instead of $ as the sigil: MyPowerShellCommand #myArray
Do note that when calling PowerShell commands it is more common - and more robust - to use hashtable- rather than an array-based splatting, as it allows you to explicitly bind to parameters by name rather than by position - and PowerShell commands often have parameters that can only be bound by name.
E.g., if MyPowerShellCommand accepts parameters -Foo and -Bar, you could use:
$myArgs = #{ Foo='foo value'; Bar='bar value '}; MyPowerShellCommand #myArgs
If you do want to handle argument-list strings that have arguments with embedded quoting:
$myString = 'arg1 "arg2 with spaces" arg3'
$myArray = (Invoke-Expression ('Write-Output -- ' + $myString -replace '\$', "`0")) -replace "`0", '$$'
Note: Invoke-Expression (iex) should generally be avoided, but the extra precautions taken in this particular command make its use safe.
$myArray is then a 3-element array with verbatim elements arg1, arg2 with spaces and arg3, which can again be used as shown above.
See this answer for an explanation of the technique.

These work for me ($args is reserved). -split on the left side splits on whitespace. Or you can get-content from a file where each argument is on a seperate line. You might run into a limit with how long a commandline can be. Piping that list in or loading it from a file might be a better approach.
echo hi > file.txt
$args2 = 'hi','file.txt'
findstr $args2
# hi
$args2 = 'hi','file.txt'
& findstr $args2
# hi
$args2 = 'hi file.txt'
findstr (-split $args2)
# hi
findstr ($args2 -split ' ')
# hi

Related

Evaluate a string read from a file in perl

Made up example (perl):
my $x = read_input_from_file();
# $x now contains string $ENV{SOMETHING}/dir/$ENV{SOMETHING_ELSE}
my $y = eval($x); # doesnt work
How can I get value of string contained in $x in the script?
So far I have tried using eval which doesn't generate any output. I am hoping that something already exists in perl and these string expressions do not need to be parsed and evaluated.
The "string" eval is a little specific:
eval in all its forms is used to execute a little Perl program.
...
In a string eval, the value of the expression (which is itself determined within scalar context) is first parsed, and if there were no errors, executed as a block within the lexical context of the current Perl program.
So this evaluates code, and with a variable to be evaluated containing a string literal we have a "bareword" ("Unquoted string") which is generally no good. In your case, those / in $x cause additional trouble.
If the content of the variable to evaluate is a string literal (not code) it need be quoted
my $y = eval q(") . $x . q("); # double-quote so that it interpolates
I use the operator form of a single quote, q(). Quoted under it is a double-quote since $x itself seems to contain variables that need be evaluated (interpolated).
Keep in mind that running code from external sources can be a serious security problem.

When do values passed to a parameter require quotes?

I have been working with Exchange Online unified groups and Microsoft Teams teams in powershell: New-UnifiedGroup, Set-UnifiedGroup, Get-UnifiedGroup, New-Team, Get-Team etc. I have been creating groups and teams from PowerShell.
I am interested in understanding a bit better when a value for a parameter requires quotes and when it does not.
When I import from CSV, it seems the values are automatically interpreted as strings. When I supply them to a parameter that requires a string, the value does not require quotes even if it has spaces e.g New-UnifiedGroup -DisplayName $item.displayName does not require quotes even when the display name has spaces.
But when I want to create a team from an existing group, and I get the ID of the group, the group id requires quotes: New-Teams -GroupId "$group.ExternalDirectoryObjectId". In this case the parameter -GroupId requires a string value, although the ExternalDirectoryObjectId that it requires is not a string.
Is there a rule that a value does not require quotes if it is a string, and a string value is expected? Does it help to declare a variable as a string before passing it to a parameter that requires a string? For example, if I have a $path variable, I usually have to provide it as -Path "$path". If I declared the path as [String]$path =, would I then not need to use the quotes in -Path $path
Generally, only ever use quoting in PowerShell to explicitly pass a value as a string ([string]).
String literals require quoting if they contain any of the following: spaces or, more generally, PowerShell metacharacters[1], and commands or expressions as part of a larger string (which must then be enclosed in $(...) - see below).
To pass the value of a variable, one of its properties, or even the result of a method call on it, you do not need quoting in PowerShell, which will pass the resulting value with its original data type; however, when the value is bound to its target parameter, PowerShell may automatically convert it to the parameter's type.
If the target parameter is [string]-typed (as is the case with New-Team's -GroupId parameter), PowerShell will automatically convert any non-string argument to a string, essentially by calling .ToString() on it[2]. If the resulting string isn't the right representation, you must perform explicit stringification, by way of an expression or command.
E.g., both -GroupId $groupId and -GroupId $group.ExternalDirectoryObjectId would work - even if the resulting string contains embedded spaces or other PowerShell metacharacters.
If you need to pass an object's property, a method call, or any type of command or expression as part of a larger string, enclose the argument in "..." and use $(...), the subexpression operator around the expression / command (e.g., "$($group.ExternalDirectoryObjectId)/more"; referencing a variable by itself inside "..." does not require $(...)).
"$group.ExternalDirectoryObjectId" definitely does not work as intended, because only variable reference $group by itself is recognized - and stringified - whereas the .ExternalDirectoryObjectId part is treated literally - see first link below for why.
Further reading:
Overview of PowerShell's expandable strings (string interpolation, "...")
String literals in PowerShell (bottom section)
How unquoted tokens are parsed as arguments.
[1] The metacharacters are (some only need quoting if at the start of the argument):
<space> ' " ` , ; ( ) { } | & < > # #
[2] The exact stringification rules, where culture-sensitivity factors in as well, are detailed in this answer.
Generally, PowerShell has a very flexible automatic type-conversion system whose rules are complex and not explicitly documented - a peek at the source code may help.
PowerShell always tries to automatically convert a given value to the target type, where the target type may be dictated by a parameter's type or the (usually) LHS operand of an operator-based expression (e.g., 42 + "1" yields 43).
That's strange. Usually you only need quotes to pass a literal string that has a space in it.
get-childitem -path 'foo 2'
I can pass an object property without quotes usually:
$a = [pscustomobject]#{path = 'foo 2'}
get-childitem -path $a.path
This is more rare, but if a string looks like an array element, I've found I have to quote it:
select-xml -XPath "//*[#a='hi']" -Path file.xml
Even using something like an integer works without quotes, because it can be 'coerced' into a string.

cant create third parameter in bash script (after $*)

I want to create a bash script, that gets 3 parameters. But the second needs to be $*, because i need later these lines. The other two parameters (first and third) doesn't need this.
for x in $* do
The first and second parameter aren't the problem, this one works:
parameter1="$1"
shift
parameter2="$*"
But i need the third parameter at the end and something like this
parameter1="$1"
parameter3="$3"
shift
parameter2="$*"
won't work. My command at the end should look like this:
bash myscript parameter1 parameter2 parameter3
For specifically three parameters, you can use substring parameter expansion in a simple way:
parameter1=$1
parameter2="${#:2:1}" # One parameter, starting with #2
parameter3=$3
Or course, that's unnecessary, since you can just use $2 instead of ${#:2:1}, but I point it out as a simple introduction to the syntax (and not at all because I overlooked the fact you would use $2, really....)
(You can also use it as a substitute for indirect parameter expansion; "${#:n:1}" and "${!n}" are basically equivalent when n is a variable with an integer value.)
For the more general case, where you want an arbitrary number of arguments between the first and last, it gets a little more complicated, although the principle is the same:
parameter1=$1
middleParameters=( "${#:2:$#-2}" ) # n - 2 parameters, starting with #2, i.e., all but $1 and ${!n} for n=$#
lastParameter="${#:$#}"
shift removes an argument from the left. If you want to remove an argument from the right, you can do that with:
set -- "${#:1:$# - 1}"
Thus:
parameter1=$1 # capture leftmost argument
shift # remove leftmost argument
parameter3=${*:$#:1} # capture rightmost argument
set -- "${#:1:$# - 1}" # remove rightmost argument
parameter2=$* # concatenate remaining arguments and store in a string
Note that $* is almost certainly the Wrong Thing. If you want to keep your arguments separate, respecting their quoting, instead use an array:
parameter2=( "$#" )
for item in "${parameter2[#]}"; do
echo "Processing item: $item"
done
If your script is run as yourscript arg1 "item A" "item B" arg3, then the above will ensure that item A and item B are treated as individual arguments, rather than treating item as an argument, A as another, etc.

Finding substring of variable length in bash

I have a string, such as time=1234, and I want to extract just the number after the = sign. However, this number could be in the range of 0 and 100000 (eg. - time=1, time=23, time=99999, etc.).
I've tried things like $(string:5:8}, but this will only work for examples of a certain length.
How do I get the substring of everything after the = sign? I would prefer to do it without outside commands like cut or awk, because I will be running this script on devices that may or may not have that functionality. I know there are examples out there using outside functions, but I am trying to find a solution without the use of such.
s=time=1234
time_int=${s##*=}
echo "The content after the = in $s is $time_int"
This is a parameter expansion matching everything matching *= from the front of the variable -- thus, everything up to and including the last =.
If intending this to be non-greedy (that is, to remove only content up to the first = rather than the last =), use ${s#*=} -- a single # rather than two.
References:
The bash-hackers page on parameter expansion
BashFAQ #100 ("How do I do string manipulations in bash?")
BashFAQ #73 ("How can I use parameter expansion? How can I get substrings? [...])
BashSheet quick-reference, paramater expansion section
if time= part is constant you can remove prefix by using ${str#time=}
Let's say you have str='time=123123' if you execute echo ${str#time=} you would get 123123

bash getopts - difference between ${OPTARG} and $OPTARG [duplicate]

In shell scripts, when do we use {} when expanding variables?
For example, I have seen the following:
var=10 # Declare variable
echo "${var}" # One use of the variable
echo "$var" # Another use of the variable
Is there a significant difference, or is it just style? Is one preferred over the other?
In this particular example, it makes no difference. However, the {} in ${} are useful if you want to expand the variable foo in the string
"${foo}bar"
since "$foobar" would instead expand the variable identified by foobar.
Curly braces are also unconditionally required when:
expanding array elements, as in ${array[42]}
using parameter expansion operations, as in ${filename%.*} (remove extension)
expanding positional parameters beyond 9: "$8 $9 ${10} ${11}"
Doing this everywhere, instead of just in potentially ambiguous cases, can be considered good programming practice. This is both for consistency and to avoid surprises like $foo_$bar.jpg, where it's not visually obvious that the underscore becomes part of the variable name.
Variables are declared and assigned without $ and without {}. You have to use
var=10
to assign. In order to read from the variable (in other words, 'expand' the variable), you must use $.
$var # use the variable
${var} # same as above
${var}bar # expand var, and append "bar" too
$varbar # same as ${varbar}, i.e expand a variable called varbar, if it exists.
This has confused me sometimes - in other languages we refer to the variable in the same way, regardless of whether it's on the left or right of an assignment. But shell-scripting is different, $var=10 doesn't do what you might think it does!
You use {} for grouping. The braces are required to dereference array elements. Example:
dir=(*) # store the contents of the directory into an array
echo "${dir[0]}" # get the first entry.
echo "$dir[0]" # incorrect
You are also able to do some text manipulation inside the braces:
STRING="./folder/subfolder/file.txt"
echo ${STRING} ${STRING%/*/*}
Result:
./folder/subfolder/file.txt ./folder
or
STRING="This is a string"
echo ${STRING// /_}
Result:
This_is_a_string
You are right in "regular variables" are not needed... But it is more helpful for the debugging and to read a script.
Curly braces are always needed for accessing array elements and carrying out brace expansion.
It's good to be not over-cautious and use {} for shell variable expansion even when there is no scope for ambiguity.
For example:
dir=log
prog=foo
path=/var/${dir}/${prog} # excessive use of {}, not needed since / can't be a part of a shell variable name
logfile=${path}/${prog}.log # same as above, . can't be a part of a shell variable name
path_copy=${path} # {} is totally unnecessary
archive=${logfile}_arch # {} is needed since _ can be a part of shell variable name
So, it is better to write the three lines as:
path=/var/$dir/$prog
logfile=$path/$prog.log
path_copy=$path
which is definitely more readable.
Since a variable name can't start with a digit, shell doesn't need {} around numbered variables (like $1, $2 etc.) unless such expansion is followed by a digit. That's too subtle and it does make to explicitly use {} in such contexts:
set app # set $1 to app
fruit=$1le # sets fruit to apple, but confusing
fruit=${1}le # sets fruit to apple, makes the intention clear
See:
Allowed characters in Linux environment variable names
The end of the variable name is usually signified by a space or newline. But what if we don't want a space or newline after printing the variable value? The curly braces tell the shell interpreter where the end of the variable name is.
Classic Example 1) - shell variable without trailing whitespace
TIME=10
# WRONG: no such variable called 'TIMEsecs'
echo "Time taken = $TIMEsecs"
# What we want is $TIME followed by "secs" with no whitespace between the two.
echo "Time taken = ${TIME}secs"
Example 2) Java classpath with versioned jars
# WRONG - no such variable LATESTVERSION_src
CLASSPATH=hibernate-$LATESTVERSION_src.zip:hibernate_$LATEST_VERSION.jar
# RIGHT
CLASSPATH=hibernate-${LATESTVERSION}_src.zip:hibernate_$LATEST_VERSION.jar
(Fred's answer already states this but his example is a bit too abstract)
Following SierraX and Peter's suggestion about text manipulation, curly brackets {} are used to pass a variable to a command, for instance:
Let's say you have a sposi.txt file containing the first line of a well-known Italian novel:
> sposi="somewhere/myfolder/sposi.txt"
> cat $sposi
Ouput: quel ramo del lago di como che volge a mezzogiorno
Now create two variables:
# Search the 2nd word found in the file that "sposi" variable points to
> word=$(cat $sposi | cut -d " " -f 2)
# This variable will replace the word
> new_word="filone"
Now substitute the word variable content with the one of new_word, inside sposi.txt file
> sed -i "s/${word}/${new_word}/g" $sposi
> cat $sposi
Ouput: quel filone del lago di como che volge a mezzogiorno
The word "ramo" has been replaced.

Resources