Given the following script:
const yargs = require('yargs');
const argv =
yargs
.usage('Usage: $0 [--whatIf]')
.alias('d', 'directory')
.alias('wi', 'whatIf')
.nargs('d', 1)
.describe('d', 'alphabetize this directory')
.describe('whatIf', 'show what would happen if run')
.demandOption(['d'])
.argv;
console.log(argv.directory);
If I invoke the script from Windows PowerShell like so: node .\alphabetizer.js -d 'l:\my folder\Files - Some Files In Here\' --whatIf I get the output l:\my folder\Files - Some Files In Here\" --whatIf where I would expect just l:\my folder\Files - Some Files In Here\. It works OK with folder names that require no escaping, but it seems to get confused by the escaping.
If I examine process.argv, I can see the same escaping issue.
I have noticed that if I remove the trailing slash it will work. However, this still points to the node script not handling the input properly, because this should not be necessary with string set off by single quotes.
Is there a way to make this work?
Both Windows PowerShell (powershell.exe) and PowerShell [Core] v6+ (pwsh) are fundamentally broken with respect to quoting arguments for external programs properly - see this answer for background info.
Generally, PowerShell on Windows has to perform re-quoting behind the scenes in order to ensure that just "..."-quoting is used, given that external programs can't be assumed to understand '...'-quoting too when parsing their command line (which on Windows every program has to do itself).
Windows PowerShell is more broken with respect to arguments that end in \ and have embedded spaces, re-quoting them improperly; e.g.:
PS> foo.exe 'c:\foo \' bar
is translated into the following command line behind the scenes:
foo.exe "c:\ foo \" bar
This is broken, in that most applications - including PowerShell's own CLI - sensibly assume that the \" is an escaped " char. to be taken verbatim, thinking that the argument continues with bar and then implicitly ends, despite the formal lack of a closing ".
PowerShell [Core] v6+ more sensibly translates the above to foo.exe "c:\foo \\" bar, where the \\ is interpreted as an escaped \, and the following " again has syntactic function.
If you're stuck with Windows PowerShell, your only choices are:
either: if possible, leave off the trailing \
otherwise: manually double it (\\), but only do so if the argument also contains spaces (otherwise, the \\ will be retained as-is, though in the case of filesystem paths that is usually benign).
Related
In pwsh call the following:
Write-Host '{"drop_attr": "name"}'
Result ok:
{"drop_attr": "name"}
Now do the same via pwsh:
pwsh -Command Write-Host '{"drop_attr": "name"}'
Result is missing quotation marks and square brackets?
drop_attr: name
Update:
PowerShell 7.3.0 mostly fixed the problem, with selective exceptions on Windows, and it seems that in some version after 7.3.1 the fix will require opt-in - see this answer for details.
For cross-version, cross-edition code, the Native module discussed at the bottom may still be of interest.
Unfortunately, PowerShell's handling of passing arguments with embedded " chars. to external programs - which includes PowerShell's own CLI (pwsh) - is fundamentally broken (and always has been), up to at least PowerShell 7.2.x:
You need to manually \-escape " instances embedded in your arguments in order for them to be correctly passed through to external programs (which happens to be PowerShell in this case as well):
# Note: The embedded '' sequences are the normal and expected
# way to escape ' chars. inside a PowerShell '...' string.
# What is *unexpected* is the need to escape " as \"
# even though " can normally be used *as-is* inside a '...' string.
pwsh -Command ' ''{\"drop_attr\": \"name\"}'' '
Note that I'm assuming your intent is to pass a JSON string, hence the inner '' ... '' quoting (escaped single quotes), which ensures that pwsh ultimately sees a single-quoted string ('...'). (No need for an explicit output command; PowerShell implicitly prints command and expression output).
Another way to demonstrate this on Windows is via the standard choice.exe utility, repurposed to simply print its /m (message) argument (followed by verbatim [Y,N]?Y):
# This *should* preserve the ", but doesn't as of v7.2
PS> choice /d Y /t 0 /m '{"drop_attr": "name"}'
{drop_attr: name} [Y,N]?Y # !! " were REMOVED
# Only the extra \-escaping preserves the "
PS> choice /d Y /t 0 /m '{\"drop_attr\": \"name\"}'
{"drop_attr": "name"} [Y,N]?Y # OK
Note that from inside PowerShell, you can avoid the need for \-escaping, if you call pwsh with a script block ({ ... }) - but that only works when calling PowerShell itself, not other external programs:
# NOTE: Works from PowerShell only.
pwsh -Command { '{"drop_attr": "name"}' }
Background info on PowerShell's broken handling of arguments with embedded " in external-program calls, as of PowerShell 7.2.1:
This GitHub docs issue contains background information.
GitHub issue #1995 discusses the problem and the details of the broken behavior as well as manual workarounds are summarized in this comment; the state of the discussion as of PowerShell [Core] 7 seems to be:
A fix is being considered as an experimental feature, which may become an official feature, in v7.3 at the earliest. Whether it will become a regular feature - i.e whether the default behavior will be fixed or whether the fix will require opt-in or even if the feature will become official at all - remains to be seen.
Fixing the default behavior would substantially break backward compatibility; as of this writing, this has never been allowed, but a discussion as to whether to allow breaking changes in the future and how to manage them has begun: see GitHub issue #13129.
See GitHub PR #14692 for the relevant experimental feature, which, however, as of this writing is missing vital accommodations for batch files and msiexec-style executables on Windows - see GitHub issue #15143.
In the meantime, you can use the PSv3+ ie helper function from the Native module (in PSv5+, install with Install-Module Native from the PowerShell Gallery), which internally compensates for all broken behavior and allows passing arguments as expected; e.g.,
ie pwsh -Command ' ''{"drop_attr": "name"}'' ' would then work properly.
Another way. Are you in Windows or Unix?
pwsh -c "[pscustomobject]#{drop_attr='name'} | convertto-json -compress"
{"drop_attr":"name"}
Another way is to use "encoded commands".
> $cmd1 = "Write-Host '{ ""description"": ""Test program"" }'"
> pwsh -encoded ([Convert]::ToBase64String([Text.Encoding]::Unicode.GetBytes($cmd1)))
{ "description": "Test program" }
I'm executing git ls-remote ssh://git#git_repo:port * in two different computers under same network, one Linux another Windows, and on Windows I'm getting the list but on Linux not. No error at all just and empty list on Linux.
Both has the SSH key added to the remote repository and both are able to clone the repository.
Update 1:
Windows Git version: 2.19.2.windows.1
Linux Git version: 2.7.4
Update 2:
The repository is in Gerrit.
Update 3:
I'm facing this problem using the Jenkins plugin Extended Choice Parameter plugin. It has no change since 2016. Any workaround for this would be also an answer.
Any idea?
You probably should use:
git ls-remote ssh://git#git_repo:port
without any suffix, as it defaults to listing everything.
You can use:
git ls-remote ssh://git#git_repo:port '*'
(or the same with double quotes—one or both of these may work on Windows as well). In a Unix/Linux-style command shell, the shell will replace * with a list of all the files in the current directory before running the command, unless you protect the asterisk from the shell.
You can also use a single backlash:
git ls-remote ssh://git#git_repo:port \*
as there are a lot of ways to protect individual characters from shells. The rules get a little complicated, but in general, single quotes are the "most powerful" quotes, while double quotes quote glob characters1 but not other expansions.2 Backslashes quote the immediate next character if you're not already inside quotes (the behavior of backslash within double quotes varies in some shells).
1The glob characters are *, [, and ?. After [, characters inside the glob run to the closing ]. So echo foo[abc] looks for files named fooa, foob, and fooc. Note that . is generally not special: foo.* matches only files whose names start with foo., i.e., including the period: a file named foo does not start with foo., only with foo, and is not matched.
Globs are very different from regular expressions: in regular expressions, . matches any character (like ? does in glob) and asterisk means "repeat previous match zero or more times", so that glob * and regular-expression .* are similar. (In regular expression matches, we also need to consider whether the expression is anchored. Globs are always anchored so that the question does not arise.)
2Most expansions occur with dollar sign $, as in $var or ${var} or $(subcommand), but backquotes also invoke command substitution, as in echo `echo bar`.
Situation:
In my Node-RED flow there is a call to another program where I use the exec node.
The arguments are set-up in a function node and passed as the msg.payload to the exec node.
This works fine and returns the expected results as long as the command has no space in it.
Problem:
A customer cannot use the call to the command on his system, because the path tp the installation of the program contains a whitespace.
What I tried and didn't work:
Escaping the whitespace with ^ as read in another question: /opt/path^ toProgram^ directory/program
Escaping the whitespace with \\: /opt/path\\ toProgram\\ directory/program
Quoting the whole string with "": "/opt/path toProgram directory/program"
Quoting the whole string and escaping the whitespace: "/opt/path\\ toProgram\\ directory/program"
Quoting the whole string with '': '/opt/path toProgram directory/program'
Leaving the command line empty/""and combining any of the above points with its arguments to one string (in the function node that sets up the arguments for that exec node) and passing it on as the msg.payload -> input parameters in the exec node config panel. No Success.
What's not the problem:
The program itself works fine, on mine and on the customers system, it's only the path that is different
Other than that specific command string with whitespace, the configuration of the exec node and its msg.payload ( = input parameters or arguments) as well as the "use spawn() instead of exec()?" is fine and works
Request
Is there any other way to escape the whitespace that I'm not aware of so the path to the program can be found? From my understanding the whitespace is interpreted as a separator for the input arguments, which it should be, but not on the command string.
This should be a quick fix in my opinion, however nothing seems to work that usually works in node, js, php or bash.. thanks in advance for any hints and ideas!
Environment:
Node-RED version: v0.15.2 |
Node.js version: v5.12.0 |
Mac OS X 10.11.6 = Darwin 15.6.0 x64 LE
Screenshots
This is the part of the flow:
This config works:
This config does not work:
I've just tested this and option 3 (Quoting the path with "") works fine.
Putting "/home/foo/foo bar/date" in the command section of the exec node executed the binary correctly
After a good nights sleep the solution (at least what I thought at the time) was to adapt the exec node in node-red itself: The original code appended the additional arguments (the ones separated by whitespace) to the command, and then slice the whole string by whitespace into an array (node.cmd is the path to the command with or without whitespace):
before: var arg = node.cmd; arg += " "+msg.payload; arg = arg.match(/(?:[^\s"]+|"[^"]*")+/g);
and the (earlier, incomplete) solution was to prepend the command with the builtin array.unshift(<value>) function to the array after the slice operation, so that the command path with the whitespace is not affected by the array creation:
after: var arg = ""; ...add msg.payload and slice arg string into array.. arg.unshift(node.cmd);
The file can be found at: nodered node 75-exec.js on github, in the lines 46-51
Update to real solution:
After further investigation it was clear, that the Use spawn() instead of exec()? option was the problem: the command part with whitespace in double quotes and using exec() worked fine. And my approach couldn't be used when the command path contained an argument, e.g. in /home/foo bar/date -r 1000.
Following the issue at the official github repository of node-red the command part to the arguments array at the beginning after the slice but instead to have the command path with whitespace in quotes and later unwrapping them, copying the behaviour when exec() is used. He added this line in the original file after line 51: if (/^".*"$/.test(cmd)) { cmd = cmd.slice(1,-1); } and now both spawn() and exec() work the same - credits to him, unfortunately I don't know his username here.
I am passing in the full path to a file as a commandline argument in perl.
For example
myscript.pl C:\Dir\myfile.txt
In myscript.pl, I have
my $full_path = shift;
print $full_path;
When I do this, my output is
C:Dirmyfile.txt
What I really want is C:\Dir\myfile.txt
But when I run my script as
myscript.pl 'C:\Dir\myfile.txt'
my output is C:/Dir/myfile.txt. Now it has forward slashes instead of backslashes. How do I get what I want? (The same text as what was passed in, file path with backslashes)
I need to run be able to run this script on Cygwin in a windows environment. Note that the script serves a larger purpose, but what I have posted is the part I am stuck with. The path is something I copy from somewhere else, so I really don't want to do the extra work of replacing backslash with forward slash or spaces.
use the File::Spec module. This simplifies passing parameters to your script, since you don't need to use slashes, and it also makes your application portable across operating systems.
use File::Spec;
my $full_path = File::Spec->catfile(#ARGV);
print $full_path, "\n";
Example:
perl myscript.pl C: Dir myfile.txt
C:\Dir\myfile.txt
Alternatively, if you need to use the full path string, then use the following line in place of the above:
my $full_path = File::Spec->canonpath($ARGV[0]);
Example 2:
perl myscript.pl C:\Dir\myfile.txt
--OR--
perl myscript.pl C:/Dir/myfile.txt
C:\Dir\myfile.txt
Example 3 (for Cygwin) - surround parameter with single quotes:
perl myscript.pl 'C:\Dir\myfile.txt'
C:\Dir\myfile.txt
I want to get some data from a HTTP server. What it sends me depends on what I put in a POST request.
What I put in the INPUT_TEXT field is a sequence of words. When I run the following command, I get good looking output.
$ curl http://localhost:59125/process -d INPUT_TEXT="here are some words"
I want a bash script to take some string as a command line argument, and pass it appropriately to curl. The first thing I tried was to put the following in a script:
sentence=$1
command="curl http://localhost:59125/process -d INPUT_TEXT=\"${sentence}\""
$command
I then run the script like so:
$ ./script "here are some words"
But then I get a curl Couldn't resolve host error for each of "are", "some", and "words". It would seem that "here" got correctly treated as the INPUT_TEXT, but the rest of the words were then considered to be hosts, and not part of the option.
So I tried:
command=("curl" "http://localhost:59125/process" "-d" "INPUT_TEXT='$sentence'")
${command[#]}
I got the same output as the first script. I finally got what I wanted with:
result=$(curl http://localhost:59125/process -d INPUT_TEXT="${sentence}")
echo $result
I'm still unsure as to what the distinction is. In the first two cases, when I echoed out the contents of command, I get exactly what I input from the interactive Bash prompt, which had worked fine. What caused the difference?
The following will work:
command=("curl" "http://localhost:59125/process"
"-d" "INPUT_TEXT=$sentence")
"${command[#]}"
That has two changes from yours:
I removed the incorrect quotes around $sentence since you don't want to send quotes to the server (as far as I can see).
I put double-quotes around the use of "${command[#]}". Without the double quotes, the array's elements are concatenated with spaces between them and then the result is word-split. With double quotes, the individual array elements are used as individual words.
The second point is well-explained in the bash FAQ and a bunch of SO answers dealing with quotes.
The important thing to understand is that quotes only quote when a command is parsed. A quote which is a character in a variable is just a character; it is not reinterpreted when the value of the variable expanded. Whitespace in the variable is used for word-splitting if the variable expansion is unquoted; the fact that the whitespace was quoted in the the command which defined the variable is completely irrelevant. In this sense, bash is just the same as any other programming language.