Situation:
In my Node-RED flow there is a call to another program where I use the exec node.
The arguments are set-up in a function node and passed as the msg.payload to the exec node.
This works fine and returns the expected results as long as the command has no space in it.
Problem:
A customer cannot use the call to the command on his system, because the path tp the installation of the program contains a whitespace.
What I tried and didn't work:
Escaping the whitespace with ^ as read in another question: /opt/path^ toProgram^ directory/program
Escaping the whitespace with \\: /opt/path\\ toProgram\\ directory/program
Quoting the whole string with "": "/opt/path toProgram directory/program"
Quoting the whole string and escaping the whitespace: "/opt/path\\ toProgram\\ directory/program"
Quoting the whole string with '': '/opt/path toProgram directory/program'
Leaving the command line empty/""and combining any of the above points with its arguments to one string (in the function node that sets up the arguments for that exec node) and passing it on as the msg.payload -> input parameters in the exec node config panel. No Success.
What's not the problem:
The program itself works fine, on mine and on the customers system, it's only the path that is different
Other than that specific command string with whitespace, the configuration of the exec node and its msg.payload ( = input parameters or arguments) as well as the "use spawn() instead of exec()?" is fine and works
Request
Is there any other way to escape the whitespace that I'm not aware of so the path to the program can be found? From my understanding the whitespace is interpreted as a separator for the input arguments, which it should be, but not on the command string.
This should be a quick fix in my opinion, however nothing seems to work that usually works in node, js, php or bash.. thanks in advance for any hints and ideas!
Environment:
Node-RED version: v0.15.2 |
Node.js version: v5.12.0 |
Mac OS X 10.11.6 = Darwin 15.6.0 x64 LE
Screenshots
This is the part of the flow:
This config works:
This config does not work:
I've just tested this and option 3 (Quoting the path with "") works fine.
Putting "/home/foo/foo bar/date" in the command section of the exec node executed the binary correctly
After a good nights sleep the solution (at least what I thought at the time) was to adapt the exec node in node-red itself: The original code appended the additional arguments (the ones separated by whitespace) to the command, and then slice the whole string by whitespace into an array (node.cmd is the path to the command with or without whitespace):
before: var arg = node.cmd; arg += " "+msg.payload; arg = arg.match(/(?:[^\s"]+|"[^"]*")+/g);
and the (earlier, incomplete) solution was to prepend the command with the builtin array.unshift(<value>) function to the array after the slice operation, so that the command path with the whitespace is not affected by the array creation:
after: var arg = ""; ...add msg.payload and slice arg string into array.. arg.unshift(node.cmd);
The file can be found at: nodered node 75-exec.js on github, in the lines 46-51
Update to real solution:
After further investigation it was clear, that the Use spawn() instead of exec()? option was the problem: the command part with whitespace in double quotes and using exec() worked fine. And my approach couldn't be used when the command path contained an argument, e.g. in /home/foo bar/date -r 1000.
Following the issue at the official github repository of node-red the command part to the arguments array at the beginning after the slice but instead to have the command path with whitespace in quotes and later unwrapping them, copying the behaviour when exec() is used. He added this line in the original file after line 51: if (/^".*"$/.test(cmd)) { cmd = cmd.slice(1,-1); } and now both spawn() and exec() work the same - credits to him, unfortunately I don't know his username here.
Related
Given the following script:
const yargs = require('yargs');
const argv =
yargs
.usage('Usage: $0 [--whatIf]')
.alias('d', 'directory')
.alias('wi', 'whatIf')
.nargs('d', 1)
.describe('d', 'alphabetize this directory')
.describe('whatIf', 'show what would happen if run')
.demandOption(['d'])
.argv;
console.log(argv.directory);
If I invoke the script from Windows PowerShell like so: node .\alphabetizer.js -d 'l:\my folder\Files - Some Files In Here\' --whatIf I get the output l:\my folder\Files - Some Files In Here\" --whatIf where I would expect just l:\my folder\Files - Some Files In Here\. It works OK with folder names that require no escaping, but it seems to get confused by the escaping.
If I examine process.argv, I can see the same escaping issue.
I have noticed that if I remove the trailing slash it will work. However, this still points to the node script not handling the input properly, because this should not be necessary with string set off by single quotes.
Is there a way to make this work?
Both Windows PowerShell (powershell.exe) and PowerShell [Core] v6+ (pwsh) are fundamentally broken with respect to quoting arguments for external programs properly - see this answer for background info.
Generally, PowerShell on Windows has to perform re-quoting behind the scenes in order to ensure that just "..."-quoting is used, given that external programs can't be assumed to understand '...'-quoting too when parsing their command line (which on Windows every program has to do itself).
Windows PowerShell is more broken with respect to arguments that end in \ and have embedded spaces, re-quoting them improperly; e.g.:
PS> foo.exe 'c:\foo \' bar
is translated into the following command line behind the scenes:
foo.exe "c:\ foo \" bar
This is broken, in that most applications - including PowerShell's own CLI - sensibly assume that the \" is an escaped " char. to be taken verbatim, thinking that the argument continues with bar and then implicitly ends, despite the formal lack of a closing ".
PowerShell [Core] v6+ more sensibly translates the above to foo.exe "c:\foo \\" bar, where the \\ is interpreted as an escaped \, and the following " again has syntactic function.
If you're stuck with Windows PowerShell, your only choices are:
either: if possible, leave off the trailing \
otherwise: manually double it (\\), but only do so if the argument also contains spaces (otherwise, the \\ will be retained as-is, though in the case of filesystem paths that is usually benign).
I'm using a bash script to send an argument to a node app like so:
testString="\nhello\nthere"
node ./myNodeScript.js $testString
The trouble comes when I use testString inside the node program after capturing it as process.argv[2] -- rather than expand the \n characters to newlines node prints them literally. I need a way to tell node to convert the argument to a javascript string, respecting the formatting characters. Is there a way to go about this?
Try to avoid confusing literal linefeeds and literal backslash followed by literal n.
If you want the string you pass to have linefeeds, you should ignore JavaScript string literal syntax and just pass the linefeeds as linefeeds:
$ cat myNodeScript.js
console.log("Node was passed this: " + process.argv[2])
$ cat myBashScript
testString='
hello
there'
printf 'Bash passes this: %s\n' "$testString"
node myNodeScript.js "$testString"
$ bash myBashScript
Bash passes this:
hello
there
Node was passed this:
hello
there
Arguments should contain data (linefeed) while script files should contain code (quoted linefeed or expanded \n as appropriate in the language). When you make sure not to confuse code and data, you can trivially handle both backslash-en and linefeeds in the same string with no surprises:
testString='
"\nhello\nthere" is JavaScript syntax for:
hello
there'
There are ways to express this on a single line in bash using \n for linefeeds and \\n for backslash-en, you just need to make sure that it remains as code, and doesn't accidentally make it into the variable as data.
Can you try this:
testString=$( printf "\nhello\nthere")
node ./myNodeScript.js "$testString"
And let me know if it works?
I want to get some data from a HTTP server. What it sends me depends on what I put in a POST request.
What I put in the INPUT_TEXT field is a sequence of words. When I run the following command, I get good looking output.
$ curl http://localhost:59125/process -d INPUT_TEXT="here are some words"
I want a bash script to take some string as a command line argument, and pass it appropriately to curl. The first thing I tried was to put the following in a script:
sentence=$1
command="curl http://localhost:59125/process -d INPUT_TEXT=\"${sentence}\""
$command
I then run the script like so:
$ ./script "here are some words"
But then I get a curl Couldn't resolve host error for each of "are", "some", and "words". It would seem that "here" got correctly treated as the INPUT_TEXT, but the rest of the words were then considered to be hosts, and not part of the option.
So I tried:
command=("curl" "http://localhost:59125/process" "-d" "INPUT_TEXT='$sentence'")
${command[#]}
I got the same output as the first script. I finally got what I wanted with:
result=$(curl http://localhost:59125/process -d INPUT_TEXT="${sentence}")
echo $result
I'm still unsure as to what the distinction is. In the first two cases, when I echoed out the contents of command, I get exactly what I input from the interactive Bash prompt, which had worked fine. What caused the difference?
The following will work:
command=("curl" "http://localhost:59125/process"
"-d" "INPUT_TEXT=$sentence")
"${command[#]}"
That has two changes from yours:
I removed the incorrect quotes around $sentence since you don't want to send quotes to the server (as far as I can see).
I put double-quotes around the use of "${command[#]}". Without the double quotes, the array's elements are concatenated with spaces between them and then the result is word-split. With double quotes, the individual array elements are used as individual words.
The second point is well-explained in the bash FAQ and a bunch of SO answers dealing with quotes.
The important thing to understand is that quotes only quote when a command is parsed. A quote which is a character in a variable is just a character; it is not reinterpreted when the value of the variable expanded. Whitespace in the variable is used for word-splitting if the variable expansion is unquoted; the fact that the whitespace was quoted in the the command which defined the variable is completely irrelevant. In this sense, bash is just the same as any other programming language.
I have been experimenting with Haskell. I am trying to write a web crawler and I need to use external curl binary (due to some proxy settings, curl needs to have some special arguments which seem to be impossible/hard to set inside the haskell code, so i rather just pass it as a command line option. but that is another story...)
In the code at the bottom, if I change the marked line with curl instead of curl --help the output renders properly and gives:
"curl: try 'curl --help' or 'curl --manual' for more information
"
otherwise the string is empty - as the `curl --help' response is multiline.
I suspect that in haskell the buffer is cleared with every new line. (same goes for other simple shell commands like ls versus ls -l etc.)
How do I fix it?
The code:
import System.Process
import System.IO
main = do
let sp = (proc "curl --help"[]){std_out=CreatePipe} -- *** THIS LINE ***
(_,Just out_h,_,_)<- createProcess sp
out <-hGetContents out_h
print out
proc takes as a first argument the name of the executable, not a shell command. That, is when you use proc "foo bar" you are not referring to a foo executable, but to an executable named exactly foo bar, with the space in its file name.
This is a useful feature in practice, because sometimes you do have spaces in there (e.g. on Windows you might have c:\Program Files\Foo\Foo.exe). Using a shell command you would have to escape spaces in your command string. Worse, a few other characters need to be escaped as well, and it's cumbersome to check what exactly those are. proc sidesteps the issue by not using the shell at all but passing the string as it is to the OS.
For the executable arguments, proc takes a separate argument list. E.g.
proc "c:\\Program Files\\Foo\\Foo.exe" ["hello world!%$"]
Note that the arguments need no escaping as well.
If you want to pass arguments to curl you have to pass that it in the list:
sp = (proc "/usr/bin/curl" ["--help"]) {std_out=CreatePipe}
Then you will get the complete output in the entire string.
I'm having a nightmare from what should be the most trivial of tasks.
My final goal is issue the following command from a bash script:
sqlite3 my_db.db '.read my_file.sql'
There are two catches here:
1. The single-quotes are obligatory, and can't be replaced by, say, double-quotes
2. my_file.sql is a variable known only at run-time.
So what I need is a way to have bash build a string that on one hand contains a variable value, while on the other hand that value should be surrounded by single quotes.
I would also much prefer a solution not relying on additional tools like AWK, Perl or the like. Maybe sed if it's really necessary.
Thanks.
Thanks Jonathan and Nelson.
I tried all three suggestions, but they all failed.
For simplicity I reduced the problem to the following:
I wrote the following script (tst.sh):
#!/bin/bash
file=/tmp/1
ls "'"$file"'"
ls \'$file\'
ls "'$file'"
Then I isuues the following commands:
$ touch /tmp/1
$ ls '/tmp/1'
/tmp/1
$ ./tst.sh
'/tmp/1': No such file or directory
'/tmp/1': No such file or directory
'/tmp/1': No such file or directory
It seems the quotes were indeed added, but the resulting command was not the same as when entered manually.
Any ideas ?
Single-quotes are not obligatory. All of the following commands run sqlite3 with exactly the same arguments:
sqlite3 my_db.db '.read my_file.sql'
sqlite3 my_db.db ".read my_file.sql"
sqlite3 my_db.db .read\ my_file.sql
sqlfile="my_file.sql"
sqlite3 my_db.db ".read $sqlfile"
In all cases, the quotes (/escape) are parsed and removed before the arguments are passed to sqlite3. This is what you want. You want sqlite3 to get two arguments: my_db.db and .read my_file.sql. You do not want sqlite3 to see the quotes around the command -- that would be the equivalent of:
$ sqlite3 my_db.db
SQLite version 3.7.7 2011-06-25 16:35:41
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> '.read my_file.sql'
...>
...which, as you can see, just confuses sqlite3.
BTW, this is the same as the problem in your ls examples: you're passing single-quotes as part of the argument to ls, so it's looking for a file with single-quotes in the name and not finding it. You want the shell to remove the quotes rather than pass them to the command as part of an argument.
This will do what you say you want to do (getting single quotes to the program), but it uses double quotes:
sqlite3 my_db.db "'".read" "my_file.sql"'"
Avoiding double quotes, you can write:
sqlite3 my_db.db \'.read\ my_file.sql\'
For both of these, the second argument will be seen by sqlite3 as a string containing:
'.read my_file.sql'
If the file name is in a variable (file=my_file.sql), then:
sqlite3 my_db.db "'".read" "$file"'"
sqlite3 my_db.db \'.read\ $file\'
These notations are vulnerable to confusion if the file name contains spaces.
However, I don't think that's likely to be what you really want. The proscription on double quotes is puzzling, and the requirement for single quotes is likewise puzzling.
You can do as follows:
VAR=my_file.sql
VAR2="'.read $VAR'"
sqlite3 my_db.db $VAR2
user1860085, if you check out documentation for sqlite3 command and you will know how shell treats quotes and white spaces, you will probably come to conclusion that you want double quotes for your case.
but if you really want single quotes, here is solution:
eval sqlite3 my_db.db \'.read $VARIABLE\'
which in the fly will change to:
sqlite3 my_db.db '.read my_file.sql'
But I don't see why you could want it...
OK, problem solved !!
All that was missing is adding a little 'eval' command before the line.
So, in the simple example script I gave, changing:
ls "'$file'" to:
eval ls "'$file'"
did the job.
Thanks to all replyers :-)