I'm trying to run something that simplifies to
(trivial-shell:shell-command
(concatenate 'string "echo -e " "one" "\n" "two"))
And trying to get it to return something like
"one
two"
Whereas what it returns instead is
"onentwo
"
I'd be open to using something different from trivial-shell (I've tried inferior-shell, but it had the same problem)
Using SBCL 1.1.18 on 64-bit Gentoo Linux
The simplest way to output a newline is to have a newline character inside your string:
(print "one
two")
You can also coerce the newline character into a string:
(concatenate 'string "one" (string #\newline) "two")
You cannot use \n in a string to insert newlines: the backslash is only used to escape double-quotes inside strings (e.g. "\""). Necessarily, it is used to escape actual backslashes character too.
So if you want to print a backslash followed by n, you need to escape the backslash:
(print "\\n")
But then, why do you need 4 backslashes?
Because trivial-shell:shell-command invokes a shell, as given by *bourne-compatible-shell*, and gives it a command as a string. And so, your string might be subject to shell escaping rules too.
(trivial-shell:shell-command "echo -e one \\\\n two")
Knowing that, you might want to take into account the fact that bourne compatible shells have single-quotes, which do not require escaping characters inside of them. The following works as expected:
(trivial-shell:shell-command "echo 'one
two'")
Note that the default value for trivial-shell:*bourne-compatible-shell* is "/bin/sh", which as far as I know does not recognize the -e option for echo. You might already know this, but you might want to use a different shell. Here is a program that calls echo with newlines between arguments:
(defun echo-args (&rest args)
(let ((trivial-shell:*bourne-compatible-shell* #P"/bin/bash"))
(trivial-shell:shell-command
(format nil "echo -e '~{~A~^~%~}'" args))))
Solved it by using four backslashes instead of one.
Related
I'm using a bash script to send an argument to a node app like so:
testString="\nhello\nthere"
node ./myNodeScript.js $testString
The trouble comes when I use testString inside the node program after capturing it as process.argv[2] -- rather than expand the \n characters to newlines node prints them literally. I need a way to tell node to convert the argument to a javascript string, respecting the formatting characters. Is there a way to go about this?
Try to avoid confusing literal linefeeds and literal backslash followed by literal n.
If you want the string you pass to have linefeeds, you should ignore JavaScript string literal syntax and just pass the linefeeds as linefeeds:
$ cat myNodeScript.js
console.log("Node was passed this: " + process.argv[2])
$ cat myBashScript
testString='
hello
there'
printf 'Bash passes this: %s\n' "$testString"
node myNodeScript.js "$testString"
$ bash myBashScript
Bash passes this:
hello
there
Node was passed this:
hello
there
Arguments should contain data (linefeed) while script files should contain code (quoted linefeed or expanded \n as appropriate in the language). When you make sure not to confuse code and data, you can trivially handle both backslash-en and linefeeds in the same string with no surprises:
testString='
"\nhello\nthere" is JavaScript syntax for:
hello
there'
There are ways to express this on a single line in bash using \n for linefeeds and \\n for backslash-en, you just need to make sure that it remains as code, and doesn't accidentally make it into the variable as data.
Can you try this:
testString=$( printf "\nhello\nthere")
node ./myNodeScript.js "$testString"
And let me know if it works?
This question already has answers here:
How can I store a command in a variable in a shell script?
(12 answers)
Closed 4 years ago.
These work as advertised:
grep -ir 'hello world' .
grep -ir hello\ world .
These don't:
argumentString1="-ir 'hello world'"
argumentString2="-ir hello\\ world"
grep $argumentString1 .
grep $argumentString2 .
Despite 'hello world' being enclosed by quotes in the second example, grep interprets 'hello (and hello\) as one argument and world' (and world) as another, which means that, in this case, 'hello will be the search pattern and world' will be the search path.
Again, this only happens when the arguments are expanded from the argumentString variables. grep properly interprets 'hello world' (and hello\ world) as a single argument in the first example.
Can anyone explain why this is? Is there a proper way to expand a string variable that will preserve the syntax of each character such that it is correctly interpreted by shell commands?
Why
When the string is expanded, it is split into words, but it is not re-evaluated to find special characters such as quotes or dollar signs or ... This is the way the shell has 'always' behaved, since the Bourne shell back in 1978 or thereabouts.
Fix
In bash, use an array to hold the arguments:
argumentArray=(-ir 'hello world')
grep "${argumentArray[#]}" .
Or, if brave/foolhardy, use eval:
argumentString="-ir 'hello world'"
eval "grep $argumentString ."
On the other hand, discretion is often the better part of valour, and working with eval is a place where discretion is better than bravery. If you are not completely in control of the string that is eval'd (if there's any user input in the command string that has not been rigorously validated), then you are opening yourself to potentially serious problems.
Note that the sequence of expansions for Bash is described in Shell Expansions in the GNU Bash manual. Note in particular sections 3.5.3 Shell Parameter Expansion, 3.5.7 Word Splitting, and 3.5.9 Quote Removal.
When you put quote characters into variables, they just become plain literals (see http://mywiki.wooledge.org/BashFAQ/050; thanks #tripleee for pointing out this link)
Instead, try using an array to pass your arguments:
argumentString=(-ir 'hello world')
grep "${argumentString[#]}" .
In looking at this and related questions, I'm surprised that no one brought up using an explicit subshell. For bash, and other modern shells, you can execute a command line explicitly. In bash, it requires the -c option.
argumentString="-ir 'hello world'"
bash -c "grep $argumentString ."
Works exactly as original questioner desired. There are two restrictions to this technique:
You can only use single quotes within the command or argument strings.
Only exported environment variables will be available to the command
Also, this technique handles redirection and piping, and other shellisms work as well. You also can use bash internal commands as well as any other command that works at the command line, because you are essentially asking a subshell bash to interpret it directly as a command line. Here's a more complex example, a somewhat gratuitously complex ls -l variant.
cmd="prefix=`pwd` && ls | xargs -n 1 echo \'In $prefix:\'"
bash -c "$cmd"
I have built command processors both this way and with parameter arrays. Generally, this way is much easier to write and debug, and it's trivial to echo the command you are executing. OTOH, param arrays work nicely when you really do have abstract arrays of parameters, as opposed to just wanting a simple command variant.
I am trying to do string substitution in bash, want to understand it better.
I crafted a success case like this:
a=abc_de_f
var=$a
echo ${var//_/-}
outout is abc-de-f. This works.
However, the following script fails:
a=abc_de_f
echo ${$a//_/-}
The error message is ${$a//_/-}: bad substitution.
It seems like related to how we can use a variable in substitution. Why this fails? How bash handles variables in this case?
Also, what is the best practice to handle escape characters in bash string substitution?
In the second case, you don't need the second $ as a is the string.
a=abc_de_f
echo ${a//_/-}
If you wanted to add a level of indirection, you can use ! before the variable as in
a=abc_de_f
b=a
echo ${b//_/-}
will output a, while
echo ${!b//_/-}
will output abc-de-f.
See here for a discussion on the art of escaping in BASH
I'm using the flag package to interpret flags entered at the command line.
I created a variable using
ptrString := flag.String("string", "", "A test string")
flat.Parse()
Then when I want to print it,
fmt.Println("You entered " + *ptrString)
If I enter something like -string=hello! as a command line argument, it prints "hello!"
If I enter something like -string=hello\Bob as a command line argument, it prints "helloBob"
Is there a recommended way to convert or interpret the flag argument to a string that doesn't remove the backslash? (This is being tested on Linux and OS X, if the shell is interfering...)
Characters that have special meaning in the shell need to be quoted or escaped. You can find complete list in the shell's man pages (under "Quoting" in man 1 bash).
In this case, you can either quote or escape the baskslash
-string=hello\\Bob
// or
-string='hello\Bob'
I want to get some data from a HTTP server. What it sends me depends on what I put in a POST request.
What I put in the INPUT_TEXT field is a sequence of words. When I run the following command, I get good looking output.
$ curl http://localhost:59125/process -d INPUT_TEXT="here are some words"
I want a bash script to take some string as a command line argument, and pass it appropriately to curl. The first thing I tried was to put the following in a script:
sentence=$1
command="curl http://localhost:59125/process -d INPUT_TEXT=\"${sentence}\""
$command
I then run the script like so:
$ ./script "here are some words"
But then I get a curl Couldn't resolve host error for each of "are", "some", and "words". It would seem that "here" got correctly treated as the INPUT_TEXT, but the rest of the words were then considered to be hosts, and not part of the option.
So I tried:
command=("curl" "http://localhost:59125/process" "-d" "INPUT_TEXT='$sentence'")
${command[#]}
I got the same output as the first script. I finally got what I wanted with:
result=$(curl http://localhost:59125/process -d INPUT_TEXT="${sentence}")
echo $result
I'm still unsure as to what the distinction is. In the first two cases, when I echoed out the contents of command, I get exactly what I input from the interactive Bash prompt, which had worked fine. What caused the difference?
The following will work:
command=("curl" "http://localhost:59125/process"
"-d" "INPUT_TEXT=$sentence")
"${command[#]}"
That has two changes from yours:
I removed the incorrect quotes around $sentence since you don't want to send quotes to the server (as far as I can see).
I put double-quotes around the use of "${command[#]}". Without the double quotes, the array's elements are concatenated with spaces between them and then the result is word-split. With double quotes, the individual array elements are used as individual words.
The second point is well-explained in the bash FAQ and a bunch of SO answers dealing with quotes.
The important thing to understand is that quotes only quote when a command is parsed. A quote which is a character in a variable is just a character; it is not reinterpreted when the value of the variable expanded. Whitespace in the variable is used for word-splitting if the variable expansion is unquoted; the fact that the whitespace was quoted in the the command which defined the variable is completely irrelevant. In this sense, bash is just the same as any other programming language.