I am trying to use TCLs builtin exec procedure to run the following sed shell command:
sed -i 's/"VALUE.${name}">.*</"VALUE.${name}">${value}</' ${dir}/imp.xml
However when I pass it to exec tcl errors out with
sed: -e expression #1, char 1: unknown command: `''
no idea how to interpret this.
I tried escaping the exec string:
exec {sed -i 's/"VALUE.${name}">.*</"VALUE.${name}">${value}</' ${dir}/imp.xml}
However this prevents the tcl variables from being expanded inside of the string.
Does anyone know what I need to do to get tcl to exec this sed program?
(my shell is csh if that is relevant)
The final solution involved 2 changes to the command string,
Escape all the double quote characters, (thanks #Chris Heithoff)
Single quotes are handled funny by TCL, replacing them with double quotes
(that are not escaped!) resolves the issue.
The final, working command string:
exec sed -i "s/\"VALUE.${name}\">.*</\"VALUE.${name}\">${alue}</" ${dir}/impl.xml
Each argument to the exec command must correspond to an individual argument at the shell command line, so enclosing everything in {} doesn't work.
Try this:, where all double quotes and dollar signs are escaped.
exec sed -i 's/\"VALUE.\${name}\">.*</\"VALUE.\${name}\">\${value}' \${dir}/impl.xml
I am looking to write a simple script to perform a SSH command on many hosts simultaneously, and which hosts exactly are generated from another script. The problem is that when I run the script using sometihng like sed it doesn't work properly.
It should run like sshall.sh {anything here} and it will run the {anything here} part on all the nodes in the list.
sshall.sh
#!/bin/bash
NODES=`listNodes | grep "node-[0-9*]" -o`
echo "Connecting to all nodes and running: ${#:1}"
for i in $NODES
do
:
echo "$i : Begin"
echo "----------------------------------------"
ssh -q -o "StrictHostKeyChecking no" $i "${#:1}"
echo "----------------------------------------"
echo "$i : Complete";
echo ""
done
When it is run with something like whoami it works but when I run:
[root#myhost bin]# sshall.sh sed -i '/^somebeginning/ s/$/,appendme/' /etc/myconfig.conf
Connecting to all nodes and running: sed -i /^somebeginning/ s/$/,appendme/ /etc/myconfig.conf
node-1 : Begin
----------------------------------------
sed: -e expression #1, char 18: missing command
----------------------------------------
node-1 : Complete
node-2 : Begin
----------------------------------------
sed: -e expression #1, char 18: missing command
----------------------------------------
node-2 : Complete
…
Notice that the quotes disappear on the sed command when sent to the remote client.
How do I go about fixing my bash command?
Is there a better way of achieving this?
Substitute an eval-safe quoted version of your command into a heredoc:
#!/bin/bash
# ^^^^- not /bin/sh; printf %q is an extension
# Put your command into a single string, with each argument quoted to be eval-safe
printf -v cmd_q '%q ' "$#"
while IFS= read -r hostname; do
# run bash -s remotely, with that string passed on stdin
ssh -q -o 'StrictHostKeyChecking no' "$hostname" "bash -s" <<EOF
$cmd_q
EOF
done < <(listNodes | grep -o -e "node-[0-9*]")
Why this works reliably (and other approaches don't):
printf %q knows how to quote contents to be eval'd by that same shell (so spaces, wildcards, various local quoting methods, etc. will always be supported).
Arguments given to ssh are not passed to the remote command individually!
Instead, they're concatenated into a string passed to sh -c.
However: The output of printf %q is not portable to all POSIX-derived shells! It's guaranteed to be compatible with the same shell locally in use -- ksh will always parse output from printf '%q' in ksh, bash will parse output from printf '%q' in bash, etc; thus, you can't safely pass this string on the remote argument vector, because it's /bin/sh -- not bash -- running there. (If you know your remote /bin/sh is provided by bash, then you can run ssh "$hostname" "$cmd_q" safely, but only under this condition).
bash -s reads the script to run from stdin, meaning that passing your command there -- not on the argument vector -- ensures that it'll be parsed into arguments by the same shell that escaped it to be shell-safe.
You want to pass the entire command -- with all of its arguments, spaces, and quotation marks -- to ssh so it can pass it unchanged to the remote shell for parsing.
One way to do that is to put it all inside single quotation marks. But then you'll also need to make sure the single quotation marks within your command are preserved in the arguments, so the remote shell builds the correct arguments for sed.
sshall.sh 'sed -i '"'"'/^somebeginning/ s/$/,appendme/'"'"' /etc/myconfig.conf'
It looks redundant, but '"'"' is a common Bourne trick to get a single quotation mark into a single-quoted string. The first quote ends single-quoting temporarily, the double-quote-single-quote-double-quote construct appends a single quotation mark, and then the single quotation mark resumes your single-quoted section. So to speak.
Another trick that can be helpful for troubleshooting is to add the -v flag do your ssh flags, which will spit out lots of text, but most importantly it will show you exactly what string it's passing to the remote shell for parsing and execution.
--
All of this is fairly fragile around spaces in your arguments, which you'll need to avoid, since you're relying on shell parsing on the opposite end.
Thinking outside the box: instead of dealing with all the quoting issues and the word-splitting in the wrong places, you could attempt to a) construct the script locally (maybe use a here-document?), b) scp the script to the remote end, then c) invoke it there. This easily allows more complex command sequences, with all the power of shell control constructs etc. Debugging (checking proper quoting) would be a breeze by simply looking at the generated script.
I recommend reading the command(s) from the standard input rather than from the command line arguments:
cmd.sh
#!/bin/bash -
# Load server_list with user#host "words" here.
cmd=$(</dev/stdin)
for h in ${server_list[*]}; do
ssh "$h" "$cmd"
done
Usage:
./cmd.sh <<'CMD'
sed -i '/^somebeginning/ s/$/,appendme/' /path/to/file1
# other commands
# here...
CMD
Alternatively, run ./cmd.sh, type the command(s), then press Ctrl-D.
I find the latter variant the most convenient, as you don't even need for here documents, no need for extra escaping. Just invoke your script, type the commands, and press the shortcut. What could be easier?
Explanations
The problem with your approach is that the quotes are stripped from the arguments by the shell. For example, the argument '/^somebeginning/ s/$/,appendme/' will be interpreted as /^somebeginning/ s/$/,appendme/ string (without the single quotes), which is an invalid argument for sed.
Of course, you can escape the command with the built-in printf as suggested in other answer here. But the command becomes not very readable after escaping. For example
printf %q 'sed -i /^somebeginning/ s/$/,appendme/ /home/ruslan/tmp/file1.txt'
produces
sed\ -i\ /\^somebeginning/\ s/\$/\,appendme/\ /home/ruslan/tmp/file1.txt
which is not very readable, and will look ugly, if you print it to the screen in order to show the progress.
That's why I prefer to read from the standard input and leave the command intact. My script prints the command strings to the screen, and I see them just in the form I have written them.
Note, the for .. in loop iterates $IFS-separated "words", and is generally not preferred way to traverse an array. It is generally better to invoke read -r in a while loop with adjusted $IFS. I have used the for loop for simplicity, as the question is really about invoking the ssh command.
Logging into multiple systems over SSH and using the same (or variations on the same) command is the basic use case behind ansible. The system is not without significant flaws, but for simple use cases is pretty great. If you want a more solid solution without too much faffing about with escaping and looping over hosts, take a look.
Ansible has a 'raw' module which doesn't even require any dependencies on the target hosts, and you might find that a very simple way to achieve this sort of functionality in a way that frees you from the considerations of looping over hosts, handling errors, marshalling the commands, etc and lets you focus on what you're actually trying to achieve.
This is my simple shell script
sample.sh
LOCALCONFIGDIR="Install"
CONFLOC="$LOCALCONFIGDIR/server.conf"
echo "Enter Web Url:"
read weburl
echo "sed 's/^\(ServerName\)$/\1 "$weburl"/' "$CONFLOC
sed "'s/^\(ServerName\)$/\1 "$weburl"/' "$CONFLOC
When I run this code, I get the result in echo command as following.
sed 's/^\(ServerName\)$/\1 www.weburl.com/' Install/server.conf
But when executing sed command in the next line, It says the below error.
sed: -e expression #1, char 1: unknown command: `''
I tried the command produced in echo statement from Terminal screen, It is working. But Line number 5, doesn't working from shell script
You need to use one set of quotes, not two, and since you want the $weburl variable expanded, you need to use double quotes:
sed "s/^\(ServerName\)$/\1 $weburl/" "$CONFLOC"
That'll be OK as long as $weburl doesn't contain any slashes. If it does, you need to use a different character than /, such as %, to separate the parts of the substitute command:
sed "s%^\(ServerName\)$%\1 $weburl%" "$CONFLOC"
I am trying to use SED command with a variable that contains several / and i got the following error :
sed: -e expression 1, char 16: unknown option to s
this is my code and this is inside a script:
thispath=$(readlink -f $0)
sudo sed -i '13s/^/'"$thispath"'/g' /etc/rc.local
and the variable contains for example: /home/user/script.sh
i do not know very well the use of sed can somebody help me
The s command is sed allows you to use any character to delimit the regex and replacement parts. "/" is often a poor choice given how often you come across it in UNIX paths:
Try:
sudo sed -i '13s:^:'"$thispath"':g' /etc/rc.local
It is dangerous to do this directly on rc.local. So make a copy first.
I have the following script (to get my current IP from an external service):
#!/bin/bash
####################################################################
# Gets the public IP address of current server
####################################################################
cmd='curl -s'
#cmd='wget -q -O'
#cmd='lynx -dump'
ipservice=checkip.dyndns.org
pipecmd="sed -e 's/.*Current IP Address: //' -e 's/<.*\$//'"
# Run command
echo $($cmd $ipservice | $pipecmd)
But sed command complains:
sed: -e expression #1, char 1: unknown command: `''
I have been googling around on how to use single quotes inside a variable without success.
Thanks!
The command is split into words sed, -e, 's/.*Current, IP, Address:, //' etc., so the first command in the sed program indeed starts with ', which is not a valid sed command. Use an array and quoting instead:
cmd=(curl -s)
pipecmd=(sed -e 's/.*Current IP Address: //' -e 's/<.*$//')
"${cmd[#]}" "$ipservice" | "${pipecmd[#]}"
Note that echo "$(command)" is equivalent to command. In general, make sure that you always quote all variables (there are a few exceptions, though).
You need to use eval to get the shell to interpret the contents of the variable
echo $($cmd $ipservice | eval $pipecmd)
You may need extra escaping because of the extra evaluation, although in this particular case I think it's okay as is.