I have the following command in my bash script:
printf '\n"runtime": %s' "$(bc -l <<<"($a - $b)")"
I need to run this script on around 100 servers and I have found that on few of them bc is not installed. I am not admin and cannot install bc on missing servers.
In that case, what alternative can i use to perform the same calculation? Please let me know how the new command should look like
In case you need a solution which works for floating-point arithmetic you can always fall back to Awk.
awk -v a="$a" -v b="$b" 'BEGIN { printf "\n\"runtime\": %s", a-b }' </dev/null
Putting the code in a BEGIN block and redirecting input from /dev/null is a common workaround for when you want to use Awk but don't have a file of lines to loop over, which is what it's really designed to do.
If you are only dealing with integers you can use bash's arithmetic expansion for this:
printf '\n"runtime": %s' $((a - b))
Note that this does assume you have bash available (as you've indicated you do). If you only have a stripped down Bourne shell (/bin/sh) arithmetic expansion is not available to you.
Related
So I wanted to make a simple script to keep checking the CPU temperature of my RasPi, which is stored in /sys/class/thermal/thermal_zone0/temp , and hence cat /sys/class/thermal/thermal_zone0/temp would give the temp, but like this :
cat /sys/class/thermal/thermal_zone0/temp
38459
which essentially means 38.459 degree Celsius.
I was unable to format the output to get 38.594 °C
My code:
tempT="$(cat /sys/class/thermal/thermal_zone0/temp)"
tempC=$($tempT / 1000)
echo "$tempC °C"
The error I get:
-bash: 38459: command not found
°C
Thanks
The simplest would be to use awk.
awk '{print $1/1000}' /sys/class/thermal/thermal_zone0/temp
or with some more control with printf
awk '{printf "%.3f\n", $1/1000}' /sys/class/thermal/thermal_zone0/temp
The error you are seeing comes from that you used $( ...), which is a command substitution and tries to run the command inside. So when you do:
$($tempT / 1000)
First $tempT expands to 38459 and then shell tries to run a command named 38459 with two arguments / and 1000. So you see the message 38459: Command not found. Use $((...)) for arithmetic expansion, but shells do not implement floating point arithmetic so you have to use other tools like awk or bc.
I'd use bc if it is available on your system.
$ CELSIUS=$(bc -l <<< $(cat /sys/class/thermal/thermal_zone0/temp)/1000)
$ echo $CELSIUS
25.00000000000000000000
TempC=$($tempT / 1000);
Resolves to:
TempC=$(38459 / 1000);
And bash treats $(...) as a command to be passed into a subshell, so it tries to run the executable 38455, which it can't find, and hence complains.
I would use bc, as #kinezan suggested, though I personally prefer the following convention:
TempC=$(echo "scale=3; $tempT / 1000" | bc)
which outputs 38.459
I was trying to do floating point arithmetic in bash, but as floats are not supported, was trying to get the solution using AWK. Below is the issue i am facing:
I see this working fine for me:
code
echo - | awk '{printf("%04.4f \n", (-225.2*7+30*6)/17 + (19^2)/9) }'
output
-42.0301
But my motive is to "read an expression" and compute value correct to 4 decimals, so tried below code inputting same expression (-225.2*7+30*6)/17 + (19^2)/9) and its giving incorrect values(i guess variable is passed as string to awk):
code
read inpt
echo - | awk -v input=$inpt '{printf("%04.4f \n", input) }'
output
0.0000
Note: Please ignore the space around second + in this example expression, that i can remove using sed or similar methods(with space i get syntactical error in awk while passing variable from bash).
Any help is highly appreciated. Thanks in advance
PS: the bash version in my case is "bash-4.2". I guess its the version of bash preventing me using from many other options.
You can't evaluate data in a variable in awk out of the box. In this case you need to write an arithmetic evaluator or use a pre-existing one, like https://github.com/radare/radare2-bindings/blob/master/awk/calc.awk . Once you fix that missing parenthesis and quote your expression properly, you can:
$ echo "((-225.2*7+30*6)/17 + (19^2)/9)" | awk -f calc.awk
((-225.2*7+30*6)/17 + (19^2)/9) = -42.0301
I recommend and have upvoted James Brown's answer. But if you need something here and now with no external dependencies, you can simply interpolate the input into the script.
awk "END { printf("%04.4f\n", $input) }" </dev/null
This is functionally equivalent to using eval so if this isn't deployed where you know you can trust the input (e.g. because it comes from a controlled process, not an actual user) you will need to perform some sort of sanitization (and even then probably cope with odd or outright misleading error messages if the input isn't a well-formed Awk expression).
read -p "Input an Awk expression: " input
case $input in
*[!-+/*()^%0-9]*)
echo "$0: invalid input" >&2
exit 1;;
esac
awk ...
Notice also the construct to avoid the basically useless echo. Redirecting input from /dev/null and putting your code in the END (or BEGIN) block is a standard technique for running an arbitrary piece of Awk script without requiring any input.
First, I don't see anything wrong in the command, it should work. Please try again the exact commands posted and provide exact details if the issue persists.
However, if all you need is formating the output, you can do it directly with printf.
$ read input
12.34
$ printf '%4.4f' $input
12.3400
EDIT:
If you need to format the output after performing some calculation, then you can alternatively use bc. (awk should still work)
$ echo "scale=4; (-225.2*7+30*6)/17+(19^2)/9" | bc
-42.0300
You can use variables in the expression as usual,
$ read inpt
1234
$ echo "scale=4; $inpt * 0.01" | bc
12.34
Is this what you are looking for ?
echo "25 50"| awk '{print $2,"/",$1}' | bc
I am looking to write a simple script to perform a SSH command on many hosts simultaneously, and which hosts exactly are generated from another script. The problem is that when I run the script using sometihng like sed it doesn't work properly.
It should run like sshall.sh {anything here} and it will run the {anything here} part on all the nodes in the list.
sshall.sh
#!/bin/bash
NODES=`listNodes | grep "node-[0-9*]" -o`
echo "Connecting to all nodes and running: ${#:1}"
for i in $NODES
do
:
echo "$i : Begin"
echo "----------------------------------------"
ssh -q -o "StrictHostKeyChecking no" $i "${#:1}"
echo "----------------------------------------"
echo "$i : Complete";
echo ""
done
When it is run with something like whoami it works but when I run:
[root#myhost bin]# sshall.sh sed -i '/^somebeginning/ s/$/,appendme/' /etc/myconfig.conf
Connecting to all nodes and running: sed -i /^somebeginning/ s/$/,appendme/ /etc/myconfig.conf
node-1 : Begin
----------------------------------------
sed: -e expression #1, char 18: missing command
----------------------------------------
node-1 : Complete
node-2 : Begin
----------------------------------------
sed: -e expression #1, char 18: missing command
----------------------------------------
node-2 : Complete
…
Notice that the quotes disappear on the sed command when sent to the remote client.
How do I go about fixing my bash command?
Is there a better way of achieving this?
Substitute an eval-safe quoted version of your command into a heredoc:
#!/bin/bash
# ^^^^- not /bin/sh; printf %q is an extension
# Put your command into a single string, with each argument quoted to be eval-safe
printf -v cmd_q '%q ' "$#"
while IFS= read -r hostname; do
# run bash -s remotely, with that string passed on stdin
ssh -q -o 'StrictHostKeyChecking no' "$hostname" "bash -s" <<EOF
$cmd_q
EOF
done < <(listNodes | grep -o -e "node-[0-9*]")
Why this works reliably (and other approaches don't):
printf %q knows how to quote contents to be eval'd by that same shell (so spaces, wildcards, various local quoting methods, etc. will always be supported).
Arguments given to ssh are not passed to the remote command individually!
Instead, they're concatenated into a string passed to sh -c.
However: The output of printf %q is not portable to all POSIX-derived shells! It's guaranteed to be compatible with the same shell locally in use -- ksh will always parse output from printf '%q' in ksh, bash will parse output from printf '%q' in bash, etc; thus, you can't safely pass this string on the remote argument vector, because it's /bin/sh -- not bash -- running there. (If you know your remote /bin/sh is provided by bash, then you can run ssh "$hostname" "$cmd_q" safely, but only under this condition).
bash -s reads the script to run from stdin, meaning that passing your command there -- not on the argument vector -- ensures that it'll be parsed into arguments by the same shell that escaped it to be shell-safe.
You want to pass the entire command -- with all of its arguments, spaces, and quotation marks -- to ssh so it can pass it unchanged to the remote shell for parsing.
One way to do that is to put it all inside single quotation marks. But then you'll also need to make sure the single quotation marks within your command are preserved in the arguments, so the remote shell builds the correct arguments for sed.
sshall.sh 'sed -i '"'"'/^somebeginning/ s/$/,appendme/'"'"' /etc/myconfig.conf'
It looks redundant, but '"'"' is a common Bourne trick to get a single quotation mark into a single-quoted string. The first quote ends single-quoting temporarily, the double-quote-single-quote-double-quote construct appends a single quotation mark, and then the single quotation mark resumes your single-quoted section. So to speak.
Another trick that can be helpful for troubleshooting is to add the -v flag do your ssh flags, which will spit out lots of text, but most importantly it will show you exactly what string it's passing to the remote shell for parsing and execution.
--
All of this is fairly fragile around spaces in your arguments, which you'll need to avoid, since you're relying on shell parsing on the opposite end.
Thinking outside the box: instead of dealing with all the quoting issues and the word-splitting in the wrong places, you could attempt to a) construct the script locally (maybe use a here-document?), b) scp the script to the remote end, then c) invoke it there. This easily allows more complex command sequences, with all the power of shell control constructs etc. Debugging (checking proper quoting) would be a breeze by simply looking at the generated script.
I recommend reading the command(s) from the standard input rather than from the command line arguments:
cmd.sh
#!/bin/bash -
# Load server_list with user#host "words" here.
cmd=$(</dev/stdin)
for h in ${server_list[*]}; do
ssh "$h" "$cmd"
done
Usage:
./cmd.sh <<'CMD'
sed -i '/^somebeginning/ s/$/,appendme/' /path/to/file1
# other commands
# here...
CMD
Alternatively, run ./cmd.sh, type the command(s), then press Ctrl-D.
I find the latter variant the most convenient, as you don't even need for here documents, no need for extra escaping. Just invoke your script, type the commands, and press the shortcut. What could be easier?
Explanations
The problem with your approach is that the quotes are stripped from the arguments by the shell. For example, the argument '/^somebeginning/ s/$/,appendme/' will be interpreted as /^somebeginning/ s/$/,appendme/ string (without the single quotes), which is an invalid argument for sed.
Of course, you can escape the command with the built-in printf as suggested in other answer here. But the command becomes not very readable after escaping. For example
printf %q 'sed -i /^somebeginning/ s/$/,appendme/ /home/ruslan/tmp/file1.txt'
produces
sed\ -i\ /\^somebeginning/\ s/\$/\,appendme/\ /home/ruslan/tmp/file1.txt
which is not very readable, and will look ugly, if you print it to the screen in order to show the progress.
That's why I prefer to read from the standard input and leave the command intact. My script prints the command strings to the screen, and I see them just in the form I have written them.
Note, the for .. in loop iterates $IFS-separated "words", and is generally not preferred way to traverse an array. It is generally better to invoke read -r in a while loop with adjusted $IFS. I have used the for loop for simplicity, as the question is really about invoking the ssh command.
Logging into multiple systems over SSH and using the same (or variations on the same) command is the basic use case behind ansible. The system is not without significant flaws, but for simple use cases is pretty great. If you want a more solid solution without too much faffing about with escaping and looping over hosts, take a look.
Ansible has a 'raw' module which doesn't even require any dependencies on the target hosts, and you might find that a very simple way to achieve this sort of functionality in a way that frees you from the considerations of looping over hosts, handling errors, marshalling the commands, etc and lets you focus on what you're actually trying to achieve.
I want to use awk to extract a single field from a list of records. For example,
Assignment1:/home/dir/:Admin:08-07-12
Assignment2:/home/dir/:Paul:09-22-13
I want to extract the 1st field from the second line, or the third field from the first line. Any ideas?
awk -F: 'NR == 2 { print $1 }'
awk -F: 'NR == 1 { print $3 }'
For the given line number, print the specified field.
It doesn't work
My apologies, I am using the Bourne shell and this does not work.
As noted in the comment, I don't believe that the Bourne shell has anything directly to do with the problem. (If $IFS is set to something odd, or some other peculiar setup applies, maybe that has an effect. But in a normal Bourne shell, there should be no problem.)
Here's what I get on my machine (an Ubuntu 14.04 derivative, but I'm confident I'd get the same result on Mac OS X 10.10.3, and pretty much any other Unix-like system, too). This is using Bash, but I don't think that's a factor — I'll go out on a limb and say that Korn shell, Zsh, Dash, and Heirloom Shell would all work the same on this; heck, it should work in the C shell family of shells too since there's nothing special about the notations used). It is using GNU awk too, but I don't think that's a factor either.
$ cat data
Assignment1:/home/dir/:Admin:08-07-12
Assignment2:/home/dir/:Paul:09-22-13
$ awk -F: 'NR == 2 { print $1 }' data
Assignment2
$ awk -F: 'NR == 1 { print $3 }' data
Admin
$
The output looks like record 2, field 1 and record 1, field 3 to me. Please find a way to demonstrate what you're doing and the result you get — probably, add something to the question. We can clean it up later when we've worked out what's going wrong with your setup. Please identify your platform reasonably clearly, too.
You can always use 'cut' from bash like so
cut -d ":" -f "1,3" < asdf
Assignment1:Admin
Assignment2:Paul
I'm writing a shell script that should be somewhat secure, i.e., does not pass secure data through parameters of commands and preferably does not use temporary files. How can I pass a variable to the standard input of a command?
Or, if it's not possible, how can I correctly use temporary files for such a task?
Passing a value to standard input in Bash is as simple as:
your-command <<< "$your_variable"
Always make sure you put quotes around variable expressions!
Be cautious, that this will probably work only in bash and will not work in sh.
Simple, but error-prone: using echo
Something as simple as this will do the trick:
echo "$blah" | my_cmd
Do note that this may not work correctly if $blah contains -n, -e, -E etc; or if it contains backslashes (bash's copy of echo preserves literal backslashes in absence of -e by default, but will treat them as escape sequences and replace them with corresponding characters even without -e if optional XSI extensions are enabled).
More sophisticated approach: using printf
printf '%s\n' "$blah" | my_cmd
This does not have the disadvantages listed above: all possible C strings (strings not containing NULs) are printed unchanged.
(cat <<END
$passwd
END
) | command
The cat is not really needed, but it helps to structure the code better and allows you to use more commands in parentheses as input to your command.
Note that the 'echo "$var" | command operations mean that standard input is limited to the line(s) echoed. If you also want the terminal to be connected, then you'll need to be fancier:
{ echo "$var"; cat - ; } | command
( echo "$var"; cat - ) | command
This means that the first line(s) will be the contents of $var but the rest will come from cat reading its standard input. If the command does not do anything too fancy (try to turn on command line editing, or run like vim does) then it will be fine. Otherwise, you need to get really fancy - I think expect or one of its derivatives is likely to be appropriate.
The command line notations are practically identical - but the second semi-colon is necessary with the braces whereas it is not with parentheses.
This robust and portable way has already appeared in comments. It should be a standalone answer.
printf '%s' "$var" | my_cmd
or
printf '%s\n' "$var" | my_cmd
Notes:
It's better than echo, reasons are here: Why is printf better than echo?
printf "$var" is wrong. The first argument is format where various sequences like %s or \n are interpreted. To pass the variable right, it must not be interpreted as format.
Usually variables don't contain trailing newlines. The former command (with %s) passes the variable as it is. However tools that work with text may ignore or complain about an incomplete line (see Why should text files end with a newline?). So you may want the latter command (with %s\n) which appends a newline character to the content of the variable. Non-obvious facts:
Here string in Bash (<<<"$var" my_cmd) does append a newline.
Any method that appends a newline results in non-empty stdin of my_cmd, even if the variable is empty or undefined.
I liked Martin's answer, but it has some problems depending on what is in the variable. This
your-command <<< """$your_variable"""
is better if you variable contains " or !.
As per Martin's answer, there is a Bash feature called Here Strings (which itself is a variant of the more widely supported Here Documents feature):
3.6.7 Here Strings
A variant of here documents, the format is:
<<< word
The word is expanded and supplied to the command on its standard
input.
Note that Here Strings would appear to be Bash-only, so, for improved portability, you'd probably be better off with the original Here Documents feature, as per PoltoS's answer:
( cat <<EOF
$variable
EOF
) | cmd
Or, a simpler variant of the above:
(cmd <<EOF
$variable
EOF
)
You can omit ( and ), unless you want to have this redirected further into other commands.
Try this:
echo "$variable" | command
If you came here from a duplicate, you are probably a beginner who tried to do something like
"$variable" >file
or
"$variable" | wc -l
where you obviously meant something like
echo "$variable" >file
echo "$variable" | wc -l
(Real beginners also forget the quotes; usually use quotes unless you have a specific reason to omit them, at least until you understand quoting.)