I am trying to write a simple bash script that will print a multiline output to another file. I am doing it through heredoc format:
#!/bin/sh
echo "Hello!"
cat <<EOF > ~/Desktop/what.txt
a=`echo $1 | awk -F. '{print $NF}'`
b=`echo $2 | tr '[:upper:]' '[:lower:]'`
EOF
I was expecting to see a file in my desktop with these contents:
a=`echo $1 | awk -F. '{print $NF}'`
b=`echo $2 | tr '[:upper:]' '[:lower:]'`
But instead, I am seeing these as the contents of my what.txt file:
a=
b=
Somehow, even though it is part of a heredoc, bash is trying to execute it line by line. How do I prevent this, and print the contents to the file as it is?
Quote EOF so that bash takes inputs literally:
cat <<'EOF' > what.txt
a=`echo $1 | awk -F. '{print $NF}'`
b=`echo $2 | tr '[:upper:]' '[:lower:]'`
EOF
Also start using $() for command substitution instead of old and problematic ``.
Related
I am running the following command in a bash script:
echo `netstat -plten | grep -i autossh | awk '{print $4}'` >> /root/logs/autossh.txt
The output displays in a single line:
127.0.0.1:25001 127.0.0.1:15501 127.0.0.1:10001 127.0.0.1:20501 127.0.0.1:15001 127.0.0.1:5501 127.0.0.1:20001
I would like each IP to display line by line. What do I need to do with the awk command to make the output display line by line
Just remove the echo and subshell:
netstat -plten | grep -i autossh | awk '{print $4}' >> /root/logs/autossh.txt
awk is already printing them one per line, but when you pass them to echo it parses its arguments and prints them each with a space between them. Every line of awk output then becomes a separate argument to echo so you lose your line endings.
Of course, awk can do pattern matching too, so no real need for grep:
netstat -plten | awk '/autossh/ {print $4}' >> /root/logs/autossh.txt
with gawk at least you can have it ignore case too
netstat -plten | awk 'BEGIN {IGNORECASE=1} /autossh/ {print $4}' >> /root/logs/autossh.txt
or as Ed Morton pointed out, with any awk you could do
netstat -plten | awk 'tolower($0) ~ /autossh/ {print $4}' >> /root/logs/autossh.txt
You can just quote the result of command substitution to prevent the shell from performing word splitting.
You can modify it as follows to achieve what you want.
echo "`netstat -plten | grep -i autossh | awk '{print $4}'`" >> /root/logs/autossh.txt
Having trouble to wrap my head around piping and potential buffering issue. I am trying to perform set of operations piped that seem to break at some piping level. To simplify , I narrowed it down to 3 piping operations that do not work correctly
tail -f | awk '{print $1}' > file
results in no data redirected to the file , however
tail -f | awk '{print $1}'
results are output to stdout fine
also
tail -10 | awk '{print $1}' > file
works fine as well.
thinking it might be buffering issue, tried
tail -f | unbuffer awk '{print $1}' > file
what produced no positive results
(note: in original request, i have more operation in between using grep --line-buffer, but the problem was narrowed down to 3 piped commands tail -f | awk > file
The following will tail -f on a given file and whenever new data is added will automatically execute the while loop:
tail -f file_to_watch | while read a; do echo "$a" |awk '{print $1}' >> file; done
or more simply if you really only need to print the first field you could read it directly to your variable like this:
tail -f file_to_watch | while read a b; do echo "$a" >> file; done
Here is how to handle log files:
tail --follow=name logfile | awk '{print $1 | "tee /var/log/file"}'
or for you this may be ok:
tail -f | awk '{print $1 | "tee /var/log/file"}'
--follow=name this prevents stop of command while log file are rolled.
| "tee /var/log/file" this is used to get the output to the file.
Is it possible to cut a string without a line break?
printf 'test.test' prints the test.test without a newline.
But if I cut the output with printf 'test.test' | cut -d. -f1 there's a newline behind test.
There are many ways. In addition to isedev and fedorqui's answers, you could also do:
perl -ne '/^([^.]+)/ && print $1' <<< "test.test"
cut -d. -f1 <<< "test.test" | tr -d $'\n'
cut -d. -f1 <<< "test.test" | perl -pe 's/\n//'
while read -d. i; do printf "%s" "$i"; done <<< "test.test
No that I know. man cut is quite short and doesn't reflect anything similar.
Instead, you can provide the cut output to printf with a here-string, so that the new line issue depends again on printf:
printf '%s' $(cut -d. -f1 <<< "test.test")
If you don't have to use cut, you can achieve the same result with awk:
printf 'test.test' | awk -F. '{printf($1)}'
I'd like to read the text after characters in a file.
For example:
MPlayer-2013-08-30-i486|MPlayer|2013-08-30-i486||Multimedia;video|4508K||MPlayer-2013-08-30-i486.pet|+ffmpeg|mplayer video player|slackware|14.0||
I'd like to read the version of the program (in the third box):
2013-08-30-i486
How I can do this in my bash script?
This is pretty easily done with cut:
echo 'MPlayer-2013-08-30-i486|MPlayer|2013-08-30-i486||Multimedia;video|4508K||MPlayer-2013-08-30-i486.pet|+ffmpeg|mplayer video player|slackware|14.0||' | cut -d '|' -f 3
2013-08-30-i486
which will split on | and choose the 3rd field.
Using BASH regex:
s='MPlayer-2013-08-30-i486|MPlayer|2013-08-30-i486||Multimedia;video|4508K||MPlayer-2013-08-30-i486.pet|+ffmpeg|mplayer video player|slackware|14.0||'
[[ "$s" =~ MPlayer-([^|]+) ]] && echo "${BASH_REMATCH[1]}"
2013-08-30-i486
Using awk:
awk -F 'MPlayer-|\\|' '{print $2}' <<< "$s"
2013-08-30-i486
To grab 3rd field using awk:
awk -F '\\|' '{print $3}' <<< "$s"
2013-08-30-i486
This is simple to do in AWK:
$ awk -F'|' '{print $3}' file
2013-08-30-i486
It seems that the same data is repeated in several places, so I assume that they are all OK to use...In the above line, the input is being split into fields on the | character and the third field is being printed. The same thing will happen for every line of input.
Through grep,
$ grep -oP 'MPlayer-\K[^|.]*(?=\|)' file
2013-08-30-i486
Through sed,
$ echo 'MPlayer-2013-08-30-i486|MPlayer|2013-08-30-i486||Multimedia;video|4508K||MPlayer-2013-08-30-i486.pet|+ffmpeg|mplayer video player|slackware|14.0||' | sed -r 's/^[^|]+\|[^|]+\|([^|]+).*$/\1/'
2013-08-30-i486
Using read (all shells):
IFS='|' read __ __ VERSION __ < file
echo "$VERSION"
Another using read -a and Bash arrays:
IFS='|' read -a FIELDS < file
echo "${FIELDS[2]}"
Output:
2013-08-30-i486
The read built-in will be most efficient for a single line:
IFS="|" read __ __ version __ <<< "$line"
although if you are processing a file full of such lines with
while IFS="|" read __ __ version __; do
# do something with $version
done < file
it might be more efficient to use cut:
while read version; do
# do something with $version
done < <(cut -d'|' -f3 file)
or awk:
awk -F'|' '{ # do something with $3 }' file
I'm trying to understand why the command below doesn't work (output is empty):
echo 'aaa\tbbb' | awk -F '\\t' '{print $2}'
I would expect the output to be 'bbb'.
Interestingly this works (output is 'bbb'):
echo 'aaa\tbbb' | awk -F 't' '{print $2}'
And this works as well (ouptut is 'tbbb'):
echo 'aaa\tbbb' | awk -F '\\' '{print $2}'
It looks as if \\\t is read as backslash followed by tab instead of escaped backslash followed by t.
Is there a proper way to write this command?
You need to tell echo to interpret backslash escapes. Try:
$ echo -e 'aaa\tbbb' | awk -F '\t' '{print $2}'
bbb
man echo would tell:
-e enable interpretation of backslash escapes