cat file_name | grep "something" results "cat: grep: No such file or directory" in shell scripting - linux

I have written shell script which reads commands from input file and execute commands. I have command like:
cat linux_unit_test_commands | grep "dmesg"
in the input file. I am getting below error message while executing shell script:
cat: |: No such file or directory
cat: grep: No such file or directory
cat: "dmesg": No such file or directory
Script:
#!/bin/bash
while read line
do
output=`$line`
echo $output >> logs
done < $1
Below is input file(example_commands):
ls
date
cat linux_unit_test_commands | grep "dmesg"
Execute: ./linux_unit_tests.sh example_commands
Please help me to resolve this issue.

Special characters like | and " are not parsed after expanding variables; the only processing done after variable expansion is word splitting and wildcard expansions. If you want the line to be parsed fully, you need to use eval:
while read line
do
output=`eval "$line"`
echo "$output" >> logs
done < $1

You might be wondering why its not working with cat command.
Then here is the answer for your question.
output=`$line` i.e. output=`cat linux_unit_test_commands | grep "dmesg"`
here the cat command will take (linux_unit_test_commands | grep "dmesg") all these as arguments i.e. fileNames.
From Man page:
SYNTAX : cat [OPTION]... [FILE]...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Script is OK!
#!/bin/bash
while read line;
do
output=`$line`
echo $output >> logs
done < $1
To make it work you need to change 'cat: "dmesg": No such file or directory' to 'grep "dmesg" linux_unit_test_commands'. It will work!
cat linux_unit_test_commands
ls
date
grep "dmesg" linux_unit_test_commands

Related

Append in file a linux command

When doing
echo "some stuff" >> file.txt
I can append to my file some stuff
But when trying to do
echo ls >> file.txt
I append ls instead of listing all my items in the current path. How can I fix this? Sorry I'm new to linux
just redirect the ls output
ls >> file.txt
echo ls >> file.txt
This is interpreted as echo the term ls to file.txt. This isn't what you want.
>> redirects the output of a command, which can be ls.
So you could do:
ls >> file.txt

Bash: how to cleanly log processed lines of ssh/ bash output?

I wrote a linux bash script with tee and grep to log and timestamp the actions I take in my various ssh sessions. It works, but the logged lines are mixed together sometimes and are full of control characters. How can I properly escape control and other characters not visible in the original sessions and log each line separately?
I am learning bash and the linux interface, so any other suggestions to improve the script would be extremely welcome!
Here is my script (used as a wrapper for the ssh command):
#! /bin/bash
logfile=~/logs/ssh.log
desc="sshlog ${#}"
tab="\t"
format_line() {
while IFS= read -r line; do
echo -e "$(date +"%Y-%m-%d %H:%M:%S %z")${tab}${desc}${tab}${line}"
done
}
echo "[START]" | format_line >> ${logfile}
# grep is used to filter out command line output while keeping commands
ssh "$#" | tee >(grep -e '\#.*\:.*\$' --color=never --line-buffered | format_line >> ${logfile})
echo "[END]" | format_line >> ${logfile}
And here is a screenshot of the jarbled output in the log file:
A note on the solution: Tiago's answer took care of the nonprinting characters very well. Unfortunately, I just realized that the jumbling is being caused by backspaces and using the up and down keys for command completion. That is, the characters are being piped to grep as soon as they appear, and not line-by-line. I will have to ask about this in another question.
Update: I figured out a way to (almost always) handle up/down completion, backspace completion, and control characters.
You can remove those characters with:
perl -lpe 's/[^[:print:]]//g'
Not filtered:
perl -e 'for($i=0; $i<=255; $i++){print chr($i);}' | cat -A
^#^A^B^C^D^E^F^G^H^I$
^K^L^M^N^O^P^Q^R^S^T^U^V^W^X^Y^Z^[^\^]^^^_ !"#$%&'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~^?M-^#M-^AM-^BM-^CM-^DM-^EM-^FM-^GM-^HM-^IM-^JM-^KM-^LM-^MM-^NM-^OM-^PM-^QM-^RM-^SM-^TM-^UM-^VM-^WM-^XM-^YM-^ZM-^[M-^\M-^]M-^^M-^_M- M-!M-"M-#M-$M-%M-&M-'M-(M-)M-*M-+M-,M--M-.M-/M-0M-1M-2M-3M-4M-5M-6M-7M-8M-9M-:M-;M-<M-=M->M-?M-#M-AM-BM-CM-DM-EM-FM-GM-HM-IM-JM-KM-LM-MM-NM-OM-PM-QM-RM-SM-TM-UM-VM-WM-XM-YM-ZM-[M-\M-]M-^M-_M-`M-aM-bM-cM-dM-eM-fM-gM-hM-iM-jM-kM-lM-mM-nM-oM-pM-qM-rM-sM-tM-uM-vM-wM-xM-yM-zM-{M-|M-}M-~M-^?
Filtered:
perl -e 'for($i=0; $i<=255; $i++){print chr($i);}' | perl -lpe 's/[^[:print:]]//g' | cat -A
$
!"#$%&'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~$
Explanation:
I am printing the whole ASCII table with:
perl -e 'for($i=0; $i<=255; $i++){print chr($i);}'
I am identifying non printable chars with:
cat -A
I am filtering non printable chars with:
perl -lpe 's/[^[:print:]]//g'
Edit: It seems to me that you need to remove ANSI color chars:
Example:
perl -MTerm::ANSIColor -e 'print colored("yellow on_magenta","yellow on_magenta"),"\n"'| sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" | perl -lpe 's/[^[:print:]]//g'
Adapting to your code:
format_line() {
while IFS= read -r line; do
line=$(sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" <<< "$line")
line=$(perl -lpe 's/[^[:print:]]//g' <<< "$line")
echo -e "$(date +"%Y-%m-%d %H:%M:%S %z")${tab}${desc}${tab}${line}"
done
}
I also edited your grep command:
ssh "$#" | tee >(grep -Po '(?<=\$).*' --color=never --line-buffered | format_line >> ${logfile})
Below the output of my test:
2014-06-26 10:11:10 +0100 sshlog tiago#localhost [START]
2014-06-26 10:11:15 +0100 sshlog tiago#localhost whoami
2014-06-26 10:11:16 +0100 sshlog tiago#localhost exit
2014-06-26 10:11:16 +0100 sshlog tiago#localhost [END]
While writing your own script is a great learning experience, you can also use script to record everything printed on your terminal to a file.
The resulting file will still contains the control characters but there are multiple ways to get rid of them as described in How to clean up output of linux 'script' command.

working fine in command prompt but not in shell script

I have to take the field count from particular line in a zip file.
when I queried on command prompt in Linux it gives me output.
gunzip -c file | grep 'good' | awk -F' ' '{prinf NF}'
when execute this query on command line it gives a output 10 which is correct.
when I assigned this to a variable in shell script and execute .sh it gives me error
cat > find.sh
cnt=`gunzip -c file | grep 'good' | awk -F' ' '{print NF}'`
echo $cnt
./ sh find.sh
find.sh: 2: find sh: 10: not found
Please help out in this..!!
Try this:
cat find.sh
#!/bin/bash
cnt=$(gunzip -c file | awk '/good/ {prinf NF}')
echo $cnt
./find.sh
10

Shell script to log output of console

I want to grep the output of my script - which itself contains call to different binaries...
Since the script has multiple binaries within I can't simply put exec and dump the output in file (it does not copy output from the binaries)...
And to let you know, I am monitoring the script output to determine if the system has got stuck!
Why don't you append instead?
mybin1 | grep '...' >> mylog.txt
mybin2 | grep '...' >> mylog.txt
mybin3 | grep '...' >> mylog.txt
Does this not work?
#!/bin/bash
exec 11>&1 12>&2 > >(exec tee /var/log/somewhere) 2>&1 ## Or add -a option to tee to append.
# call your binaries here
exec >&- 2>&- >&11 2>&12 11>&- 12>&-

Bash Script : Unwanted Output

I have this simple bash script:
I run ns simulator on each file passed in argument where last argument is some text string to search for.
#!/bin/bash
nsloc="/home/ashish/ns-allinone-2.35/ns-2.35/ns"
temp="temp12345ashish.temp"
j=1
for file in "$#"
do
if [ $j -lt $# ]
then
let j=$j+1
`$nsloc $file > $temp 2>&1`
if grep -l ${BASH_ARGV[0]} $temp
then
echo "$file Successful"
fi
fi
done
I expected:
file1.tcl Successful
I am getting:
temp12345ashish.temp
file1.tcl Successful
When i run the simulator command myself on the terminal i do not get the file name to which output is directed.
I am not getting from where this first line of output is getting printed.
Please explain it.
Thanks in advance.
See man grep, and see specifically the explanation of the -l option.
In your script (above), you are using -l, so grep is telling you (as instructed) the filename where the match occurred.
If you don't want to see the filename, don't use -l, or use -q with it also. Eg:
grep -ql ${BASH_ARGV[0]} $temp
Just silence the grep:
if grep -l ${BASH_ARGV[0]} $temp &> /dev/null

Resources