I am trying to execute the following command in a Bash script:
grep 1001 -w /etc/passwd | cut -d ':' -f 1,4,5
grep 1004 -w /etc/passwd | cut -d ':' -f 1,4,5
it works fine from the command line in Linux, and if I remove the latter portion of the pipeline it executes properly from Bash as well.
here is my script thus far:
#/bin/bash
#find the group number correlated to reader and user
reader=`grep reader /etc/group | cut -d ":" -f3`
user=`grep user /etc/group | cut -d ":" -f3`
echo reader: $reader #prints 1004
echo user: $user #prints 1001
cmdRead="grep ${reader} -w /etc/passwd | cut -d \":\" -f 1,4,5"
cmdUser="grep ${user} -w /etc/passwd | cut -d \":\" -f 1,4,5"
echo executing command: ${cmdRead}
echo `${cmdRead}`
echo executing command: ${cmdUser}
echo `${cmdUser}`
the output of this code yields:
reader: 1004
user: 1001
executing command: grep 1004 -w /etc/passwd | cut -d ":" -f 1,4,5
grep: invalid argument ‘":"’ for ‘--directories’
Valid arguments are:
- ‘read’
- ‘recurse’
- ‘skip’
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
executing command: grep 1001 -w /etc/passwd | cut -d ":" -f 1,4,5
grep: invalid argument ‘":"’ for ‘--directories’
Valid arguments are:
- ‘read’
- ‘recurse’
- ‘skip’
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
I only started learning Bash yesterday so I apologize for the noob-esque question but greatly appreciate any help :)
Enclose your command in $( ... ), not quotes. Also, no need to quote the colon as the value of the -f parameter in cut, therefore no need to escape the quotes:
cmdRead=$(grep ${reader} -w /etc/passwd | cut -d: -f 1,4,5)
Related
I'm trying to write a script that builds a list of nodes then ssh into the first node of that list
and runs a checknodes.sh script which it's self is just a for i loop that calls checknode.sh
The first 2 lines seems to work ok, the list builds successfully, but then I get either get just the echo line of checknodes.sh to print out or an error saying cat: gpcnodes.txt: No such file or directory
MYSCRIPT.sh:
#gets the master node for the job
MASTERNODE=`qstat -t -u \* | grep $1 | awk '{print$8}' | cut -d'#' -f 2 | cut -d'.' -f 1 | sed -e 's/$/.com/' | head -n 1`
#builds list of nodes in job
ssh -qt $MASTERNODE "qstat -t -u \* | grep $1 | awk '{print$8}' | cut -d'#' -f 2 | cut -d'.' -f 1 | sed -e 's/$/.com/' > /users/issues/slow_job_starts/gpcnodes.txt"
ssh -qt $MASTERNODE cd /users/issues/slow_job_starts/
ssh -qt $MASTERNODE /users/issues/slow_job_starts/checknodes.sh
checknodes.sh
for i in `cat gpcnodes.txt `
do
echo "### $i ###"
ssh -qt $i /users/issues/slow_job_starts/checknode.sh
done
checknode.sh
str=`hostname`
cd /tmp
time perf record qhost >/dev/null 2>&1 | sed -e 's/^/${str}/'
perf report --pretty=raw | grep % | head -20 | grep -c kernel.kallsyms | sed -e "s/^/`hostname`:/"
When ssh -qt $MASTERNODE cd /users/issues/slow_job_starts/ is finished, the changed directory is lost.
With the backquotes replaced by $(..) (not an error here, but get used to it), the script would be something like
for i in $(cat /users/issues/slow_job_starts/gpcnodes.txt)
do
echo "### $i ###"
ssh -nqt $i /users/issues/slow_job_starts/checknode.sh
done
or better
while read -r i; do
echo "### $i ###"
ssh -nqt $i /users/issues/slow_job_starts/checknode.sh
done < /users/issues/slow_job_starts/gpcnodes.txt
Perhaps you would also like to change your last script (start with cd /users/issues/slow_job_starts)
You will find more problems, like sed -e 's/^/${str}/' (the ${str} inside single quotes won't be replaced by a host), but this should get you started.
EDIT:
I added option -n to the ssh call.
Redirects stdin from /dev/null (actually, prevents reading from stdin).
Without this option only one node is checked.
I have created a shell script to execute 5 commands from a file called elist.txt
ps -ef | grep user | grep 'process -s 9000' | cut -c -15 | cut -c 10-
ps -ef | grep user | grep 'process -s 9001' | cut -c -15 | cut -c 10-
ps -ef | grep user | grep 'process -s 9002' | cut -c -15 | cut -c 10-
ps -ef | grep user | grep 'process -s 9003' | cut -c -15 | cut -c 10-
The shell script is as follows
export PATH="/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/user/.local/bin:/home/user/bin"
input="/home/user/script/list.txt"
while IFS= read -r line
do
echo $($line)
done < "$input"
output:
error: garbage option
Usage:
ps [options]
Try 'ps --help <simple|list|output|threads|misc|all>'
or 'ps --help <s|l|o|t|m|a>'
for additional help text.
For more details see ps(1).
You can do this too. People usually do not recommend to use eval at all.
export PATH="/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/user/.local/bin:/home/user/bin"
input="/home/user/script/list.txt"
while IFS= read -r line
do
bash -c "$line"
done < "$input"
Regards!
I would use this script with eval which is a similar solution described in this answer https://stackoverflow.com/a/6002329/6778826
export PATH="/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/user/.local/bin:/home/user/bin"
input="/home/user/script/list.txt"
while read -r line
do
eval "$line"
done <$input
I would like to create a perl or bash script that will read keyboard input and assign a variable, perform a fixed string grep recursively within the current directory filled with Snort logs, and then automatically tcpdump the matched files, grep its output, and print the specified lines to the terminal. Does anyone have a good idea of how this should work?
Here is an example of the methodology I want from the script:
step 1: Read keyboard input and assign it to variable named string.
step 2 command: grep -Fr "$string"
step 2 output: snort.log.1470609906 matches
step 3 command: tcpdump -r snort.log.1470609906 | grep -F "$string" C-10
step 3 output:
Snort log
Here's some bash code that does that:
s="google.com"
grep -Frl "$s" | \
while IFS= read -r x; do
tcpdump -r "$x" | grep -F "$s" -C10
done
idk about perl but you can do it easily enough just in shell:
str="google.com"
find . -type f -name 'snort.log.*' -exec grep -FlZ "$str" {} + |
xargs -0 -I {} sh -c 'tcpdump -r "{}" | grep -F '"$str"' -C10'
I have to concatenate a set of files. Directory structure is like this:
root/features/xxx/multiple_files... -> root/xxx/single_file
what i have written (and it works fine):
for dirname in $(ls -d root/features/*|awk -F/ '{print $NF}');do;mkdir root/${dirname};cat root/features/${dirname}/* > root/${dirname}/final.txt;done
But when i run the same thing via sh shell
/bin/sh -c "for dirname in $(ls -d root/features/*|awk -F/ '{print $NF}');do;mkdir root/${dirname};cat root/features/${dirname}/* > root/${dirname}/final.txt;done"
it gives me errors:
/bin/sh: -c: line 1: syntax error near unexpected token `201201000'
/bin/sh: -c: line 1: `201201000'
My process always appends /bin/sh -c before running any commands. Any suggestions what might be going wrong here? Any alternate ways? I have spent a really long time on this ,without making much headway!
EDIT:
`ls -d root/features/*|awk -F/ '{print $NF}' returns
201201
201201000
201201001
201201002
201201003
201201004
201201005
201201006
201201007
201202000
201205000
201206000
201207000
201207001
201207002
Always use sh -c 'cmd1 | cmd2' with single quotes.
Always use sh -eu -xv -c 'cmd1 | cmd2' to debug.
Always use bash -c 'cmd1 | cmd2' if your code is Bash-specific (cf. process substitution, ...).
Remove ; after do in for ... ; do; mkdir ....
Escape possible single quotes within single quotes like so: ' --> '\''.
(And sometimes just formatting your code clarifies a lot.)
Applied to your command this should look somewhat like this ...
# test version
/bin/sh -c '
for dirname in $(ls -d /* | awk -F/ '\''{print $NF}'\''); do
printf "%s\n" "mkdir root/${dirname}";
printf "%s\n" "cat root/features/${dirname}/* > root/${dirname}/final.txt";
echo
done
' | nl
# test version using 'printf' instead of 'ls'
sh -c '
printf "%s\000" /*/ | while IFS="" read -r -d "" file; do
dirname="$(basename "$file")"
printf "%s\n" "mkdir root/${dirname}";
printf "%s\n" "cat root/features/${dirname}/* > root/${dirname}/final.txt";
echo
done
' | nl
I got this to run in the little test environment I set up on my box. Turns out it didn't like the double quotes. The issue I ran into was the quotes around the awk statement...if you wrap it in double quotes it prints the whole thing.....I used cut to get the desired result, but my guess is you'll have to change the -f arg to 3 instead of 2..I think.
/bin/sh -c 'for dirname in $(ls -d sh_test/* | awk -F/ '\''{print $NF}'\''); do mkdir sh_test_root/${dirname}; cat sh_test/${dirname}/* > sh_test_root/${dirname}/final.txt;done'
edit: Tested edit proposed by nadu and it works fine. The above reflects that change.
I have started writing a small piece of code to print all the list of users available in the linux box. But I want to pass one by one user into my command to display each user details together.
to list all users
root#bt# getent passwd | grep /home/ | cut -d ':' -f 1
root
san
postgres
Now I want to pass one by user in to the below command to display each user details together.
root#bt# chage -l ${user1} ; chage -l ${user2} etcc.
should I need to user for loop or while loop here?
can any one help me in suggesting how to write the same?
You can use the while loop:
getent passwd | grep /home/ | cut -d ':' -f 1 | \
while read user ; do
chage -l "$user"
done
or the for loop:
for user in $(getent passwd | grep /home/ | cut -d ':' -f 1) ; do
chage -l "$user"
done
or xargs:
getent passwd | grep /home/ | cut -d ':' -f 1 | \
xargs -n1 chage -l
I would use xargs, which runs a command on each output item of the previous pipe:
getent passwd | grep /home/ | cut -d ':' -f 1 | sudo xargs -I % sh -c '{ echo "User: %"; chage -l %; echo;}'
sudo is used to get information about all users, if you don't have access to this information then you can remove sudo
-I % is used to specify that % is a placeholder for the input item (in your case a user)
sh -c '{ command1; command2; ...;}' is the command executed by xargs on every % item; in turn, the command sh -c allows multiple shell commands to be executed
'{ echo "User: %"; chage -l %; echo;}' echoes the current user in %, then runs chage -l on this user and finished with a final empty echo to format the ouput