I have files one is /etc/passwd that contains three new user methun, salam and kalam and have another file in /methunfiles/mypractice/myfile/passwd which contains input as methun:xxx salam:firstboy kalam:secondboy in a tow columns. first column contains methun, salam, kalam and second column contains xxx, firstboy, secondboy. Now my job is to matches the /etc/passwd files first column with the first column of /methunfiles/mypractice/myfile/passwd 's first colimn. If any matches found then insert the /etc/passwd 's comment field
with the second column of /methunfiles/mypractice/myfile/passwd file in same name found in first column of both. i have tried with the following code but no output found. I want to use loop here . Anybody help ? my output should like methun:x:501:502:xxx:......, salam:x:439:439:firstboy ...etc.
mainUser=cat /etc/passwd | awk -F ':' '{print $1}'
modifyUser=cat /methunfiles/mypractice/myfile/passwd | awk -F ':' '{print $1}'
modifyComment=cat /methunfiles/mypractice/myfile/passwd | awk -F ':' '{print $2}'
for muser in $mainUser
do
for moduser in $modifyUser
do
for mcomment in $modifyComment
do
if ["$muser" == "$moduser" ]
chmod -c "$mcomment" $muser
fi
done
done
done
the join command is what you need.
f1=/etc/passwd
f2=/methunfiles/mypractice/myfile/passwd
join -t: -j1 -o 2.1,2.2 <(sort -t: -k1,1 $f1) <(sort -t: -k1,1 $f2) |
while IFS=: read user new_comment; do
if usermod -c "$new_comment" $user; then
getent passwd $user
else
echo "could not modify comment field for $user"
fi
done
Related
i have a shell script
#/bin/bash
var1=`cat log.json | grep "accountnumber" | awk -F ' ' '{print $1}'`
echo $var
output of shell script is :-
23466
283483
324932
87374
I want match the above number which is already store in another file (below is the file format ) and print its value .
23466=account-1
283483=account-2
324932=account-3
87374=account-4
127632=account-5
1324237=account-6
73642=account-7
324993284=account-8
.
.
4543454=account-200
exapected output
account-1
account-2
account-3
account-4
a Compact one line solution can be:
join -t "=" <(sort bf) <(sort fa) | cut -d '=' -f 2
here fa is a file containing out-put of your bash script and bf is the file that has 23466=account-1 format
the output is:
account-1
account-2
account-3
account-4
#!/bin/bash
for var1 in $(awk -F ' ' '/accountnumber/{print $1}' log.json)
do
awk -F= '$1=="'"$var1"'"{print $2}' anotherfile
done
For a moment there was another answer that almost worked that I think is much slicker than what I wrote. Probably faster / more efficient on large files too. Here it is fixed.
awk -F ' ' '/accountnumber/{print $1}' log.json \
| sort -n \
| join -t= - accountfile \
| cut -d= -f2
I want to print the longest and shortest username found in /etc/passwd. If I run the code below it works fine for the shortest (head -1), but doesn't run for (sort -n |tail -1 | awk '{print $2}). Can anyone help me figure out what's wrong?
#!/bin/bash
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n |head -1 | awk '{print $2}'
sort -n |tail -1 | awk '{print $2}'
Here the issue is:
Piping finishes with the first sort -n |head -1 | awk '{print $2}' command. So, input to first command is provided through piping and output is obtained.
For the second command, no input is given. So, it waits for the input from STDIN which is the keyboard and you can feed the input through keyboard and press ctrl+D to obtain output.
Please run the code like below to get desired output:
#!/bin/bash
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n |head -1 | awk '{print $2}'
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n |tail -1 | awk '{print $2}
'
All you need is:
$ awk -F: '
NR==1 { min=max=$1 }
length($1) > length(max) { max=$1 }
length($1) < length(min) { min=$1 }
END { print min ORS max }
' /etc/passwd
No explicit loops or pipelines or multiple commands required.
The problem is that you only have two pipelines, when you really need one. So you have grep | while read do ... done | sort | head | awk and sort | tail | awk: the first sort has an input (i.e., the while loop) - the second sort doesn't. So the script is hanging because your second sort doesn't have an input: or rather it does, but it's STDIN.
There's various ways to resolve:
save the output of the while loop to a temporary file and use that as an input to both sort commands
repeat your while loop
use awk to do both the head and tail
The first two involve iterating over the password file twice, which may be okay - depends what you're ultimately trying to do. But using a small awk script, this can give you both the first and last line by way of the BEGIN and END blocks.
While you already have good answers, you can also use POSIX shell to accomplish your goal without any pipe at all using the parameter expansion and string length provided by the shell itself (see: POSIX shell specifiction). For example you could do the following:
#!/bin/sh
sl=32;ll=0;sn=;ln=; ## short len, long len, short name, long name
while read -r line; do ## read each line
u=${line%%:*} ## get user
len=${#u} ## get length
[ "$len" -lt "$sl" ] && { sl="$len"; sn="$u"; } ## if shorter, save len, name
[ "$len" -gt "$ll" ] && { ll="$len"; ln="$u"; } ## if longer, save len, name
done </etc/passwd
printf "shortest (%2d): %s\nlongest (%2d): %s\n" $sl "$sn" $ll "$ln"
Example Use/Output
$ sh cketcpw.sh
shortest ( 2): at
longest (17): systemd-bus-proxy
Using either pipe/head/tail/awk or the shell itself is fine. It's good to have alternatives.
(note: if you have multiple users of the same length, this just picks the first, you can use a temp file if you want to save all names and use -le and -ge for the comparison.)
If you want both the head and the tail from the same input, you may want something like sed -e 1b -e '$!d' after you sort the data to get the top and bottom lines using sed.
So your script would be:
#!/bin/bash
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n | sed -e 1b -e '$!d'
Alternatively, a shorter way:
cut -d":" -f1 /etc/passwd | awk '{ print length, $0 }' | sort -n | cut -d" " -f2- | sed -e 1b -e '$!d'
For example, I have below log files from the 16th-20th of Feb 2015. Now I want to create a single file named, mainentrywatcherReport_2015-02-16_2015-02-20.log. So in other words, I want to extract the date format from the first and last file of week (Mon-Fri) and create one output file every Saturday. I will be using cron to trigger the script every Saturday.
$ ls -l
mainentrywatcher_2015-02-16.log
mainentrywatcher_2015-02-17.log
mainentrywatcher_2015-02-18.log
mainentrywatcher_2015-02-19.log
mainentrywatcher_2015-02-20.log
$ cat *.log >> mainentrywatcherReport_2015-02-16_2015-02-20.log
$ mv *.log archive/
Can anybody help on how to rename the output file to above format?
Perhaps try this:
parta=`ls -l | head -n1 | cut -d'_' -f2 | cut -d'.' -f1`
partb=`ls -l | head -n5 | cut -d'_' -f2 | cut -d'.' -f1`
filename=mainentrywatcherReport_${parta}_${partb}.log
cat *.log >> ${filename}
"ls -l" output is described in the question
"head -nX" takes the Xth line of the output
"cut -d'_' -f2" takes everything (that remains) after the first underscore
"cut -d'.' -f1" times everything (that remains) before the first period
both commands are surrounded by ` marks (above tilde ~) to capture the output of the command to a variable
file name assembles the two dates stripped of the unnecessary with the other formatting desired for the final file name.
the cat command demonstrates one possible way to use the resulting filename
Happy coding! Leave a comment if you have any questions.
You can try this if you want to introduce simple looping...
FROM=ls -lrt mainentrywatcher_* | awk '{print $9}' | head -1 | cut -d"_" -f2 | cut -d"." -f1
TO=ls -lrt mainentrywatcher_* | awk '{print $9}' | tail -1 | cut -d"_" -f2 | cut -d"." -f1
FINAL_LOG=mainentrywatcherReport_${FROM}_${TO}.log
for i in ls -lrt mainentrywatcher_* | awk '{print $9}'
do
cat $i >> $FINAL_LOG
done
echo "All Logs Stored in $FINAL_LOG"
Another approach given your daily files and test contents as follows:
mainentrywatcher_2015-02-16.log -> a
mainentrywatcher_2015-02-17.log -> b
mainentrywatcher_2015-02-18.log -> c
mainentrywatcher_2015-02-19.log -> d
mainentrywatcher_2015-02-20.log -> e
That utilizes bash parameter expansion/substring extraction would be a simple loop:
#!/bin/bash
declare -i cnt=0 # simple counter to determine begin
for i in mainentrywatcher_2015-02-*; do # loop through each matching file
tmp=${i//*_/} # isolate date
tmp=${tmp//.*/}
[ $cnt -eq 0 ] && begin=$tmp || end=$tmp # assign first to begin, last to end
((cnt++)) # increment counter
done
cmbfname="${i//_*/}_${begin}_${end}.log" # form the combined logfile name
cat ${i//_*/}* > $cmbfname # cat all into combined name
## print out begin/end/cmbfname & contents to verify
printf "\nbegin: %s\nend : %s\nfname: %s\n\n" $begin $end $cmbfname
printf "contents: %s\n\n" $cmbfname
cat $cmbfname
exit 0
use/output:
alchemy:~/scr/tmp/stack/tmp> bash weekly.sh
begin: 2015-02-16
end : 2015-02-20
fname: mainentrywatcher_2015-02-16_2015-02-20.log
contents: mainentrywatcher_2015-02-16_2015-02-20.log
a
b
c
d
e
You can, of course, modify the for loop to accept a positional parameter containing the partial filename and pass the partial file name from the command line.
Something like this:
#!/bin/sh
LOGS="`echo mainentrywatcher_2[0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9].log`"
HEAD=
TAIL=
for logs in $LOGS
do
TAIL=`echo $logs | sed -e 's/^.*mainentrywatcher_//' -e 's/\.log$//'`
test -z "$HEAD" && HEAD=$TAIL
done
cat $LOGS >mainentrywatcherReport_${HEAD}_${TAIL}.log
mv $LOGS archive/
That is:
get a list of the existing logs (which happen to be sorted) in a variable $LOGS
walk through the list, getting just the date according to the example
save the first date as $HEAD
save the last date as $TAIL
after the loop, cat all of those files into the new output file
move the used-up log-files into the archive directory.
This is just an example of what I run into a lot:
I would like to copy all .bash_histories to one directory.
grep "/bin/bash" /etc/passwd | awk -F: '{ print "cp " $6"/.bash_history /backup" $6 ".bash_history" }
Output:
cp /home/peter/.bash_history /backup/home/peter/.bash_history
cp /home/john/.bash_history /backup/home/john/.bash_history
What I would like is an output like this:
cp /home/peter/.bash_history /backup/_home_peter_.bash_history
cp /home/john/.bash_history /backup/_home_john_.bash_history
And that this output will be executed.
(It's not specifically about this issue, but just in general how to reformat with awk and sed and execute the new created command line, without really creating a script for it)
The awk script to obtain a similar output will be
grep "/bin/bash" /etc/passwd |head -2 | awk -F: '{ print "cp " $6 "/.bash_history backup/_home_"$1".bash_history" }'
giving an output like
cp /root/.bash_history backup/_home_root.bash_history
cp /home/xxx/.bash_history backup/_home_xxx.bash_history
Now inorder to excecute the commands, the system() function within the awk would be helpfull
system(command) would excecute any command, and return value being the exit status of the command.
The above script can be modified as
grep "/bin/bash" /etc/passwd |head -2 | awk -F: '{ system("cp " $6 "/.bash_history backup/_home_"$1".bash_history;") }'
Test run:
$ grep "/bin/bash" /etc/passwd |head -2 | awk -F: '{ system("cp " $6 "/.bash_history backup/_home_"$1".bash_history;") }'
$ ls backup/
_home_xxx.bash_history _home_root.bash_history
PS: It is not recommend to create directories in your root folder. So i intentionally replaced /backup in your script to backup.
Also inorder for the script to be successful, the backup folder must be created before hand.
getent passwd | grep \/bin\/bash | cut -d ":" -f 6 | while read a; do eval "cp $a/.bash_history /backup/$(echo $a | sed 's#/#_#g')_.bash_history"; done
This uses getent to fetch the passwd file and cut gets the 6th field like your awk statement did, then it reads each entry line by line and builds the string and executes it with eval.
getent passwd | grep \/bin\/bash | cut -d ":" -f 6 | while read a; do eval "cp $a/.bash_history /backup/$(echo $a | sed 's#/#_#g')_.bash_history"; done
Worked perfectly! Issue solved!
I have a jar file, i need to execute the files in it in Linux.
So I need to get the result of the unzip -l command line by line.
I have managed to extract the files names with this command :
unzip -l package.jar | awk '{print $NF}' | grep com/tests/[A-Za-Z] | cut -d "/" -f3 ;
But i can't figure out how to obtain the file names one after another to execute them.
How can i do it please ?
Thanks a lot.
If all you need the first row in a column, add a pipe and get the first line using head -1
So your one liner will look like :
unzip -l package.jar | awk '{print $NF}' | grep com/tests/[A-Za-Z] | cut -d "/" -f3 |head -1;
That will give you first line
now, club head and tail to get second line.
unzip -l package.jar | awk '{print $NF}' | grep com/tests/[A-Za-Z] | cut -d "/" -f3 |head -2 | tail -1;
to get second line.
But from scripting piont of view this is not a good approach. What you need is a loop as below:
for class in `unzip -l el-api.jar | awk '{print $NF}' | grep javax/el/[A-Za-Z] | cut -d "/" -f3`; do echo $class; done;
you can replace echo $class with whatever command you wish - and use $class to get the current class name.
HTH
Here is my attempt, which also take into account Daddou's request to remove the .class extension:
unzip -l package.jar | \
awk -F'/' '/com\/tests\/[A-Za-z]/ {sub(/\.class/, "", $NF); print $NF}' | \
while read baseName
do
echo " $baseName"
done
Notes:
The awk command also handles the tasks of grep and cut
The awk command also handles the removal of the .class extension
The result of the awk command is piped into the while read... command
baseName represents the name of the class file, with the .class extension removed
Now, you can do something with that $baseName