I want to format output of a user account from /etc/passwd to display only the name, role, and directory path, all separated by commas. I know this should be easy, but for the life of me I cannot figure out how to display between certain colons. (Note this should work with any username, not just the ex)
Ex of grep joe /etc/passwd:
joe:x:1001:1001:System Admin:/home/joe:/bin/bash
Desired Output:
joe, System Admin, /home/joe
Thank you!
awk 'BEGIN{ FS=":"; OFS=", " } $1=="joe" { print $1,$5,$6; exit }' /etc/passwd
(but you should show a little more effort next time -- your question is very downvotable :))
With cut using comma as --output-delimiter:
cut -d: -f1,5,6 --output-delimiter=, /etc/passwd
Try:
grep joe /etc/passwd | cut -d: -f1,5,6
If you really need "," as a delimiter:
grep "^joe:" /etc/passwd | cut -d: -f1,5,6 | tr : ,
The "^" ensures only matches to "joe" at the beginning of the line are intercepted (Thanks PSkocik for the reminder). grep by defaults accepts a regex.
Translation in bash, but two lines:
x=`grep joe /etc/passwd | cut -d: -f1,5,6`
echo ${x/:/,}
Adding to the variations, in bash itself, you can pipe the output of grep directly to a brace enclosed read and separate the fields using IFS as required. Using short names for the fields, you could do:
grep '^joe:' /etc/passwd |
{ IFS=: read -r u x uid gid n h s; echo "$u, $n, $h"; }
Which would give the output you seek (if /etc/passwd contained the entry)
If you want to do it purely in shell (POSIX sh is enough, Bash not needed), then read is your friend:
while IFS=: read user pw uid gid gecos home shell ; do
if [ "$user" = "joe" ] ; then
echo "$user, $gecos, $home"
fi
done < /etc/passwd
Related
Iam fairly new here, and looking to learn more about bash programming.
So first I need some help with the finger command.
When I use just "finger" thats what I get as output, obviously with some data sets.
Login Name Tty Idle Login Time Where
What I want, is that I adapt the finger command so it only outputs the "Name" with its associated data sets, like that:
Name
...
You can use awk for this:
finger | awk '{print $2}'
Edit: New approach using a combination of awk and cut that's somewhat more robust to arbitrarily formatted names.
#!/bin/bash
#parse_finger.sh
#Read first line from stdin
IFS='$\n' read -r line
#Count the number of chars until 'Name'
str=$(echo "$line" | awk -F "Name" '{print $1}')
start=${#str}
start=$((start+1))
#Count the number of chars until 'Tty'
str=$(echo "$line" | awk -F "Tty" '{print $1}')
stop=${#str}
stop=$((stop-1))
#Print out the 'Name' header
echo "$line" | cut -c $start-$stop
#Read in the rest of our lines and print the cols we care about
while IFS='$\n' read -r line; do
echo "$line" | cut -c $start-$stop
done
Run it with finger | parse_finger.sh
This line:
echo $(grep Uid /proc/1/status) | cut -d ' ' -f 2
Produces output:
0
This line:
grep Uid /proc/1/status | cut -d ' ' -f 2
Produces output:
Uid: 0 0 0 0
My goal was the first output. My question is, why the second command does not produce the output I expected. Why am I required to echo it?
One way to do this is to change the Output Field Separator or OFS variable in the bash shell
IFSOLD="$IFS" # IFS for Internal field separator
IFS=$'\t'
grep 'Uid' /proc/1/status | cut -f 2
0 # Your result
IFS="$IFSOLD"
or the easy way
grep 'Uid' /proc/1/status | cut -d $'\t' -f 2
Note : By the way tab is the default delim for cut as pointed out [ here ]
Use awk
awk '/Uid/ { print $2; }' /proc/1/status
You should almost never need to write something like echo $(...) - it's almost equivalent to calling ... directly. Try echo "$(...)" (which you should always use) instead, and you'll see it behaves like ....
The reason is because when the $() command substitution is invoked without quotes the resulting string is split by Bash into separate arguments before being passed to echo, and echo outputs each argument separated by a single space, regardless of the whitespace generated by the command substitution (in your case tabs).
As sjsam suggested, if you want to cut tab-delimited output, just specify tabs as the delimiter instead of spaces:
cut -d $'\t' -f 2
grep Uid /proc/1/status |sed -r “s/\s+/ /g” | awk ‘{print $3}’
Output
0
I want to print the longest and shortest username found in /etc/passwd. If I run the code below it works fine for the shortest (head -1), but doesn't run for (sort -n |tail -1 | awk '{print $2}). Can anyone help me figure out what's wrong?
#!/bin/bash
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n |head -1 | awk '{print $2}'
sort -n |tail -1 | awk '{print $2}'
Here the issue is:
Piping finishes with the first sort -n |head -1 | awk '{print $2}' command. So, input to first command is provided through piping and output is obtained.
For the second command, no input is given. So, it waits for the input from STDIN which is the keyboard and you can feed the input through keyboard and press ctrl+D to obtain output.
Please run the code like below to get desired output:
#!/bin/bash
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n |head -1 | awk '{print $2}'
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n |tail -1 | awk '{print $2}
'
All you need is:
$ awk -F: '
NR==1 { min=max=$1 }
length($1) > length(max) { max=$1 }
length($1) < length(min) { min=$1 }
END { print min ORS max }
' /etc/passwd
No explicit loops or pipelines or multiple commands required.
The problem is that you only have two pipelines, when you really need one. So you have grep | while read do ... done | sort | head | awk and sort | tail | awk: the first sort has an input (i.e., the while loop) - the second sort doesn't. So the script is hanging because your second sort doesn't have an input: or rather it does, but it's STDIN.
There's various ways to resolve:
save the output of the while loop to a temporary file and use that as an input to both sort commands
repeat your while loop
use awk to do both the head and tail
The first two involve iterating over the password file twice, which may be okay - depends what you're ultimately trying to do. But using a small awk script, this can give you both the first and last line by way of the BEGIN and END blocks.
While you already have good answers, you can also use POSIX shell to accomplish your goal without any pipe at all using the parameter expansion and string length provided by the shell itself (see: POSIX shell specifiction). For example you could do the following:
#!/bin/sh
sl=32;ll=0;sn=;ln=; ## short len, long len, short name, long name
while read -r line; do ## read each line
u=${line%%:*} ## get user
len=${#u} ## get length
[ "$len" -lt "$sl" ] && { sl="$len"; sn="$u"; } ## if shorter, save len, name
[ "$len" -gt "$ll" ] && { ll="$len"; ln="$u"; } ## if longer, save len, name
done </etc/passwd
printf "shortest (%2d): %s\nlongest (%2d): %s\n" $sl "$sn" $ll "$ln"
Example Use/Output
$ sh cketcpw.sh
shortest ( 2): at
longest (17): systemd-bus-proxy
Using either pipe/head/tail/awk or the shell itself is fine. It's good to have alternatives.
(note: if you have multiple users of the same length, this just picks the first, you can use a temp file if you want to save all names and use -le and -ge for the comparison.)
If you want both the head and the tail from the same input, you may want something like sed -e 1b -e '$!d' after you sort the data to get the top and bottom lines using sed.
So your script would be:
#!/bin/bash
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n | sed -e 1b -e '$!d'
Alternatively, a shorter way:
cut -d":" -f1 /etc/passwd | awk '{ print length, $0 }' | sort -n | cut -d" " -f2- | sed -e 1b -e '$!d'
let's say my file /etc/passwd contains
ntp:x:38:40::/etc/ntp:/sbin/nologin
avahi:x:70:70:Avahi mDNS/DNS-SD Stack:/var/run/avahi-daemon:/sbin/nologin
haldaemon:x:38:68:HAL daemon:/:/sbin/nologin
pulse:x:497:495:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
gdm:x:42:38::/var/lib/gdm:/sbin/nologin
sshd:x:388:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
tcpdump:x:38:72::/:/sbin/nologin
what i'm trying to do is print the line containing a "38" in the third column, something which will print this:
ntp:x:38:40::/etc/ntp:/sbin/nologin haldaemon:x:38:68:HAL
daemon:/:/sbin/nologin gdm:x:42:38::/var/lib/gdm:/sbin/nologin
tcpdump:x:38:72::/:/sbin/nologin
I tried something like
cat "/etc/passwd" | cut -d ":" -f3 | grep "38"
but it only show the "38" not the entire line
Thanks
you may test this:
awk -F: '$3~/38/' /etc/passwd
note that 3rd column with 338 or 838 will be printed as well.
You could use grep
grep ^.*:.*:38: /etc/passwd
Improved version after tripleee's comment:
egrep ^[^:]*:[^:]*:38: /etc/passwd
You can use wk:
awk -F: '$3==38{print}' file
In general, I would suggest you avoid parsing /etc/passwd directly. Instead you can use getent passwdto read the passwd database.
You can do this:
cat /etc/passwd | egrep "^[[:alnum:]]*:[[:alnum:]]*:38:.*"
Using the alphanumeric character class.
In pure bash (awk is the way to go though!):
$ while read line; do array=(${line//:/ }); [ ${array[2]} -eq 38 ] && echo $line; done < input
ntp:x:38:40::/etc/ntp:/sbin/nologin
haldaemon:x:38:68:HAL daemon:/:/sbin/nologin
only sed was remaining :)
sed -n '/^[^:]*:[^:]:*38:/p' /etc/passwd
I am able to find the number of times a word occurs in a text file, like in Linux we can use:
cat filename|grep -c tom
My question is, how do I find the count of multiple words like "tom" and "joe" in a text file.
Since you have a couple names, regular expressions is the way to go on this one. At first I thought it was as simple as just a grep count on the regular expression of joe or tom, but fount that this did not account for the scenario where tom and joe are on the same line (or tom and tom for that matter).
test.txt:
tom is really really cool! joe for the win!
tom is actually lame.
$ grep -c '\<\(tom\|joe\)\>' test.txt
2
As you can see from the test.txt file, 2 is the wrong answer, so we needed to account for names being on the same line.
I then used grep -o to show only the part of a matching line that matches the pattern where it gave the correct pattern matches of tom or joe in the file. I then piped the results into number of lines into wc for the line count.
$ grep -o '\(joe\|tom\)' test.txt|wc -l
3
3...the correct answer! Hope this helps
Ok, so first split the file into words, then sort and uniq:
tr -cs '[:alnum:]' '\n' < testdata | sort | uniq -c
You use uniq:
sort filename | uniq -c
Use awk:
{for (i=1;i<=NF;i++)
count[$i]++
}
END {
for (i in count)
print count[i], i
}
This will produce a complete word frequency count for the input.
Pipe tho output to grep to get the desired fields
awk -f w.awk input | grep -E 'tom|joe'
BTW, you do not need cat in your example, most programs that acts as filters can take the filename as an parameter; hence it's better to use
grep -c tom filename
if not, there is a strong possibility that people will start throwing Useless Use of Cat Award at you ;-)
The sample you gave does not search for words "tom". It will count "atom" and "bottom" and many more.
Grep searches for regular expressions. Regular expression that matches word "tom" or "joe" is
\<\(tom\|joe\)\>
You could do regexp,
cat filename |tr ' ' '\n' |grep -c -e "\(joe\|tom\)"
Here is one:
cat txt | tr -s '[:punct:][:space:][:blank:]'| tr '[:punct:][:space:][:blank:]' '\n\n\n' | tr -s '\n' | sort | uniq -c
UPDATE
A shell script solution:
#!/bin/bash
file_name="$2"
string="$1"
if [ $# -ne 2 ]
then
echo "Usage: $0 <pattern to search> <file_name>"
exit 1
fi
if [ ! -f "$file_name" ]
then
echo "file \"$file_name\" does not exist, or is not a regular file"
exit 2
fi
line_no_list=("")
curr_line_indx=1
line_no_indx=0
total_occurance=0
# line_no_list contains loc k the line number loc k+1 the number
# of times the string occur at that line
while read line
do
flag=0
while [[ "$line" == *$string* ]]
do
flag=1
line_no_list[line_no_indx]=$curr_line_indx
line_no_list[line_no_indx+1]=$((line_no_list[line_no_indx+1]+1))
total_occurance=$((total_occurance+1))
# remove the pattern "$string" with a null" and recheck
line=${line/"$string"/}
done
# if we have entered the while loop then increment the
# line index to access the next array pos in the next
# iteration
if (( flag == 1 ))
then
line_no_indx=$((line_no_indx+2))
fi
curr_line_indx=$((curr_line_indx+1))
done < "$file_name"
echo -e "\nThe string \"$string\" occurs \"$total_occurance\" times"
echo -e "The string \"$string\" occurs in \"$((line_no_indx/2))\" lines"
echo "[Occurence # : Line Number : Nos of Occurance in this line]: "
for ((i=0; i<line_no_indx; i=i+2))
do
echo "$((i/2+1)) : ${line_no_list[i]} : ${line_no_list[i+1]} "
done
echo
I completely forgot about grep -f:
cat filename | grep -fc names
AWK solution:
Assuming the names are in a file called names:
cat filename | awk 'NR==FNR {h[NR] = $1;ct[i] = 0; cnt=NR} NR !=FNR {for(i=1;i<=cnt;++i) if(match($0,h[i])!=0) ++ct[i] } END {for(i in h) print h[i], ct[i]}' names -
Note that your original grep doesn't search for words. e.g.
$ echo tomorrow | grep -c tom
1
You need grep -w
gawk -vRS='[^[:alpha:]]+' '{print}' | grep -c '^(tom|joe|bob|sue)$'
The gawk program sets the record separator to anything non-alphabetic, so every word will end up on a separate line. Then grep counts lines that match one of the words you want exactly.
We use gawk because the POSIX awk doesn't allow regex record separator.
For brevity, you can replace '{print}' with 1 - either way, it's an Awk program that simply prints out all input records ("is 1 true? it is? then do the default action, which is {print}.")
To find all hits in all lines
echo "tom is really really cool! joe for the win!
tom is actually lame." | akw '{i+=gsub(/tom|joe/,"")} END {print i}'
3
This will count "tomtom" as 2 hits.