Iam fairly new here, and looking to learn more about bash programming.
So first I need some help with the finger command.
When I use just "finger" thats what I get as output, obviously with some data sets.
Login Name Tty Idle Login Time Where
What I want, is that I adapt the finger command so it only outputs the "Name" with its associated data sets, like that:
Name
...
You can use awk for this:
finger | awk '{print $2}'
Edit: New approach using a combination of awk and cut that's somewhat more robust to arbitrarily formatted names.
#!/bin/bash
#parse_finger.sh
#Read first line from stdin
IFS='$\n' read -r line
#Count the number of chars until 'Name'
str=$(echo "$line" | awk -F "Name" '{print $1}')
start=${#str}
start=$((start+1))
#Count the number of chars until 'Tty'
str=$(echo "$line" | awk -F "Tty" '{print $1}')
stop=${#str}
stop=$((stop-1))
#Print out the 'Name' header
echo "$line" | cut -c $start-$stop
#Read in the rest of our lines and print the cols we care about
while IFS='$\n' read -r line; do
echo "$line" | cut -c $start-$stop
done
Run it with finger | parse_finger.sh
Related
I want to print the longest and shortest username found in /etc/passwd. If I run the code below it works fine for the shortest (head -1), but doesn't run for (sort -n |tail -1 | awk '{print $2}). Can anyone help me figure out what's wrong?
#!/bin/bash
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n |head -1 | awk '{print $2}'
sort -n |tail -1 | awk '{print $2}'
Here the issue is:
Piping finishes with the first sort -n |head -1 | awk '{print $2}' command. So, input to first command is provided through piping and output is obtained.
For the second command, no input is given. So, it waits for the input from STDIN which is the keyboard and you can feed the input through keyboard and press ctrl+D to obtain output.
Please run the code like below to get desired output:
#!/bin/bash
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n |head -1 | awk '{print $2}'
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n |tail -1 | awk '{print $2}
'
All you need is:
$ awk -F: '
NR==1 { min=max=$1 }
length($1) > length(max) { max=$1 }
length($1) < length(min) { min=$1 }
END { print min ORS max }
' /etc/passwd
No explicit loops or pipelines or multiple commands required.
The problem is that you only have two pipelines, when you really need one. So you have grep | while read do ... done | sort | head | awk and sort | tail | awk: the first sort has an input (i.e., the while loop) - the second sort doesn't. So the script is hanging because your second sort doesn't have an input: or rather it does, but it's STDIN.
There's various ways to resolve:
save the output of the while loop to a temporary file and use that as an input to both sort commands
repeat your while loop
use awk to do both the head and tail
The first two involve iterating over the password file twice, which may be okay - depends what you're ultimately trying to do. But using a small awk script, this can give you both the first and last line by way of the BEGIN and END blocks.
While you already have good answers, you can also use POSIX shell to accomplish your goal without any pipe at all using the parameter expansion and string length provided by the shell itself (see: POSIX shell specifiction). For example you could do the following:
#!/bin/sh
sl=32;ll=0;sn=;ln=; ## short len, long len, short name, long name
while read -r line; do ## read each line
u=${line%%:*} ## get user
len=${#u} ## get length
[ "$len" -lt "$sl" ] && { sl="$len"; sn="$u"; } ## if shorter, save len, name
[ "$len" -gt "$ll" ] && { ll="$len"; ln="$u"; } ## if longer, save len, name
done </etc/passwd
printf "shortest (%2d): %s\nlongest (%2d): %s\n" $sl "$sn" $ll "$ln"
Example Use/Output
$ sh cketcpw.sh
shortest ( 2): at
longest (17): systemd-bus-proxy
Using either pipe/head/tail/awk or the shell itself is fine. It's good to have alternatives.
(note: if you have multiple users of the same length, this just picks the first, you can use a temp file if you want to save all names and use -le and -ge for the comparison.)
If you want both the head and the tail from the same input, you may want something like sed -e 1b -e '$!d' after you sort the data to get the top and bottom lines using sed.
So your script would be:
#!/bin/bash
grep -Eo '^([^:]+)' /etc/passwd |
while read NAME
do
echo ${#NAME} ${NAME}
done |
sort -n | sed -e 1b -e '$!d'
Alternatively, a shorter way:
cut -d":" -f1 /etc/passwd | awk '{ print length, $0 }' | sort -n | cut -d" " -f2- | sed -e 1b -e '$!d'
For example
echo "abc-1234a :" | grep <do-something>
to print only abc-1234a
I think these are closer to what you're getting at, but without knowing what you're really trying to achieve, it's hard to say.
echo "abc-1234a :" | egrep -o '^[^:]+'
... though this will also match lines that have no colon. If you only want lines with colons, and you must use only grep, this might work:
echo "abc-1234a :" | grep : | egrep -o '^[^:]+'
Of course, this only makes sense if your echo "abc-1234a :" is an example that would be replace with possibly multiple lines of input.
The smallest tool you could use is probably cut:
echo "abc-1234a :" | cut -d: -f1
And sed is always available...
echo "abc-1234a :" | sed 's/ *:.*//'
For this last one, if you only want to print lines that include a colon, change it to:
echo "abc-1234a :" | sed -ne 's/ *:.*//p'
Heck, you could even do this in pure bash:
while read line; do
field="${line%%:*}"
# do stuff with $field
done <<<"abc-1234a :"
For information on the %% bit, you can man bash and search for "Parameter Expansion".
UPDATE:
You said:
It's the characters in the first line of input before the colon. The
input could have multiple line though.
The solutions with grep probably aren't your best choice, then, since they'll also print data from subsequent lines that might include colons. Of course, there are many ways to solve this requirement as well. We'll start with sample input:
$ function sample { printf "abc-1234a:foo\nbar baz:\nNarf\n"; }
$ sample
abc-1234a:foo
bar baz:
Narf
You could use multiple pipes, for example:
$ sample | head -1 | grep -Eo '^[^:]*'
abc-1234a
$ sample | head -1 | cut -d: -f1
abc-1234a
Or you could use sed to process only the first line:
$ sample | sed -ne '1s/:.*//p'
abc-1234a
Or tell sed to exit after printing the first line (which is faster than reading the whole file):
$ sample | sed 's/:.*//;q'
abc-1234a
Or do the same thing but only show output if a colon was found (for safety):
$ sample | sed -ne 's/:.*//p;q'
abc-1234a
Or have awk do the same thing (as the last 3 examples, respectively):
$ sample | awk '{sub(/:.*/,"")} NR==1'
abc-1234a
$ sample | awk 'NR>1{nextfile} {sub(/:.*/,"")} 1'
abc-1234a
$ sample | awk 'NR>1{nextfile} sub(/:.*/,"")'
abc-1234a
Or in bash, with no pipes at all:
$ read line < <(sample)
$ printf '%s\n' "${line%%:*}"
abc-1234a
It is possible to do what you want with only sed.
Here is an example:
#!/bin/sh
filename=$1
pattern=yourpattern
# flag -n disables print everyline (default behavior)
sed -n "
1,/$pattern/ {
/$pattern/n # skip line containing pattern
p # print lines ranging from line 1 untill pattern
}
" $filename
exit 0
This works at least for GNU's sed. It should work for other sed too, except
regarding the comments (some implementations of sed don't support comments).
Source: https://www.grymoire.com/Unix/Sed.html
In the below code, the array length is 1.
Could anyone explain why, as grep output will displayed in each new line but when it is stored in the array, the array length will be 1.
How to display each line reading the array?
#!/bin/bash
NUM=()
SHORT_TEXT=()
LONG_TEXT=()
#cat /tmp/dummy2 |
while read NUM
do
LONG_TEXT+=$(grep $NUM -A4 RtpLogShm.Msg | grep -vi abate | grep ^LG)
done < /tmp/dummy2
#cat /tmp/dummy1 |
while read LINE
do
NUM+=$(echo $LINE | awk -F':' '{print $1}')
SHORT_TEXT+=$(echo $LINE | awk -F':' '{print $2}')
done < /tmp/dummy1
printf "[%s]\n" "${LONG_TEXT[#]}"
done
done
In bash, the syntax of appending to an array is (say we want to append an element stored in ${new_element} to an existing array ${array[#]}):
array=("${array[#]}" "${new_element}")
I have a text files with a line like this in them:
MC exp. sig-250-0 events & $0.98 \pm 0.15$ & $3.57 \pm 0.23$ \\
sig-250-0 is something that can change from file to file (but I always know what it is for each file). There are lines before and above this, but the string "MC exp. sig-250-0 events" is unique in the file.
For a particular file, is there a good way to extract the second number 3.57 in the above example using bash?
use awk for this:
awk '/MC exp. sig-250-0/ {print $10}' your.txt
Note that this will print: $3.57 - with the leading $, if you don't like this, pipe the output to tr:
awk '/MC exp. sig-250-0/ {print $10}' your.txt | tr -d '$'
In comments you wrote that you need to call it in a script like this:
while read p ; do
echo $p,awk '/MC exp. sig-$p/ {print $10}' filename | tr -d '$'
done < grid.txt
Note that you need a sub shell $() for the awk pipe. Like this:
echo "$p",$(awk '/MC exp. sig-$p/ {print $10}' filename | tr -d '$')
If you want to pass a shell variable to the awk pattern use the following syntax:
awk -v p="MC exp. sig-$p" '/p/ {print $10}' a.txt | tr -d '$'
More lines would've been nice but I guess you would like to have a simple use awk.
awk '{print $N}' $file
If you don't tell awk what kind of field-separator it has to use it will use just a space ' '. Now you just have to count how many fields you have got to get your field you want to get. In your case it would be 10.
awk '{print $10}' file.txt
$3.57
Don't want the $?
Pipe your awk result to cut:
awk '{print $10}' foo | cut -d $ -f2
-d will use the $ als field-separator and -f will select the second field.
If you know you always have the same number of fields, then
#!/bin/bash
file=$1
key=$2
while read -ra f; do
if [[ "${f[0]} ${f[1]} ${f[2]} ${f[3]}" == "MC exp. $key events" ]]; then
echo ${f[9]}
fi
done < "$file"
let's say my file /etc/passwd contains
ntp:x:38:40::/etc/ntp:/sbin/nologin
avahi:x:70:70:Avahi mDNS/DNS-SD Stack:/var/run/avahi-daemon:/sbin/nologin
haldaemon:x:38:68:HAL daemon:/:/sbin/nologin
pulse:x:497:495:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
gdm:x:42:38::/var/lib/gdm:/sbin/nologin
sshd:x:388:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
tcpdump:x:38:72::/:/sbin/nologin
what i'm trying to do is print the line containing a "38" in the third column, something which will print this:
ntp:x:38:40::/etc/ntp:/sbin/nologin haldaemon:x:38:68:HAL
daemon:/:/sbin/nologin gdm:x:42:38::/var/lib/gdm:/sbin/nologin
tcpdump:x:38:72::/:/sbin/nologin
I tried something like
cat "/etc/passwd" | cut -d ":" -f3 | grep "38"
but it only show the "38" not the entire line
Thanks
you may test this:
awk -F: '$3~/38/' /etc/passwd
note that 3rd column with 338 or 838 will be printed as well.
You could use grep
grep ^.*:.*:38: /etc/passwd
Improved version after tripleee's comment:
egrep ^[^:]*:[^:]*:38: /etc/passwd
You can use wk:
awk -F: '$3==38{print}' file
In general, I would suggest you avoid parsing /etc/passwd directly. Instead you can use getent passwdto read the passwd database.
You can do this:
cat /etc/passwd | egrep "^[[:alnum:]]*:[[:alnum:]]*:38:.*"
Using the alphanumeric character class.
In pure bash (awk is the way to go though!):
$ while read line; do array=(${line//:/ }); [ ${array[2]} -eq 38 ] && echo $line; done < input
ntp:x:38:40::/etc/ntp:/sbin/nologin
haldaemon:x:38:68:HAL daemon:/:/sbin/nologin
only sed was remaining :)
sed -n '/^[^:]*:[^:]:*38:/p' /etc/passwd