This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 1 year ago.
good morning, I am a newbie in the world of scripts and I have this problem and I don't know why it happens to me and why I have it wrong, thanks in advance.
For example, I have this command that searches for users with X less letters:
cut -d: -f1 /etc/passwd | awk 'length($1) <= 4'
It works correctly but when I substitute a 4 for a variable with the same value it doesn't do it well:
number=4
echo -e $(cut -d: -f1 /etc/passwd | awk 'length($1) <= $number')
The same error happens to me here too, when I search for users who have an old password
awk -F: '{if($3<=18388)print$1}' < /etc/shadow
Works, but when I use the variable it stops working
variable=18388
awk -F: '{if($3<=$variable)print$1}' < /etc/shadow
Consider using awk's ability to import variables via the -v option, eg:
number=4
cut -d: -f1 /etc/passwd | awk -v num="${number}" 'length($1) <= num'
Though if the first field from /etc/passwd contains white space, and you want to consider the length of the entire field, you should replace $1 with $0, eg:
number=4
cut -d: -f1 /etc/passwd | awk -v num="${number}" 'length($0) <= num'
Even better, eliminate cut and the subprocess (due to the pipe) and use awk for the entire operation by designating the input field separator as a colon:
number=4
awk -F':' -v num="${number}" 'length($1) <= num { print $1 }' /etc/passwd
You need to use double quotes:
echo -e $(cut -d: -f1 /etc/passwd | awk "length(\$1) <= $number")
In single quotes, variables are not interpreted.
Updated: Here is an illustrative example:
> X=1; echo '$X'; echo "$X"
$X
1
Update 2: as rightfully pointed out in the comment, when using double quotes one need to make sure that the variables that are meant for awk to interpret are escaped, so they are not interpreted during script evaluation. Command updated above.
Alternatively, this task could be done using only a single GNU sed one-liner:
num=4
sed -E "s/:.*//; /..{$num}/d" /etc/passwd
Another one-liner, using only grep would be
grep -Po "^[^:]{1,$num}(?=:)" /etc/passwd
but this requires a grep supporting perl regular expressions, for the lookahead construct (?=...).
Related
All the users in the system that dosen't have as an ending character on their names a, s, t, r, m, z must be shown in Bash.
The users names can be obtained from the /etc/passwd file, in the first column. But I can not perceive the correct approach to exclude those characters from the search.
Should I use grep? Or just a cut?
Something like
grep -o '^[^:]*[^astrmz:]:' /etc/passwd | tr -d :
or
cut -d: -f1 /etc/passwd | grep '[^astrmz]$'
[^blah] matches any character but the ones listed, the opposite of [blah].
GNU grep using a lookahead:
grep -Po '^[^:]*[^astrmz:](?=:)' /etc/passwd
Or using awk instead:
awk -F: '$1 ~ /[^astrmz]$/ { print $1 }' /etc/passwd
Or in pure bash without external commands:
while IFS=: read -r name rest; do
if [[ $name =~ [^astrmz]$ ]]; then
echo "$name"
fi
done < /etc/passwd
As you can see, there's lots of potential approaches.
Simple one liner when using bash:
compgen -u | grep -v '[astrmz]$'
The compgen -u command will produce a list of users (without all of the extra fields present in /etc/passwd); compgen is a builtin in bash where it's normally used for username completion.
This should do the trick:
cut -d: -f1 /etc/passwd | grep -vE 's$|t$|r$|m$|z$'
The cut command strips out the username from the password file.
Then grep -v (does the UNMATCHING)
grep -E does multiple matching (OR OR OR)
the $ sign indicates the last character to match
For example , on my mac, I get:
_gamecontrollerd
_ondemand
_wwwproxy
_findmydevice
_ctkd
_applepay
_hidd
_analyticsd
_fpsd
_timed
_reportmemoryexception
(You see no names end with those 5 letters).
Good Luck.
Basically, I have a file formatted like
ABC:123
And I would like to flip the strings around the delimiter, so it would look like this
123:ABC
I would prefer to do this with bash/linux tools.
Thanks for any help!
That's reasonably easy with internal bash commands, assuming two fields, as per the following transcript:
pax:~$ x='abc:123'
pax:~$ echo "${x#*:}:${x%:*}"
123:abc
The first substitution ${x#*:} removes everything from the start up to the colon. The second, ${x%:*}, removes everything from the colon to the end.
Then you just re-join them with the colon in-between.
It doesn't matter for your particular data but % and # use the shortest possible pattern. The %% and ## variants will give you the longest possible pattern (greedy).
As an aside, this is ideal if you doing it for one string at a time since you don't need to kick up an external process to do the work for you. But, if you're processing an entire file, there are better ways to do it, such as with awk:
pax:~$ printf "abc:123\ndef:456\nghi:789\n" | awk -F: '{print $2 FS $1}'
123:abc
456:def
789:ghi
#!/bin/sh -x
var1=$(echo -e 'ABC:123' | cut -d':' -f1)
var2=$(echo -e 'ABC:123' | cut -d':' -f2)
echo -e "${var2}":"${var1}"
I use cut to split the string into two parts, and store both of those parts as variables.
From there, it's possible to use echo to re-arrange the variables as you see fit.
Using sed.
sed -E 's/(.*):(.*)/\2:\1/' file.txt
Using paste and cut with process substitution.
paste -d: <(cut -d : -f2 file.txt) <(cut -d : -f1 file.txt)
A slower/slowest shell solution on large set of data/files.
while IFS=: read -r left rigth; do printf '%s:%s\n' "$rigth" "$left"; done < file.txt
I have this bash statement for printing a specific cell from a .csv file.
set `cat $filename | awk -v FS=',' '{print $2}' | head -5 | tail -n 1`
The '{print $2}' part determines the column and the head -5 part determines the row.
Can I substitute a $counter variable in place of $2 (e.g., '{print $counter}')?
The answer is "yes" -- and there are a couple ways to do what you want. The proper way is to declare an awk variable using -v:
awk -F',' -v c=$counter 'NR==6 { print $c; exit }' "$filename"
(You will forgive me for moving some things around to do everything in awk, for passing "$filename" to awk safely, and for getting rid of set and back ticks -- that were doing nothing for the cause.)
Another way to do this is a bit of a "hackish" way -- leveraging shell quoting rules. This method requires some escaping to ensure that the first $ character (that references the intended field in awk) is not interpreted by the shell... The following works in bash (and POSIX sh):
awk -F',' "NR==6 { print \$$counter; exit }" "$filename"
Yes and all pipes could be removed. Variables are passed to awk with -v var=value.
Give a try to this tested version. Provide a value to the ̀€col and row variables:
set $(awk -F "," -v col=2 -v row=5 'NR==row {print $col; exit}' "${filename}")
$(command) is prefered to `command`, this later is deprecated.
NR is the current line number.
"${filename}" is expanded by the shell to its value: the double quotes will help if the filename contains some special chars.
I need a grep or another like command that will pull the usernames only from the /etc/passwd file in linux. Anything before the colon. I know this is doing with reg ex however I am not nearly experienced enough...
The following command will give all ACTUAL users, I need a way to pipe to grep or another line of code to only display the username portion.
awk -v LIMIT=500 -F: '{print $1}' '($3>=LIMIT) && ($3!=65534)' /etc/passwd
This should do:
awk -F: '$3>=LIMIT && $3!=65534 {print $1}' LIMIT=500 /etc/passwd
To do this in mostly plain bash:
limit=500
nfsnobody_id=65534
cut -d: -f1,3 /etc/passwd | while IFS=: read username uid; do
(( uid >= limit && uid != nfsnobody_id )) && echo $username
done
Get out of the habit of using VARNAMES_IN_CAPS: one day you'll write PATH=$(dirname $FILE) and then wonder why commands can no longer be found.
You simply need to rewrite the filter part of your awk command a bit. For instance, the following should work:
awk -v LIMIT=500 -F: '{if (($3>=LIMIT) && ($3!=65534)) print $1}' /etc/passwd
Otherwise, to answer your question strictly, if you want to use grep, the command would be
... | grep -o "^[^:]*"
but that would not be the way to go.
This question already has an answer here:
Shell script, saving the command value to a variable
(1 answer)
Closed 9 years ago.
Can anyone help me out with this problem?
I'm trying to save the awk output into a variable.
variable = `ps -ef | grep "port 10 -" | grep -v "grep port 10 -"| awk '{printf "%s", $12}'`
printf "$variable"
EDIT: $12 corresponds to a parameter running on that process.
Thanks!
#!/bin/bash
variable=`ps -ef | grep "port 10 -" | grep -v "grep port 10 -" | awk '{printf $12}'`
echo $variable
Notice that there's no space after the equal sign.
You can also use $() which allows nesting and is readable.
I think the $() syntax is easier to read...
variable=$(ps -ef | grep "port 10 -" | grep -v "grep port 10 -"| awk '{printf "%s", $12}')
But the real issue is probably that $12 should not be qouted with ""
Edited since the question was changed, This returns valid data, but it is not clear what the expected output of ps -ef is and what is expected in variable.
as noted earlier, setting bash variables does not allow whitespace between the variable name on the LHS, and the variable value on the RHS, of the '=' sign.
awk can do everything and avoid the "awk"ward extra 'grep'. The use of awk's printf is to not add an unnecessary "\n" in the string which would give perl-ish matcher programs conniptions. The variable/parameter expansion for your case in bash doesn't have that issue, so either of these work:
variable=$(ps -ef | awk '/port 10 \-/ {print $12}')
variable=`ps -ef | awk '/port 10 \-/ {print $12}'`
The '-' int the awk record matching pattern removes the need to remove awk itself from the search results.
variable=$(ps -ef | awk '/[p]ort 10/ {print $12}')
The [p] is a neat trick to remove the search from showing from ps
#Jeremy
If you post the output of ps -ef | grep "port 10", and what you need from the line, it would be more easy to help you getting correct syntax