eval: syntax error at line 1: `then' unexpected
Hi, I am having issues with this particular loop and couldn't find a solution to it, any ideas why?
getent passwd | cut -f1 -d":" | sort -n | uniq -c |\ while read x ; do [ -z "${x}" ] && break set - $x if [ $1 -gt 1 ]; then gids=`getent passwd |\ nawk -F: '($1 == n) { print $3 }' n=$2 | xargs` echo "Duplicate Group Name ($2): ${gids}" fi done
If you run the code through shellcheck and correct the errors which it shows (except for one problematic warning), the code will become:
getent passwd | cut -f1 -d":" | sort -n | uniq -c |
while read -r x ; do
[ -z "${x}" ] && break
set - $x
if [ "$1" -gt 1 ]; then
gids=$(getent passwd | nawk -F: '($1 == n) { print $3 }' n="$2" | xargs)
echo "Duplicate Group Name ($2): ${gids}"
fi
done
The code still seems to have issues, one of which is that it looks for duplicate user names but the print out claims that it found duplicate group names.
I would suggest replacing the above with:
getent passwd | awk -F: '{c[$1]++; uids[$1]=uids[$1]" "$3} END{for (name in c) if (c[name]>1) printf "Duplicate User Name (%s):%s\n",name, uids[name]}'
How the awk code works
In the output of getent passwd, the user name will be in field 1 and the user ID will be in field 3.
c[$1]++; uids[$1]=uids[$1]" "$3
This counts the number of times that user name $1 occurs and saves the count in array c. It also saves the user ID, $3, associated with each name in array uids.
END{for (name in c) if (c[name]>1) printf "Duplicate User Name (%s):%s\n",name, uids[name]}
After we have finished processing getent's output, this looks for user names for which the count is greater than 1 and prints the info.
Multiline version of suggested code
getent passwd | awk -F: '
{
c[$1]++
uids[$1] = uids[$1] " " $3
}
END{
for (name in c)
if (c[name]>1)
printf "Duplicate User Name (%s):%s\n", name, uids[name]
}'
Related
I am writing a function in a BASH shell script, that should return lines from csv-files with headers, having more commas than the header. This can happen, as there are values inside these files, that could contain commas. For quality control, I must identify these lines to later clean them up. What I have currently:
#!/bin/bash
get_bad_lines () {
local correct_no_of_commas=$(head -n 1 $1/$1_0_0_0.csv | tr -cd , | wc -c)
local no_of_files=$(ls $1 | wc -l)
for i in $(seq 0 $(( ${no_of_files}-1 )))
do
# Check that the file exist
if [ ! -f "$1/$1_0_${i}_0.csv" ]; then
echo "File: $1_0_${i}_0.csv not found!"
continue
fi
# Search for error-lines inside the file and print them out
echo "$1_0_${i}_0.csv has over $correct_no_of_commas commas in the following lines:"
grep -o -n '[,]' "$1/$1_0_${i}_0.csv" | cut -d : -f 1 | uniq -c | awk '$1 > $correct_no_of_commas {print}'
done
}
get_bad_lines products
get_bad_lines users
The output of this program is now all the comma-counts with all of the line numbers in all the files,
and I suspect this is due to the input $1 (foldername, i.e. products & users) conflicting with the call to awk with reference to $1 as well (where I wish to grab the first column being the count of commas for that line in the current file in the loop).
Is this the issue? and if so, would it be solvable by either referencing the 1.st column or the folder name by different variable names instead of both of them using $1 ?
Example, current output:
5 6667
5 6668
5 6669
5 6670
(should only show lines for that file having more than 5 commas).
Tried variable declaration in call to awk as well, with same effect
(as in the accepted answer to Awk field variable clash with function argument)
:
get_bad_lines () {
local table_name=$1
local correct_no_of_commas=$(head -n 1 $table_name/${table_name}_0_0_0.csv | tr -cd , | wc -c)
local no_of_files=$(ls $table_name | wc -l)
for i in $(seq 0 $(( ${no_of_files}-1 )))
do
# Check that the file exist
if [ ! -f "$table_name/${table_name}_0_${i}_0.csv" ]; then
echo "File: ${table_name}_0_${i}_0.csv not found!"
continue
fi
# Search for error-lines inside the file and print them out
echo "${table_name}_0_${i}_0.csv has over $correct_no_of_commas commas in the following lines:"
grep -o -n '[,]' "$table_name/${table_name}_0_${i}_0.csv" | cut -d : -f 1 | uniq -c | awk -v table_name="$table_name" '$1 > $correct_no_of_commas {print}'
done
}
You can use awk the full way to achieve that :
get_bad_lines () {
find "$1" -maxdepth 1 -name "$1_0_*_0.csv" | while read -r my_file ; do
awk -v table_name="$1" '
NR==1 { num_comma=gsub(/,/, ""); }
/,/ { if (gsub(/,/, ",", $0) > num_comma) wrong_array[wrong++]=NR":"$0;}
END { if (wrong > 0) {
print(FILENAME" has over "num_comma" commas in the following lines:");
for (i=0;i<wrong;i++) { print(wrong_array[i]); }
}
}' "${my_file}"
done
}
For why your original awk command failed to give only lines with too many commas, that is because you are using a shell variable correct_no_of_commas inside a single quoted awk statement ('$1 > $correct_no_of_commas {print}'). Thus there no substitution by the shell, and awk read "$correct_no_of_commas" as is, and perceives it as an undefined variable. More precisely, awk look for the variable correct_no_of_commas which is undefined in the awk script so it is an empty string . awk will then execute $1 > $"" as matching condition, and as $"" is a $0 equivalent, awk will compare the count in $1 with the full input line. From a numerical point of view, the full input line has the form <tab><count><tab><num_line>, so it is 0 for awk. Thus, $1 > $correct_no_of_commas will be always true.
You can identify all the bad lines with a single awk command
awk -F, 'FNR==1{print FILENAME; headerCount=NF;} NF>headerCount{print} ENDFILE{print "#######\n"}' /path/here/*.csv
If you want the line number also to be printed, use this
awk -F, 'FNR==1{print FILENAME"\nLine#\tLine"; headerCount=NF;} NF>headerCount{print FNR"\t"$0} ENDFILE{print "#######\n"}' /path/here/*.csv
I'm pretty new to bash scripting so some of the syntaxes may not be optimal. Please do point them out if you see one.
I have files in a directory named sequentially.
Example: prob01_01 prob01_03 prob01_07 prob02_01 prob02_03 ....
I am trying to have the script iterate through the current directory and count how many extensions each problem has. Then print the pre-extension name then count
Sample output for above would be:
prob01 3
prob02 2
This is my code:
#!/bin/bash
temp=$(mktemp)
element=''
count=0
for i in *
do
current=${i%_*}
if [[ $current == $element ]]
then
let "count+=1"
else
echo $element $count >> temp
element=$current
count=1
fi
done
echo 'heres the temp:'
cat temp
rm 'temp'
The Problem:
Current output:
prob1 3
Desired output:
prob1 3
prob2 2
The last count isn't appended because it's not seeing a different element after it
My Guess on possible solutions:
Have the last append occur at the end of the for loop?
Your code has 2 problems.
The first problem doesn't answer your question. You make a temporary file, the filename is stored in $temp. You should use that one, and not the file with the fixed name temp.
The problem is that you only write results when you see a new problem/filename. The last one will not be printed.
Fixing only these problems will result in
results() {
if (( count == 0 )); then
return
fi
echo $element $count >> "${temp}"
}
temp=$(mktemp)
element=''
count=0
for i in prob*
do
current=${i%_*}
if [[ $current == $element ]]
then
let "count+=1" # Better is using ((count++))
else
results
element=$current
count=1
fi
done
results
echo 'heres the temp:'
cat "${temp}"
rm "${temp}"
You can do without the script with
ls prob* | cut -d"_" -f1 | sort | uniq -c
When you want the have the output displayed as given, you need one more step.
ls prob* | cut -d"_" -f1 | sort | uniq -c | awk '{print $2 " " $1}'
You may use printf + awk solution:
printf '%s\n' *_* | awk -F_ '{a[$1]++} END{for (i in a) print i, a[i]}'
prob01 3
prob02 2
We use printf to print each file that has at least one _
We use awk to get a count of each file's first element delimited by _ by using an associative array.
I would do it like this:
$ ls | awk -F_ '{print $1}' | sort | uniq -c | awk '{print $2 " " $1}'
prob01 3
prob02 2
I don't know why I get an extra 0 when I run my script.
This is my script: I run a SQL query and save it ta an file valor.txt.
This is my array: array=(50 60 70)
Valor.txt:
count | trn_hst_id | trn_msg_host
-------+------------+--------------
11 | 50 | Aprobada
2 | 70 | Aprobada
(2 rows)
Code:
function service_status {
cd
cat valor.txt | grep $1 | gawk '{print $1}' FS="|" | sed "s/ //g"
if [ $? -eq 0 ]; then
echo -n 0
else
echo -n $1
fi
}
echo "<prtg>"
# <-- Start
for i in "${array[#]}"
do
echo -n " <result>
<channel>$i</channel>
<value>"
service_status $i
echo "</value>
</result>"
done
# End -->
echo "</prtg>"
exit
And this is my output.
<prtg>
<result>
<channel>50</channel>
<value>11
0</value>
</result>
<result>
<channel>60</channel>
<value>0</value>
</result>
<result>
<channel>70</channel>
<value>2
0</value>
</result>
</prtg>
Why do I get the 0 here? —
<value>2
0</value>
If I understand your comment correctly, you want to print the count. That is the value of the count column, if present in valor.txt, or 0 if the trn_hst_id in array is not in valor.txt. This should work (though not tested):
function service_status {
val=$(cat ~/valor.txt | grep $1 | gawk '{print $1}' FS="|" | sed "s/ //g")
# ^^ so you don't need to "cd" each time
# Save the value into "$val"
echo -n "${val:-0}" # If there is nothing in $val, print a 0
}
The "${val:-0}" sequence expands as "$val", if $val has text in it, or as a literal 0 otherwise. If the $1 wasn't in valor.txt, $val will be empty, so you will get a zero. See the wiki for more about how :- and friends work.
The "0" is the result of the echo -n 0 which is executed inside the function in case the awk command works properly (which is usually the case).
From the code it is not clear why is it written like it is. It's clear that is supposed to extract certain values from a file, but the 'if' condition seems to be checking the wrong thing, the return code of 'sed' which I bet is not what is intended. (better candidate would be the return code of 'grep'.
So I would write the function like this:
function service_status {
cd
var=$(cat valor.txt | grep $1 | gawk '{print $1}' FS="|" | sed "s/ //g")
if [ -z "$var" ]; then
echo -n 0
else
echo -n "$var"
fi
}
The variable 'var' will contain the result of the "search command". If the search would not return any value then 'var' will be empty and '0' will be the output of the function, otherwise the content of 'var' will be on the output.
i have this bash script here that i'm trying to modify to check if there is only one root id, is it vulnerable and currently, this script only checks if there is a duplicate uid and display the users that shares the same uid. Thanks in advance! :)
Bash Script:
#!/bin/bash
/bin/cat /etc/passwd| /bin/cut -f3 -d":" | /bin/sort -n | /usr/bin/uniq-c | while
read x ; do
[ -z "${x}" ] && break
set -$x
if [ $1 -gt1 ]; then
users=`/bin/gawk -F: '($3 == n) { print $1 }' n=$2 /etc/passwd| /usr/bin/xargs`
echo "Duplicate UID ($2): ${users}"
fi
done
Expected Output:
Audit criteria: There is only one root id
Vulnerability: Yes
Details: See below
root:!:0:0::/:/usr/bin/bash
jdoe:*:0:1:John Doe:/home/jdoe:/usr/bin/bash
You can simplify your script greatly because all you are looking for is user id 0, which is root:
#!/bin/bash
root_count=$(cut -f3 -d":" /etc/passwd | grep -wc 0)
if [[ $root_count > 1 ]]; then
users=$(awk -F: '($3 == 0) { print $1 }' /etc/passwd | xargs)
echo "Duplicate roots: ${users}"
fi
You can use awk to find that out:
if ! awk -F: '$3==0{c++}END{exit !(c<2)}' /etc/passwd ; then
echo "More than one user with uid 0"
fi
I have a file called names.txt that holds a list of names. Some of these names do not correspond to names in /etc/passwd (5th field) and some do. For the names in the file that have users with the name I want to print their user name. For example if the name Bill Gates was in the names.txt file and this line is in /etc/passwd bgates:x:23246:879:Bill Gates:/co/bgates:/bin/bash I would print out "Bill Gates exists and has the username 'bgates'"
This is what I've been trying, but it just prints out the entire /etc/passwd file.
while read name; do
if cut -d: -f5 '/etc/passwd' | grep -q "$name"; then
userName=$(cat /etc/passwd | cut -d: -f6)
echo "$name exists and has the username $userName"
else
echo "no such person '$line'"
fi
done < names.txt
Thank you
Maybe something like this?
#!/bin/bash
#set -x
set -eu
set -o pipefail
function get_pwent_by_name
{
full_name="$1"
while read pwent
do
pw_full_name=$(echo "$pwent" | awk -F':' ' { print $5 }')
if echo "$pw_full_name" | egrep -iq "$full_name"
then
echo "$pwent"
break
fi
done < /etc/passwd
}
while read name
do
pwent=$(get_pwent_by_name "$name")
if [ "$pwent" != "" ]
then
userName=$(echo "$pwent" | awk -F':' ' { print $1 }')
echo "$name exists and has the username $userName"
else
echo "No such person as $name"
fi
done < names.txt
Do you accept to use awk to resolve your problem?
awk -F: 'NR==FNR{a[$5]=$1;next}
{print ($0 in a)?$0 " exists and has the username " a[$0]:"no such person " $0}' /etc/passwd names.txt