I am trying to run a Shell Script, but got stuck on an issue. I want to Run certain set of code when i supply arguments and remaining should run, if i dont pass any argument.
Part which i want to run with args:
#!/bin/bash
while [[ "$1" != "" ]]; do
case "$1" in
-c ) cat /proc/cpuinfo | grep cores
;;
-d ) fdisk -l | grep Disk | awk '{print $1,$2,$3,$4}' #fdisk -l 2> /dev/null | grep Disk | grep -v identifier
;;
esac
shift
done
and this part without any args
while [[ $# -eq 0 ]]; do
echo PART 2 $#
cat /proc/cpuinfo | grep cores
fdisk -l | grep Disk | awk '{print $1,$2,$3,$4}' #fdisk -l 2> /dev/null | grep Disk | grep -v identifier
break
done
I believe issue is with the Loop condition, but i cant understand what?
if [[ -n "$1" ]]; then
#
# "$1" is not empty. This is the part which runs when one or more
# arguments are supplied.
#
while [[ -n "$1" ]]; do
case "$1" in
-c) cat /proc/cpuinfo | grep cores
;;
-d) LC_ALL=C fdisk -l | grep Disk | awk '{print $1,$2,$3,$4}'
#LC_ALL=C fdisk -l 2> /dev/null | grep Disk | grep -v identifier
;;
esac
shift
done
exit
fi
#
# "$1" is empty. The following code runs when no arguments are supplied.
#
echo PART 2 $#
cat /proc/cpuinfo | grep cores
LC_ALL=C fdisk -l | grep Disk | awk '{print $1,$2,$3,$4}'
#LC_ALL=C fdisk -l 2> /dev/null | grep Disk | grep -v identifier
Note 1: Not tested.
Note 2: Whenever you feel a need to parse the output of a command looking for certain words or phrases it is a good idea to run the command in the default locale by prefixing it with LC_ALL=C. In this way you won't be surprised when in a French locale, for example, fdisk says Disque...
Related
I'm trying to write a script that builds a list of nodes then ssh into the first node of that list
and runs a checknodes.sh script which it's self is just a for i loop that calls checknode.sh
The first 2 lines seems to work ok, the list builds successfully, but then I get either get just the echo line of checknodes.sh to print out or an error saying cat: gpcnodes.txt: No such file or directory
MYSCRIPT.sh:
#gets the master node for the job
MASTERNODE=`qstat -t -u \* | grep $1 | awk '{print$8}' | cut -d'#' -f 2 | cut -d'.' -f 1 | sed -e 's/$/.com/' | head -n 1`
#builds list of nodes in job
ssh -qt $MASTERNODE "qstat -t -u \* | grep $1 | awk '{print$8}' | cut -d'#' -f 2 | cut -d'.' -f 1 | sed -e 's/$/.com/' > /users/issues/slow_job_starts/gpcnodes.txt"
ssh -qt $MASTERNODE cd /users/issues/slow_job_starts/
ssh -qt $MASTERNODE /users/issues/slow_job_starts/checknodes.sh
checknodes.sh
for i in `cat gpcnodes.txt `
do
echo "### $i ###"
ssh -qt $i /users/issues/slow_job_starts/checknode.sh
done
checknode.sh
str=`hostname`
cd /tmp
time perf record qhost >/dev/null 2>&1 | sed -e 's/^/${str}/'
perf report --pretty=raw | grep % | head -20 | grep -c kernel.kallsyms | sed -e "s/^/`hostname`:/"
When ssh -qt $MASTERNODE cd /users/issues/slow_job_starts/ is finished, the changed directory is lost.
With the backquotes replaced by $(..) (not an error here, but get used to it), the script would be something like
for i in $(cat /users/issues/slow_job_starts/gpcnodes.txt)
do
echo "### $i ###"
ssh -nqt $i /users/issues/slow_job_starts/checknode.sh
done
or better
while read -r i; do
echo "### $i ###"
ssh -nqt $i /users/issues/slow_job_starts/checknode.sh
done < /users/issues/slow_job_starts/gpcnodes.txt
Perhaps you would also like to change your last script (start with cd /users/issues/slow_job_starts)
You will find more problems, like sed -e 's/^/${str}/' (the ${str} inside single quotes won't be replaced by a host), but this should get you started.
EDIT:
I added option -n to the ssh call.
Redirects stdin from /dev/null (actually, prevents reading from stdin).
Without this option only one node is checked.
I want to check if multiple lines in a file exist in bash.
so for that I use grep -q which works with only one line:
if grep -q string1 "/path/to/file";then
echo 'exists'
else
echo 'does not exist'
fi
I tried many things in various combinations, for example:
if grep -q [ string1 ] && grep -q [ string2 ] "path/to/file";then
I also tried it with -E:
grep -E 'pattern1' filename | grep -E 'pattern2'
but nothing seems to work. Any ideas?
Rather than running multiple grep commands you can use this gnu-awk command to assert presence of multiple strings in a file:
awk -v RS='\\Z' '/string1/ && /string2/ && /string3/{e=1} END{exit !e}' file &&
echo 'exists' || echo 'does not exist'
RS=\Z will make awk read all the input in a single record separator
Using && between multiple search terms will make sure all the search words exist in input file
This will print exists only if all 3 search terms exists in the input file.
since #iruvar hasn't posted his comment as answer, i'll put it here:
grep -q string_1 file && grep -q string_2 file
now, here is my contribution. is #anubhava's more computationally complex awk answer, which reads the file only once, any faster than #iruvar's simpler answer, which reads the file three times?
awk 11.730 s
grep && grep 0.258 s
no.
this surely will depend on the speed of the filesystem vs the cpu, and on how much caching goes on, but on my system, which is probably a typical B+/A- workstation, grep kw1 file && grep kw2 file && grep kw3 file is ~50x as fast as #anubhava's awk solution. this held true both on ssd and spindle raid. (details: test file was 5,000,000 lines, 160M, and had kw1 on the first line, kw2 on the 2.5 millionth, and kw3 on the 5 millionth.)
some easy optimization is possible, for example, if you can solve your problem by matching whole lines, do so (with grep -x); it's twice as fast in this case.
for many (e.g., >1,000) files, it is faster to use grep -l and xargs:
grep -l kw1 *.txt | xargs grep -l kw2 | xargs grep -q kw3
as opposed to a loop:
for f in *.txt; do
grep -q kw1 $f && grep -q kw2 $f && grep -q kw3 $f
done
with the same test file, grep -l | xargs grep took 0.258 s, just like grep && grep. with two test files, it was still no faster than grep && grep. with 2000 test files of 5,000 lines each, none of which contained any matches, grep -l | xargs grep was ~10x as faster as grep && grep.
There are a couple ambiguities in your question, but assuming you want pattern_1 and pattern_2 to exist in a file (not on the same line) then you can do this.
for file in *; do
egrep -q pattern_1 $file && egrep -q pattern_2 $file && echo $file
done
With grep -p you can match multiply patterns in the same line:
grep -P '(?=.*string1)(?=.*string2)' file
The above will print lines that matches string1 and string2.
(?=...) is a positive lookaheads which matches a pattern without making it a part of the match.
And -z will slurp the whole file:
% seq 1 100 | grep -qzP '(?=.*1)(?=.*5)'; echo $?
0
% seq 1 100 | grep -qzP '(?=.*1)(?=.*a)'; echo $?
1
You can do it like this:
if grep -q 'string1' /path/to/file; then
if grep -q 'string2' /path/to/file; then
echo exists
else
echo 'does not exist'
else
echo 'does not exist'
fi
Or:
grep -q 'string1' /path/to/file &&
grep -q 'string2' /path/to/file &&
echo exists ||
echo 'does not exist'
you can use "-q" to search using grep
if grep -q string1 "/path/to/file" && grep -q string2 "/path/to/file";then
echo 'exists'
else
echo 'does not exist'
fi
I've got two test files, namely, ttt.txt and ttt2.txt, the Content of which is shown as below:
#ttt.txt
(132) 123-2131
543-732-3123
238-3102-312
#ttt2.txt
1
2
3
I've already tried the following commands in bash and it works fine:
if grep -oE "(\(\d{3}\)[ ]?\d{3}-\d{4})|(\d{3}-\d{3}-\d{4})" ttt1.txt ; then echo "found"; fi
# with output 'found'
if grep -oE "(\(\d{3}\)[ ]?\d{3}-\d{4})|(\d{3}-\d{3}-\d{4})" ttt2.txt ; then echo "found"; fi
But when I combine the above command with xargs, it complains error '-bash: syntax error near unexpected token `then''. Could anyone give me some explanation? Thanks in advance!
ll | awk '{print $9}' | grep ttt | xargs -I $ if grep --quiet -oE "(\(\d{3}\)[ ]?\d{3}-\d{4})|(\d{3}-\d{3}-\d{4})" $; then echo "found"; fi
$ is a special character in bash (it marks variables) so don't use it as your xargs marker, you'll only get confused.
The real problem here though is that you are passing if grep --quiet -oE "(\(\d{3}\)[ ]?\d{3}-\d{4})|(\d{3}-\d{3}-\d{4})" $ as the argument to xargs, and then the remainder of the line is being treated as a new command, because it breaks at the ;.
You can wrap the whole thing in a sub-invocation of bash, so that xargs sees the whole command:
$ ll | awk '{print $9}' | grep ttt | xargs -I xx bash -c 'if grep --quiet -oE "(\(\d{3}\)[ ]?\d{3}-\d{4})|(\d{3}-\d{3}-\d{4})" xx; then echo "found"; fi'
found
Finally, ll | awk '{print $9}' | grep ttt is a needlessly complicated way of listing the files that you're looking for. You actually you don't need any of the code above, just do this:
$ if grep --quiet -oE "(\(\d{3}\)[ ]?\d{3}-\d{4})|(\d{3}-\d{3}-\d{4})" ttt*; then echo "found"; fi
found
Alternatively, if you want to process each file in turn (which you don't need here, but you might want when this gets more complicated):
for file in ttt*
do
if grep --quiet -oE "(\(\d{3}\)[ ]?\d{3}-\d{4})|(\d{3}-\d{3}-\d{4})" "$file"
then
echo "found"
fi
done
I bought a NAS box which has a cut down version of debian on it.
It ran out of space the other day and I did not realise. I am basically wanting to write a bash script that will alert me whenever the disk gets over 90% full.
Is anyone aware of a script that will do this or give me some advice on writing one?
#!/bin/bash
source /etc/profile
# Device to check
devname="/dev/sdb1"
let p=`df -k $devname | grep -v ^File | awk '{printf ("%i",$3*100 / $2); }'`
if [ $p -ge 90 ]
then
df -h $devname | mail -s "Low on space" my#email.com
fi
Crontab this to run however often you want an alert
EDIT: For multiple disks
#!/bin/bash
source /etc/profile
# Devices to check
devnames="/dev/sdb1 /dev/sda1"
for devname in $devnames
do
let p=`df -k $devname | grep -v ^File | awk '{printf ("%i",$3*100 / $2); }'`
if [ $p -ge 90 ]
then
df -h $devname | mail -s "$devname is low on space" my#email.com
fi
done
I tried to use Erik's answer but had issues with devices having long names which wraps the numbers and causes script to fail, also the math looked wrong to me and didn't match the percentages reported by df itself.
Here's an update to his script:
#!/bin/bash
source /etc/profile
# Devices to check
devnames="/dev/sda1 /dev/md1 /dev/mapper/vg1-mysqldisk1 /dev/mapper/vg4-ctsshare1 /dev/mapper/vg2-jbossdisk1 /dev/mapper/vg5-ctsarchive1 /dev/mapper/vg3-muledisk1"
for devname in $devnames
do
let p=`df -Pk $devname | grep -v ^File | awk '{printf ("%i", $5) }'`
if [ $p -ge 70 ]
then
df -h $devname | mail -s "$devname is low on space" my#email.com
fi
done
Key changes are changed df -k to df -Pk to avoid line wrapping and simplified the awk to use pre-calc'd percent instead of recalcing.
You could also use Monit for this kind of job. It's a "free open source utility for managing and monitoring, processes, programs, files, directories and filesystems on a UNIX system".
Based on #Erik answer, here is my version with variables :
#!/bin/bash
DEVNAMES="/ /home"
THRESHOLD=80
EMAIL=you#email.com
host=$(hostname)
for devname in $DEVNAMES
do
current=$(df $devname | grep / | awk '{ print $5}' | sed 's/%//g')
if [ "$current" -gt "$THRESHOLD" ] ; then
mail -s "Disk space alert on $host" "$EMAIL" << EOF
WARNING: partition $devname on $host is $current% !!
To list big files (>100Mo) :
find $devname -xdev -type f -size +100M
EOF
fi
done
And if you do not have the mail command on your server, you can send email via SMPT with swaks :
swaks --from "$EMAIL" --to "$EMAIL" --server "TheServer" --auth LOGIN --auth-user "TheUser" --auth-password "ThePasswrd" --h-Subject "Disk space alert on $host" --body - << EOF
#!/bin/bash
DEVNAMES=$(df --output=source | grep ^/dev)
THRESHOLD=90
EMAIL=your#email
HOST=$(hostname)
for devname in $DEVNAMES
do
current=$(df $devname | awk 'NR>1 {printf "%i",$5}')
[ "$current" -gt "$THRESHOLD" ] && warn="WARNING: partition $devname on $HOST is $current% !! \n$warn"
done
[ "$warn" ] && echo -e "$warn" | mail -s "Disk space alert on $HOST" $EMAIL
Based on previous answers, here's my version with following changes:
Automatically checks all mounted devices
Sends only one mail per check, regardless of how many devices are over the threshold
Code generally tidied up
How can I get the size of a file into a variable?
ls -l | grep testing.txt | cut -f6 -d' '
gave the size, but how can I store it in a shell variable?
filesize=$(stat -c '%s' testing.txt)
You can do it this way with ls (check the man page for the meaning of -s)
var=$(ls -s1 testing.txt | awk '{print $1}')
Or you can use stat with -c '%s'.
Or you can use find (GNU):
var=$(find testing.txt -printf "%s")
size() {
file="$1"
if [ -b "$file" ]; then
/sbin/blockdev --getsize64 "$file"
else
wc -c < "$file" # Handles pseudo files like /proc/cpuinfo
# stat --format %s "$file"
# find "$file" -printf '%s\n'
# du -b "$file" | cut -f1
fi
}
fs=$(size testing.txt)
size=`ls -l | grep testing.txt | cut -f6 -d' '`
You can get the file size in bytes with the command wc, which is fairly common on Linux systems since it's part of GNU coreutils:
wc -c < file
In a Bash script you can read it into a variable like this:
FILESIZE=$(wc -c < file)
From man wc:
-c, --bytes
print the byte counts
a=\`stat -c '%s' testing.txt\`;
echo $a