I have CentOS and this bash script:
#!/bin/sh
files=$( ls /vps_backups/site )
counter=0
for i in $files ; do
echo $i | grep -o -P '(?<=-).*(?=.tar)'
let counter=$counter+1
done
In the site folder I have compressed backups with the following names :
site-081916.tar.gz
site-082016.tar.gz
site-082116.tar.gz
...
The code above prints :
081916
082016
082116
I want to put each extracted date to a variable so I replaced this line
echo $i | grep -o -P '(?<=-).*(?=.tar)'
with this :
dt=$($i | grep -o -P '(?<=-).*(?=.tar)')
echo $dt
however I get this error :
./test.sh: line 6: site-090316.tar.gz: command not found
Any help please?
Thanks
you still need the echo inside the $(...):
dt=$(echo $i | grep -o -P '(?<=-).*(?=.tar)')
Don't use ls in a script. Use a shell pattern instead. Also, you don't need to use grep; bash has a built-in regular expression operator.
#!/bin/bash
files=$( /vps_backups/site/* )
counter=0
for i in "${files[#]#/vps_backups/site/}" ; do
[[ $i =~ -(.*).tar.gz ]] && dt=${BASH_REMATCH[1]}
counter=$((counter + 1))
done
Related
I have created a bash script that is used to modify the ulimit of open files in the RHEL server.
so i have reading the lines in the file /etc/security/limits.conf and if the soft/hard limit of the open files are less than 10000 for '*' domain i am commenting the line and adding a new line with soft/hard limit as 10000.
The Script is working as designed but the sed command to comment a line in the script is not working.
Please find the full script below :-
#!/bin/sh
#This script would be called by '' to set ulimit values for open files in unix servers.
#
configfile=/etc/security/limits.conf
help(){
echo "usage: $0 <LimitValue>"
echo -e "where\t--LimitValue= No of files you want all the users to open"
exit 1
}
modifyulimit()
{
grep '*\s*hard\s*nofile\s*' $configfile | while read -r line ; do
firstChar="$(echo $line | xargs | cut -c1-1)"
if [ "$firstChar" != "#" ];then
hardValue="$(echo $line | rev | cut -d ' ' -f1 | rev)"
if [[ "$hardValue" -ge "$1" ]]; then
echo ""
else
sed -i -e 's/$line/#$line/g' $configfile
echo "* hard nofile $1" >> $configfile
fi
else
echo ""
fi
done
grep '*\s*soft\s*nofile\s*' $configfile | while read -r line ; do
firstChar="$(echo $line | xargs | cut -c1-1)"
if [ "$firstChar" != "#" ];then
hardValue="$(echo $line | rev | cut -d ' ' -f1 | rev)"
if [[ "$hardValue" -ge "$1" ]]; then
echo ""
else
sed -i -e 's/$line/#$line/g' $configfile
echo "* hard nofile $1" >> $configfile
fi
else
echo ""
fi
done
}
deleteEofTag(){
sed -i "/\b\(End of file\)\b/d" $configfile
}
addEofTag()
{
echo "#################End of file###################" >> $configfile
}
#-------------Execution of the script starts here ----------------------
if [ $# -ne 1 ];
then
help
else
modifyulimit $1
deleteEofTag
addEofTag
fi
The command sed -i -e 's/$line/#$line/g' $configfile when executed from the terminal is working absolutely fine and it is commenting the line but it is not working when i am executing it from the unix shell script.
interpolation does not work in single quote
use double quote and try
sed -i -e 's/$line/#$line/g'
sed -i -e "s/$line/#$line/g"
also you might try:
sed -i -e s/${line}/#${line}/g
as this will tell the script to take the value of the variable instead of variable as such.
I'm writing a bash script which shall search in multiple files.
The problem I'm encountering is that I can't egrep an undetermined number of variables passed as parameters to the bash script
I want it to do the following:
Given a random number of parameters. i.e:
./searchline.sh A B C
Do a grep on the first one, and egrep the result with the rest:
grep "A" * | egrep B | egrep C
What I've tried to do is to build a string with the egreps:
for j in "${#:2}";
do
ADDITIONALSEARCH="$ADDITIONALSEARCH | egrep $j";
done
grep "$1" * "$ADDITIONALSEARCH"
But somehow that won't work, it seems like bash is not treating the "egrep" string as an egrep.
Do you guys have any advice?
By the way, as a side note, I'm not able to create any auxiliary file so grep -f is out of the line I guess. Also note, that the number of parameters passed to the bash script is variable, so I can't do egrep "$2" | egrep "$3".
Thanks in advance.
Fernando
You can use recursion here to get required number of pipes:
#!/bin/bash
rec_egrep() {
if [ $# -eq 0 ]; then
exec cat
elif [ $# -eq 1 ]; then
exec egrep "$1"
else
local pat=$1
shift
egrep "$pat" | rec_egrep "$#"
fi
}
first_arg="$1"
shift
grep "$first_arg" * | rec_egrep "$#"
A safe eval can be a good solution:
#!/bin/bash
if [[ $# -gt 0 ]]; then
temp=("grep" "-e" "\"\$1\"" "*")
for (( i = 2; i <= $#; ++i )); do
temp=("${temp[#]}" "|" "egrep" "-e" "\"\$$i\"")
done
eval "${temp[#]}"
fi
To run it:
bash script.sh A B C
I'm working on a script that takes commands from MySQL and executes them one by one. My problem is I can't execute the commands from script:
./bash.sh: line 26: /myscript.sh -c "": no such file or directory
Line 26 is the command I wish to execute ("$com"). if i try to run the command manually, by echoing the content of "$com" and than run it from terminal, it's working.
What am I doing wrong?
if [ ! "${#array[*]}" -eq "0" ]; then
for (( i=0 ; i<cnt ; i++ )); do
id=$(echo "${array[$i]}" | sed 's#\t#^#g' | cut -f'1' -d'^')
com=$(echo "${array[$i]}" | sed 's#\t#^#g' | cut -f'2' -d'^')
imp=$(echo "${array[$i]}" | sed 's#\t#^#g' | cut -f'3' -d'^')
if [[ "$id" = [0-9]* ]]; then
"$com"
echo "DELETE FROM list WHERE id='$id'" | mysql "$DB_USER" -u "$DB_USER" -p"$DB_PASS"
fi
done
else
echo "The list is empty"
fi
Added as Community Wiki
This question was solved in the comments by Vaughn Cato and the OP hasn't been seen since March.
The accept answer there was to use eval so the if statement in the script would be:
if [[ "$id" = [0-9]* ]]; then
eval "$com"
echo "DELETE FROM list WHERE id='$id'" | mysql "$DB_USER" -u "$DB_USER" -p"$DB_PASS"
fi
Change
"$com"
rather to
$com
(remember: too many quotes can be as harmful as too few)
than to
eval "$com"
it is unnecessarily complicated to use eval just to undo the unwanted effect of quoting.
I found similar questions but not in Linux/Bash
I want my script to create a file with a given name (via user input) but add number at the end if filename already exists.
Example:
$ create somefile
Created "somefile.ext"
$ create somefile
Created "somefile-2.ext"
The following script can help you. You should not be running several copies of the script at the same time to avoid race condition.
name=somefile
if [[ -e $name.ext || -L $name.ext ]] ; then
i=0
while [[ -e $name-$i.ext || -L $name-$i.ext ]] ; do
let i++
done
name=$name-$i
fi
touch -- "$name".ext
Easier:
touch file`ls file* | wc -l`.ext
You'll get:
$ ls file*
file0.ext file1.ext file2.ext file3.ext file4.ext file5.ext file6.ext
To avoid the race conditions:
name=some-file
n=
set -o noclobber
until
file=$name${n:+-$n}.ext
{ command exec 3> "$file"; } 2> /dev/null
do
((n++))
done
printf 'File is "%s"\n' "$file"
echo some text in it >&3
And in addition, you have the file open for writing on fd 3.
With bash-4.4+, you can make it a function like:
create() { # fd base [suffix [max]]]
local fd="$1" base="$2" suffix="${3-}" max="${4-}"
local n= file
local - # ash-style local scoping of options in 4.4+
set -o noclobber
REPLY=
until
file=$base${n:+-$n}$suffix
eval 'command exec '"$fd"'> "$file"' 2> /dev/null
do
((n++))
((max > 0 && n > max)) && return 1
done
REPLY=$file
}
To be used for instance as:
create 3 somefile .ext || exit
printf 'File: "%s"\n' "$REPLY"
echo something >&3
exec 3>&- # close the file
The max value can be used to guard against infinite loops when the files can't be created for other reason than noclobber.
Note that noclobber only applies to the > operator, not >> nor <>.
Remaining race condition
Actually, noclobber does not remove the race condition in all cases. It only prevents clobbering regular files (not other types of files, so that cmd > /dev/null for instance doesn't fail) and has a race condition itself in most shells.
The shell first does a stat(2) on the file to check if it's a regular file or not (fifo, directory, device...). Only if the file doesn't exist (yet) or is a regular file does 3> "$file" use the O_EXCL flag to guarantee not clobbering the file.
So if there's a fifo or device file by that name, it will be used (provided it can be open in write-only), and a regular file may be clobbered if it gets created as a replacement for a fifo/device/directory... in between that stat(2) and open(2) without O_EXCL!
Changing the
{ command exec 3> "$file"; } 2> /dev/null
to
[ ! -e "$file" ] && { command exec 3> "$file"; } 2> /dev/null
Would avoid using an already existing non-regular file, but not address the race condition.
Now, that's only really a concern in the face of a malicious adversary that would want to make you overwrite an arbitrary file on the file system. It does remove the race condition in the normal case of two instances of the same script running at the same time. So, in that, it's better than approaches that only check for file existence beforehand with [ -e "$file" ].
For a working version without race condition at all, you could use the zsh shell instead of bash which has a raw interface to open() as the sysopen builtin in the zsh/system module:
zmodload zsh/system
name=some-file
n=
until
file=$name${n:+-$n}.ext
sysopen -w -o excl -u 3 -- "$file" 2> /dev/null
do
((n++))
done
printf 'File is "%s"\n' "$file"
echo some text in it >&3
Try something like this
name=somefile
path=$(dirname "$name")
filename=$(basename "$name")
extension="${filename##*.}"
filename="${filename%.*}"
if [[ -e $path/$filename.$extension ]] ; then
i=2
while [[ -e $path/$filename-$i.$extension ]] ; do
let i++
done
filename=$filename-$i
fi
target=$path/$filename.$extension
Use touch or whatever you want instead of echo:
echo file$((`ls file* | sed -n 's/file\([0-9]*\)/\1/p' | sort -rh | head -n 1`+1))
Parts of expression explained:
list files by pattern: ls file*
take only number part in each line: sed -n 's/file\([0-9]*\)/\1/p'
apply reverse human sort: sort -rh
take only first line (i.e. max value): head -n 1
combine all in pipe and increment (full expression above)
Try something like this (untested, but you get the idea):
filename=$1
# If file doesn't exist, create it
if [[ ! -f $filename ]]; then
touch $filename
echo "Created \"$filename\""
exit 0
fi
# If file already exists, find a similar filename that is not yet taken
digit=1
while true; do
temp_name=$filename-$digit
if [[ ! -f $temp_name ]]; then
touch $temp_name
echo "Created \"$temp_name\""
exit 0
fi
digit=$(($digit + 1))
done
Depending on what you're doing, replace the calls to touch with whatever code is needed to create the files that you are working with.
This is a much better method I've used for creating directories incrementally.
It could be adjusted for filename too.
LAST_SOLUTION=$(echo $(ls -d SOLUTION_[[:digit:]][[:digit:]][[:digit:]][[:digit:]] 2> /dev/null) | awk '{ print $(NF) }')
if [ -n "$LAST_SOLUTION" ] ; then
mkdir SOLUTION_$(printf "%04d\n" $(expr ${LAST_SOLUTION: -4} + 1))
else
mkdir SOLUTION_0001
fi
A simple repackaging of choroba's answer as a generalized function:
autoincr() {
f="$1"
ext=""
# Extract the file extension (if any), with preceeding '.'
[[ "$f" == *.* ]] && ext=".${f##*.}"
if [[ -e "$f" ]] ; then
i=1
f="${f%.*}";
while [[ -e "${f}_${i}${ext}" ]]; do
let i++
done
f="${f}_${i}${ext}"
fi
echo "$f"
}
touch "$(autoincr "somefile.ext")"
without looping and not use regex or shell expr.
last=$(ls $1* | tail -n1)
last_wo_ext=$($last | basename $last .ext)
n=$(echo $last_wo_ext | rev | cut -d - -f 1 | rev)
if [ x$n = x ]; then
n=2
else
n=$((n + 1))
fi
echo $1-$n.ext
more simple without extension and exception of "-1".
n=$(ls $1* | tail -n1 | rev | cut -d - -f 1 | rev)
n=$((n + 1))
echo $1-$n.ext
I have few long commands that I will be using on a day to day basis. So I though it would be better to have a bash script where I could pass arguments, thus saving typing. I guess this is the norm in Linux but I am kind of new to it. Could someone show me how to do it. A example is the following command
cut -f <column_number> <filename> | sort | uniq -c |
sort -r -k1 -n | awk '{printf "%-15s %-10d\n", $2,$1}'
so i want this in a script where i can pass the filename and column number (preferably in any order) and get the desired ouput instead of having to type the whole thing everytime.
Create a file say myscript.sh -
#!/bin/bash
if [ $# -ne 2 ]; then
echo Usage: myscript.sh column_number file_path
exit
fi
if ! [ -f $2 ]; then
echo File doesnt exist
exit
fi
if [ `echo $1 | grep -E ^[0-9]+$ | wc -l` -ne 1 ]; then
echo First argument must be a number
exit
fi
cut -f 10 $1 $2 | sort | uniq -c |
sort -r -k1 -n | awk '{printf "%-15s %-10d\n", $2,$1}'
Make sure this file is executable using command chmod +x mytask.sh
You can invoke it like sh myscript.sh 30 myfile.sh or ./myscript.sh 30 myfile.sh
The first line of above script specifies the shell you would like your script to be executed in. $1 and $2 refer to the first and second command line arguments.
About argument validity checks:
First check ensures that there are exactly two arguments passed to the script.
Second check ensures the file pointed by the argument two is existing
Third check ensures that the number passed as first argument is really a number. It uses regular expression for that purpose. May be someone provide a better replacement for this check but this is what came to my mind instantly.
To accept the filename and column number in any order, you'll need to use option switches. Bash's getopts allows you to specify and process options so you can call your script using scriptname -f filename -c 12 or scriptname -c 12 -f filename for example.
#!/bin/bash
options=":f:c:"
while getopts $options option
do
case $option in
f)
filename=$OPTARG
;;
c)
col_num=$OPTARG
;;
\?)
usage_function # not shown
exit 1
;;
*)
echo "Invalid option"
usage_function
exit 1
;;
esac
done
shift $((OPTIND - 1))
if [[ -z $filename || -z $col_num ]]
then
echo "Missing option"
usage_function
exit 1
fi
if [[ $col_num == *[^0-9]* ]]
then
echo "Invalid integer"
usage_function
exit 1
fi
# other checks
cut -f 10 $col_num "$filename" | ...