I am relatively new to bash scripting, and I have the following script, which is not giving me results I expect. So, my script looks like so:
#!/bin/bash
echo "Today is $(date)"
shopt -s nullglob
FILES=/some/empty/dir/with/no/text/files/*.txt
#echo $FILES
if [ -z "$FILES" ]
then
echo 'FILES variable is empty'
exit
else
echo 'FILES variable is not empty'
echo 'done' > write_file_out.dat
fi
So, the directory I am trying to use FILES on is completely empty - and still, when I do if [ -z "$FILES" ] it seems to say that it is not empty.
Baffled by this - wondering if someone can point me in the right direction.
Instead of:
FILES=/some/empty/dir/with/no/text/files/*.txt
You need to use:
FILES="$(echo /some/empty/dir/with/no/text/files/*.txt)"
Otherwise $FILES will be set to: /some/empty/dir/with/no/text/files/*.txt and this condition [ -z "$FILES" ] will always be false (not empty).
You can also use BASH arrays for shell expansion:
FILES=(/some/empty/dir/with/no/text/files/*.txt)
And check for:
[[ ${#FILES[#]} == 0 ]]
for empty check.
Related
I'm trying to learn the bash language. How do I execute a Linux command with a dynamic argument and check whether or not the returning string is empty. For example:
if ls "my_directory123" == `emtpy string` then
....
end
If you are testing for an empty directory pass in the first positional parameter "$1", you can test:
if test -z "$(ls -A "$1")" ; then
or
if [ -z "$(ls -A "$1")" ]; then
which are equivalent uses of the test of [ keywords.
Assign the result of the command to a variable and compare it as a string.
result=$(ls "my_directory123")
if [ "$result" = "" ]
then
echo empty
fi
TEST=`ls`
if [$TEST == ""]; then
echo "do something"
fi
if you want to run this in the terminal you key in each line at a time. If you prefer you can put everything in a file prepend with
#!/bin/bash
to make an executable string once you run
chmod +x FILENAME
Need to call Command Substitution and check for returned value. Some think like this
x=$(ls path)
if [ -z "$x" ] ;then echo empty ; else echo no empty; fi
Below is my script to check root path integrity, to ensure there is no vulnerability in PATH variable.
#! /bin/bash
if [ ""`echo $PATH | /bin/grep :: `"" != """" ]; then
echo "Empty Directory in PATH (::)"
fi
if [ ""`echo $PATH | /bin/grep :$`"" != """" ]; then echo ""Trailing : in PATH""
fi
p=`echo $PATH | /bin/sed -e 's/::/:/' -e 's/:$//' -e 's/:/ /g'`
set -- $p
while [ ""$1"" != """" ]; do
if [ ""$1"" = ""."" ]; then
echo ""PATH contains ."" shift
continue
fi
if [ -d $1 ]; then
dirperm=`/bin/ls -ldH $1 | /bin/cut -f1 -d"" ""`
if [ `echo $dirperm | /bin/cut -c6 ` != ""-"" ]; then
echo ""Group Write permission set on directory $1""
fi
if [ `echo $dirperm | /bin/cut -c9 ` != ""-"" ]; then
echo ""Other Write permission set on directory $1""
fi
dirown=`ls -ldH $1 | awk '{print $3}'`
if [ ""$dirown"" != ""root"" ] ; then
echo $1 is not owned by root
fi
else
echo $1 is not a directory
fi
shift
done
The script works fine for me, and shows all vulnerable paths defined in the PATH variable. I want to also automate the process of correctly setting the PATH variable based on the above result. Any quick method to do that.
For example, on my Linux box, the script gives output as:
/usr/bin/X11 is not a directory
/root/bin is not a directory
whereas my PATH variable have these defined,and so I want to have a delete mechanism, to remove them from PATH variable of root. lot of lengthy ideas coming in mind. But searching for a quick and "not so complex" method please.
No offense but your code is completely broken. Your using quotes in a… creative way, yet in a completely wrong way. Your code is unfortunately subject to pathname expansions and word splitting. And it's really a shame to have an insecure code to “secure” your PATH.
One strategy is to (safely!) split your PATH variable into an array, and scan each entry. Splitting is done like so:
IFS=: read -r -d '' -a path_ary < <(printf '%s:\0' "$PATH")
See my mock which and How to split a string on a delimiter answers.
With this command you'll have a nice array path_ary that contains each fields of PATH.
You can then check whether there's an empty field, or a . field or a relative path in there:
for ((i=0;i<${#path_ary[#]};++i)); do
if [[ ${path_ary[i]} = ?(.) ]]; then
printf 'Warning: the entry %d contains the current dir\n' "$i"
elif [[ ${path_ary[i]} != /* ]]; then
printf 'Warning: the entry %s is not an absolute path\n' "$i"
fi
done
You can add more elif's, e.g., to check whether the entry is not a valid directory:
elif [[ ! -d ${path_ary[i]} ]]; then
printf 'Warning: the entry %s is not a directory\n' "$i"
Now, to check for the permission and ownership, unfortunately, there are no pure Bash ways nor portable ways of proceeding. But parsing ls is very likely not a good idea. stat can work, but is known to have different behaviors on different platforms. So you'll have to experiment with what works for you. Here's an example that works with GNU stat on Linux:
read perms owner_id < <(/usr/bin/stat -Lc '%a %u' -- "${path_ary[i]}")
You'll want to check that owner_id is 0 (note that it's okay to have a dir path that is not owned by root; for example, I have /home/gniourf/bin and that's fine!). perms is in octal and you can easily check for g+w or o+w with bit tests:
elif [[ $owner_id != 0 ]]; then
printf 'Warning: the entry %s is not owned by root\n' "$i"
elif ((0022&8#$perms)); then
printf 'Warning: the entry %s has group or other write permission\n' "$i"
Note the use of 8#$perms to force Bash to understand perms as an octal number.
Now, to remove them, you can unset path_ary[i] when one of these tests is triggered, and then put all the remaining back in PATH:
else
# In the else statement, the corresponding entry is good
unset_it=false
fi
if $unset_it; then
printf 'Unsetting entry %s: %s\n' "$i" "${path_ary[i]}"
unset path_ary[i]
fi
of course, you'll have unset_it=true as the first instruction of the loop.
And to put everything back into PATH:
IFS=: eval 'PATH="${path_ary[*]}"'
I know that some will cry out loud that eval is evil, but this is a canonical (and safe!) way to join array elements in Bash (observe the single quotes).
Finally, the corresponding function could look like:
clean_path() {
local path_ary perms owner_id unset_it
IFS=: read -r -d '' -a path_ary < <(printf '%s:\0' "$PATH")
for ((i=0;i<${#path_ary[#]};++i)); do
unset_it=true
read perms owner_id < <(/usr/bin/stat -Lc '%a %u' -- "${path_ary[i]}" 2>/dev/null)
if [[ ${path_ary[i]} = ?(.) ]]; then
printf 'Warning: the entry %d contains the current dir\n' "$i"
elif [[ ${path_ary[i]} != /* ]]; then
printf 'Warning: the entry %s is not an absolute path\n' "$i"
elif [[ ! -d ${path_ary[i]} ]]; then
printf 'Warning: the entry %s is not a directory\n' "$i"
elif [[ $owner_id != 0 ]]; then
printf 'Warning: the entry %s is not owned by root\n' "$i"
elif ((0022 & 8#$perms)); then
printf 'Warning: the entry %s has group or other write permission\n' "$i"
else
# In the else statement, the corresponding entry is good
unset_it=false
fi
if $unset_it; then
printf 'Unsetting entry %s: %s\n' "$i" "${path_ary[i]}"
unset path_ary[i]
fi
done
IFS=: eval 'PATH="${path_ary[*]}"'
}
This design, with if/elif/.../else/fi is good for this simple task but can get awkward to use for more involved tests. For example, observe that we had to call stat early before the tests so that the information is available later in the tests, before we even checked that we're dealing with a directory.
The design may be changed by using a kind of spaghetti awfulness as follows:
for ((oneblock=1;oneblock--;)); do
# This block is only executed once
# You can exit this block with break at any moment
done
It's usually much better to use a function instead of this, and return from the function. But because in the following I'm also going to check for multiple entries, I'll need to have a lookup table (associative array), and it's weird to have an independent function that uses an associative array that's defined somewhere else…
clean_path() {
local path_ary perms owner_id unset_it oneblock
local -A lookup
IFS=: read -r -d '' -a path_ary < <(printf '%s:\0' "$PATH")
for ((i=0;i<${#path_ary[#]};++i)); do
unset_it=true
for ((oneblock=1;oneblock--;)); do
if [[ ${path_ary[i]} = ?(.) ]]; then
printf 'Warning: the entry %d contains the current dir\n' "$i"
break
elif [[ ${path_ary[i]} != /* ]]; then
printf 'Warning: the entry %s is not an absolute path\n' "$i"
break
elif [[ ! -d ${path_ary[i]} ]]; then
printf 'Warning: the entry %s is not a directory\n' "$i"
break
elif [[ ${lookup[${path_ary[i]}]} ]]; then
printf 'Warning: the entry %s appears multiple times\n' "$i"
break
fi
# Here I'm sure I'm dealing with a directory
read perms owner_id < <(/usr/bin/stat -Lc '%a %u' -- "${path_ary[i]}")
if [[ $owner_id != 0 ]]; then
printf 'Warning: the entry %s is not owned by root\n' "$i"
break
elif ((0022 & 8#$perms)); then
printf 'Warning: the entry %s has group or other write permission\n' "$i"
break
fi
# All tests passed, will keep it
lookup[${path_ary[i]}]=1
unset_it=false
done
if $unset_it; then
printf 'Unsetting entry %s: %s\n' "$i" "${path_ary[i]}"
unset path_ary[i]
fi
done
IFS=: eval 'PATH="${path_ary[*]}"'
}
All this is really safe regarding spaces and glob characters and newlines inside PATH; the only thing I don't really like is the use of the external (and non-portable) stat command.
I'd recommend you get a good book on Bash shell scripting. It looks like you learned Bash from looking at 30 year old system shell scripts and by hacking away. This isn't a terrible thing. In fact, it shows initiative and great logic skills. Unfortunately, it leads you down to some really bad code.
If statements
In the original Bourne shell the [ was a command. In fact, /bin/[ was a hard link to /bin/test. The test command was a way to test certain aspects of a file. For example test -e $file would return a 0 if the $file was executable and a 1 if it wasn't.
The if merely took the command after it, and would run the then clause if that command returned an exit code of zero, or the else clause (if it exists) if the exit code wasn't zero.
These two are the same:
if test -e $file
then
echo "$file is executable"
fi
if [ -e $file ]
then
echo "$file is executable"
fi
The important idea is that [ is merely a system command. You don't need these with the if:
if grep -q "foo" $file
then
echo "Found 'foo' in $file"
fi
Note that I am simply running grep and if grep is successful, I'm echoing my statement. No [ ... ] are necessary.
A shortcut to the if is to use the list operators && and ||. For example:
grep -q "foo" $file && echo "I found 'foo' in $file"
is the same as the above if statement.
Never parse ls
You should never parse the ls command. You should use stat instead. stat gets you all the information in the command, but in an easily parseable form.
[ ... ] vs. [[ ... ]]
As I mentioned earlier, in the original Bourne shell, [ was a system command. In Kornshell, it was an internal command, and Bash carried it over too.
The problem with [ ... ] is that the shell would first interpolate the command before the test was performed. Thus, it was vulnerable to all sorts of shell issues. The Kornshell introduced [[ ... ]] as an alternative to the [ ... ] and Bash uses it too.
The [[ ... ]] allows Kornshell and Bash to evaluate the arguments before the shell interpolates the command. For example:
foo="this is a test"
bar="test this is"
[ $foo = $bar ] && echo "'$foo' and '$bar' are equal."
[[ $foo = $bar ]] && echo "'$foo' and '$bar' are equal."
In the [ ... ] test, the shell interpolates first which means that it becomes [ this is a test = test this is ] and that's not valid. In [[ ... ]] the arguments are evaluated first, thus the shell understands it's a test between $foo and $bar. Then, the values of $foo and $bar are interpolated. That works.
For loops and $IFS
There's a shell variable called $IFS that sets how read and for loops parse their arguments. Normally, it's set to space/tab/NL, but you can modify this. Since each PATH argument is separated by :, you can set IFS=:", and use a for loop to parse your $PATH.
The <<< Redirection
The <<< allows you to take a shell variable and pass it as STDIN to the command. These both more or less do the same thing:
statement="This contains the word 'foo'"
echo "$statement" | sed 's/foo/bar/'
statement="This contains the word 'foo'"
sed 's/foo/bar/'<<<$statement
Mathematics in the Shell
Using ((...)) allows you to use math and one of the math function is masking. I use masks to determine whether certain bits are set in the permission.
For example, if my directory permission is 0755 and I and it against 0022, I can see if user read and write permissions are set. Note the leading zeros. That's important, so that these are interpreted as octal values.
Here's your program rewritten using the above:
#! /bin/bash
grep -q "::" <<<"$PATH" && echo "Empty directory in PATH ('::')."
grep -q ":$" <<<$PATH && "PATH has trailing ':'"
#
# Fix Path Issues
#
path=$(sed -e 's/::/:/g' -e 's/:$//'<<<$PATH);
OLDIFS="$IFS"
IFS=":"
for directory in $PATH
do
[[ $directory == "." ]] && echo "Path contains '.'."
[[ ! -d "$directory" ]] && echo "'$directory' isn't a directory in path."
mode=$(stat -L -f %04Lp "$directory") # Differs from system to system
[[ $(stat -L -f %u "$directory") -eq 0 ]] && echo "Directory '$directory' owned by root"
((mode & 0022)) && echo "Group or Other write permission is set on '$directory'."
done
I'm not 100% sure what you want to do or mean about PATH Vulnerabilities. I don't know why you care whether a directory is owned by root, and if an entry in the $PATH is not a directory, it won't affect the $PATH. However, one thing I would test for is to make sure all directories in your $PATH are absolute paths.
[[ $directory != /* ]] && echo "Directory '$directory' is a relative path"
The following could do the whole work and also removes duplicate entries
export PATH="$(perl -e 'print join(q{:}, grep{ -d && !((stat(_))[2]&022) && !$seen{$_}++ } split/:/, $ENV{PATH})')"
I like #kobame's answer but if you don't like the perl-dependency you can do something similar to:
$ cat path.sh
#!/bin/bash
PATH="/root/bin:/tmp/groupwrite:/tmp/otherwrite:/usr/bin:/usr/sbin"
echo "${PATH}"
OIFS=$IFS
IFS=:
for path in ${PATH}; do
[ -d "${path}" ] || continue
paths=( "${paths[#]}" "${path}" )
done
while read -r stat path; do
[ "${stat:5:1}${stat:8:1}" = '--' ] || continue
newpath="${newpath}:${path}"
done < <(stat -c "%A:%n" "${paths[#]}" 2>/dev/null)
IFS=${OIFS}
PATH=${newpath#:}
echo "${PATH}"
$ ./path.sh
/root/bin:/tmp/groupwrite:/tmp/otherwrite:/usr/bin:/usr/sbin
/usr/bin:/usr/sbin
Note that this is not portable due to stat not being portable but it will work on Linux (and Cygwin). For this to work on BSD systems you will have to adapt the format string, other Unices don't ship with stat at all OOTB (Solaris, for example).
It doesn't remove duplicates or directories not owned by root either but that can easily be added. The latter only requires the loop to be adapted slightly so that stat also returns the owner's username:
while read -r stat owner path; do
[ "${owner}${stat:5:1}${stat:8:1}" = 'root--' ] || continue
newpath="${newpath}:${path}"
done < <(stat -c "%A:%U:%n" "${paths[#]}" 2>/dev/null)
May be a novice question but anyways in my intro to linux/unix class were touching on bash scripting and in one of the problems I got the it tasked me with making a script so if the user searched to a name in a file that wasn't there it would output a messaged saying 'your_input is not in the directory'
It says to use if statements and the exit status $?.
So far I got the input portion but I'm not sure how to properly use the $? in a if statement if its possible.
#!/bin/bash
name=$1
if [ "$name" = "" ]
then echo -n "Enter a name to search for: "
read name
fi
grep -i $name ~uli101/2014c/phonebook
if [ "$?" < "0" ]
then echo "error"
fi
I get the error:
./phone4: line 14: 0: No such file or directory
My question is: How can I use the $? with and if statement, and If I can't, can you explain me how to use the $? in this problem?
Note: I did use echo $? to see how $? gave a 0 if grep worked and a 1 if it didn't.
There's two bugs in it. The one you already see is that in the [] expression, the < is interpreted not as "less than" but as stream redirection operator. The reason is that [ is just another program (an alias for test), so [ "$?" < "0" ] is similar to cat < filename. The other error is that you don't want to check for "less than" but for "not equal". In sum:
if [ "$?" < "0" ]
should be
if [ "$?" -ne "0" ]
Or you could write
if ! grep "$name" ~uli101/2014c/phonebook
...because if interprets a return code of zero as true and everything else as false.
never mind one of my friends pushed me in the right direction:
all I had to do is:
if [ "$?" = "1" ]
then echo "error"
fi
pretty much I was over thinking it, I just needed to say if $? = 1 then error, because in the readings it said $? can be greater then 1 I was trying to compensate for that.
It should be
if [ "$?" -gt 0 ]
The symbol '<' is a redirection operator, and it's not a Python or C - everything in a shell script is a command, including the text after 'if', and you are executing a command named '[' here, you may find it at the location /usr/bin/[, and this command uses -gt and -lt parameters to compare numbers, instead of '>' and '<', which are special shell operators.
You can rewrite this code like this:
if grep -i "$name" ~uli101/2014c/phonebook
then true # 'true' is also a command, which does nothing and returns success
else echo "Error"
fi
or even like this, using '||' operator, which will execute following command only if previous command returned an error:
grep -i "$name" ~uli101/2014c/phonebook || echo "Error"
The "$?" doesn't need quotes, as it is a number really. If you want better script, check on existance of the Phonebook file, and exit before asking the Name input if the file is missing. Also, if you reply nothing (enter only) on the READ command, you may need to do something.
#!/bin/bash
name=$1
phonebook=~/phonebook
if [ "$name" = "" ]
then
echo -n "Enter a name to search for: "
read name
fi
grep -i "$name" $phonebook
if [ $? -gt 0 ]
then
echo "error, no \"$name\" in $phonebook"
fi
This works just fine
if [[ -e img.png ]]
then
echo "exist"
else
echo "doesn't exist"
fi
but what if I know that there might be imgage with name img but I do not know if the file is .jpg , .gif , .jpeg , .tff and so on.
I do not care what is the extension I just want to know if there is a file with name 'img'
How can I do this ?
You can do:
files=$(ls img.* 2> /dev/null | wc -l)
if [ "$files" != "0" ]
then
echo "exist"
else
echo "doesn't exist"
fi
You can use the following scripts
files=`ls img.* 2>/dev/null`
if [ "$files" -a ${#files[#]} ]; then
echo "exist"
else
echo "doesn't exist"
fi
In this snippet, you use ls img.* to list all the files in current working directory whose name match the pattern img.*.
The result is stored into an array named files.
Then check size of the array to determine whether required files exist.
See this for how to get length of the array.
Something like this should do the job:
if [[ $(ls img.*) ]]; then
echo "file exist";
else
echo "file does not exist";
fi
I recommend to have a look at bash's pattern matching:
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_04_03.html
Without any external command:
$ for i in img.*
> do
> [ -f $i ] && echo exist || echo not exist
> break
> done
Check if any file is present. If present print exist , else not exist, and break immediately. The "-f" check is needed because if no files are present, still the loop runs once with i as "img.*" itself.
shopt -s nullglob
files=( img.* )
if (( ${#files[#]} == 0 )); then
echo "there are no 'img' files"
fi
If you don't use nullglob then, if there are no such files, the array will have 1 element, the literal string "img.*".
This question already has answers here:
Test whether a glob has any matches in Bash
(22 answers)
Closed 1 year ago.
How would you go about telling whether files of a specific extension are present in a directory, with bash?
Something like
if [ -e *.flac ]; then
echo true;
fi
#!/bin/bash
count=`ls -1 *.flac 2>/dev/null | wc -l`
if [ $count != 0 ]
then
echo true
fi
#/bin/bash
myarray=(`find ./ -maxdepth 1 -name "*.py"`)
if [ ${#myarray[#]} -gt 0 ]; then
echo true
else
echo false
fi
This uses ls(1), if no flac files exist, ls reports error and the script exits; othewise the script continues and the files may be be processed
#! /bin/sh
ls *.flac >/dev/null || exit
## Do something with flac files here
shopt -s nullglob
if [[ -n $(echo *.flac) ]] # or [ -n "$(echo *.flac)" ]
then
echo true
fi
#!/bin/bash
files=$(ls /home/somedir/*.flac 2> /dev/null | wc -l)
if [ "$files" != "0" ]
then
echo "Some files exists."
else
echo "No files with that extension."
fi
You need to be carful which flag you throw into your if statement, and how it relates to the outcome you want.
If you want to check for only regular files and not other types of file system entries then you'll want to change your code skeleton to:
if [ -f file ]; then
echo true;
fi
The use of the -f restricts the if to regular files, whereas -e is more expansive and will match all types of filesystem entries. There are of course other options like -d for directories, etc. See http://tldp.org/LDP/abs/html/fto.html for a good listing.
As pointed out by #msw, test (i.e. [) will choke if you try and feed it more than one argument. This might happen in your case if the glob for *.flac returned more than one file. In that case try wrapping your if test in a loop like:
for file in ./*.pdf
do
if [ -f "${file}" ]; then
echo 'true';
break
fi
done
This way you break on the first instance of the file extension you want and can keep on going with the rest of the script.
The top solution (if [ -e *.flac ];) did not work for me, giving: [: too many arguments
if ls *.flac >/dev/null 2>&1; then it will work.
You can use -f to check whether files of a specific type exist:
#!/bin/bash
if [ -f *.flac ] ; then
echo true
fi
bash only:
any_with_ext () (
ext="$1"
any=false
shopt -s nullglob
for f in *."$ext"; do
any=true
break
done
echo $any
)
if $( any_with_ext flac ); then
echo "have some flac"
else
echo "dir is flac-free"
fi
I use parentheses instead of braces to ensure a subshell is used (don't want to clobber your current nullglob setting).
shopt -s nullglob
set -- $(echo *.ext)
if [ "${#}" -gt 0 ];then
echo "got file"
fi
For completion, with zsh:
if [[ -n *.flac(#qN) ]]; then
echo true
fi
This is listed at the end of the Conditional Expressions section in the zsh manual. Since [[ disables filename globbing, we need to force filename generation using (#q) at the end of the globbing string, then the N flag (NULL_GLOB option) to force the generated string to be empty in case there’s no match.
Here is a solution using no external commands (i.e. no ls), but a shell function instead. Tested in bash:
shopt -s nullglob
function have_any() {
[ $# -gt 0 ]
}
if have_any ./*.flac; then
echo true
fi
The function have_any uses $# to count its arguments, and [ $# -gt 0 ] then tests whether there is at least one argument. The use of ./*.flac instead of just *.flac in the call to have_any is to avoid problems caused by files with names like --help.
Here's a fairly simple solution:
if [ "$(ls -A | grep -i \\.flac\$)" ]; then echo true; fi
As you can see, this is only one line of code, but it works well enough. It should work with both bash, and a posix-compliant shell like dash. It's also case-insensitive, and doesn't care what type of files (regular, symlink, directory, etc.) are present, which could be useful if you have some symlinks, or something.
I tried this:
if [ -f *.html ]; then
echo "html files exist"
else
echo "html files dont exist"
fi
I used this piece of code without any problem for other files, but for html files I received an error:
[: too many arguments
I then tried #JeremyWeir's count solution, which worked for me:
count=`ls -1 *.flac 2>/dev/null | wc -l`
if [ $count != 0 ]
then
echo true
fi
Keep in mind you'll have to reset the count if you're doing this in a loop:
count=$((0))
This should work in any borne-like shell out there:
if [ "$(find . -maxdepth 1 -type f | grep -i '.*\.flac$')" ]; then
echo true
fi
This also works with the GNU find, but IDK if this is compatible with other implementations of find:
if [ "$(find . -maxdepth 1 -type f -iname \*.flac)" ]; then
echo true
fi