This if my first attempt at bash scripting. I am trying to create a script to check on every single file owner and group starting under a certain directory.
For example if I have this:
files=/*
for f in $files; do
owner=$(stat -c %U $f)
if [ "$owner" != "someone" ]; then
echo $f $owner
fi
done
The ultimate goal is to fix permission problems. However, I am not able to get the /* variable to go underneath everything in /, it will only check the files under / and stop at any new directories. Any pointers on how I could check for permissions over everything under / and any of its sub-directories?
You can shopt -s globstar and use for f in yourdir/** to expand recursively in bash4+, or you can use find:
find yourdir ! -user someone
If you want the same output format with username and filename, you have to get system specific:
GNU$ find yourdir ! -user someone -printf '%p %u\n'
OSX$ find yourdir ! -user someone -exec stat -f '%N %Su' {} +
List all files recursively in list format and hidden files which shows ownership and permissions
ls -Rla *
you can try this one, it is a recursive one:
function playFiles {
files=$1
for f in $files; do
if [ ! -d $f ]; then
owner=$(stat -c %U $f)
echo "Simple FILE=$f -- OWNER=$owner"
if [ "$owner" != "root" ]; then
echo $f $owner
fi
else
playFiles "$f/*"
fi
done
}
playFiles "/root/*"
Play a little with in a another directory before replacing playFiles "/root/" with : playFiles "/". Btw playFiles is a bash function. Hopefully this will help you.
Related
I am working on a shell script which should validate .gz files in multiple folders in linux and then gzip them if a particular file not zipped and if already the file is zipped, purge them with following condition.
a) All these files in folders have *.log..gz as extension
So i was using functions and find cmd to achieve the same.
Script seems to be working fine but its not logging the zipped files information to log file, however its spooling about already zipped files in the folder to log. is this the correct way using functions?
#!/bin/bash
DIR_PATH="/var/log"
LOG="/tmp/test.log"
VARLOG_PATH=("$DIR_PATH"{"Kevin","John","Robin","Pavan"})
fun_zip_log(){
for i in `find "$i" -type f \( -name "*.log.20*" 2>/dev/null \) `; do
gzip "$i" ; done >> $LOG
}
fun_purge_log(){
for i in `find "$i" -type f \( -name "log.20*" 2>/dev/null \) `; do rm -f
"$i" ; done >> $LOG
}
validate_zip(){
for file in $i/*.gz
do
if ! [ -f "$file" ];
then
echo "$file is getting zipped" >> $LOG
fun_zip_log "$i"
else
echo "$file is already zipped" >> $LOG
fun_purge_log "$i"
fi
done
}
#MainBlock
for i in "${VARLOG_PATH[#]}"
do
if [ -d "$i" ] && [ "$(ls -A "$i" |wc -l )" -gt 0 ]; then
echo "Searching for files in directory : "$i" " >> $LOG
validate_zip "$i"
else
echo "No files exist in directory : "$i" " >> $LOG
fi
done
exit
####LOG FILE###
Searching for files in directory : /var/log/Kevin
[*.gz] is getting zipped.
Searching for files in directory : /var/log/John
/var/log/John/instrumentation.log.2018-06-20.gz is already zipped
/var/log/John/instrumentation.log.2018-06-21.gz is already zipped
No files exist in directory : /var/log/Robin
Searching for files in directory : /var/log/Pavan
[*.gz] is getting zipped.
Your code is very muddled and confusing. For example in this:
fun_purge_log(){
for i in `find "$i" -type f \( -name "log.20*" 2>/dev/null \) `; do rm -f
"$i" ; done >> $LOG
}
for file in $i/*.gz
do
...
fun_purge_log "$i"
In the calling code you're looping setting a variable file but then passing the directory "$i" to your function to then try to find the files again.
Within fun_purge_log() you're ignoring the argument being passed in and then using the global variable i as both the directory argument to find and also to loop through the list of files output by find - why not pick a new variable name and use some local variables?
I can't imagine what you think 2>/dev/null is going to do in \( -name "*log.20*" 2>/dev/null \).
You're trying to append something to $LOG but you aren't printing anything to append to it.
Run your code through shellcheck (e.g. at shellcheck.net), read http://mywiki.wooledge.org/BashFAQ/001, https://mywiki.wooledge.org/Quotes and https://mywiki.wooledge.org/ParsingLs, and really just THINK about what each line of your code is doing. Correct the issues yourself and then let us know if you still have a problem. Oh, and by convention and to avoid clashing with other variables don't use all capitals for non-exported variable names and lastly use $(command) instead of `command`.
I am trying to move the directories from $DIR1 to $DIR2 if $DIR2 does not have the same directory name
if [[ ! $(ls -d /$DIR2/* | grep test) ]] is what I currently have.
then
mv $DIR1/test* /$DIR2
fi
first it gives
ls: cannot access //data/lims/PROCESSING/*: No such file or directory
when $DIR2 is empty
however, it still works.
secondly
when i run the shell script twice.
it doesn't let me move the directories with the similar name.
for example
in $DIR1 i have test-1 test-2 test-3
when it runs for the first time all three directories moves to $DIR2
after that i do mkdir test-4 at $DIR1 and run the script again..
it does not let me move the test-4 because my loop thinks that test-4 is already there since I am grabbing all test
how can I go around and move test-4 ?
Firstly, you can check whether or not a directory exists using bash's built in 'True if directory exists' expression:
test="/some/path/maybe"
if [ -d "$test" ]; then
echo "$test is a directory"
fi
However, you want to test if something is not a directory. You've shown in your code that you already know how to negate the expression:
test="/some/path/maybe"
if [ ! -d "$test" ]; then
echo "$test is NOT a directory"
fi
You also seem to be using ls to get a list of files. Perhaps you want to loop over them and do something if the files are not a directory?
dir="/some/path/maybe"
for test in $(ls $dir);
do
if [ ! -d $test ]; then
echo "$test is NOT a directory."
fi
done
A good place to look for bash stuff like this is Machtelt Garrels' guide. His page on the various expressions you can use in if statements helped me a lot.
Moving directories from a source to a destination if they don't already exist in the destination:
For the sake of readability I'm going to refer to your DIR1 and DIR2 as src and dest. First, let's declare them:
src="/place/dir1/"
dest="/place/dir2/"
Note the trailing slashes. We'll append the names of folders to these paths so the trailing slashes make that simpler. You also seem to be limiting the directories you want to move by whether or not they have the word test in their name:
filter="test"
So, let's first loop through the directories in source that pass the filter; if they don't exist in dest let's move them there:
for dir in $(ls -d $src | grep $filter); do
if [ ! -d "$dest$dir" ]; then
mv "$src$dir" "$dest"
fi
done
I hope that solves your issue. But be warned, #gniourf_gniourf posted a link in the comments that should be heeded!
If you need to mv some directories to another according to some pattern, than you can use find:
find . -type d -name "test*" -exec mv -t /tmp/target {} +
Details:
-type d - will search only for directories
-name "" - set search pattern
-exec - do something with find results
-t, --target-directory=DIRECTORY move all SOURCE arguments into DIRECTORY
There are many examples of exec or xargs usage.
And if you do not want to overwrite files, than add -n option to mv command:
find . -type d -name "test*" -exec mv -n -t /tmp/target {} +
-n, --no-clobber do not overwrite an existing file
I need a Bash Script to Execute a program for all directories that do not have a specific file and create the output file on the same directory.This program needs an input file which exist in every directory with the name *.DNA.fasta.Suppose I have the following directories that may contain sub directories also
dir1/a.protein.fasta
dir2/b.protein.fasta
dir3/anyfile
dir4/x.orf.fasta
I have started by finding the directories that don't have that specific file whic name is *.protein.fasta
in this case I want the dir3 and dir4 to be listed (since they do not contain *.protein.fasta)
I have tried this code:
find . -maxdepth 1 -type d \! -exec test -e '{}/*protein.fasta' \; -print
but it seems I missed some thing it does not work.
also I do not know how to proceed for the whole story.
This is a tricky one.
I can't think of a good solution. But here's a solution, nevertheless. Note that this is guaranteed not to work if your directory or file names contain newlines, and it's not guaranteed to work if they contain other special characters. (I've only tested with the samples in your question.)
Also, I haven't included a -maxdepth because you said you need to search subdirectories too.
#!/bin/bash
# Create an associative array
declare -A excludes
# Build an associative array of directories containing the file
while read line; do
excludes[$(dirname "$line")]=1
echo "excluded: $(dirname "$line")" >&2
done <<EOT
$(find . -name "*protein.fasta" -print)
EOT
# Walk through all directories, print only those not in array
find . -type d \
| while read line ; do
if [[ ! ${excludes[$line]} ]]; then
echo "$line"
fi
done
For me, this returns:
.
./dir3
./dir4
All of which are directories that do not contain a file matching *.protein.fasta. Of course, you can replace the last echo "$line" with whatever you need to do with these directories.
Alternately:
If what you're really looking for is just the list of top-level directories that do not contain the matching file in any subdirectory, the following bash one-liner may be sufficient:
for i in *; do test -d "$i" && ( find "$i" -name '*protein.fasta' | grep -q . || echo "$i" ); done
#!/bin/bash
for dir in *; do
test -d "$dir" && ( find "$dir" -name '*protein.fasta' | grep -q . || Programfoo"$dir/$dir.DNA.fasta");
done
I created a script and it moves files with different extensions to their specified directories.
If the directory is not there, it creates another one (where the files will go), and it creates another directory where the remaining files with different extensions will go.
My first problem is that I want when I put -d and full path on the terminal it should move only media files, -l and full path to move all text files, then -x to change the extension to uppercase, then -u to lowercase.
Can somebody modify it for me and show me how to overcome this problem?
#!/bin/bash
From="/home/elg19/lone/doc"
To="/home/elg19/mu"
WA="/home/elg19/du"
MA="/home/elg19/dq"
WQ="/home/elg19/d2"
# this function checks if the directory exits and creates one if it does not then moves all doc files
function mama(){
if [[ ! -d "$WA" ]]; then
mkdir -p "$WA"
fi
cd "$From"
for i in pdf txt doc; do
find . -type f -name "*.${i}" -exec mv "{}" "$WA" \;
done
}
# this function checks if the directory exits and creates one if it does not then moves all media files
function so(){
if [[ ! -d "$To" ]]; then
mkdir -p "$To"
fi
cd "$From"
for i in mp3 mp4 swf; do
find . -type f -name "*.${i}" -exec mv "{}" "$To" \;
done
}
# this function checks if the directory exits and creates one if it does not then moves all image files
function soa(){
if [[ ! -d "$MA" ]]; then
mkdir -p "$MA"
fi
cd "$From"
for i in jpg gif png; do
find . -type f -name "*.${i}" -exec mv "{}" "$MA" \;
done
}
# this function checks if the directory exits and creates one if it does not then moves all the remaining files
function soaq(){
if [[ ! -d "$WQ" ]]; then
mkdir -p "$WQ"
fi
cd "$From"
for i in *; do
find . -type f -name "*.${i}" -exec mv "{}" "$WQ" \;
done
}
mama
so
soa
soaq
I don't know if the options suggested are mnemonic in your native language, but they are counter-mnemonic in English. I would suggest something more like:
-m path Move media files
-t path Move text files
-u Change extensions to upper-case
-l Change extensions to lower-case
The command to use for regular argument parsing like this is getopts (plural - many systems also have a command getopt, singular, which has different characteristics altogether).
The referenced page gives an example of how to use it:
The following example script parses and displays its arguments:
aflag=
bflag=
while getopts ab: name
do
case $name in
a) aflag=1;;
b) bflag=1
bval="$OPTARG";;
?) printf "Usage: %s: [-a] [-b value] args\n" $0
exit 2;;
esac
done
if [ ! -z "$aflag" ]; then
printf "Option -a specified\n"
fi
if [ ! -z "$bflag" ]; then
printf 'Option -b "%s" specified\n' "$bval"
fi
shift $(($OPTIND - 1))
printf "Remaining arguments are: %s\n" "$*"
The option -a doesn't take an argument; the option -b requires an argument.
I'm trying to move media and other files which are in a specified directory to another directory and create another one if it does not exits (where the files will go), and create a directory the remaining files with different extensions will go. My first problem is that my script is not making a new directory and it is not moving the files to other directories and what code can I use to move files with different extensions to one directory?
This is what i have had so far, correct me where I'm wrong and help modify my script:
#!/bin/bash
From=/home/katy/doc
To=/home/katy/mo #directory where the media files will go
WA=/home/katy/do # directory where the other files will go
if [ ! -d "$To" ]; then
mkdir -p "$To"
fi
cd $From
find path -type f -name"*.mp4" -exec mv {} $To \;
I'd solve it somewhat like this:
#!/bin/bash
From=/home/katy/doc
To=/home/katy/mo # directory where the media files will go
WA=/home/katy/do # directory where the other files will go
cd "$From"
find . -type f \
| while read file; do
dir="$(dirname "$file")"
base="$(basename "$file")"
if [[ "$file" =~ \.mp4$ ]]; then
target="$To"
else
target="$WA"
fi
mkdir -p "$target/$dir"
mv -i "$file" "$target/$dir/$base"
done
Notes:
mkdir -p will not complain if the directory already exists, so there's no need to check for that.
Put double quotes around all filenames in case they contain spaces.
By piping the output of find into a while loop, you also avoid getting bitten by spaces, because read will read until a newline.
You can modify the regex according to taste, e.g. \.(mp3|mp4|wma|ogg)$.
In case you didn't know, $(...) will run the given command and stick its output back in the place of the $(...) (called command substitution). It is almost the same as `...` but slightly better (details).
In order to test it, put echo in front of mv. (Note that quotes will disappear in the output.)
cd $From
find . -type f -name "*.mp4" -exec mv {} $To \;
^^^
or
find $From -type f -name "*.mp4" -exec mv {} $To \;
^^^^^
cd $From
mv *.mp4 $To;
mv * $WA;