I have a bash script that isn't properly working, I am new to this type of coding. Any suggestions or help would be great.
I am attempting to change email addresses of an old user with a new user across our PuTTy platform. However not all directories have a certain file, I think there may be a problem with my IF statement. Here is the script I have attempted using:
#!/bin/bash
#set -x
for dirs in *
do
echo $dirs
cd $dirs/
if [ -d "Seadont"]
then
sed -i 's/USER1/USER2/g'
ls
fi
done
My results are not correct.
Try this:
#!/bin/bash
#set -x
for dirs in * ;do
echo $dirs
if [ -d "Seadont"] ;then
sed -i.bak 's/USER1/USER2/g' $dirs
fi
done
Mind you that it won't work if there are white spaces on "$dirs". I can propose a better solution if it does, but I would need to know current directory.
Related
I need to write a script that checks some >20k files for some >2k search text and it needs to be flexible, so I came up with this script:
#!/bin/bash
# This script checks all files in a given directory against a list of criteria
shopt -s expand_aliases
source ~/.bashrc
TIMESTAMP=$(date "+%Y-%m-%d-%T")
ROOT_DIR=/data
PROJECT_NAME=$1
FILE_DIR=$ROOT_DIR/projects/$1/$2
RESULT_DIR=$ROOT_DIR/projects/$1/check_result
SEARCHTEXT_FILE=$ROOT_DIR/scripts/$3
OIFS="$IFS"
IFS=$'\n'
files=$(find $FILE_DIR -type f -name '*.json')
for file in $files; do
while read line; do
grep -H -o $line "$file" >> $RESULT_DIR/check_result_$TIMESTAMP.log
done < $SEARCHTEXT_FILE
done
IFS="$OIFS"
This script only produces the empty $RESULT_DIR/check_result_$TIMESTAMP.log log file with correct name.
Because the file names sometimes contain spaces I added the IFS... statements and I enclosed $file in " quotes (copied from another post).
The content of the $SEARCHTEXT_FILE is for example:
'Tel alt........'
'City ..........'
If I place an echo before the grep like this
echo grep -H -o $line "$file"
then output I get is
grep -H -o 'Tel alt........' /data/projects/DNAR/input/report-157538.json
and I can execute this line as is and get the correct result.
I tried to put various combinations of " or ' or ` or () or {} around any part of this grep command but nothing changed.
Somewhere I did read about alias and the alias set for grep is
alias grep='grep --color=auto'
After many hours of searching on the internet I couldn't find any post that helped me as most of them are covering issues around wrong quotes or inline bash issues.
What are I missing here?
The simple and obvious workaround is to remove all that complexity and simply use the features of the commands you are running anyway.
find "$FILE_DIR" -type f -name '*.json' \
-exec grep -H -o -f "$SEARCHTEXT_FILE" {} + > "$RESULT_DIR/check_result_$TIMESTAMP.log"
Notice also the quoting fixes; see When to wrap quotes around a shell variable; to avoid mishaps, you should switch to lower case for your private variables (see Correct Bash and shell script variable capitalization).
shopt -s expand_aliases
and source ~/.bashrc merely look superfluous, but could contribute to whatever problem you are trying to troubleshoot; they should basically never be part of a script you plan to use in production.
I have made a script that takes all of the files in the current directory, checks if it is a regular file or a folder and sets permissions to them.
My problem is when I encounter hidden files\folders and files\folders that have spaces in the name.
Here is my script:
#!/bin/bash
FILES=$(pwd)/*
for f in $FILES
do
if [ -f $f ]; then chmod u+x $f; fi
if [ -d $f ]; then chmod u=w,g+r,o-rwx $f; fi
done
Here is an example of an error I get from the testing computer:
'test/.bwhajtbzmu xswxcgqsvz' has incorrect permissions: expected 250, got 414.
The other errors are basically the same.
I am not actually sure what is the problem here, if it is the fact that it is a hidden file or that it has a space in the name. I guess both things are the problem.
How can I modify the script so that it can work with hidden files and files that have space in the name ?
Thank you
PS. Please don't question the usefulness of the script, it is a school homework.
Handling whitespace in file names is tricky. First rule is: doublequotes around all usages of variables. Otherwise the shell interprets the spaces as separators. Unfortunately you cannot simply hold a list in a variable. You have to use array variables for this.
#!/bin/bash
FILES=( "$(pwd)"/* )
for f in "${FILES[#]}"
do
if [ -f "$f" ]; then chmod u+x "$f"; fi
if [ -d "$f" ]; then chmod u=w,g+r,o-rwx "$f"; fi
done
For handling hidden files and folders (the ones starting with a dot .) you should best set the shell option dotglob which makes * also match these (which it otherwise doesn't). (Using .* is not good as it matches also . and .. which normally aren't wanted and things like .??* will not match stuff like .a which is normally wanted.):
shopt -s dotglob
FILES=( "$(pwd)"/* )
shopt -u dotglob
I would not recommend leaving it on, so I switch it off after using it.
I am trying to write a bash/shell script to zip up a specific folder and ignore certain sub-dirs in that folder.
This is the folder I am trying to zip "sync_test5":
My bash script generates an ignore list (based on) and calls the zip function like this:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR="dir_to_ignore dir2_to_ignore"
ignorelist=""
if [ "$SYNC_WEB_ROOT_IGNORE_DIR" != "" ];
then
for ignoredir in $SYNC_WEB_ROOT_IGNORE_DIR
do
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip"
cd $SYNC_WEB_ROOT_BASE_DIR;
zip -r $FILE $SYNC_WEB_ROOT_BACKUP_DIR -x $ignorelist >/dev/null
echo "Done"
Now this script runs without error, however it is not ignoring/excluding the dirs I've specified.
So, I had the shell script output the command it tried to run, which was:
zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
Now If I run the above command directly in putty like this, it works:
So, why doesn't my shell script exclude working as intended? the command that is being executed is identical (in shell and putty directly).
Because backslash quotings in a variable after word splitting are not evaluated.
If you have a='123\4', echo $a would give
123\4
But if you do it directly like echo 123\4, you'd get
1234
Clearly the arguments you pass with the variable and without the variables are different.
You probably just meant to not quote your argument with backslash:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Btw, what actual works is a non-evaluated glob pattern:
zip -r 12-08-2014_072810.website.zip sync_test5 -x 'sync_test5/dir_to_ignore/***' 'sync_test5/dir2_to_ignore/***'
You can verify this with
echo zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
And this is my suggestion:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR=("dir_to_ignore" "dir2_to_ignore")
IGNORE_LIST=()
if [[ -n $SYNC_WEB_ROOT_IGNORE_DIR ]]; then
for IGNORE_DIR in "${SYNC_WEB_ROOT_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/***") ## "$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/*" perhaps is enough?
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip" ## Where is $SYNC_BACKUP_DIR set?
cd "$SYNC_WEB_ROOT_BASE_DIR";
zip -r "$FILE" "$SYNC_WEB_ROOT_BACKUP_DIR" -x "${IGNORE_LIST[#]}" >/dev/null
echo "Done"
This is what I ended up with:
#!/bin/bash
# This script zips a directory, excluding specified files, types and subdirectories.
# while zipping the directory it excludes hidden directories and certain file types
[[ "`/usr/bin/tty`" == "not a tty" ]] && . ~/.bash_profile
DIRECTORY=$(cd `dirname $0` && pwd)
if [[ -z $1 ]]; then
echo "Usage: managed_directory_compressor /your-directory/ zip-file-name"
else
DIRECTORY_TO_COMPRESS=${1%/}
ZIPPED_FILE="$2.zip"
COMPRESS_IGNORE_FILE=("\.git" "*.zip" "*.csv" "*.json" "gulpfile.js" "*.rb" "*.bak" "*.swp" "*.back" "*.merge" "*.txt" "*.sh" "bower_components" "node_modules")
COMPRESS_IGNORE_DIR=("bower_components" "node_modules")
IGNORE_LIST=("*/\.*" "\.* "\/\.*"")
if [[ -n $COMPRESS_IGNORE_FILE ]]; then
for IGNORE_FILES in "${COMPRESS_IGNORE_FILE[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_FILES/*")
done
for IGNORE_DIR in "${COMPRESS_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_DIR/")
done
fi
zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
# echo zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
echo $DIRECTORY_TO_COMPRESS "compressed as" $ZIPPED_FILE.
fi
After a few trial and error, I have managed to fix this problem by changing this line:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
to:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Not sure why this worked, but it does :)
How can I list the path of the output of this script?
This is my command:
(ls -d */ ); echo -n $i; ls -R $i | grep "wp-config.php" ;
This is my current output:
/wp-config.php
It seems you want find the path to a file called "wp-config.php".
Does the following help?
find $PWD -name 'wp-config.php'
Your script is kind of confusing: Why does ls -d */ does not show any output? What's the value of $i? Your problem in fact seems to be that ls -R lists the contents of all subdirectories but doesn't give you full paths for their contents.
Well, find is the best tool for that, but you can simulate it in this case via a script like this:
#!/bin/bash
searchFor=wp-config.php
startDir=${1:-.}
lsSubDir() {
local actDir="$1"
for entry in $(ls "$actDir"); do
if [ -d "$actDir/$entry" ]; then
lsSubDir "$actDir/$entry"
else
[ $entry = $searchFor ] && echo "$actDir/$entry"
fi
done
}
lsSubDir $startDir
Save it in a file like findSimulator, make it executable and call it with the directory where to start searching as parameter.
Be warned: this script is not very efficient, it may stop on large subdirectory-trees because of recursion. I would strongly recommend the solution using find.
I have a file that looks like this:
...
%ldirs
(list of line-separated directories)
...
With a shell script, I need to add a directory to the list in that file, but only if that directory is not already in the list. Here's the catch: The directory in question must come from a variable $SOME_PATH.
I thought about using the patch utility, but to do that I would have to generate the patch file dynamically to add "+$SOME_PATH". The other problem is that I do not know the "after context" or the line number of "%ldirs", so generating the patch file is problematic.
Is there another option?
Tweaked answer - Thanks to Rob:
line=$(grep "$SOME_PATH" /path/to/file)
if [ $? -eq 1 ]
then
sed -i "/%ldirs/ a\\$SOME_PATH" /path/to/file
fi
Final answer - Thanks to tripleee:
fgrep -xq "$SOME_PATH" /path/to/file || sed -i "/%ldirs/ a\\$SOME_PATH" /path/to/file
line=$(grep "$SOME_PATH" %ldirs)
if [ $? -eq 1 ]
then
echo "$SOME_PATH" >> %ldirs
fi
something like this should work, it worked fine for me. I'm sure there are other ways to write it, too.
line=$(grep "$SOME_PATH" /path/to/file)
if [ $? -eq 1 ]
then
sed -i 's/%lsdir/%lsdir\n"$SOME_PATH"/' /path/to/file
fi
should work. It'll find %lsdir and replace it with %lsdir(newline)$SOME_PATH (not sure if quotes are needed on $SOME_PATH here, pretty sure they aren't)