I need help for bash scripting
I have a folder with log files(number may vary)
I have to test those files for content and if one or more of those files has content, I want to send the content by mail (I use mailx) being formatted like so:
The content of "filename" is:
------content---------
The content of "filename2" is:
------content---------
There was no content in "filename3".
Any practical way using bash to do this?
Thanks for your help.
shopt -s nullglob
for i in *; do
if [[ -s "$i" ]]; then
echo "file $i exists and is not empty"
fi
done
See: man bash
Related
related to: Loop through all the files with a specific extension
I want to loop through files that matches a pattern:
for item in ./bob* ; do
echo $item
done
I have a file list like:
bob
bobob
bobob.log
I only want to list files that have no extension:
bob
bobob
what is the best way to archive this? - can I do it in the loop somehow or do I need an if statement within the loop?
In bash you can use features of xtended globbing:
shopt -s extglob
for item in ./bob!(*.*) ; do
echo $item
done
You can put shopt -s extglob in your .bashrc file to enable it.
Recent Bash versions have regular expression support:
for f in *
do
if [[ "$f" =~ .*\..* ]]
then
: ignore
else
echo "$f"
fi
done
I need a script that will check any modification in the /etc/passwd file, if there's a modification I get alerted immediately via email telling me that a user that has been added or removed from the passwd file
Without using inotifywait command
You could use the inotifywait utility, something like this:
inotifywait -e modify /etc/passwd
inotifywait will exit when the selected file is written, so you can take whatever action you need.
you could check for modifications with md5sum or sha256...
make your base checksum, hardcode it in a scrip
if [[ "$(sha256 /etc/passwd)" != "the hash of ori file" ]]; then
mail yourself
fi
you could also have a copy of /etc/passwd somewhere and run a diff that you send by mail as to have the content of the changes
if [[ "$(diff /etc/passwd /root/passwdbackup)" != "" ]]; then
mail yourself
fi
I am trying to write a bash/shell script to zip up a specific folder and ignore certain sub-dirs in that folder.
This is the folder I am trying to zip "sync_test5":
My bash script generates an ignore list (based on) and calls the zip function like this:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR="dir_to_ignore dir2_to_ignore"
ignorelist=""
if [ "$SYNC_WEB_ROOT_IGNORE_DIR" != "" ];
then
for ignoredir in $SYNC_WEB_ROOT_IGNORE_DIR
do
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip"
cd $SYNC_WEB_ROOT_BASE_DIR;
zip -r $FILE $SYNC_WEB_ROOT_BACKUP_DIR -x $ignorelist >/dev/null
echo "Done"
Now this script runs without error, however it is not ignoring/excluding the dirs I've specified.
So, I had the shell script output the command it tried to run, which was:
zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
Now If I run the above command directly in putty like this, it works:
So, why doesn't my shell script exclude working as intended? the command that is being executed is identical (in shell and putty directly).
Because backslash quotings in a variable after word splitting are not evaluated.
If you have a='123\4', echo $a would give
123\4
But if you do it directly like echo 123\4, you'd get
1234
Clearly the arguments you pass with the variable and without the variables are different.
You probably just meant to not quote your argument with backslash:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Btw, what actual works is a non-evaluated glob pattern:
zip -r 12-08-2014_072810.website.zip sync_test5 -x 'sync_test5/dir_to_ignore/***' 'sync_test5/dir2_to_ignore/***'
You can verify this with
echo zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
And this is my suggestion:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR=("dir_to_ignore" "dir2_to_ignore")
IGNORE_LIST=()
if [[ -n $SYNC_WEB_ROOT_IGNORE_DIR ]]; then
for IGNORE_DIR in "${SYNC_WEB_ROOT_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/***") ## "$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/*" perhaps is enough?
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip" ## Where is $SYNC_BACKUP_DIR set?
cd "$SYNC_WEB_ROOT_BASE_DIR";
zip -r "$FILE" "$SYNC_WEB_ROOT_BACKUP_DIR" -x "${IGNORE_LIST[#]}" >/dev/null
echo "Done"
This is what I ended up with:
#!/bin/bash
# This script zips a directory, excluding specified files, types and subdirectories.
# while zipping the directory it excludes hidden directories and certain file types
[[ "`/usr/bin/tty`" == "not a tty" ]] && . ~/.bash_profile
DIRECTORY=$(cd `dirname $0` && pwd)
if [[ -z $1 ]]; then
echo "Usage: managed_directory_compressor /your-directory/ zip-file-name"
else
DIRECTORY_TO_COMPRESS=${1%/}
ZIPPED_FILE="$2.zip"
COMPRESS_IGNORE_FILE=("\.git" "*.zip" "*.csv" "*.json" "gulpfile.js" "*.rb" "*.bak" "*.swp" "*.back" "*.merge" "*.txt" "*.sh" "bower_components" "node_modules")
COMPRESS_IGNORE_DIR=("bower_components" "node_modules")
IGNORE_LIST=("*/\.*" "\.* "\/\.*"")
if [[ -n $COMPRESS_IGNORE_FILE ]]; then
for IGNORE_FILES in "${COMPRESS_IGNORE_FILE[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_FILES/*")
done
for IGNORE_DIR in "${COMPRESS_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_DIR/")
done
fi
zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
# echo zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
echo $DIRECTORY_TO_COMPRESS "compressed as" $ZIPPED_FILE.
fi
After a few trial and error, I have managed to fix this problem by changing this line:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
to:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Not sure why this worked, but it does :)
I have a bash script that isn't properly working, I am new to this type of coding. Any suggestions or help would be great.
I am attempting to change email addresses of an old user with a new user across our PuTTy platform. However not all directories have a certain file, I think there may be a problem with my IF statement. Here is the script I have attempted using:
#!/bin/bash
#set -x
for dirs in *
do
echo $dirs
cd $dirs/
if [ -d "Seadont"]
then
sed -i 's/USER1/USER2/g'
ls
fi
done
My results are not correct.
Try this:
#!/bin/bash
#set -x
for dirs in * ;do
echo $dirs
if [ -d "Seadont"] ;then
sed -i.bak 's/USER1/USER2/g' $dirs
fi
done
Mind you that it won't work if there are white spaces on "$dirs". I can propose a better solution if it does, but I would need to know current directory.
How to write simple shell script to create zip file.
I want to create zip file by collecting files with same string pattern in their file names from a folder.
For example, there may be many files under a folder.
xxxxx_20140502_xxx.txt
xxxxx_20140502_xxx.txt
xxxxx_20140503_xxx.txt
xxxxx_20140503_xxx.txt
xxxxx_20140504_xxx.txt
xxxxx_20140504_xxx.txt
After running the shell script, the result must be following three zip files.
20140502.zip
20140503.zip
20140504.zip
Please give me right direction to create simple shell script to output the result as above.
#!/bin/bash
for file in *_????????_*.csv *_????????_*.txt; do
[ -f "${file}" ] || continue
date=${file#*_} # adjust this and next line depending
date=${date%_*} # on your actual prefix/suffix
echo "${date}"
done | sort -u | while read date; do
zip "${date}.zip" *${date}*
done
Since zip will update the archive, this will do:
shopt -s nullglob
for file in *.{txt,csv}; do [[ $file =~ _([[:digit:]]{8})_ ]] && zip "${BASH_REMATCH[1]}.zip" "$file"; done
The shopt -s nullglob is because you don't want to have unexpanded globs if there are no matching files.
Everything below this line is my old answer...
First, get all the possible dates. Heuristically, this could be the files ending in .txt and .csv that match the regex _[[:digit:]]{8}_:
#!/bin/bash
shopt -s nullglob
declare -A dates=()
for file in *.{csv,txt}; do
[[ $file =~ _([[:digit:]]{8})_ ]] && dates[${BASH_REMATCH[1]}]=
done
printf "Date found: %s\n" "${!dates[#]}"
This will output to stdout all the dates found in the files. E.g. (I called the previous snipped gorilla and I chmod +x gorilla and touched a few files for demo):
$ ls
banana_20010101_gorilla.csv gorilla_20140502_bonobo.csv
gorilla notthisone_123_lol.txt
gorilla_20140502_banana.txt
$ ./gorilla
Date found: 20140502
Date found: 20010101
Next step, for each date found, get all the files ending in .txt and .csv and zip them in the archive corresponding to the date: appending this to gorilla will do the job:
for date in "${!dates[#]}"; do
zip "$date.zip" *"_${date}_"*.{csv,txt}
done
Full script after removing the flooding part:
#!/bin/bash
shopt -s nullglob
declare -A dates=()
for file in *.{csv,txt}; do
[[ $file =~ _([[:digit:]]{8})_ ]] && dates[${BASH_REMATCH[1]}]=
done
for date in "${!dates[#]}"; do
zip "$date.zip" *"_${date}_"*.{csv,txt}
done
Edit. I overlooked your requirement with one line command. Then here's the one-liner:
shopt -s nullglob; declare -A dates=(); for file in *.{csv,txt}; do [[ $file =~ _([[:digit:]]{8})_ ]] && dates[${BASH_REMATCH[1]}]=; done; for date in "${!dates[#]}"; do zip "$date.zip" *"_${date}_"*.{csv,txt}; done
:)
#! /bin/bash
dates=$(ls ?????_[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]_???.{csv,txt} \
| cut -f2 -d_ | sort -u)
for date in $dates ; do
zip $date.zip ?????_"$date"_???.{csv,txt}
done