My question is about Bash, Shell. I am writing a script and I have the following problem:
I have a case when user declares that he or she will extract a file into a dir. But I have to test if the existence and if exist a need to check if that file is a *.tar file. I searched for similar like when checking if the file is executable:
if [ -x "file" ]; then
echo "file is executable"
else
echo "file is not executable"
# will this if test work?
case $1
"--extract")
if [ -e $2 ] && [ tar -tzf $2 >/dev/null ]; then
echo "file exists and is tar archive"
else
echo "file either does not exists or it is not .tar arcive"
fi
;;
esac
Code from above doesn't work it is totally ignored. Any ideas?
file command can determine file type:
file my.tar
if it is a tar file it will output:
my.tar: POSIX tar archive (GNU)
Then you can use grep to check the output (whether or not contains tar archive):
file my.tar | grep -q 'tar archive; && echo "I'm tar" || echo "I'm not tar"
In case the file does not exis, file output will be (with exit code 0):
do-not-exist.txt: cannot open `do-not-exist.txt' (No such file or directory).
You could use a case statement to handle several types of files.
I would just see if tar can list the file:
if ! { tar ztf "$file" || tar tf "$file"; } >/dev/null 2>&1; then
echo "$file is not a tar file"
fi
I usually use a construct like this based off of the file command.
gzipped tarballs
$ file somefile1.tar.gz | grep -q 'gzip compressed data' && echo yes || echo no
yes
$ file somefile2.tar.gz | grep -q 'gzip compressed data' && echo yes || echo no
no
tarballs
The above handles gzipped tarball files, for uncompressed change out the string that grep detects:
$ file somefile1.tar | grep -q 'POSIX tar archive' && echo yes || echo no
yes
$ file somefile2.tar | grep -q 'POSIX tar archive' && echo yes || echo no
no
OK, I found the answer. I know that this is not most optimal, however, it works as I intended.
I put case $1 from user into a variable and create another variable equal to *.tar.gz then in if statement I compare var1 (string from user input) with var2 equal to *tar.gz and it works.
Related
I am trying to run a command
mv /var/www/my_folder/reports.html /tmp/
it is running properly. But I want to put a condition like if that file exists then only run the command. Is there anything like that?
I can put a shell file instead.
for shell a tried below thing
if [ -e /var/www/my_folder/reports.html ]
then
mv /var/www/my_folder/reports.html /tmp/
fi
But I need a command. Can some one help me with this?
Moving the file /var/www/my_folder/reports.html only if it exists and regular file:
[ -f "/var/www/my_folder/reports.html" ] && mv "/var/www/my_folder/reports.html" /tmp/
-f - returns true value if file exists and regular file
if exist file and then move or echo messages through standard error output
test -e /var/www/my_folder/reports.html && mv /var/www/my_folder/reports.html /tmp/ || echo "not existing the file" >&2
You can do it simply in a shell script
#!/bin/bash
# Check for the file
ls /var/www/my_folder/ | grep reports.html > /dev/null
# check output of the previous command
if [ $? -eq 0 ]
then
# echo -e "Found file"
mv /var/www/my_folder/reports.html /tmp/
else
# echo -e "File is not in there"
fi
Hope it helps
Maybe your use case is "Create if not exist, then copy always".
then: touch myfile && cp myfile mydest/
I am trying to iterate in a loop, tar a couple of directories with each iteration and then compare the md5 sums of both of them. I notice that my first tar statement produces the tar files one level above the actual path of the directory. i.e. the statement:
tar -czvf ${folder_name}.tar.gz /tmp/psk1/hadoop_validation$ENV/${folder_name}
produces the ${folder_name}.tar.gz in /tmp/psk1/ rather than /tmp/psk1/hadoop_validation$ENV/
and the second tar statement:
tar -czvf ${folder_name}.tar.gz ${edge_base_dir}/wlossf$ENV/app/${folder_name}
doesn't produce the tar file at all. I can't find it even on one level above the actual path.
hdfs dfs -ls /haas/wlf/wlossf$ENV/app | while read rec; do
echo $rec
folder_path=`echo ${rec} | awk -F ' ' '{print $8}'`
folder_name=`echo ${folder_path} | awk -F '/' '{print $6}'`
if [ ! -z ${folder_name} ] && [ ! -z ${folder_path} ]; then
hdfs dfs -get ${folder_path} /tmp/psk1/hadoop_validation$ENV/
if [ $? -eq 0 ]; then
echo "Hadoop to local copy job Successful"
else
echo "Hadoop to local copy job Failed"
fi
tar -czvf ${folder_name}.tar.gz /tmp/psk1/hadoop_validation$ENV/${folder_name}
hadoop_md5=$(md5sum /tmp/psk1/hadoop_validation$ENV/${folder_name}.tar.gz)
tar -czvf ${folder_name}.tar.gz ${edge_base_dir}/wlossf$ENV/app/${folder_name}
edge_md5=$(md5sum ${edge_base_dir}/wlossf$ENV/app/${folder_name}.tar.gz)
if [ ${hadoop_md5} == ${edge_md5} ]; then
echo "${folder_name} is good"
else
echo "${folder_name} is bad"
fi
fi
echo ${folder_name}
echo ${folder_path}
done
What am I missing here? Any help would be appreciated.
Thank you.
As mouviciel said in the comments, tar by default creates the file in the current working directory.
Simply prefix the tar.gz file with the folder and it will create it where you want it:
tar -czvf /tmp/psk1/hadoop_validation$ENV/${folder_name}.tar.gz /tmp/psk1/hadoop_validation$ENV/${folder_name}
Note that as you will be creating the tar inside the same folder that you are archiving, you'll get a file changed as we read it warning as part of the output. Nothing to worry about.
Good Morning,
I am trying to write a korn shell script to look inside a directory that contains loads of files and check that each file also exists with .orig on the end.
For example if a file inside the directory is called 'mercury_1' there must also be a file called 'mercury_1.orig'
If there isn't, it needs to move the mercury_1 file to another location. However if the .orig file exists do nothing and move onto the next file.
I am sure it is really simple but I am not that experienced in writing Linux scripts and help would be greatly appreciated!!
Here's a small ksh snippet to check if a file exists in the current directory
fname=mercury_1
if [ -f $fname ]
then
echo "file exists"
else
echo "file doesn't exit"
fi
Edit:
The updated script that does the said functionality
#/usr/bin/ksh
if [ ! $# -eq 1 ]
then
echo "provide dir"
exit
fi
dir=$1
cd $dir
#process file names not ending with orig
for fname in `ls | grep -v ".orig$"`
do
echo processing file $fname
if [ -d $fname ] #skip directory
then
continue
fi
if [ -f "$fname.orig" ] #if equiv. orig file present
then
echo "file exist"
continue
else
echo "moving"
mv $fname /tmp
fi
done
Hope its of help!
You can use the below script
script.sh :
#!/bin/sh
if [ ! $# -eq 2 ]; then
echo "error";
exit;
fi
for File in $1/*
do
Tfile=${File%%.*}
if [ ! -f $Tfile.orig ]; then
echo "$File"
mv $File $2/
fi
done
Usage:
./script.sh <search directory> <destination dir if file not present>
Here, for each file with extension stripped check if "*.orig" is present, if not then move file to different directory, else do nothing.
Extension is stripped because you don't want to repeat the same steps for *.orig files.
I tested this on OSX (basically mv should not differ to much from linux). My test directory is zbar and destination is /tmp directory
#!/bin/bash
FILES=zbar
cd $FILES
array=$(ls -p |grep -v "/") # we search for file without extension so put them in array and ignore directory
echo $array
for f in $array #loop in array and find .orig file
do
#echo $f
if [ -e "$f.orig" ]
then
echo "found $f.orig"
else
mv -f "$f" "/tmp"
fi
done
I am trying to write a bash/shell script to zip up a specific folder and ignore certain sub-dirs in that folder.
This is the folder I am trying to zip "sync_test5":
My bash script generates an ignore list (based on) and calls the zip function like this:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR="dir_to_ignore dir2_to_ignore"
ignorelist=""
if [ "$SYNC_WEB_ROOT_IGNORE_DIR" != "" ];
then
for ignoredir in $SYNC_WEB_ROOT_IGNORE_DIR
do
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip"
cd $SYNC_WEB_ROOT_BASE_DIR;
zip -r $FILE $SYNC_WEB_ROOT_BACKUP_DIR -x $ignorelist >/dev/null
echo "Done"
Now this script runs without error, however it is not ignoring/excluding the dirs I've specified.
So, I had the shell script output the command it tried to run, which was:
zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
Now If I run the above command directly in putty like this, it works:
So, why doesn't my shell script exclude working as intended? the command that is being executed is identical (in shell and putty directly).
Because backslash quotings in a variable after word splitting are not evaluated.
If you have a='123\4', echo $a would give
123\4
But if you do it directly like echo 123\4, you'd get
1234
Clearly the arguments you pass with the variable and without the variables are different.
You probably just meant to not quote your argument with backslash:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Btw, what actual works is a non-evaluated glob pattern:
zip -r 12-08-2014_072810.website.zip sync_test5 -x 'sync_test5/dir_to_ignore/***' 'sync_test5/dir2_to_ignore/***'
You can verify this with
echo zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
And this is my suggestion:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR=("dir_to_ignore" "dir2_to_ignore")
IGNORE_LIST=()
if [[ -n $SYNC_WEB_ROOT_IGNORE_DIR ]]; then
for IGNORE_DIR in "${SYNC_WEB_ROOT_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/***") ## "$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/*" perhaps is enough?
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip" ## Where is $SYNC_BACKUP_DIR set?
cd "$SYNC_WEB_ROOT_BASE_DIR";
zip -r "$FILE" "$SYNC_WEB_ROOT_BACKUP_DIR" -x "${IGNORE_LIST[#]}" >/dev/null
echo "Done"
This is what I ended up with:
#!/bin/bash
# This script zips a directory, excluding specified files, types and subdirectories.
# while zipping the directory it excludes hidden directories and certain file types
[[ "`/usr/bin/tty`" == "not a tty" ]] && . ~/.bash_profile
DIRECTORY=$(cd `dirname $0` && pwd)
if [[ -z $1 ]]; then
echo "Usage: managed_directory_compressor /your-directory/ zip-file-name"
else
DIRECTORY_TO_COMPRESS=${1%/}
ZIPPED_FILE="$2.zip"
COMPRESS_IGNORE_FILE=("\.git" "*.zip" "*.csv" "*.json" "gulpfile.js" "*.rb" "*.bak" "*.swp" "*.back" "*.merge" "*.txt" "*.sh" "bower_components" "node_modules")
COMPRESS_IGNORE_DIR=("bower_components" "node_modules")
IGNORE_LIST=("*/\.*" "\.* "\/\.*"")
if [[ -n $COMPRESS_IGNORE_FILE ]]; then
for IGNORE_FILES in "${COMPRESS_IGNORE_FILE[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_FILES/*")
done
for IGNORE_DIR in "${COMPRESS_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_DIR/")
done
fi
zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
# echo zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
echo $DIRECTORY_TO_COMPRESS "compressed as" $ZIPPED_FILE.
fi
After a few trial and error, I have managed to fix this problem by changing this line:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
to:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Not sure why this worked, but it does :)
#!/bin/sh
#My script
echo "Are you sure you want to reorganize your files?"
echo "Type y or Y to continue. Anything else will stop the process"
read response
if [ "$response" = "y" ] || [ "$response" = "Y" ]; then
mkdir video
mkdir audio
mkdir text
mv -v *.txt text >> log.txt
mv -v *.wmv video >> log.txt
mv -v *.mov video >> log.txt
mv -v *.mpg video >> log.txt
mv -v *.mp3 audio >> log.txt
mv -v *.wma audio >> log.txt
echo "Yay, it worked!"
else
echo "Nothing happened."
fi
I wrote the script above to organize files into subfolders. For instance the music files will go into an audio folder. Now I would like to take a step further and make it more global.I would like to allow the script to accept a command line argument, which is the folder that contains the unorganized files. This should allow the script to be located and run from anywhere in the file system, and accept any folder of unorganized files.
Example:
organizefiles.sh mystuff/media // subfolders would go inside "media"
the folder media contains all of the media files.
Thank you!
A portion of your script could use the first positional parameter like this:
if [ -d $1 ]
then
mkdir video
mkdir audio
mkdir text
mv -v $1/*.txt text >> log.txt
mv -v $1/*.wmv video >> log.txt
mv -v $1/*.mov video >> log.txt
mv -v $1/*.mpg video >> log.txt
mv -v $1/*.mp3 audio >> log.txt
mv -v $1/*.wma audio >> log.txt
else
echo "The destination directory does not exist"
exit 1
fi
You can refer to the command line parameters as $1, $2, etc. The first one is $1. Here's a good description of how to pass arguments to a script: http://docsrv.sco.com:507/en/OSUserG/_Passing_to_shell_script.html
Scripts has access to arguments on the command line via some variables like this:
$1, $2, ..., $n - refers to first, second up to n arguments.
Example: Typing myscript.sh foo will set foo to the $1 variable.
Bash arguments are fairly straightforward, using a $# format. So for example, you could access the first argument of the command line from your script with $1
In your script, you could do something like so:
if [ -z $1 ]
then
dir = $1
else
dir = './'
fi
Then just add the new $dir variable to the paths in your mv commands. I recommend checking out Bash By Example from IBM. A great article series to teach you Bash.
Note that there may be a petter better to do what I suggested but I am nowhere near an expert in Bash. :-)
here's a simple system. you can use case/esac instead of if/else for neatness. also, rearranged the mv commands a bit
#!/bin/bash
dir=$1
cd $dir
while true
do
echo "Are you sure you want to reorganize your files?"
printf "Type y or Y to continue. Anything else will stop the process: "
read response
case "$response" in
y|Y )
mv -v *.txt text >> log.txt
for vid in "*.mov" "*.wmv" "*.mpg" "*.wma"
do
mv $vid video >> log.txt
done
echo "yay"
break;;
*) echo "Invalid choice";;
esac
done