breaking the loop in bash programming - linux

This code is working fine with me I am new to bash programming I want loop to run only once and I dont want loop to run and watermark the all videos. Instead I want to break the loop when first video is done.
#!/bin/bash
#!!!!!!VARIABLES!!!!!!
VIDEOS_PATH='/home/danny/public_html/videowork/'
LOGO_PATH='/home/danny/public_html/watermark_pics/'
DATABASE_INFORMATION='/home/danny/public_html/videowork/db.txt'
#!!!!!!!!!!!!!!!!!!!!!
if find $VIDEOS_PATH -name '*.mp4' 2> /dev/null
then
for file in $(find $VIDEOS_PATH -name '*.mp4')
do
echo "User: ".$(whoami)
echo "File: "$file" has been detected"
sitename=$(echo $file | awk -F $VIDEOS_PATH '{print $2}')
sitename=$(echo $sitename |awk -F '/' '{print $1}')
echo "File sitename is: "$sitename
logo=$LOGO_PATH$sitename.png
echo "Watermark picture has been located in: "$logo
echo "Encoding "$file" to /home/"$sitename"/public_html/yt/"$(basename $file)
ffmpeg -i $file -vcodec libx264 -acodec copy -vf "movie=$logo [watermark]; [in][watermark] overlay=10:10 [out]" /home/$sitename/public_html/yt/$(basename $file)
thumbnail=$(basename $file .mp4)
thumbnail=$VIDEOS_PATH$sitename"/"$(echo $thumbnail).jpg
mv $thumbnail /home/$sitename/public_html/yt/
rm $file
echo "/home/$sitename/public_html/yt/$(basename $file)"
for line in $(cat $DATABASE_INFORMATION)
do
database=$(echo $line | awk -F '|' '{print $1}')
password=$(echo $line | awk -F '|' '{print $2}')
echo "Database detected: "$database
echo "Password: "$password
videoid=$(basename $file .mp4)
if [[ $(echo $database) == $(echo $sitename) ]]
then
php -f /home/danny/public_html/videowork/database_job.php -- $(echo $database) $(echo $password) $(echo $videoid)
#echo "php -f database_job.php -- "$(echo $database)" "$(echo $password)" "$(echo $videoid)
fi
done
done
fi
If I add "break" before last fi? would it be fine? I tried but it didnt work well

Using a looping construct only to break after the first iteration is silly. Instead, don't use that construct at all. For instance, with bash 4.x or newer, you can completely avoid use of find entirely:
#!/bin/bash
shopt -s globstar nullglob
videos=( "$VIDEOS_PATH"/**/*.mp4 )
(( ${#videos[#]} )) || { echo "No videos found" >&2; exit 1; }
file=${videos[0]}
# ...transcode "$file" here
For compatibility with bash 3.x, one might change this a bit to still use find (albeit with -print0 and an appropriate read construct to handle filenames with unusual characters):
#!/bin/bash
IFS= read -r -d '' file < <(find "$VIDEOS_PATH" -name '*.mp4' -print0)
[[ $file ]] || { echo "No videos found" >&2; exit 1; }
# ...transcode "$file" here

Related

sed is not working for commenting a line in a file using bash script

I have created a bash script that is used to modify the ulimit of open files in the RHEL server.
so i have reading the lines in the file /etc/security/limits.conf and if the soft/hard limit of the open files are less than 10000 for '*' domain i am commenting the line and adding a new line with soft/hard limit as 10000.
The Script is working as designed but the sed command to comment a line in the script is not working.
Please find the full script below :-
#!/bin/sh
#This script would be called by '' to set ulimit values for open files in unix servers.
#
configfile=/etc/security/limits.conf
help(){
echo "usage: $0 <LimitValue>"
echo -e "where\t--LimitValue= No of files you want all the users to open"
exit 1
}
modifyulimit()
{
grep '*\s*hard\s*nofile\s*' $configfile | while read -r line ; do
firstChar="$(echo $line | xargs | cut -c1-1)"
if [ "$firstChar" != "#" ];then
hardValue="$(echo $line | rev | cut -d ' ' -f1 | rev)"
if [[ "$hardValue" -ge "$1" ]]; then
echo ""
else
sed -i -e 's/$line/#$line/g' $configfile
echo "* hard nofile $1" >> $configfile
fi
else
echo ""
fi
done
grep '*\s*soft\s*nofile\s*' $configfile | while read -r line ; do
firstChar="$(echo $line | xargs | cut -c1-1)"
if [ "$firstChar" != "#" ];then
hardValue="$(echo $line | rev | cut -d ' ' -f1 | rev)"
if [[ "$hardValue" -ge "$1" ]]; then
echo ""
else
sed -i -e 's/$line/#$line/g' $configfile
echo "* hard nofile $1" >> $configfile
fi
else
echo ""
fi
done
}
deleteEofTag(){
sed -i "/\b\(End of file\)\b/d" $configfile
}
addEofTag()
{
echo "#################End of file###################" >> $configfile
}
#-------------Execution of the script starts here ----------------------
if [ $# -ne 1 ];
then
help
else
modifyulimit $1
deleteEofTag
addEofTag
fi
The command sed -i -e 's/$line/#$line/g' $configfile when executed from the terminal is working absolutely fine and it is commenting the line but it is not working when i am executing it from the unix shell script.
interpolation does not work in single quote
use double quote and try
sed -i -e 's/$line/#$line/g'
sed -i -e "s/$line/#$line/g"
also you might try:
sed -i -e s/${line}/#${line}/g
as this will tell the script to take the value of the variable instead of variable as such.

Incorrect results from Md5 sum in bash shell script

I'm missing some images that should have been archived when this script runs. I think this may be to do with my indentations or my Md5 sum. I have tried everything I can think of.
here is the code with out the correct indentations:
#!/bin/sh
if [ ! -d "$1" ]; then
echo Directory "$1" cannot be found. Please try again.
exit
fi
if [ $# -eq 1 ]; then
echo "usage: Phar image_path archive_path"
exit
fi
if [ -d "$2" ]; then
echo "archive exists"
else
echo "the directory 'archive' does't exist. Creating directory 'archive'."
mkdir -p ~/archive
fi
find $1 -iname "IMG_[0-9][0-9][0-9][0-9].JPG" | cat > list.txt
[ -f ~/my-documents/md5.txt ] && rm md5.txt || break
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
sort -k 1,1 -u md5.txt | cat > uniquemd5.txt
cut -d " " -f 2- uniquemd5.txt > uniquelist.txt
sort uniquelist.txt -r -o uniquelist.txt
for line in $(cat uniquelist.txt)
do
file=$(basename $line) path="$2/file"
if [ ! -f $path ];
then
cp $line $2
else
cp $line $path.JPG
fi
rm uniquelist.txt md5.txt uniquemd5.txt list.txt
done
This loop
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
should probably be
while read line;
do md5sum "$line"
done < list.txt > md5.txt
Quote parameter expansions, and it's unclear why you needed.

How to use the case statement to call functions?

Hi I have used this code to find the .java file from the given path and generate the list of the .java files from the path to output file,and count total number of tabs and spaces in each java file.now i need to write the two while loops in function and call that function in case statement, how to do this?
Code:
#!/bin/sh
#
output=/home/user/Desktop/file-list.txt
path=/home/user/Desktop
find $path -type f -name \*.java > $output
echo "Generating files list..."
echo "Done"
while IFS= read file
do
if [ -f "$file" ]; then
spaces=$(tr -cd '\s' < "$file" | wc -c);
echo "$spaces spaces in file $file" >> "/home/user/Desktop/space-count.txt"
fi
done < "$output"
echo "Space Count Done!"
while IFS= read file
do
if [ -f "$file" ]; then
tabs=$(tr -cd '\t' < "$file" | wc -c);
echo "$tabs tabs in file $file" >> "/home/user/Desktop/tab-count.txt"
fi
done < "$output"
echo "Tab Count Done!"
Here is an example of case, but the name of the character could be passed to the function
thereby eliminating the need to case the char.
function count_char
{
flist=$1
ch=$2
case $ch in
" ") chName=space;;
"\t") chName=tab;;
esac
while IFS= read file
do
if [ -f "$file" ]; then
tot=$(tr -cd $ch < "$file" | wc -c);
echo "$tot $chName in file $file" >> "/home/user/Desktop/tab-count.txt"
fi
done < "$flist"
}
# setup code, find command here
# ....
count_char $output " "
count_char $output "\t"

breaking the parameters of shell script into several causes errors

I have a shell script using a phantom js function
The Phantom JS function crashes when it is run on too many urls apparently, I tried to rewrite it to only do part at a time but I get an error where the $url variable does not change so it always copies down the same url though it appears to write down the actual size of the site.
If I could be helped through that error that would be glorious.
#!/bin/bash
echo "Importing URLs..."
file=sizeurls.csv
url=""
while IFS= read -r line
do
url+=" "
url+=$line
done < "$file"
echo "Gathering page sizes..."
phantomjs yslow.js --info basic --format plain $url | grep 'url\|size' > temp.txt
echo "Formatting data..."
sed -i 's/size://g' temp.txt
sed -i 's/url://g' temp.txt
paste - - -d, < temp.txt > pagesize.csv
echo "Done!"
version that is supposed to do part at a time, it may be mortally screwed because I just messed with it a little bit an d I am not sure I returned it to the previous state
#!/bin/bash
echo "Importing URLs..."
file=sizeurls.csv
url=""
i=0;
while IFS= read -r line
do
while [ $i -le 10 ] #10 at a time i < 10
do
url+=" "
url+=$line
i=$((i+1));
done < "$file"
phantomjs yslow.js --info basic --format plain $url | grep 'url\|size' >> temp.txt
#echo "Formatting data..."
sed -i 's/size://g' temp.txt
sed -i 's/url://g' temp.txt
paste - - -d, < temp.txt >> pagesize.csv
done < "$file"
i = 0
echo "Done!"
Why not simply do one at a time?
#!/bin/bash
echo "Importing URLs..."
file=sizeurls.csv
echo "Gathering page sizes..."
while IFS= read -r url
do
phantomjs yslow.js --info basic --format plain $url | grep 'url\|size'
done < "$file" > temp.txt
echo "Formatting data..."
sed -i -e 's/size://g' -e 's/url://g' temp.txt
paste - - -d, < temp.txt > pagesize.csv
echo "Done!"
This may give you some ideas (untested, but looks about right). The processing is moved to a function where it is called every 10th URL, and again at the end if there are any leftover URLs.
#!/bin/bash
echo "Importing URLs..."
file=sizeurls.csv
rm pagesize.csv
ProcessURLs () {
echo "Gathering page sizes..."
phantomjs yslow.js --info basic --format plain $# | grep 'url\|size' > temp.txt
echo "Formatting data..."
sed -i 's/size://g' temp.txt
sed -i 's/url://g' temp.txt
paste - - -d, < temp.txt >> pagesize.csv
}
url=""
count=0
while IFS= read -r line
do
url+="$line$ "
(( count++ ))
# Procss URLs in 10-URL chunks
if [[ $count -ge 10 ]] ; then
ProcessURLs $url
url=''
count=0
fi
done < "$file"
# Handle any remaining URLs
[ -n "$url" ] && ProcessURLs $url
echo "Done!"

Need help with my bash script

need some help with my script. I wanna convert ISO files to UTF-8. The problem is I don't know how to write the IF:
if [ `file -b {}` = "$UTF8" ] \
right and how can tell the sed program - that it ignores # comments ?
Here's my script :
#!/bin/bash
clear
echo -e '\E[37mThis script encodes recursively files from \E[31mISO-8859-1 \E[37mto \E[31mUTF-8 \E[37musing iconv.'
echo "Files of the following coded character sets will be encode: "
echo -e '\E[32m'
a='*.java'
b='*.txt'
c='*.php'
d='*.html'
e='*.aj'
f='*.patch'
g='*.css'
h='*.js'
i='*.conf'
j='*.jsp'
k='*.sh'
l='*.py'
m='*.pl'
n='*.rb'
for x in "$a" "$b" "$c" "$d" "$e" "$f" "$g" "$h" "$i" "$j" "$k" "$l" "$m" "$n"
do
echo $x
done
echo
tput sgr0
#
# TODO: COMMENTS aren't ignored
# TOOD: IF-THEN aren't working right
#
for y in "$a" "$b" "$c" "$d" "$e" "$f" "$g" "$h" "$i" "$j" "$k" "$l" "$m" "$n"
do
echo -e "\E[37mencoding all <\E[32m$y\E[37m> files ..."
find . -name "$y" -exec sh -c "( \
UTF=".*UTF-8 Unicode.*" \
FILE={} \
if [ `file -b {}` = "$UTF8" ] \
then \
iconv -f latin1 -t UTF-8 {} -o {}.iconv ; \
sed -n '
{
s/^ *#/#/#.*//g;
s/ä/0xE4;/g;
s/Ä/0xC4;/g;
s/ü/0xFC;/g;
s/Ü/0xDC;/g;
s/ö/0xF6;/g;
s/Ö/0xD6;/g;
s/ß/0xDF;/g;
p;
} {}.iconv > {}.iconv_sed \ '
mv {}.iconv_sed {} && rm {}.iconv ; \
else \
echo "$FILE is a UTF8 file. " \
fi \
)" \;
echo -e '\E[33m*** done ***'
done
echo
tput sgr0
exit 0
thanks
There appear to be more than a few things wrong in your script (for example, I don't see the "UTF8" variable defined anywhere), but you've made it extremely difficult on yourself in terms of debugging it. If it were me, I would:
put all the find's sh -c "... crap in a separate script so you can test it separately
if [ "`file -b $1`" = ...
probably put the sed stuff in a separate function and test that
not use sed -n and then explicitly p; every line, that's silly
properly quote the sed script; I believe you are trying to do redirection inside it
... five suggestions should be enough to get you started. Suggestion 0 is "write a more specific title for your question"

Resources