I'm building a bash script to send an email based off the last command. I seem to be having difficulties. Outside of a script the command works fine but when putting it in script it doesn't give the desired outcome.
Here is snippet of script:
grep -vFxf /path/to/first/file /path/to/second/file > /path/to/output/file.txt
if [ -s file.txt ] || echo "file is empty";
then
swaks -t "1#email.com" -f "norply#email.com" --header "Subject: sample" --body "Empty"
else
swaks -t "1#email.com" -f "norply#email.com" --header "subject: sample" --body "Not Empty"
fi
I ran the commands outside of script and I can see that there is data but when I add the commands within script I get the empty output. Please advise . Thank you in advance .
Your condition will always be true, because if [ -s file.txt ] fails, the exit status of the ||-list is the exit status of echo, which is almost guaranteed to be 0. You want to move the echo out of the condition and into the body of the if statement. (And to simplify further, just set the body to a variable and call swaks after the if completes.
if [ -s file.txt ];
then
body="Not Empty"
else
echo "file is empty"
body="Empty"
fi
swaks -t "1#email.com" -f "norply#email.com" --header "subject: sample" --body "$body"
If the only reason you create file.txt is to check if it is empty or not, you can just put the grep command directly in the if condition:
if grep -vFxfq /atph/to/first/file /path/to/second/file; then
body="Not Empty"
else
echo "No output"
body="Empty"
fi
swaks -t "1#email.com" -f "norply#email.com" --header "subject: sample" --body "$body"
Related
I am currently trying to download data from (https://ladsweb.modaps.eosdis.nasa.gov/search/order). They provide a bash file (https://ladsweb.modaps.eosdis.nasa.gov/tools-and-services/data-download-scripts/) which I need to try and edit. I have a .csv file with all of the file extensions I need to download (specifically I need all VNP46A1 data for China from 2015 until now.
In pseudo-code form I would like to add the following:
FOR url_path IN url_list:
recurse "https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/5000/VNP46A1/“+”$url_path"+”.h5" “your_directory”+”$url_path" "TOKEN_HERE"
I need to edit this bash to iterate over the files in the csv and download them into a folder for use later.
The bash file is as follows:
#!/bin/bash
function usage {
echo "Usage:"
echo " $0 [options]"
echo ""
echo "Description:"
echo " This script will recursively download all files if they don't exist"
echo " from a LAADS URL and stores them to the specified path"
echo ""
echo "Options:"
echo " -s|--source [URL] Recursively download files at [URL]"
echo " -d|--destination [path] Store directory structure to [path]"
echo " -t|--token [token] Use app token [token] to authenticate"
echo ""
echo "Dependencies:"
echo " Requires 'jq' which is available as a standalone executable from"
echo " https://stedolan.github.io/jq/download/"
}
function recurse {
local src=$1
local dest=$2
local token=$3
echo "Querying ${src}.json"
for dir in $(curl -s -H "Authorization: Bearer ${token}" ${src}.json | jq '.[] | select(.size==0) | .name' | tr -d '"')
do
echo "Creating ${dest}/${dir}"
mkdir -p "${dest}/${dir}"
echo "Recursing ${src}/${dir}/ for ${dest}/${dir}"
recurse "${src}/${dir}/" "${dest}/${dir}"
done
for file in $(curl -s -H "Authorization: Bearer ${token}" ${src}.json | jq '.[] | select(.size!=0) | .name' | tr -d '"')
do
if [ ! -f ${dest}/${file} ]
then
echo "Downloading $file to ${dest}"
# replace '-s' with '-#' below for download progress bars
curl -s -H "Authorization: Bearer ${token}" ${src}/${file} -o ${dest}/${file}
else
echo "Skipping $file ..."
fi
done
}
POSITIONAL=()
while [[ $# -gt 0 ]]
do
key="$1"
case $key in
-s|--source)
src="$2"
shift # past argument
shift # past value
;;
-d|--destination)
dest="$2"
shift # past argument
shift # past value
;;
-t|--token)
token="$2"
shift # past argument
shift # past value
;;
*) # unknown option
POSITIONAL+=("$1") # save it in an array for later
shift # past argument
;;
esac
done
if [ -z ${src+x} ]
then
echo "Source is not specified"
usage
exit 1
fi
if [ -z ${dest+x} ]
then
echo "Destination is not specified"
usage
exit 1
fi
if [ -z ${token+x} ]
then
echo "Token is not specified"
usage
exit 1
fi
recurse "$src" "$dest" "$token"
and a shortened (for testing purposes) csv file is given as:
/archive/allData/5000/VNP46A1/2015/001/VNP46A1.A2015001.h30v05.001.2019135185504.h5
/archive/allData/5000/VNP46A1/2015/002/VNP46A1.A2015002.h30v05.001.2019136091632.h5
/archive/allData/5000/VNP46A1/2015/003/VNP46A1.A2015003.h30v05.001.2019136075625.h5
/archive/allData/5000/VNP46A1/2015/004/VNP46A1.A2015004.h30v05.001.2019136081706.h5
/archive/allData/5000/VNP46A1/2015/005/VNP46A1.A2015005.h30v05.001.2019136084155.h5
/archive/allData/5000/VNP46A1/2015/006/VNP46A1.A2015006.h30v05.001.2019136084128.h5
/archive/allData/5000/VNP46A1/2015/007/VNP46A1.A2015007.h30v05.001.2019136085336.h5
/archive/allData/5000/VNP46A1/2015/008/VNP46A1.A2015008.h30v05.001.2019136103147.h5
/archive/allData/5000/VNP46A1/2015/009/VNP46A1.A2015009.h30v05.001.2019136100110.h5
Any help or suggestions will be much appreciated.
Kind regards
You want to create a bash script that will loop over each line to download the data that you need, using the script provided by NASA.
For example say the below script was a file called save-data.sh:
#!/bin/bash
while read p; do
./laads-data-download.sh -s $P -d "destination" -t "token"
echo "$p"
done <paths.txt
Example tree structure:
nasa-satellite-data
├── laads-data-download.sh
├── paths.txt
└── save-data.sh
I have a shell Unix running every hour (crontab on CentOS 7).
Inside that shell, a loop read and proceed treatment for all new files find in a defined folder.
At the end of each files's treatment a CURL command is send with some parameters, for example :
curl https://aaaaaa.com/website -d param1=value1 -d param2=value2 ....
Each time the shell is run by crontab, the 1st CURL is correctly converted to a true URL and received by Apache/Tomcat, but all the others are bad. In fact the 2nd and the following CURLs seem not converted in the correct format like
https://aaaaaa.com/website?param1=value1¶m2=value2
but they are sent like
https://aaaaaa.com/website -d param1=value1 -d param2=value2
So the website is unable to treat the parameters properly.
Why the 1st command is correctly converted to a correct URL format and not the following ?
EDIT - EDIT
The part of shell :
#!/bin/bash
...
#======================================================
# FUNCTIONS
#======================================================
UpdateStatus () {
CMD_CURL="${URL_WEBSITE} -d client=CLIENT -d site=TEST -d produit=MEDIASFILES -d action=update"
CMD_CURL="${CMD_CURL} -d codecmd=UPDATE_MEDIA_STATUS"
CMD_CURL="${CMD_CURL} -d idmedia=$4"
CMD_CURL="${CMD_CURL} -d idbatch=$3"
CMD_CURL="${CMD_CURL} -d statusmedia=$2"
if [[ ! -z "$5" ]]; then
CMD_CURL="${CMD_CURL} -d filename=$5"
fi
echo " ${CMD_CURL}" >> $1
CURL_RESULT=`curl -k ${CMD_CURL}`
CURL_RESULT=`echo ${CURL_RESULT} | tr -d ' '`
echo " Result CURL = ${CURL_RESULT}" >> $1
if [ "${CURL_RESULT}" = "OK" ]; then
return 0
fi
return 1
}
#======================================================
# MAIN PROGRAM
#======================================================
echo "----- Batch in progress : `date '+%d/%m/%y - %H:%M:%S'` -----"
for file in $( ls ${DIR_FACTORY_BATCHFILES}/*.batch )
do
...
old_IFS=$IFS
while IFS=';' read <&3 F_STATUS F_FILEIN F_TYPE F_CODE F_ID F_IDPARENT F_TAGID3 F_PROF F_YEARMEDIA F_DATECOURS F_TIMEBEGINCOURS F_LANG || [[ -n "$F_STATUS $F_FILEIN $F_TYPE $F_CODE $F_ID $F_IDPARENT $F_TAGID3 $F_PROF $F_YEARMEDIA $F_DATECOURS $F_TIMEBEGINCOURS $F_LANG" && $F_STATUS ]];
do
...
UpdateStatus ${LOG_FILENAME} ${STATUS_ERROR} ${F_ID} ${F_IDPARENT}
...
done 3< $file
IFS=$Old_IFS
...
done
You need to provide the "-d" flags and values before the URL so:
curl -d param1=value1 -d param2=value2 https://aaaaaa.com/website
Moreover, this command is going to send the parameters/values as POST parameters, not query parameters. You can use the "-G" flag, possibly combined with "--url-encode" to send as query parameters, see:
https://unix.stackexchange.com/questions/86729/any-way-to-encode-the-url-in-curl-command
I have a Linux system running OpenWRT. I am trying to get a shell script to post to a remote server and then split the return string into an array. I get the following error when I try to run it: "line 18: syntax error: unexpected redirection" I know this has been asked before but all the solutions pointed to it being a bash syntax issue. So I changed "#!/bin/sh" to "#!/bin/ash" and have been unable to fix it. I am new to OpenWrt.
Script:
#!/bin/ash
#SetupScript
LAN_IP="$(ifconfig | awk 'FNR==31 {print $2}')"
SerialNo="$(cat /etc/config/example/ID/device_ID.txt)"
curl --request POST 'https://example.com/op_scripts/SetupRequest.php' --data "my_key=mykey" --data "serial=$SerialNo" --data "lan_ip=$LAN_IP"
SetupReply="$(curl http://example.com/resource)"
if [[ "$SetupReply" = *"false" ]]
then
echo 'setup_failure'
else
OIFS="$IFS"
IFS=':'
read -r -a SetupVals <<< "${SetupReply}"
echo ${SetupVals[0]} > /etc/config/FilterWatch/network/curr_http_port
echo ${SetupVals[1]} > /etc/config/FilterWatch/RunTime/RemoteDatabases/db_name
echo ${SetupVals[2]} > /etc/config/FilterWatch/RunTime/RemoteDatabases/db_username
echo ${SetupVals[3]} > /etc/config/FilterWatch/RunTime/RemoteDatabases/db_password
echo 'completed'
IFS="$OIFS"
fi
I am new to shell programming. I have two files:
eg.txt
file.sh
The eg.txt file has some HTML content and file.sh cotains a shell script.
In the script a value is assigned to the temp variable, and that value should be injected into the HTML file.
eg.txt
<html>
Hi MGM ,<br/>
One alert has been received !!<br/>
Here is the event Data.<br/><br/>
<font size=‘1’>{temp}</font>
<br/><br/>
Regards,
WDTS Supports.
</html>
file.sh
echo $1
temp=56
(
echo "To:"$1
echo "Subject: Alert Updates !! "
echo "Content-Type: text/html"
echo cat eg.txt
) | /usr/sbin/sendmail -t
echo "Mail sent !!"
With sed :
sed "s/{temp}/$temp/" eg.txt | /usr/sbin/sendmail -t
You can also use printf to inject variables in your template :
file.sh
temp=56
tpl=$(cat "eg.txt")
printf "$tpl" "$1" "$temp" | /usr/sbin/sendmail -t
eg.txt
To:%s
Subject: Alert Updates !!
Content-Type: text/html
<html>
Hi MGM ,<br/>
One alert has been received !!<br/>
Here is the event Data.<br/><br/>
<font size=‘1’>%s</font>
<br/><br/>
Regards,
WDTS Supports.
</html>
Update:
If multiple variables, just write multiple substitute commands (and update placeholders in eg.txt):
sed "s/{temp1}/$temp1/;s/{temp2}/$temp2/" eg.txt | /usr/sbin/sendmail -t
I have introduced some error checking to your code :
#!/bin/bash
temp=56
if [ -z "$1" ]
then
echo "Usage : ./file.sh user_name_to_mail to"
exit -1
else
if id "$1" >/dev/null 2>&1 #Check if user exists, suppress stdout, stderr
then
mail_header=$(echo -e "To: $1\nSubject: Alert Updates"'!!'"\nContent-Type: text/html\n")
mail_body=$(awk -v var="$temp" '{print gensub(/{temp}/,var,"g"$0)}' eg.txt)
echo -e "$mail_header\n$mail_body" | sendmail -t
else
echo -e "Sorry! Invalid User\n"
exit -1 # The error code is set to detect failure
fi
fi
To prevent the mails going to spam you need to have a valid SPF record for domain from which you're sending email. Check [ this ] for a starting point.
Note:
! is a special character to bash, it is used to refer to previous command. To work around this problem I have used ..Updates"'!!'"\nContent-Type...
Inside the single quotes the ! loses its special meaning.
Interesting reads :
What is an [ SPF ] record?
Open SPF [ documentation ] .
echo $1
temp=56
(
echo "To:"$1
echo "Subject: Alert Updates !! "
echo "Content-Type: text/html"
awk -F "" -v var=$temp '{gsub(/{temp}/,var,$0); print}' < eg.txt
) | /usr/sbin/sendmail -t
echo "Mail sent !!"
added awk to '|' where the temp is stored in awk variable : var which is later replaced
awk -F "" -v var=$temp '{gsub(/{temp}/,var,$0); print}'
Looking for a bash script that will accomplish the followings:
Check a URL (ex. www.google.com)
Looks for a specific text string
if it exists, it does nothing
if it doesnt, then sends out an email to alert someone
I tried the following script, it doesnt do anything, I dont get any email or anything.
#!/bin/sh
URL="URL"
TMPFILE=`mktemp /string_watch.XXXXXX`
curl -s -o ${TMPFILE} ${URL} 2>/dev/null
if [ "$?" -ne "0" ];
then
echo "Unable to connect to ${URL}"
exit 2
fi
RES=`grep -i "StringToLookFor" ${TMPFILE}`
if [ "$?" -ne "0" ];
then
echo "String not found in ${URL}" | mail -s "Alert" your#email
exit 1
fi
echo "String found"
exit 0;
The command
mail -s "Alert" your#email
is pausing to let you enter the text of your email. If you want to send an email with the indicated subject and no text you need to do
mail -s "Alert" your#email < /dev/null