How to check if the file is downloaded with curl - linux

Question

I am writing a shell script, I need to download 6 files from internet, In the script I have done it as
curl a
curl b
curl c
and so on
It works, but sometimes curl: (7) couldn't connect to host for some files in the middle of script for example, It will successfully download a but miss the file b with the above error and will download the file c. I would like to catch this error so that my code will execute with the successful download of all the files.

Answer 1

You can use a loop:
while true; do
curl --fail b && break
done
The loop won't break until b is downloaded. You can make it a retry function which you can call if a download fails on the first try:
retry(){
while true; do
curl --fail "$1" && break ||
echo "Download failed for url: $1
retrying..."
done
}
Then do this:
curl --fail a || retry a
curl --fail b || retry b
curl --fail c || retry c
If you just want to silent the error messages then:
curl a 2>/dev/null
curl b 2>/dev/null
...
Or if you want to just detect the error then:
if ! curl --fail a; then
echo "Failed"
fi
or, a one liner:
curl --fail a || echo Failed
If you want to exit after a failure and also show your own message:
curl --fail a 2>/dev/null || { echo failed; exit 1; }

Answer 2

You could chain them with &&...
curl --fail a && curl --fail b && curl --fail c...
Update: as #nwk pointed out below, we need to add --fail to make curl fail on bad http codes.

Answer 3

Put set -e at the beginning of the shell script and use curl --fail. E.g.,
#!/bin/sh
set -e
curl --fail http://example.com/a
curl --fail http://example.com/b
curl --fail http://example.com/c
The set -e will make the script stop with an error on the first unsuccessful command (one with an exit status ≠ zero).

Answer 4

You can check the status of execution of the last command by looking at the $? shell variable.
Execute command below to check the status of the last command anything other 0 shall indicate an error.
echo $?
curl a
if [ "$?" -gt 0 ]
then
echo "Error downloading file a. Exiting"
exit
fi
curl b
if [ "$?" -gt 0 ]
then
echo "Error downloading file b. Exiting"
exit
fi
...
A simple modified form after #andlrc suggestion:
if ! curl a ; then echo "Got error downloading a"; fi
if ! curl b ; then echo "Got error downloading b"; fi
if ! curl c ; then echo "Got error downloading c"; fi

Answer 5

for file2get in a b c d;
do
do :;
until curl --fail $file2get;
done
Or add an iterator counter to prevent endless looping