Handling exceptions in TThread.Execute to make it uninterruptible - multithreading

Question

There is a multi-threaded app, which runs 24/7. A correct disposal of resources as well as a proper exceptions handling (including EAccessViolation) are key factors.
I am a little bit stuck on understanding how to correctly nest exception-handling blocks within a thread functions.
Among TMyThread.Execute there are two helper-function:
function LoadHtml(const AUrl: sting): string - a simple wrapper for TIdHTTP.Get()
function ParsePage(const Id: string): TOffers - a parser/DB-updater function
Execute starts querying database for the initial recordset of IDs. Then it starts a while not rs.Eof do loop where calls ParsePage which is a main processor.
ParsePage loads HTML (care of LoadHtml), then performs some string parsing operations and finally updates database.
Here is a code structure: (details omitted for brevity)
{Wrapper-function to load HTML page}
function TMyThread.LoadHtml(const AUrl: string): string;
var
Response: TStringStream;
HTTP: TIdHTTP;
begin
Result := '';
Response := TStringStream.Create('');
try
try
HTTP := TIdHTTP.Create(nil);
HTTP.ReadTimeout := 10000;
HTTP.Response.KeepAlive := false;
try
HTTP.Get(AUrl, Response);
if HTTP.ResponseCode = 200 then Result := Response.DataString;
finally
HTTP.Free;
end;
finally
Response.Free;
end;
except
//This code will run only on exception and *after* freeing all resources?
on E: EIdHTTPProtocolException do
if E.ErrorCode = 404 then
raise EMyOwnHTTPNotFoundError.Create('Page not found');
else
HandleErrorAndLogItToDB(E.Class);
end;
end;
{Loads HTML, processes it and updates DB}
function TMyThread.ParsePage(const Id: string): TOffers;
var
RawHTML: string;
Offer: TOffer; //a simple record to store key offer details;
begin
Result := TOffers.Create;
try {top-level try..except block}
try {Critical request. If it fails I want to move}
RawHTML := LoadHtml('http://onlinetrade.com/offer.html?id=' + Id);
except
on E: EMyOwnHTTPNotFoundError do {Defined in function LoadHtml()}
//Update DB: product does not exist.
else
HandleErrorAndLogItToDB(E.Class);
end;
end;
try
//Preform some basing string operations on RawHTML
except
on E: Exception do HandleErrorAndLogItToDB(E.Class);
end;
try {Iterate through some blocks of data and put them in the Offers: TList}
for i := 0 to N do
begin
//Set up TOffer record
Result.Add(Offer);
end
finally
FreeAndNil(Offer);
end;
except
on E: Exception do
begin
HandleErrorAndLogItToDB(E.Class);
FreeAndNil(Result);
raise; {does this return control to Execute?}
end;
end;
end;
Now Execute:
procedure TMyThread.Execute;
var
j: Integer;
s: string;
Offers: TOffers; {Is a simple TList to store a collection of TOffer (record)}
begin
inherited;
CoInitialize(nil); {ADO is in da house}
try {top-level try..except block}
try {nested try..finally to call CoUninitialize}
try {A critical operation which sources all further operations}
rs := AdoQuery('GetSomeRecords ' + IntToStr(SomeId));
except
on E: Exception do
begin
HandleErrorAndLogItToDB(E.Class);
Exit; {DB-query error means no reason to continue}
end;
end;
while not rs.EOF do
begin
try //a loop top-level try..except handler
Offers := ParsePage(rs.Fields['Id'].Value);
try //nested resource freeer
begin
try //nested try..except to handle DB queries
for j := 0 to N do with Offers.Items[j] do
AdoUpdateDB; //Update DB
Synchronize(UpdateProgressBar);
except
on E: Exception do
begin
HandleErrorAndLogItToDB(E.Class);
Continue; //as suggested
raise; //as suggested
end;
end;
rs.MoveNext;
end;
finally
FreeAndNil(Offers);
end;
except
on E: Exception do HandleErrorAndLogItToDB(E.Class);
end;
end; //end while..do loop
Synchronize(ResetProgressBar);
finally
CoUnitialize;
end;
except
on E: Exception do
begin
//Make everything possible to keep the thread running. No matter of:
//- HTTP/404 - Not Found exceptions (which I handle)
//- UpdateDatabase fails
//- String operation exceptions
//If anything critical occurs, Execute() shall just go to the next offer
//even if the current one is not properly processed.
end;
end;
Looking at this code I think, I try to handle too many exceptions which I probably do not need to handle, just passing them to the most outer try..except handler in Execute. There are just a couple of exceptions I really need to handle: initial database query and EMyOwnHTTPNotFoundError (to set a flag that the offer doesn't exist). I read somewhere a suggestion not to explicitly chase for exception handling unless you really need it...
However it is very important that the thread keeps on running no matter of which exceptions are thrown inside/outside any code blocks. The idea is to completely ignore exceptions and never break either while..do loop or stop the thread. At the same time, correctly disposing resources is also a must.
I would be grateful for any suggestions/comments on how to improve this code.

Answer 1

There is no good way to "handle" an access violation, or really any exception that does not indicate a specific condition that your code has already planned for. (For example, a File Not Found exception can be handled in a good way if you can simply tell the user to ask for another file.)
If an exception is raised as the result of a bug, it means something is happening in your code that you did not plan for. Your code rests on a bunch of assumptions about things going right, things working as planned, and when an unexpected exception is raised, it means that those assumptions no longer necessarily hold. The best thing to do at that point is produce an error report to send back to you, and then shut down as quickly as possible.
Why? Because one of your assumptions that may no longer hold is that "critical data is in a valid, non-corrupted state." If the program keeps going, blindly following the assumption that all its data is good and then acting on it, it can turn a small problem into a much bigger one very, very quickly.
I completely understand the desire to make the program keep on going no matter what, but unfortunately it fundamentally conflicts with reality. The only sane thing to do when you get an unhandled exception--especially something like an access violation that can only be the result of buggy code of some variety--is to produce an error report and shut down.
If downtime is a Very Bad Thing, you could have something in place to make sure to start back up again as quickly as possible. This will keep you running, but it will reset your invariants (fundamental assumptions) and clear out the corrupt data. But for the love of all that is binary, shut the program down, and do it right away.
Then take the error report and fix your bug.