Retry http request with backoff - (Nestjs - axios - rxjs) throws socket hang up - node.js

Question

[second-Update]: i solved the issue by implmentig retry with promises and try & catch
[first-Update]:
I tried the retrial mechanism with HTTP request with content-type: application/json and it works!! but my issue Is with content type form-data.
I guess it's similar to this problem: Axios interceptor to retry sending FormData
Services architecture
I'm trying to make an HTTP request to service-a from NestJS-app.
and I want to implement retry with backoff logic.
to re-produce service-a failure, I'm restarting its docker container and make the HTTP request.
retry logic implemented as 3 retries.
first time as service-a is restarting.. throws 405 service not available error and make retry.
all 3 retries failed with a socket hang up error.
HTTP request code using axios nestjs wrapper lib
retryWithBackOff rxjs operator implementation
the first call throws a 405 Service Unavailable error.
then application starts retries.
first retry fires after service-a started, failed with error socket hang up
first, second, and third retries failed with socket hang up.
3 sockets hang up errors
my expected behavior is:
when service-a started then the first retry fires, it should work with a successful response.
notice that 3 retries don't log to the Nginx server anything!

Answer 1

While your solution probably works, it could be improved in terms of single responsibility, which RxJS can help with. I use an adapted solution of a code snippet I found once on the web (I can't find the original source any more).
interface GenericRetryStrategy {
getAttempt?(): number;
maxRetryAttempts?: number;
scalingDuration?: number;
maxDuration?: number;
retryFormula?: RetryFormula;
excludedStatusCodes?: number[]; // All errors with these codes will circumvent retry and just return the error
}
const genericRetryStrategy$ =
({
getAttempt,
maxRetryAttempts = 3,
scalingDuration = 1000,
maxDuration = 64000,
retryFormula = 'constant', // time-to-retry-count interpolation
excludedStatusCodes = [], // All errors with these codes will circumvent retry and just return the error
}: GenericRetryStrategy = {}) =>
(error$: Observable<unknown>): Observable<number> =>
error$.pipe(
switchMap((error, i) => {
const retryAttempt = getAttempt ? getAttempt() : i + 1;
// if maximum number of retries have been met
// or response is a error code we don't wish to retry, throw error
if (
retryAttempt > maxRetryAttempts ||
excludedStatusCodes.find(e => e === error.code)
) {
return throwError(error);
}
const retryDuration = getRetryCurve(retryFormula, retryAttempt);
const waitDuration = Math.min(
maxDuration,
retryDuration * scalingDuration,
);
// retry after 1000ms, 2000ms, etc …
return timer(waitDuration);
}),
);
You would then call it like this:
const retryThreeTimes$ = genericRetryStrategy$({
maxRetryAttempts: 3,
excludedStatusCodes: [HttpStatus.PayloadTooLarge, HttpStatus.NotFound] // This will throw the error straight away
});
this.setupUploadAttachements(url, clientApiKey, files, toPoTenantId).pipe(retryWhen(retryThreeTimes$))
This function/operator can now be re-used for all kinds of requests. It is very flexible. It also makes your operator logic more readable, since the complex retry logic sits somewhere else and does not “pollute” your pipe.
You might have to do some adjustment, since axios does return a different error payload, it seems (at least judging from your code examples). Also, if I understood your code correctly, you actually don't want to throw and error when the above error codes apply. In that case, you could add another catchError after the retryWhen and filter these codes, while returning of([]).