TooManyRequestsException for Boto3 Client Organization - python-3.x

Question

I am fetch all child account from the Master AWS Account by boto3 Organization.
Code is working fine. I am able to get child account list.
But if you run my AWS Lambda function again then it fail to get Child Accounts.
Getting following error:
Error while getting AWS Accounts : An error occurred (TooManyRequestsException) when calling the ListAccounts operation: AWS Organizations can't complete your request because another request is already in progress. Try again later
After 20 to 30 minutes, I can see my code work for once and again raise above exception.
I am Run this code by AWS Gateway + AWS Lambda.
Any idea?
Code:
import boto3
class Organizations(object):
"""AWS Organization"""
def __init__(self, access_key, secret_access_key, session_token=None):
self.client = boto3.client('organizations',
aws_access_key_id=access_key,
aws_secret_access_key=secret_access_key,
aws_session_token=session_token
)
def get_accounts(self, next_token=None, max_results=None):
"""Get Accounts List"""
if next_token and max_results:
result = self.client.list_accounts(NextToken=next_token,
MaxResults=max_results)
elif next_token:
result = self.client.list_accounts(NextToken=next_token)
elif max_results:
result = self.client.list_accounts(MaxResults=max_results)
else:
result = self.client.list_accounts()
return result
class AWSAccounts(object):
""" Return AWS Accounts information. """
def get_aws_accounts(self, access_key, secret_access_key, session_token):
""" Return List of AWS account Details."""
org_obj = Organizations(access_key=access_key,
secret_access_key=secret_access_key,
session_token=session_token)
aws_accounts = []
next_token = None
next_result = None
while True:
response = org_obj.get_accounts(next_token, next_result)
for account in response['Accounts']:
account_details = {"name": account["Name"],
"id": account["Id"],
"admin_role_name": self.account_role_name
}
aws_accounts.append(account_details)
if "NextToken" not in response:
break
next_token = response["NextToken"]
return aws_accounts

Answer 1

By Exception Handling, my code is running successfully.
Catch TooManyRequestsException exception by ClientError exception and retry to call AWS list_accounts API by boto3.
We can add time sleep of 0.1 seconds.
Code:
class AWSAccounts(object):
""" Return AWS Accounts information. """
def get_accounts(self, next_token=None, max_results=None):
"""Get Accounts List"""
# If Master AWS account contain more child accounts(150+) then
# Too-Many-Request Exception is raised by the AWS API(boto3).
# So to fix this issue, we are calling API again by Exception Handling.
result = None
while True:
try:
if next_token and max_results:
result = self.client.list_accounts(NextToken=next_token,
MaxResults=max_results)
elif next_token:
result = self.client.list_accounts(NextToken=next_token)
elif max_results:
result = self.client.list_accounts(MaxResults=max_results)
else:
result = self.client.list_accounts()
except botocore.exceptions.ClientError as err:
response = err.response
print("Failed to list accounts:", response)
if (response and response.get("Error", {}).get("Code") ==
"TooManyRequestsException"):
print("Continue for TooManyRequestsException exception.")
continue
break
return result

Answer 2

Configure your boto3 client to use the built-in standard retry mode:
import boto3
from botocore.config import Config
config = Config(
retries = {
'max_attempts': 10,
'mode': 'standard'
}
)
ec2 = boto3.client('ec2', config=config)
Per the documentation, the default mode is 'legacy' which doesn't handle TooManyRequestsException.
See boto3 documentation about retry configuration here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html