How to manipulate nested GET calls - python-3.x

Question

My goal is to make multiple Get calls from the results of the first call, then concatenate the clients informations into dataframe. Preferable a faster way because I have a million clients ids
--------------------------------------
| id | name | country | city | phone |
--------------------------------------
| 1 | Leo | France | Paris | 212...|
| . | .. | .. | .. | .. |
| 100| Bale | UK | London| 514...|
The basic request / results (all clients):
import requests
from requests.auth import HTTPBasicAuth
# the initial request which returns all clients
res0 = requests.get('https://x.y.z/api/v1/data/clients', auth=HTTPBasicAuth('me', 'blabla'))
# results
<?xml version="1.0" ?>
<queryResponse>
<entityId type="clients" url="https://x.y.z/api/v1/data/clients/1">1</entityId>
...
...
<entityId type="clients" url="https://x.y.z/api/v1/data/clients/100">100</entityId>
</queryResponse>
The detailed request / results (client infos)
# this request allows to get client informations
res1 = requests.get('https://x.y.z/api/v1/data/clients/1', auth=HTTPBasicAuth('me', 'blabla'))
# results
<queryResponse>
<entity type="client_infos" url="https://x.y.z/api/v1/data/clients/1">
<client_infos displayName="1" id="1">
<name>Leo Massina</name>
<country>France</country>
<city>1607695021057</city>
<phone>[phone]</phone>
</client_infos >
</entity>

Answer 1

You can use lxml to parse the response and make the new calls, retrieve the tags and text in a dictionary and create the dataframe:
(I used fors for clarity, you can optimize the code if needed)
Also, I did not retrieve ids, if needed they can be retrieved as attribute of client_infos tags.
from lxml import etree
root = etree.fromstring(res0)
reqentity = []
data = {"name":[], "country":[], "city":[], "phone":[]}
for entity in root.findall('./entityId'):
reqentity.append(requests.get(entity.attrib['url'], auth=HTTPBasicAuth('me', 'blabla')))
for entity in reqentity:
entity = etree.fromstring(entity)
for item in entity.findall(".//client_infos//"):
data[item.tag].append(item.text)
df = pd.DataFrame(data)