Error 502 while trying to use geopy and ArcGIS - python-3.x

Question

So I keep running into an issue where my code gets a HTTP Error 502:Bad Gateway. I have essentially the same code that works fine, just with a smaller data file. Is it just the amount of data that is messing it up or have I created another issue. Thanks for your help!
import pandas
from geopy.geocoders import ArcGIS
nom = ArcGIS(timeout=300)
info = pandas.read_csv('file here')
info['Address'] = info['city'] + ', ' + info['state'] + ', ' + 'USA'
info['Coordinates'] = info['Address'].apply(nom.geocode)
print(info)
Traceback (most recent call last):
File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\site-packages\geopy\geocoders\base.py", line 367, in _call_geocoder
page = requester(req, timeout=timeout, **kwargs)
File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 531, in open
response = meth(req, response)
File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 640, in http_response
response = self.parent.error(
File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 569, in error
return self._call_chain(*args)
File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 502, in _call_chain
result = func(*args)
File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/roger/PycharmProjects/", line 9, in <module>
info['Coordinates'] = info['Address'].apply(nom.geocode)
File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\series.py", line 3848, in apply
mapped = lib.map_infer(values, f, convert=convert_dtype)
File "pandas\_libs\lib.pyx", line 2329, in pandas._libs.lib.map_infer
File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\site-packages\geopy\geocoders\arcgis.py", line 197, in geocode
response = self._call_geocoder(url, timeout=timeout)
File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\site-packages\geopy\geocoders\base.py", line 389, in _call_geocoder
raise ERROR_CODE_MAP[http_code](message)
geopy.exc.GeocoderServiceError: HTTP Error 502: Bad Gateway

Answer 1

I wound up solving the problem by batching out the data in groups of 50. This was the most I could do without ArcGIS giving me a 502 error. Hope this helps
info['Address'] = info['city'] + ', ' + info['state'] + ', ' + 'USA'
total_length = len(info)
def make_file(data):
data.to_csv('file here', mode='a')
query_start = 0
query_end = query_start + 50
while query_start < total_length + 1:
brevinfo = info[query_start:query_end]
brevinfo['Coordinates'] = brevinfo['Address'].apply(nom.geocode)
brevinfo["Latitude"] = brevinfo["Coordinates"].apply(lambda x:
x.latitude if x != None else None)
brevinfo["Longitude"] = brevinfo["Coordinates"].apply(lambda x:
x.longitude if x != None else None)
print(brevinfo)
make_file(brevinfo)
query_start += 50
query_end += 50
time.sleep(1)