Error with batch geocoding list

460
9
05-12-2021 11:12 AM
LouieRodriguez
New Contributor III

I have a script that pulls in project data from a source, creates a list of unique address strings to geocode, passes those to the arcgis.geocoding batch_geocode method to obtain a list of the geocoded information for each address from the projects.

At smaller scales, the script runs fine (10-100 records), but when I scale the amount of addresses to geocode over a thousand, I find that some addresses that are perfectly geocoded before are now just returned as 'None' and over the course of 1000-2000 records, many are returned as None which breaks the script.

Any ideas on why this is or if there is some limit to the amount of records that can be geocoded at one time? Thanks in advance!

0 Kudos
9 Replies
JoeBorgione
MVP Esteemed Contributor

I guess you are geocoding against the ArcGIS Online Locators?  Are you out of credits?

can't wait to retire....
0 Kudos
LouieRodriguez
New Contributor III

Yes I am geocoding against the ArcGIS Online Locators but my organization has a few thousand credits and I am not close to reaching that number yet at least. I'll include two files, one from a smaller batch of geocoding that worked fine and one from a batch that was bigger and broke. When formatting these in VS Code you can see the bigger file breaks formatting at some point and I'm not sure if that is exactly why the geocoding is breaking but it seems to be part of it.

I will lastly note that in these recent examples none of the address come back as 'None' like I had mentioned before but still break when trying to call the individual Lat/Long's from this list.

0 Kudos
JoeBorgione
MVP Esteemed Contributor

When formatting these in VS Code...

I'm confused here: I thought you posted this to the Python space.  The files you zipped a json files.  What is it you need help with?

can't wait to retire....
0 Kudos
LouieRodriguez
New Contributor III

I'm sorry, I guess to elaborate a bit, this is a Python script that pulls project information from an API as a JSON object. I put the unique addressed into a Python list to be passed to the batch_geocode method which returns its own list of geocode information (address, x, y, etc.). When trying to read that list for specific values, I am getting an error, usually this: 

latitude = geocodeList[index]["location"]["y"]
KeyError: 'location'

Location is just part of the specific address information from the geocode list and usually works fine with smaller batches. I hope this helps or I can try to share more to help show what I am trying to do.

0 Kudos
JoeBorgione
MVP Esteemed Contributor

Sorry, can't help you...

can't wait to retire....
0 Kudos
KUMBIRAIMATINGO1
New Contributor II

I just ran into the same error today while building a geocoder with Python. Any luck on a way around this since you posted this?

0 Kudos
LouieRodriguez
New Contributor III

So for me, it turned out the issue was in the MaxBatchSize for the geocoder. I found this line of code to tell you that number:

print("MaxBatchSize : " + str(geocoder.properties.locatorProperties.MaxBatchSize))
 
MaxBatchSize tells you how many records you can geocode in one go, and I found that when printing the amount of records I was potentially geocoding, it was greater than that number (1000 for me, I don't know if it is different for others)
KumbiraiMatingo
New Contributor

Thanks very much. I suppose the maxBatchSize depends on one's user role and access. 

0 Kudos
LouieRodriguez
New Contributor III

Yes, in my case it was based on how my organizations ESRI account was set up. I assume you can look into how it is set up for you or if it can change somewhere in your account settings.