python geocoder api memory leak

261
1
Jump to solution
11-25-2019 08:45 AM
Jean-BernardGariépy
New Contributor III

Hi!

I'm batch geocoding millions of adresses using our dedicated Enterprise ArcGIS Server.

Some of the batch geocoding results end up being pretty bad, so we geocode them again one by one using the same geocoding service... and usually get much better results than batch geocoding (but that's a whole different issue).

This sample code is just to prove the point.

Every single call to the geocode() function make my python process add an extra 3MB of RAM. Eventually it reaches 40GIG+ of RAM. That is not suitable for a production environment. 

But sys.getsizeof(geocode) doesn't change.

Any clue?

I've implemented 2 different ways and both do the same.

By the way, the batch geocoder also have a memory leak, but much smaller.

from arcgis.geocoding import geocode
from arcgis.gis import GIS
import sys

gis = GIS('https://our_server/portal')

while True:
    single_result = geocode('3623 Rue Saint-Denis, Montréal, QC H2X 3L6')[0]
    print(sys.getsizeof(geocode))
# from arcgis.gis import GIS
# from arcgis.geocoding import Geocoder, get_geocoders, geocode
# from time import time
# import datetime
# import sys

# geocoderlist = get_geocoders(GIS())
# geocoder = Geocoder(geocoderlist[0].url)

# #geocoderlist = get_geocoders(GIS('https://our_server/portal'))
# #geocoder = Geocoder(geocoderlist[0].url)

# while True:
#     single_result = geocoder._geocode('3623 Rue Saint-Denis, Montréal, QC H2X 3L6')[0]
#     print(sys.getsizeof(geocoder))
0 Kudos
1 Solution

Accepted Solutions
Jean-BernardGariépy
New Contributor III

I was using python v3.7

Using python v3.6.8 works fine

View solution in original post

0 Kudos
1 Reply
Jean-BernardGariépy
New Contributor III

I was using python v3.7

Using python v3.6.8 works fine

0 Kudos