python geocoder api memory leak

Jean-BernardGariépy — Mon, 25 Nov 2019 16:45:00 GMT

Hi!

I'm batch geocoding millions of adresses using our dedicated Enterprise ArcGIS Server.

Some of the batch geocoding results end up being pretty bad, so we geocode them again one by one using the same geocoding service... and usually get much better results than batch geocoding (but that's a whole different issue).

This sample code is just to prove the point.

Every single call to the geocode() function make my python process add an extra 3MB of RAM. Eventually it reaches 40GIG+ of RAM. That is not suitable for a production environment.

But sys.getsizeof(geocode) doesn't change.

Any clue?

I've implemented 2 different ways and both do the same.

By the way, the batch geocoder also have a memory leak, but much smaller.

from arcgis.geocoding import geocode

from arcgis.gis import GIS

import sys

gis = GIS('https://our_server/portal')

while True:

single_result = geocode('3623 Rue Saint-Denis, Montréal, QC H2X 3L6')[0]

print(sys.getsizeof(geocode))

# from arcgis.gis import GIS

# from arcgis.geocoding import Geocoder, get_geocoders, geocode

# from time import time

# import datetime

# import sys

# geocoderlist = get_geocoders(GIS())

# geocoder = Geocoder(geocoderlist[0].url)

# #geocoderlist = get_geocoders(GIS('https://our_server/portal'))

# #geocoder = Geocoder(geocoderlist[0].url)

# while True:

# single_result = geocoder._geocode('3623 Rue Saint-Denis, Montréal, QC H2X 3L6')[0]

# print(sys.getsizeof(geocoder))

Re: python geocoder api memory leak

Jean-BernardGariépy — Tue, 26 Nov 2019 16:52:29 GMT

I was using python v3.7

Using python v3.6.8 works fine

topic Re: python geocoder api memory leak in ArcGIS API for Python Questions

python geocoder api memory leak

Re: python geocoder api memory leak