topic Re: Using a Custom Geocoder in State & Local Government Questions

Using a Custom Geocoder

ScottGeis1 — Fri, 07 Sep 2018 16:43:22 GMT

In NYS we have a custom ESRI geocoding service that can geocode to rooftop level accuracy, and I am trying to use for batch geocoding in a Jupyter notebook using python. However when I attempt to perform a batch_geocode operation I am not convinced that the address is being sent to the NYS service rather than the ESRI universal geocoding service. Here is my code:

In [1]: # Import the necessary libraries

from arcgis.gis import GIS
from arcgis.geocoding import Geocoder, get_geocoders, batch_geocode, geocode

In [2]: #Set custom ESRI geocoding service url to the NYS Geocoding Service

nys_gcdr_url = 'Locators/Street_and_Address_Composite (GeocodeServer) '

In [3]: #Establish my connection to ArcGIS Online and set the preferred geocoder to the NYS geocoder

gis = GIS("http://www.arcgis.com", "username", "password")

esrinl_geocoder = Geocoder(nys_gcdr_url, gis)

esrinl_geocoder

Out [3]: <Geocoder url:"https://gisservices.its.ny.gov/arcgis/rest/services/Locators/Street_and_Address_Composite/GeocodeServe r">

In [4]:   addresses = ["80 South Swan Street, Albany, NY, 12201",
                                 "10 B Airline Drive, Albany, NY, 12205",
                                 "800 North Pearl Street, Albany, NY, 12204",
                                 "5826 Fayetteville Rd, Durham, NC 27713"]

In [6]: results = batch_geocode(addresses, esrinl_geocoder)

In [7]: for result in results:
print("Score " + str(result['score']) + " : " + result['address'])

Out [7]:

Score 99.52 : 80 S Swan St, Albany, New York, 12210 Score 100 : 10 Airline Dr, Albany, New York, 12205 Score 100 : 800 N Pearl St, Albany, New York, 12204 Score 100 : 5826 Fayetteville Rd, Durham, North Carolina, 27713

In [8]: results[1]['location']

Out [8]: {'x': -73.81899493299994, 'y': 42.73686765800005}

My issue...

From what I can gather, the gis = GIS() in Input [3]: establishes a connection to ArcGIS Online using my developer account credentials. Even though I set the Geocoder to the NYS geocoder url with esrinl_geocoder, I’m wondering which geocoder is actually being used? The results/scores I output in the code are higher than I expect them to be, for example I get a score of 95.0 when I run the first address against the NYS geocoder here (Find Address Candidates: (Locators/Street_and_Address_Composite) ). Also the x,y for the address vary based on method:

{'x': -73.81899493299994, 'y': 42.73686765800005} for ESRI Python and

{'x': -73.75989873712614, 'y': 42.654069473771784} for NYS Geocoder

In addition, the North Carolina address should not provide a match against the NYS geocoder, however it returns a score of 100 in Out [7].

So:

1) Does the batch_geocode operation default to the ESRI Geocoder despite the custom geocoder url being set with esrinl_geocoder?

2) Are there certain circumstances when this occurs, such as the ESRI geocoder kicks in when a match cannot be returned with the specified geocoder (e.g.; NC address)?

Thanks for any insight!

Re: Using a Custom Geocoder

MichaelVolz — Fri, 22 Mar 2019 15:14:41 GMT

Can you use Fiddler software against Jupyter notebook to see the url of the geocode service that is being called when retrieving the NC address?

Re: Using a Custom Geocoder

ScottGeis1 — Tue, 25 Feb 2020 14:53:21 GMT

I found my error. Thanks to the code documentation at Understanding geocoders | ArcGIS for Developers I found I had a missing component where I called the geocode function. The Code should be as follows...

In [1]: # Import the necessary libraries

from arcgis.gis import GIS
from arcgis.geocoding import Geocoder, get_geocoders, batch_geocode, geocode

In [2]: #Log into ArcGIS Online with user credentials

gis = GIS("http://www.arcgis.com", "username", "password")

In [3]: #Set custom ESRI geocoding service url to the NYS Geocoding Service

nys_gcdr_url = "Locators/Street_and_Address_Composite (GeocodeServer) "

In [4]: #Set esrinl_geocoder to the NYS geocoder

esrinl_geocoder = Geocoder(nysGeocoderUrl, gis)

In [5]: #Provide batch addresses

            addresses = ["80 South Swan Street, Albany, NY, 12201",
                                 "10 B Airline Drive, Albany, NY, 12205",
                                 "800 North Pearl Street, Albany, NY, 12204",
                                 "5826 Fayetteville Rd, Durham, NC 27713"]

In [6]: #This is where I was missing code, I need to set the geocoder function to the esrinl_geocoder

results = batch_geocode(addresses, geocoder = esrinl_geocoder)

results

#Now I see the results I was expecting with the NC address results empty

Out [6]:

[{'address': '80 S Swan St, Albany, NY, 12210',   'location': {'x': 601644.6007360676, 'y': 4723145.388080139},   'score': 95,   'attributes': {'ResultID': 0,    'Loc_name': '1A_SAM_AP_ZipN',    'Status': 'M',    'Score': 95,    'Match_addr': '80 S Swan St, Albany, NY, 12210',    'Side': '',    'SubAdd': '',    'User_fld': '1',    'Addr_type': 'StreetAddress'}},  {'address': '10 B Airline Dr, Albany, NY, 12205',   'location': {'x': 596835.2465288276, 'y': 4732027.504390771},   'score': 100,   'attributes': {'ResultID': 1,    'Loc_name': '1A_SAM_AP_ZipN',    'Status': 'M',    'Score': 100,    'Match_addr': '10 B Airline Dr, Albany, NY, 12205',    'Side': '',    'SubAdd': '',    'User_fld': '1',    'Addr_type': 'StreetAddress'}},  {'address': '800 N Pearl St, Albany, NY, 12204',   'location': {'x': 603476.4214812117, 'y': 4726078.037474971},   'score': 100,   'attributes': {'ResultID': 2,    'Loc_name': '1A_SAM_AP_ZipN',    'Status': 'T',    'Score': 100,    'Match_addr': '800 N Pearl St, Albany, NY, 12204',    'Side': '',    'SubAdd': '',    'User_fld': '1',    'Addr_type': 'StreetAddress'}},  {'address': '',   'location': {'x': 'NaN', 'y': 'NaN'},   'score': 0,   'attributes': {'ResultID': 3,    'Loc_name': '',    'Status': 'U',    'Score': 0,    'Match_addr': '',    'Side': '',    'SubAdd': '',    'User_fld': '',    'Addr_type': ''}}]

In [7]: #Another way to visualize this

for result in results:
print("Score " + str(result['score']) + " : " + result['address'])

         Score 95 : 80 S Swan St, Albany, NY, 12210          Score 100 : 10 B Airline Dr, Albany, NY, 12205          Score 100 : 800 N Pearl St, Albany, NY, 12204          Score 0 :