Batch geocoding using CSV file in ArcGIS Online produced incorrect results

GeorgianaBostean · ‎07-17-2018

I geocoded a .csv file of 22,000+ addresses in ArcGIS Online by publishing the file as a hosted feature service (and accurately specifying the fields to be used for geocoding). After doing a spatial join of the points to a polygon in Desktop and zooming to an arbitrary location, by chance I realized that within one city (City of Vernon) the 9 locations were geocoded to 2 points. I am concerned that this may have happened with other points.

Any thoughts on what I may be doing wrong and suggestions on how to verify that the geocoding process is happening without errors, other than to manually check individual points?

I called ESRI support and they were not able to figure out why this occurred, except to suggest that I take out the spaces in the column headers (there was one space in the header for the address field). I don't believe this is the problem because when I uploaded the file with just those 9 rows, but without editing the column headers to eliminate spaces, the points geocoded correctly.

I'd like to figure out why this occurred-- I have large files with 100,000 rows to geocode and need to make sure (and find a valid way to verify) that they are geocoding correctly.

The screenshot of the table with the 9 rows selected (the space in the column header visible for the address field) and the map zoomed to the two points:

Anonymous User · ‎08-03-2018

I spoke with the analyst who owned this case and I was able to get a copy of the data. I found that I could only reproduce the issue with the 9 records in Vernon being geocoded to two locations when I did not manually map the Licensed Locations field as the address field. This must be done manually because the geocoding dialog does not do it automatically. The fields that were mapped automatically were City, State, Zip and Zip4.

My testing involved publishing two services with this data, one with the address field mapped and one without. What I found was that when the address field was not mapped and there were two locations for Vernon there were 8 records at one point and 1 at the other point. These were based on zip code.

With both feature layers added to a map I got a better idea of how close to accurate the points were located without an address. In some situations, the locator matched without the address which is pretty impressive. In other cases, it got very close because it located down to the 4 digit zip extension.

If we look at the locations in Pro using the locate pane we can see detailed information down to what level the location matched. This is noted in the AddrType field. When the address is provided it matches down to AddrType: PointAddress and if it is not provided it matched down to AddrType:PostalExt which is still very precise.

Here is a video demonstrating how you can see that information.

(view in My Videos)

I want you to feel confident that the World Geocoding Service is extremely reliable and accurate. You have my email, please feel free to reach out if you have any further questions.

View solution in original post

ShanaBritt · ‎07-23-2018

Just curious, do you get the same unexpected behavior when the csv contains 15-20 record? It's possible that there might be a value in one of the fields after record 9 that is causing a problem with the csv geocoding the locations correctly. I also noticed that another field contained a space in the name "Area Code". I would suggest removing spaces from the column headers of all the fields in the csv, even those not used for geocoding to see if you get different results.

GeorgianaBostean · ‎07-26-2018

Thank you. No, it doesn't happen when uploading a small number of the addresses, even if there are spaces in the header. I did fix the spaces in the CSV and deleted Area Code when I geocoded the second time and everything *seems* fine. Hope that fixed it, but with thousands of addresses, it is hard to tell.

MichaelVolz · ‎07-26-2018

Is this geocoding operation costing your org credits? If so, I'm just curious if ESRI allows for testing this type of scenario before putting it into production as it might take many test runs to get the expected results so you could potentially eat up a good amount of credits.

GeorgianaBostean · ‎07-26-2018

Unfortunately, yes, it costs us credits. I agree that there should be some exception for troubleshooting and mentioned this to the ESRI tech support person, who was not able to provide any suggestions about how to avoid using credits while testing.

MichaelVolz · ‎07-26-2018

Can you run this operation within your own org's network and get the expected results? If so, then I would think you might be able to say that there is a bug with AGOL geocoding.

GeorgianaBostean · ‎07-26-2018

Michael Volz, we rely on the World Geocoding Service. Do you have any suggestions how I might test this outside the AGOL environment?

MichaelVolz · ‎07-27-2018

How about using the Geocode Addresses tool on the ArcMap Geocoding toolbar to test how well your csv file gets geocoded as you would not be using credits in that environment as that toolbar should have access to the ArcGIS World Geocoding service? Once you have identified the issue, then you can run the same process against ArcGIS World Geocoding service in AGOL. If the results differ, then that might indicate a bug with how AGOL does the geocoding as opposed to ArcMap.

GeorgianaBostean · ‎07-27-2018

I was under the impression that you consume credits even in ArcMap if you use the World geocoding service, since you have to sign-in to use it. I will look into this further. I see your logic so will test it out that way regardless of credits.

Thank you, Michael.

Anonymous User · ‎07-30-2018

Geocoding in ArcMap will consume credits if the World Geocoding Service is being used.

Can you please send me the case number that you had opened to investigate this?