topic Re: Geocoding Table with Anonymized Addresses & Lat/Lon in ArcGIS Pro Questions

Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Mon, 28 Nov 2022 18:31:14 GMT

Hello, I am trying to set up a process to map locations to a street centerline by either an address or or by lat/lon if no address exists or if the address cannot be found. I have created a geocoder that only uses the Street Address Role and have a process set up to change addresses from 1234 N Main to 1200 N Main in an effort to anonymize them. All locations have a lat/lon even if no address exists. The geocoder is configured properly and geocodes the events to the street centerline with the address. The issue I am having is that the geocoder will not locate events by lat/lon where no addresses match can be found. Even if I only specify the geocoder to use the Coordinate System Category in the Geocode Table dialogue box, it will return zero matches.

Just wondering if anyone has any input on how I can geocode based on lat/lon. Everything I've tried seems to not work.

Thanks!

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

jcarlson — Mon, 28 Nov 2022 18:35:13 GMT

You should be able to use X/Y in the Rematch Addresses tool. Have you tried that?

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Mon, 28 Nov 2022 18:40:45 GMT

Yes, but unfortunately, this will become an automated process that will run daily, so there will be no interaction for manual matching. We already map the data internally by lat/lon and locations that do not have a lat/lon from creation get moved to a fixed location depending on the district they are assigned.

I'm looking publish the data publicly, which is why I am altering the addresses and hoping to use lat/lon where no address exists.

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

jcarlson — Mon, 28 Nov 2022 18:55:39 GMT

Perhaps there is a way with the ArcGIS Python API? arcgis.features.GeoAccessor.from_df() will geocode the input table, and you can specify the geocoder used. There is also from_xy(), which would handle the lat/lon input.

So perhaps something like this? Haven't tested this out, but it might be a starting point.

import arcgis import pandas as pd # load your address table as a DataFrame; this will depend on its format, location, etc # get the blank addresses no_addrs = df[df['address_field'].isna() == True] # get addresses addrs = df[df['address_field'].isna() == False] sdf_addrs = arcgis.features.GeoAccessor.from_df(addrs, 'address column', 'your geocoder') sdf_noaddrs = arcgis.features.GeoAccessor.from_xy(no_addrs, 'lon', 'lat') # merge into one dataframe geocoded = pd.concat([sdf_addrs, sdf_noaddrs], ignore_index=True) # then export / upload as needed

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Mon, 28 Nov 2022 21:17:18 GMT

Thanks, I will look into this option!

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Wed, 30 Nov 2022 17:39:57 GMT

Maybe I should simplify the question. Can a geocoder geocode lat/lon or XY's? Because the geocoders I create seem to not be able to perform that function.

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

ShanaBritt — Tue, 20 Dec 2022 17:01:16 GMT

Building a locator with the Create Locator or Create Feature Locator tools suppors global search for coordinates (latitude/longitude, MGRS, DD, or USNG). Support for coordinate searching is disabled or enabled under Categories to support on the Geocoding options page of the Locator Properties dialog box for the locator. What is the format of the coordinates you are using? Can you enter the same coordinates into the Locate pane and get results returned by the StreetAddress role locator you created? What is the coordinate system of the locator (the reference data)?

Also, are you not able to use Make XY Event Layer to generate points from the coordinates?

-Shana

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Wed, 21 Dec 2022 17:43:27 GMT

Hello @ShanaBritt Making an event layer works out fine, but that's not what I am trying to do. We have many records that map an exact location and I am trying to geocode masked addresses (to hundred block - EX: 2352 = 2300) back to the street centerline. Most of the locations do geocode back to the centerline, but many that do not have an address, but have Lat/Lon, to do not get processed by the geocoder even with coordinate category selected. I have investigated other methods like reverse geocoding the layer, which yields better results (sort of), but also causes issues when the reverse geocode goes to the wrong street since it was closer than where the address specifies.

I'm really trying to find how precise data is anonymized and I'm not having very good luck it seems.

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

ShanaBritt — Wed, 21 Dec 2022 20:00:52 GMT

It's possible to search for Street blocks, which is described here for StreetAddress role, https://pro.arcgis.com/en/pro-app/3.0/help/data/geocoding/introduction-to-locator-roles.htm#ESRI_SECTION2_6D3D70C9F7AA4871ABDBCD7F98D363D1. You will need ArcGIS Pro 2.7 or later to build a locator that supports searching for street blocks, for example "100 block of New York St, Redlands, CA". How do you have the block address formatted in your input address table? You will need to modify your address table to change addresses from "1234 N Main" to '1200 block of N Main' in order to get the matches you are looking for. You can also test this out using the following tutorial and the test addresses at the bottom of the tutorial. https://pro.arcgis.com/en/pro-app/latest/help/data/geocoding/tutorial-create-a-locator.htm#

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Thu, 22 Dec 2022 00:35:00 GMT

@ShanaBritt Correct, I do have the addresses as "700 Block E Olive Ave" for example. Those will geocode to the centerline, but if there is no address (null) the locator does not resort to plotting the lat/lon of the data:

See below:

I can create an event layer just fine with this test layer.

Also, my geocoding settings:

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

ShanaBritt — Tue, 03 Jan 2023 05:58:15 GMT

@WalidAlmasri1 If the Address field is the input address field and it contains 'Null' values for the record, there is no input that is geocoded, so the record with a 'Null' value will be unmatched. I would suggest moving the coordinate pair to the address field, by selecting all of the records that have 'Null' values, the using Calculate Field concatenate the Lat and Lon fields into the Address field. [Lon] + ","+ [Lat] Then geocode the table with the locator that has the Coordinates category enabled.

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Tue, 03 Jan 2023 16:51:43 GMT

@ShanaBritt THANK YOU, this is exactly the information I was looking for. Why I was unable to find this answer is beyond me. I guess it makes sense now because that's how you have to type it in the Locate box in Pro...

It is now matching my coordinates 🙂

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

ShelbyZelonisRoberson — Fri, 24 Mar 2023 09:29:36 GMT

Hi! I'm trying to do something similar. Do you geocode to the centerline using 1234 N Main or to 1200 N Main? Would you mind sharing your process for anonymizing the addresses to the block level? Thank you!

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Fri, 24 Mar 2023 15:23:15 GMT

Hi @ShelbyZelonisRoberson, exactly correct. Build a geocoder using only centerlines with Street Address Role. You need to have address ranges in the centerline though in order for this to work.

On the backend, the data is converted into a new table. So addresses like 1264 N Main St, are converted to 1300 Block of N Main St and if the address is 1231 N Main St, we convert it to 1200 Block of N Main St. You don't need to have 'block of' text in your address, but the geocoder will understand it if it's there. Then run those new addresses through the geocoder to get the new location. I'm not certain how having block information in the centerline would affect geocoding accuracy since we don't have that, but let's assume the street had an address range of 1000-3000, addresses that get converted to 1000 Block are placed on the beginning point of the line, addresses that are 2000 block would be placed on the center of the line, and addresses on the 3000 block will be placed on the other end of the line.

The only downside I'm seeing here is if you have multiple addresses on one street that get converted with the same block information, they will stack. One way around that would be to geocode the actual address but only display the 'block of' address, but then that could potentially place the address close to where you are trying to mask. Or limit the data by date range.

I hope this helps, let me know if you have any questions.

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

ShanaBritt — Fri, 24 Mar 2023 19:22:29 GMT

For stacked addresses, you could cartographically disperse them using https://pro.arcgis.com/en/pro-app/latest/tool-reference/cartography/disperse-markers.htm.

-Shana

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Tue, 04 Apr 2023 16:03:49 GMT

@ShanaBritt is there a particular way the data needs to be in order to return the "Addr_type" output field to equal "StreetMidBlock"?

I've actually removed our "Block of" information from the address and now only do the full hundred block address and am having good results, but in areas where ranges are limited, mapping to StreetMidBlock would be helpful as some get missed, but I'm not sure how to return that result from the geocoder. Everything I return is LatLon, StreetAddress or StreetAddressExt no matter how I type using the examples ESRI provides.

100 block of New York St, Redlands, CA
1600 blk E Cliff Dr, El Paso
200-500 block Taylor St, San Francisco
1700-1900 blk of Locust St, Philadelphia, Pennsylvania

Thanks!!

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

ShanaBritt — Tue, 04 Apr 2023 21:34:15 GMT

@WalidAlmasri1 ,

Just to clarify, are you saying that whenever you try to search for an address in one of the following street block formats that the Addr_type value returned is not "StreetMidBlock"?

100 block of New York St, Redlands, CA
1600 blk E Cliff Dr, El Paso
200-500 block Taylor St, San Francisco
1700-1900 blk of Locust St, Philadelphia, Pennsylvania

Or are you asking about the reference data to build the StreetAddress role locator?

-Shana

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Tue, 04 Apr 2023 22:06:15 GMT

@ShanaBritt That is correct, but I just figured out why. Under Locator Properties > Geocoding Options, once alter the Categories to support from "All categories supported by the Locator" to "Only categories selected here", the ability to search with any of the block formats above is unsupported.

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

ShanaBritt — Wed, 05 Apr 2023 15:09:05 GMT

@WalidAlmasri1 , did you have specific categories selected and not the default of all supported categories? In the current released versions of ArcGIS Pro, StreetMidBlock is not listed as a category you can select in the 'Categories to support' property; however, it is something we are looking into for a future release.

Thanks for the update on your situation.

-Shana

Re: Geocoding Table with Anonymized Addresses & Lat/Lon

WalidAlmasri1 — Wed, 05 Apr 2023 15:23:38 GMT

Yes, I did have specific categories selected, mainly removing StreetName as that is problematic. Including StreetMidBlock as a category would definitely be my recommendation. You also lose the ability to search by 'StreetMidBlock' when you disable categories in the coordinates too.

I hope to see this added in the very near future.

Thanks!