topic Re: Comparing two point files with one polygon in Geoprocessing Questions

Comparing two point files with one polygon

HannahSmith3 — Thu, 01 Feb 2018 17:47:22 GMT

I am doing some QA to some addressing. I have two address point files.

One from several years ago that the address point is placed on the buildings (which I want the newer data to do). The second is a month old that the address points are at the centroid of the parcel layer.

I need to see where adjustments are needed. I was hoping to find a method to compare the two address points for changes in the address number and road name and the differences of the placement such as that number is not found within that parcel (polygon).

Anyone have any ideas? I have a Basic Licenses for ArcMap 10.5

Re: Comparing two point files with one polygon

DanPatterson_Retired — Thu, 01 Feb 2018 19:29:22 GMT

Hannah, can it be safe to assume the OBJECTID numbers between the two files aren't in direct correspondence?

If they were, then a join based on that common key would make things easier.

If there is not and the address is in one field then you could join on that field

If that is not the case, then you could concatenate the full address in a new file and try a join using those two common fields. This would at least give you the addresses that were the same, leaving you with the addresses that are different.

Re: Comparing two point files with one polygon

HannahSmith3 — Thu, 01 Feb 2018 20:02:06 GMT

They do not have the same object ID. That would make my life too easy. When trying to Concatenate the fields I get a ton of spaces when a field is empty. What is the process should I do to only get information if there is some in it?

Re: Comparing two point files with one polygon

DanPatterson_Retired — Thu, 01 Feb 2018 20:16:42 GMT

You may have to do some prep work... like for now, select only the records that have all the data and don't concatenate those.

I don't know what the data looks like, but if it is a real mess and you have less than a 100 or so, it may just be faster to do a side-by-each comparison and provide a common 'key' field to both. A tax roll number would work, anything, but at some stage every address should have some kind of 'key' to tie it to attributes and location.

If you have a couple of made up records from each table (anonymize them) then some people that work specifically in that area may be able to leap in

jborgion‌ any ideas?

Re: Comparing two point files with one polygon

HannahSmith3 — Thu, 01 Feb 2018 21:09:59 GMT

Re: Comparing two point files with one polygon

HannahSmith3 — Thu, 01 Feb 2018 21:11:26 GMT

Re: Comparing two point files with one polygon

DanPatterson_Retired — Thu, 01 Feb 2018 21:25:26 GMT

Grief... well, I would certainly work with a selection to test the process first.

Your address column on the left, should be able to join with the address column on the right...

Have you tried that? The big thing to check is to ensure everything is exact.

There are a few things that you should consider when assessing the results

hopefully your data columns don't have preceeding/trailing spaces
there is no point in concatenating columns that contain <null>, so you might want to do a query for the records that are not null in one of the prominent fields that contain tons of nulls
that will enable you to simplify the concat.. process.

so check the data... run a join, .... select by attributes a street or two and manually inspect the result and report back on any issues.

Re: Comparing two point files with one polygon

JoeBorgione — Thu, 01 Feb 2018 21:40:03 GMT

Hooray for addressing changes! They keep us gainfully employed!

I'm thinking along the same lines as Dan: first thing I'd do is look for those that are identical in the components. That whittles down the ones you'll need to examine as far as the actual address (1234 S MAIN ST) goes. Additionally for those that are duplicate records, you need to see if they have the same location, and that might be tricky with a basic license. Add X and Y coordinates and calculate them accordingly in each of point in both feature classes.

Dan has bestowed upon me the way of numpy arrays for making fast comparisons and I'm currently working on an addressing project that deploys them; I won't go into them right now, but they do make comparisons a lot faster, and you'll amaze your co-workers with how cool they are.

First things first: if you don't have a "complete address" field in both feature classes add it. Calculate them accordingly. Then you can perform a relate between the two feature classes using the "complete address" field as the key. Select all of the 'old' and then see which of the new get selected via the relate. You can do the same sort of thing to look for duplicate x and y coordinates.

You'll then need to start looking at just street names: S MAIN ST in the old and S MAIN ST in the new: are there spatially near matches with different house numbers? Are there some new S MAIN ST addresses that are miles away from the old ones?

Does the parcel data have an address in them as well? You could perform a spatial join and then look to see if the address point address is different than the parcel address.

I could go on ad nauseam; there are lots of different ways to skin this cat. Give some of these ideas a spin and when you get stuck, let us know; if I don't see your post, Dan will, and then I'll know.

Re: Comparing two point files with one polygon

HannahSmith3 — Thu, 01 Feb 2018 22:08:30 GMT

The newer version of the address points was created from the parcel data. So once I do a spatial join what should I do?

Re: Comparing two point files with one polygon

JoeBorgione — Thu, 01 Feb 2018 22:27:48 GMT

What about joining the old data to the parcel data? If old address <> parcel address, that address point has changed with respect to address itself, right? Does knowing that have any value in your analysis?

If the new address points were created from the parcel data, is it safe to assume the addresses between them are the same? Does knowing if there might be a difference have any value in your analysis?

I suspect that what you really want to do is move any new address points that have the same address (and/or improved address) to the location of the old address, but before you do that, you really need to be sure the new one qualifies to replace the old one. That can be accomplished with a topology (I think). I can't recall offhand but does a spatial join for points allow for an off set distance? That is, if you were spatially join the new points to the old points and they are not exactly coincident, can the search radius be adjusted.

Re: Comparing two point files with one polygon

DanPatterson_Retired — Thu, 01 Feb 2018 22:48:14 GMT

spatial join is to the closest feature Joe.. so no problem unless there are spatial options with the same distance.

ps... see the concatenation post to see the form of the fields and the concatenated fields to see if you have any suggestions as well

Re: Comparing two point files with one polygon

HannahSmith3 — Thu, 01 Feb 2018 23:07:28 GMT

Yes exactly. Your whole last paragraph is correct. That is exactly what/why. I just know some of these addresses (the house numbers) have been changed and I want to make sure that is represented as well. I am doing this for 911 dispatch. I came to this city a month ago and it turns out they did not have anyone doing GIS for over 4 years. So I received the new addresses from the county from the parcel data. Looking back 4 years to what had previously been done. So a lot of updates. But I do not want to loose the majority of those who have been placed over the building. Thank you for your assistance. I will try these out tomorrow! I will let you know how it goes!

Re: Comparing two point files with one polygon

JoeBorgione — Thu, 01 Feb 2018 23:47:56 GMT

I just got done with a 15 year stint doing 9-1-1 addressing; I feel your pain. Something you might consider is forget the old and go with the new. Period. Four years is long enough to go between updates.

Re: Comparing two point files with one polygon

RichardDaniels — Fri, 02 Feb 2018 00:01:13 GMT

Other have addressed dealing with the addressing, but your second question was about 'two point files with one polygon | differences of the placement'.

I suggest trying the Near (ArcMap) or Generate Near Table (ArcGIS Pro) command.

Summary

Calculates distances and other proximity information between features in one or more feature class or layer. Unlike the Near tool, which modifies the input, Generate Near Table writes results to a new stand-alone table and supports finding more than one near feature.

This, along with the address information, should help you associate the points correctly.

Re: Comparing two point files with one polygon

JoeBorgione — Fri, 02 Feb 2018 01:54:49 GMT

Isn't the near tool/command available only at the Advanced License level? That's what I was alluding to with my comment about finding coincident points being tricky with a Basic license....

Near—Help | ArcGIS Desktop

Re: Comparing two point files with one polygon

HannahSmith3 — Fri, 02 Feb 2018 01:58:58 GMT

In my last job I had an advanced license. So having to figure out things

without my awesome tools, I loved. Lol

Re: Comparing two point files with one polygon

DanPatterson_Retired — Fri, 02 Feb 2018 02:14:57 GMT

I have a 'Near-ish' tool that can be used Joe and Hannah... if it comes down to it (no advanced license required). Let me know and I will dredge it up and get it into a 'tool' form

Re: Comparing two point files with one polygon

HannahSmith3 — Mon, 05 Feb 2018 16:05:23 GMT

Okay so I have done the spatial joins for the Old addressing and the parcel. How do I select based on the differences between the two addresses?

Re: Comparing two point files with one polygon

JoeBorgione — Mon, 05 Feb 2018 17:58:27 GMT

I have to assume that the parcel data has its own parcel address.

You can now query your data such that:

OldAddress_Field <> ParcelAddressField

The result tells me that the parcel address has been updated from what you previously had.

OldAddress_Field = ParcelAddressField

Tells me that no change has been made.

Re: Comparing two point files with one polygon

HannahSmith3 — Mon, 05 Feb 2018 18:45:38 GMT

I realized my issue was the fact that when I made the new field for the full address. It left spaces when there was nothing in that field. Such as

How do I do the concatenate so that there is not a space in the way screwing up the data?

Thank you for all of your help. You have been a life saver!