topic Re: ArcGIS Pro Conflation short fall? in ArcGIS Pro Questions

ArcGIS Pro Conflation short fall?

SalemOR_BAP — Sun, 27 Nov 2022 19:08:16 GMT

I have directional road linework. Conflation refuses to transfer attribution to both linework features...only the closest even though both are clearly within the search distance. Fat grey is Source. Red & thin black are Targets. The black Target is the one that received the attributes. The red received none. My search distance is 100' and the greatest distance from the Source to target missed is <50'. My entire state network shows this issue. Conflation seems to 'stop' for an area once is finds the first match. Suggestions? Thanks all. Greatly appreciated.

Spatial join is too messy in the metro areas with all the crossing highways & ramps. The results are too invalid.

Re: ArcGIS Pro Conflation short fall?

DanLee — Mon, 28 Nov 2022 19:40:01 GMT

It sounds like you have a centerline (the grey line) against dual-lines situation. Our Transfer Attributes tool currently only transfers to one side of the dual-line. We are aware of this shortcoming and are trying to address that in a future release along with some other enhancements.

Would you be able to share a sample data so we can test/verify that our code change would work on your data as expected? Thanks!

Re: ArcGIS Pro Conflation short fall?

SalemOR_BAP — Mon, 28 Nov 2022 20:08:00 GMT

Thank you for your speedy response. You are correct - Source dataset is single line & Target dataset is double.

I have attached the Target & Source data.

Greatly appreciated.

🙂 bap

Re: ArcGIS Pro Conflation short fall?

DanLee — Fri, 02 Dec 2022 23:08:56 GMT

Hi Bap,

Thanks for the sample data. I found that your source data has many relatively short lines. Below is the upper left corner of the source data. Do you know why they are not merged into longer lines?

I also found that your target data has many overlapping or duplicate lines. That really makes feature matching difficult. Are they intended this way? Here is the upper left corner of your target data; you can tell where overlapping features are by the double labels:

Re: ArcGIS Pro Conflation short fall?

SalemOR_BAP — Thu, 08 Dec 2022 17:36:10 GMT

Sorry - Is there a flag on this community that lets me know when there is a response to my post?

1: Short lines are correct. This is highway data and is split for several reasons/attributes, e.g. change of speed, functional classification, traffic count, etc. No way around this.

2: Overlapping linework is correct. This indicates directional traffic. Transportation software (INRO EMME, PTV VISUM) manages transportation networks this way. Directionality of linework drawn indicates direction when exported out of the transportation software (in the transportation software it is depicted by a From Node -> To Node). No way around this.

I greatly appreciate your efforts.

Re: ArcGIS Pro Conflation short fall?

DanLee — Thu, 08 Dec 2022 17:58:03 GMT

As far as I know, you should get an email notification for any new post on a topic you are involved, unless you selected "Unsubscribe" or "Mute" option from "Post Options".

Thanks for your clarifications on the data situations. Much appreciated!

Re: ArcGIS Pro Conflation short fall?

SalemOR_BAP — Tue, 13 Dec 2022 22:16:14 GMT

Should have mentioned my 'work-around' as it seemed to produce successful results and may be something to work in behind the scenes in Arc.

After I ran the Conflation Attribute transfer and the overlapping linework only identified one feature, I queried out those segments that didn't transfer and re-ran the process on that subset. I did that for a 3rd or 4th time until the results were acceptable. The final step was to Union all the Attributed subsets back into a single output. A model can be created to do this repetitive process if necessary although it wasn't much manual work.

Perhaps the iteration process can be applied behind the scenes for your software.

Just thought I'd pass on my work-around and let you know the geoprocess works regardless.

Re: ArcGIS Pro Conflation short fall?

DanLee — Fri, 16 Dec 2022 18:25:54 GMT

Glad your workaround worked. Thank you for the suggestion.

The tool does "see" multiple candidates internally. It currently transfers to only one best matched target is by design for the time being. Future enhancement could be made to support transfer to multiple candidates. It's a matter of time and resource to do that. 🙂

Re: ArcGIS Pro Conflation short fall?

SalemOR_BAP — Mon, 19 Dec 2022 14:57:55 GMT

You mentioned ESRI is aware of this shortcoming...should we not expect a 'fix' anytime soon? It sounds like it might be a low priority item.

Either way, I greatly appreciate the your responding and addressing my post here. Knowing it wasn't me doing something 'wrong' is useful to know so a work-around is needed and I can document it for co-workers who may also struggle.

Re: ArcGIS Pro Conflation short fall?

DanLee — Mon, 19 Dec 2022 16:33:11 GMT

It's not a low priority, and it is also not as simple as just transfer to any candidates found. I have not verified whether all the overlaps in your data are identical. Some of them seem partially overlap. In other users' data, candidate lines within search distance could be totally unrelated roads and need to be unmatched or flagged. We definitely want to make some enhancements to simplify the workflow at user's end. It's great that you brought this up and shared your use case data. Really appreciated! I will update this post when I have new information. 🙂

Re: ArcGIS Pro Conflation short fall?

SalemOR_BAP — Wed, 25 Jan 2023 19:06:46 GMT

Agree:

"...candidate lines within search distance could be totally unrelated roads and need to be unmatched or flagged..."

I find ArcPro conflation does a better job than simply spatial join. Does it include a process of matching the directionality of a line segment? With a spatial join, a 4-way intersection could take on the attribute(s) of any number of segments (Arc choses the first it runs across). This conflation specific process seems to do much better as if it compares the direction/angle/etc. of linework, e.g. a NB line will not take on a EB attribute.

Am I imagining that? I'm not certain how a simple spatial join differs from the conflation process.

Thanks again for your input/assistance.

🙂 bap

Re: ArcGIS Pro Conflation short fall?

DanLee — Wed, 25 Jan 2023 22:01:37 GMT

Thanks for sharing your agreement on that point.

You are right, conflation tools perform deeper analysis than just searching for candidates by a distance and matching/grabbing the first one found, as Spatial Join does. The underlying feature matching takes into account a number of factors, including similarity of line shapes, and makes matching decisions. So an EB line and a NB line would not match if they fail the matching criteria, therefore no attribute transfer between them.

Re: ArcGIS Pro Conflation short fall?

SalemOR_BAP — Thu, 26 Jan 2023 18:43:14 GMT

So I'm not imagining it 🙂 it's always nice to know that.

It is g-r-e-a-t to hear all the other factors taken into account beyond a simple spatial join. Other than improving (if possible/needed) on the existing factors, I expect the only addition that would help in my case is for there to be a toggle OPTION to attribute overlapping linework the same if desired.

Because our road network runs from node to node (from node-to node) as built/required in the transportation software, most highways run the same path. Divided highways & couplets will usually never fit this format and realize there will always be a certain amount of manual modifications.

Because the majority of our road network is overlapping lines, I ran the conflation 5 time (removing successful attributed segments before each run) before attributing all within my distant parameter. Then all those attributed subsets needed to be unioned back together into a single dataset. Kind of a pain, but worth it to gain more accurate results than a simple spatial join.

Knowing my dilemma and having a copy of my Source/Target datasets, I'd love any input/suggestions I might try to simplify my deduced process. Greatly appreciated.

One other thing for your input. I started my conflation work in ArcMap but it ran all night then crashed. ArcGIS Pro hasn't had any problem and wizzes through it in 15-20 minutes. The majority of our unit uses ArcMap so I'm hoping to clarify whether or not ArcGIS is a bit weaker with this task than ArcGIS Pro.

Again.... thanks.

🙂 bap

Re: ArcGIS Pro Conflation short fall?

DanLee — Thu, 26 Jan 2023 19:08:20 GMT

I fully understand the pain you are talking about and we will try to incorporate your need to transfer to multiple matched features in future enhancement. Regarding feedback to your data processing, I will let you know if I see anything.

It's good to know that your data ran through with Pro. We have fixed some crashes or potential problematic code in recent years; most of them are in Pro, not in ArcMap. So that might be the reason you ran into the crash in ArcMap, not in Pro. How big is your data?

Re: ArcGIS Pro Conflation short fall?

SalemOR_BAP — Thu, 26 Jan 2023 20:25:19 GMT

Not very, 16MB.

I previously provided you with copies of my Source/Target datasets (SalemOR_BAP_SampleData.zip).

Re: ArcGIS Pro Conflation short fall?

DanLee — Thu, 26 Jan 2023 23:34:16 GMT

Oh, that data; it's not very big. 🙂 I thought the shared data was just a sample of a larger area.

Re: ArcGIS Pro Conflation short fall?

SalemOR_BAP — Wed, 01 Feb 2023 14:19:56 GMT

We have plans to do the remainder of the state. Right now it is only a selection of the upper functionally classed roads. I have to come up with at working process before attempting the entire state road system.

Re: ArcGIS Pro Conflation short fall?

DanLee — Wed, 01 Feb 2023 17:15:38 GMT

Hi SalemOR_BAP,

I can understand that multiple routes may share the same road segments. However, I see duplicate lines with the same RoadNumb values in your target data. The 8 features in the close up below are in the red box area in the overview image. They are actually 4 pairs of identical lines, each pair with the same RoadNumb value. Are they all expected?

Re: ArcGIS Pro Conflation short fall?

DanLee — Wed, 01 Feb 2023 18:03:44 GMT

Here is another area you can find by the RoadNumb value. The 5 features are not identical but form two overlapping paths. The first path has three features (top example) and the second path has two features (bottom example). The red arrows indicate where the connections are within each path. And all 5 features have the same RoadNumb. Are they all correct/expected?

Re: ArcGIS Pro Conflation short fall?

SalemOR_BAP — Thu, 16 Feb 2023 16:11:35 GMT

The yellow highlighted are the same highway. Directional. Two individual segments, each with a directional line (eastbound/westbound).