topic Re: ArcGIS 10 SP4 Spatial Join Bug in Geoprocessing Questions

ArcGIS 10 SP4 Spatial Join Bug

BenHolmes — Wed, 04 Apr 2012 07:57:03 GMT

Hi

Just to let you know I've experienced the spatial join bug found in SP4.

I was joining 740000+ points to 13 polygons
Just under 19000 records had attribute data come across. The rest were all null.

I've run the exact same data on another machine with ArcGIS 10 SP3 and it worked fine.

I see there was a bug listed ( NIM079126 ) but for some reason it's been declined. Not sure why this is, but you should know this is happening to others.

One thing I noticed was the last record to have an entry before the attributes failed to populate was a point that was outside all polygons I was trying to join to. I'd suggest creating a point dataset with one point outisde the main body of a join polygon to test this. Maybe then we can see some work towards a solution.

Thanks
Ben

Re: ArcGIS 10 SP4 Spatial Join Bug

KimOllivier — Wed, 04 Apr 2012 10:13:05 GMT

Worked fine for me, with a small dataset.

Maybe your environments are different. Scratch workspaces, swap space, full disk, default workspace not a filegeodatabase...

Re: ArcGIS 10 SP4 Spatial Join Bug

DanLee — Wed, 04 Apr 2012 23:23:14 GMT

See this similar discussion:
http://forums.arcgis.com/threads/54102-Spatial-join-tool-vs.-right-click-spatial-join-producing-different-results?p=185943#post185943

I have confirmed that we introduced a bug in 10 Sp4. Really sorry about that! The problem will be fixed in 10 SP5.

Just FYI (according to my colleague):
The tool works fine if all input points find spatialy related polygons (i.e. Join_Count > 0). The problem happens when a point is encountered without any related (e.g., INTERSECT) join features, all subsequenct records in the output gets null values from the join features. One workaround (if you want to continue using Spatial Join) would be to uncheck 'Keep All Target Features' parameter on tool dialog or use 'KEEP_COMMON' option in script.

Re: ArcGIS 10 SP4 Spatial Join Bug

BenHolmes — Thu, 05 Apr 2012 08:45:40 GMT

Thanks for that Dan

All points in this case should really of been within the polygons!

I'll can just make an envelope polygon to pick up any that are just outside the admin boundaries.
Should do the job until SP5

Thanks
Ben

Re: ArcGIS 10 SP4 Spatial Join Bug

SanderSchaminee — Thu, 05 Apr 2012 13:11:23 GMT

See this similar discussion:
http://forums.arcgis.com/threads/54102-Spatial-join-tool-vs.-right-click-spatial-join-producing-different-results?p=185943#post185943
<SNIP>
One workaround (if you want to continue using Spatial Join) would be to uncheck 'Keep All Target Features' parameter on tool dialog or use 'KEEP_COMMON' option in script.

Yes, that is correct. However, I noticed something else as well.
When joining polyline features with a (circular) polygon feature class in a file geodatabase, the SHAPE_Length is not transferred/recalculated properly (even with the 'Keep All Target Features' parameter unchecked).
This happens with the default field mapping. Seems that the SHAPE_Length from the polygon join features is transferred to the polylines instead, which should not happen, since this is a reserved field that belongs exclusively to the output geometry. SHAPE_Leng_1 on the other hand (which should contain the joined polygon perimeters), is <Null>.
I cannot calculate the proper length of the polylines and copying to another feature class within the gdb does not fix it. Only thing that works is an export and re-import to/from a shapefile.

See picture below (unfortunately, I've already deleted the SHAPE_Leng_1 field).

[ATTACH=CONFIG]13268[/ATTACH]

Sander

Re: ArcGIS 10 SP4 Spatial Join Bug

JenniferLishman — Thu, 05 Apr 2012 13:13:28 GMT

See this similar discussion:
http://forums.arcgis.com/threads/54102-Spatial-join-tool-vs.-right-click-spatial-join-producing-different-results?p=185943#post185943

I have confirmed that we introduced a bug in 10 Sp4. Really sorry about that! The problem will be fixed in 10 SP5.

Just FYI (according to my colleague):
The tool works fine if all input points find spatialy related polygons (i.e. Join_Count > 0). The problem happens when a point is encountered without any related (e.g., INTERSECT) join features, all subsequenct records in the output gets null values from the join features. One workaround (if you want to continue using Spatial Join) would be to uncheck 'Keep All Target Features' parameter on tool dialog or use 'KEEP_COMMON' option in script.

I am using a spatial join in geoprocessing model and I am having this problem joining points to points (using intersect and a specified distance) even when the Join Count > 0. Attribute values for most features come across as NULL. One of the outputs that we need is the join counts so running an Intersect instead is not an option.

I tried the suggested workaround to uncheck "Keep All Target Features " but I am getting the same NULL values output. This processworked fine in SP3. This is causing quite a hiccup in workflow for our project, and the deliverable is due soon, way before the anticipated July release of SP5. We can't downgrade back to SP3. Any other suggestions to get this to work?

Re: ArcGIS 10 SP4 Spatial Join Bug

DanMcCoy — Thu, 05 Apr 2012 18:44:11 GMT

I have confirmed that we introduced a bug in 10 Sp4. Really sorry about that! The problem will be fixed in 10 SP5.

Dan,

Is this NIM079126 or a different BUG#?
http://support.esri.com/en/bugs/nimbus/TklNMDc5MTI2

May just be a matter of timing, but NIM079126 is listed as "Declined" in the link above and "Non-Reproducible" in our Support Bugs in Customer Care Portal.

Thanks,

Dan M.

Re: ArcGIS 10 SP4 Spatial Join Bug

JenniferLishman — Thu, 05 Apr 2012 18:49:43 GMT

Perhaps it is a different bug. Whatever the case, I am getting NULL attribute values when running the spatial join tool. I will file an incident report tomorrow and see if we can figure out what is going on.

Thanks,
Jen

Re: ArcGIS 10 SP4 Spatial Join Bug

DanLee — Thu, 05 Apr 2012 20:10:41 GMT

We are checking on this matter. I will update once I know what's going on.

Re: ArcGIS 10 SP4 Spatial Join Bug

NobbirAhmed — Thu, 05 Apr 2012 20:44:59 GMT

This happens with the default field mapping.

From the graphics I guess your lines are your "target" features, right?

What is your operating system?

Could you please post a screenshot of your default field map - just the wo fields for SHAPE_Length. For example, here is my field map:

[ATTACH=CONFIG]13283[/ATTACH]

Re: ArcGIS 10 SP4 Spatial Join Bug

NobbirAhmed — Thu, 05 Apr 2012 21:31:05 GMT

Any other suggestions to get this to work?

Sorry that you are having trouble. For a workaround you can call tech support. I'll also lookup for a suggestion.

Re: ArcGIS 10 SP4 Spatial Join Bug

DanLee — Thu, 05 Apr 2012 23:37:54 GMT

I cannot calculate the proper length of the polylines and copying to another feature class within the gdb does not fix it. Only thing that works is an export and re-import to/from a shapefile.

Sander,
Could you try Copy Features to see if it results in a recalculation of the shape_length?

Re: ArcGIS 10 SP4 Spatial Join Bug

SanderSchaminee — Mon, 09 Apr 2012 11:55:39 GMT

@Nobbir:
I'm currently running Windows XP 32 bits Professional (but will be moving to 7 64 bits soon).
The lines are indeed my target features. I've attached a screendump of my field mapping below.
I realize my feature class names are quite long, but the mapping seems to go right here. However, the result is wrong.
[ATTACH=CONFIG]13336[/ATTACH]

Maybe my long feature class names are abbreviated in such a way that their names become identical somehow, causing the tool to mix up the SHAPE_Length fields?

@Dan:
I already tried Copy Features, but strangely enough, that didn't reset the SHAPE_Length. Only export and (re)import via shapefile did the trick.

I would say that the issues related to the Spatial Join are severe enough (disruptive for workflow, maybe even for data integrity etc.) to issue a little patch until SP5 (or 10.1) comes out.

Re: ArcGIS 10 SP4 Spatial Join Bug

DanLee — Mon, 09 Apr 2012 17:15:22 GMT

Hi Sander,

First: we are working hard on a special patch on 10 SP4 to fix the problems. Hope it will come out soon.

Second: Copy Features tool worked for my test data. I reproduced the wrong shape_length (for the target lines) and then run Copy Features to gdb output. The output shape_length values reflect the original/input line lengths. Will you be able to share your data so I can try?

Re: ArcGIS 10 SP4 Spatial Join Bug

TonyAlmeida — Mon, 09 Apr 2012 19:22:08 GMT

I get Nulls when i run Spatial join with service 10 sp4 and it runs VERY SLOWWWW now.

Re: ArcGIS 10 SP4 Spatial Join Bug

SanderSchaminee — Tue, 10 Apr 2012 07:58:37 GMT

First: we are working hard on a special patch on 10 SP4 to fix the problems. Hope it will come out soon.

Ok, that's great news!

Second: Copy Features tool worked for my test data. I reproduced the wrong shape_length (for the target lines) and then run Copy Features to gdb output. The output shape_length values reflect the original/input line lengths. Will you be able to share your data so I can try?

Good to know that you were able to reproduce the SHAPE_Length problem.
As for the Copy Features bug: this issue might have been a freak occurence, related to my geodatabase. My gdb is quite large (too big to share), so I tried to create a "light" copy for you holding only the relevant data - and test Copy Features again at the same time. This time, the tool worked as expected. But that's not the only thing: I cannot reproduce the problem anymore with my original gdb. Even when I run the Spatial Join again and run Copy Features afterwards, SHAPE_Length will be properly recalculated.

Re: ArcGIS 10 SP4 Spatial Join Bug

JohnBryant — Tue, 10 Apr 2012 12:55:59 GMT

Howdy, count me in as waiting for a patch. This bug has disrupted a number of geoprocessing models I rely on for several ongoing projects.

I gotta say, it is refreshing to hear an ESRI staffer taking the blame on the company's behalf for a bug. ESRI has really picked up their game in the past few years as far as customer support goes.

Anyway - would love to see that patch ASAP! Thanks.

Re: ArcGIS 10 SP4 Spatial Join Bug

DanLee — Tue, 10 Apr 2012 15:30:22 GMT

We have heard all of you. My colleagues and I thank you for your patience.

Re: ArcGIS 10 SP4 Spatial Join Bug

ChrisFox3 — Tue, 10 Apr 2012 17:59:08 GMT

The attached Toolbox contains a model that may be helpful in working around this issue until the bug is resolved. It has proved helpful for some customers.

Re: ArcGIS 10 SP4 Spatial Join Bug

MaliaSpaid-Reitz — Wed, 11 Apr 2012 16:53:07 GMT

When will SP5 be available?