Spatial Join CRASHING at 10

TedCronin · ‎07-22-2010

So, I can run about 5 consecutive processes for a spatial join before Wing crashes and processes no more. Using a Point and a Poly layer to Join. Point layer is 774,796 records. Poly layers vary, but are much smaller than point. Spatial Join in a list, which makes this a tedious process in that I get to wait for the crash and then remove the layer from the list and then reprocess the list, so it kinda makes using lists not optimal. Physical Memory drops to about 34,000 K before the crash. It steadily goes down from the first to the sixth layer, with the sixth layer not completing successfully.

Good Times.

oh, this is running in a File GDB.

There is NO error number generated, it just stops. Believe it is runaway memory.

NobbirAhmed · ‎11-21-2011

@Hornbydd

Hi Duncan, thanks a lot for describing the issue elaborately. I'll try to repro it locall and submit a bug if necessary.

BrianBatten · ‎02-09-2012

Last post mentioned this was fixed in 10 SP2,
I'm running SP3 and experiencing a similar issue. A model I've created that uses Spatial Join works fine in 9.3.1 but crashes in Arc10. I isolated it down to just the Spatial Join command, and created a model that just executes spatial join and it crashes every time. I tested the background geoprocessing on/off solution with no success. Same process works fine in 10 when executed outside of ModelBuilder.

The spatial join is one-to-many, closest with a set maximum search distance.

Any advice on solutions would be most appreciated.

KeithAdams · ‎02-28-2012

Hi all-

We have now stumbled onto the same land mine after (finally) completing our migration from 9.3.1 to 10.0 SP 3. Our situation is:

We have a series of Python-based GP scripts that get launched from inside Informatica workflows. The spatial join step (joining county data to point features, where the size of the point feature class ranges from a few dozen to about 10,000 features) fails and throws an o/s level exception. When the same code is run from a Windows CMD shell, we have no problem; it only happens when executed from within the Informatica workflows.

Our environment is:

Windows Server 2003 with all the latest and greatest service packs, KB patches, etc.

ArcGIS 10.0 SP 3

MS SQL Server 2005 plus ArcSDE

Informatica 8.1.1 (which Informatica no longer supports- that's next up on the modernization list)

Data are being processed from a combination of file GDB (point features) and ArcSDE layers (counties)

Text of the Windows Application event log is as follows:

[INDENT]Event Type: Error
Event Source: .NET Runtime 2.0 Error Reporting
Event Category: None
Event ID: 1000
Date: 2/25/2012
Time: 1:27:48 PM
User: N/A
Computer: XXXXXXX
Description:
Faulting application python.exe, version 0.0.0.0, stamp 4ba3e443, faulting module atio2kad.dll, version 4.0.0.1119, stamp 3bcc6880, debug? 0, fault address 0x000d0d49.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
[/INDENT]

Thanks for any help / insights you have that fixes this problem. Our data managers will greatly appreciate not having to start their work day at 2 a.m. every day!

Last post mentioned this was fixed in 10 SP2,
I'm running SP3 and experiencing a similar issue. A model I've created that uses Spatial Join works fine in 9.3.1 but crashes in Arc10. I isolated it down to just the Spatial Join command, and created a model that just executes spatial join and it crashes every time. I tested the background geoprocessing on/off solution with no success. Same process works fine in 10 when executed outside of ModelBuilder.

The spatial join is one-to-many, closest with a set maximum search distance.

Any advice on solutions would be most appreciated.

NobbirAhmed · ‎02-28-2012

Last post mentioned this was fixed in 10 SP2,
.......
The spatial join is one-to-many, closest with a set maximum search distance.

Just tried your scenario (spatial join in MB) on 10 sp3 - does not crash on my machine. I can further investigate if you can provide the data + model.

AaronBarkhurst · ‎03-03-2012

Last post mentioned this was fixed in 10 SP2,
I'm running SP3 and experiencing a similar issue. A model I've created that uses Spatial Join works fine in 9.3.1 but crashes in Arc10. I isolated it down to just the Spatial Join command, and created a model that just executes spatial join and it crashes every time. I tested the background geoprocessing on/off solution with no success. Same process works fine in 10 when executed outside of ModelBuilder.

The spatial join is one-to-many, closest with a set maximum search distance.

Any advice on solutions would be most appreciated.

I am having the same problem with spatial join in a python script. Setting the match_option parameter to "CLOSEST" is the only thing that crashes the script. I absolutely need that parameter as "CLOSEST" and am not crazy about having to write the code to loop through the geometries to accomplish that without the Spatial Join tool. Has ANYONE found a cause/solution to this, or is it another overseen esri bug that allows tools to be used as stand-alone from the desktop applications only????

NathanJennings · ‎03-12-2012

I am getting a problem with the spatial join using Arc10.0, SP4 using ArcSDE 9.2 SP6 feature classes using ~150,000 points overlaid with a polygon file of ~18 polygons, both SDE feature classes. All of the records appear, but all but the first row of attributes are "null". In the first row all of the input fields are populated, but the spatial join field(s) are null, too. Output is SDE feature class.

Running from OS is Windows 7 Pro, 32-bit, SP-1 accessing an Oracle 10g relational database.

I selected about 20000 records interactively in ArcMap of the point feature class, then ran the spatial join and the data was populated as expected for the selected records. The output was only of the selected point features.

Was running fine on Arc10 SP2 with the full data set.

Ran successfully with the full data set on Arc 9.3.1 using the same ArcSDE 9.2 SP6 data.

Ran successfully with full data set in Arc10 SP 3 using the same ArcSDE 9.2 SP6 data.

Ran approximately 12000 out of the 150,000 of the records were processed. It appears the process broke when the spatial join encountered a null value in the polygon layer for the shapefile data. This was not the case for SDE feature classes.

I have a Python script that performs a number of Spatial Joins that I perform the Spatial Joins as well as other routines using SDE data. Scripts and ArcGIS Toolbox routines run successfully to completion, but produce "null" output as described above. We have typically run these scripts over the last 4 years on a monthly basis.

The "Name" field is the only field that is added as part of the Spatial Join operation, in this case. See the attached document.

I would need to FTP or send data via a different method to provide data.

Thank you,
Nate

DanMcCoy · ‎03-30-2012

The problem Nate is experiencing was logged as NIM079126:
http://support.esri.com/en/bugs/nimbus/TklNMDc5MTI2

However, it looks like maybe it was listed as "declined" and that maybe ESRI was not able to reproduce it. This particular bug only seems to affect the �??Spatial Join�?� geoprocessing task (used by ArcToolbox, ModelBuilder, and Python), but not the �??Join data from another layer based on spatial location�?� option of the join data dialog available from clicking a layer in the table of contents (TOC)->joins & relates->join�?�

Also, I believe that you will only experience the bug with the geoprocessing task, if any of the �??target features�?� do not have a spatial relationship to any �??join features�?�. For example, if you are trying a point-in-poly intersect, any points that are not within a polygon could throw off the results of the entire operation. You can avoid this by unchecking �??Keep All Target Features�?� in the tool dialog or by using the KEEP_COMMON join_type option in your code.

If anyone else is running into this same problem after installing SP4, please contact ESRI Support to ensure that they are working to get this fixed.

Thanks,

Dan

PamTurner · ‎04-01-2012

Same here. Model Builder process crashes when it hits the "Spatial Join" task. Problem appears to be with the "closest" option. I need to use this option because of the licensing level on our server. Anyone have a work around? I'd like to use "Near" or one of the simplier processing tools, but no dice. Using ArcGIS 10.0 SP4.

I'll try the bug route with ESRI.

RussCoffey · ‎04-04-2012

Using ArcGIS 10 sp4, performing a polygon/points one-to-one spatial join, I am experiencing the same issue described where only the first record has any of the field mappings applied property. All remaining records are null on the joined fields (although the "Join_Count" field is correctly populated for all rows).

I am seeing the same behaviour whether I run the tool manually in ArcMap or as code in a stand-alone Python script. Also, storage format (SDE, File GDB, shapefile) doesn't seem to matter.
I saw the bug report mentioned above, and status of "Declined" sounds like it's not going to be fixed, so I'll submit a bug report as well.

However, unchecking the "Keep All Target Features" option was successful as a work-around for me in all cases I tried, so thanks to Dan for that suggestion!

DanMcCoy · ‎04-23-2012

The problem Nate is experiencing was logged as NIM079126:
http://support.esri.com/en/bugs/nimbus/TklNMDc5MTI2

However, it looks like maybe it was listed as "declined" and that maybe ESRI was not able to reproduce it....

FYI - ESRI has resurrected this bug and will be soon releasing a patch...

Bug: The Spatial Join tool produces NULL results for some fields/features on ArcGIS 10 SP4
http://support.esri.com/en/knowledgebase/techarticles/detail/40174