Intersect polygons and segments does not complete normally

1483
13
07-01-2017 07:12 AM
EricMartin
New Contributor II

I have shapefiles with segments (representing roads, but they're just regular polylines, not a network dataset) and polygons (~14,000 polygons generated with service areas). I'm running intersect to get the segments (and their lengths) within each polygon. On the first run, I got errors, and the tool did not complete. I reduced the segments dataset (from lines representing the whole county to just those within the city) and ran "repair geometry," which did find some problems and fixed them. When I ran it again, the analysis took 12+ hours and completed. The table looked normal, but the resulting featureset was suspicious. It had exactly 64,000 segments in it, and only a few tiny segments mapped. The rest did not appear on the map. Repair geometry on the results featureset had no effect. I ran the analysis again from python. Again it took 12+ hours. This time there were exactly 70,000 segments in the results, and none of them mapped.  So, I don't have confidence that these are complete results. 

I ran a similar analysis with the same polygons shapefile and another set of segments from the same road system; this one had many, many more segments, and the analysis completed normally. It took maybe four hours to run on my university's machine. 

Seems like maybe the analysis is maxing out the computing resources, causing ARCMap to choke and produce bad results. But why could it run the previous, much more complex analysis? Advice? 

Tags (2)
0 Kudos
13 Replies
shan_sarkar
Occasional Contributor III

Eric,

  1. What version of ArcMap are you running on your machines? Make sure you check out the system requirements for that particular version just to be on the safer side.
  2. Refer: 

As per the observation you have posted in your query, it is possible the issue may be with your data. Try to export all the line features in a feature class and execute the Check Geometry—Help | ArcGIS Desktop and then back up your data and execute the Repair Geometry—Help | ArcGIS Desktop . 

Once you have completed this try to run the Intersect tool.

I hope this helps

~Shan


~Shan
EricMartin
New Contributor II

Thanks for your reply. The version is 10.4.1. It's a university install on university machines run by the research computing people, so I assume it's going to be set up correctly and adequately for most normal and even moderately intensive uses.

I will try your suggestions and report back. Thanks! 

shan_sarkar
Occasional Contributor III
0 Kudos
EricMartin
New Contributor II

Repeating my reply to another user so you'll see it.

I doubt that the polygons are very complex. They are generated with service areas on a mostly sqare street grid.

I exported the line features to a new features class and checked the geometry. It was fine. (I had previously run repair on both the segments and the polygons.) I projected the polygons so that shapefile was in the same system as the line segments. As a precaution, I moved both shapefiles into the same file geodatabase.  

Again the intersect ran a long time--overnight. This time the results had 2048 features, and all the data in the table was blank. Nothing mapped.

0 Kudos
DanPatterson_Retired
MVP Emeritus

can you provide some information on the coordinate system of the files... and I am assuming that they are the same

EricMartin
New Contributor II

Ah! I think they are not the same! I'll project them into the same system and try that.

0 Kudos
EricMartin
New Contributor II

I exported the line features to a new features class and checked the geometry. It was fine. (I had previously run repair on both the segments and the polygons.) I projected the polygons so that shapefile was in the same system as the line segments. As a precaution, I moved both shapefiles into the same file geodatabase.  

Again the intersect ran a long time--overnight. This time the results had 2048 features, and all the data in the table was blank. Nothing mapped.

0 Kudos
DanPatterson_Retired
MVP Emeritus

Eric...

If you created two feature classes in the same coordinate system in a file geodatabase AND you have sufficient memory, Intersect will dice the area automatically

http://desktop.arcgis.com/en/arcmap/latest/tools/supplement/tiled-processing-of-large-datasets.htm

You are either choking on memory, the location of the file (ie not on a local machine) or something else.  

Repeating for another overnighter without indicating how big the files are wouldn't be prudent.  If tiling the area manually by selecting a smaller area subset of the two for testing, doesn't reveal anything, you might want to contact tech support to see if there is anything that the suggestions and you haven't ruled out.

shan_sarkar
Occasional Contributor III

I agree with Dan_Patterson in fact that exactly what I had suggested in the first place.


~Shan
0 Kudos