nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How to generate Elevation summary for huge data set in 1 go?

deleted-user-_D_hZi4SxV2S

Hi,

I am trying to generate summarized elevation for water pipes using SummarizeElevaion tool. But this tool has a limitation of 1000 records at a time. I have a huge dataset that has more than a 100,000 records. When I feed my dataset to the tool, it fails. Is there any workaround to this problem?

Find more posts tagged with

Accepted answers

All comments

RobertBorchert

I suppose the tool is limited to 1000 records because it charges 1 credit per 1000 records.

This prevents users from burning through numerous credits at one time.

Any way you can get the data into a database and do it from there?

deleted-user-_D_hZi4SxV2S

But the input to this tool should be a shape file right. I am not sure how to input the records from database to this particular tool. Thanks for the reply.

RobertBorchert

I thought this tool was an ArcGIS Online tool.

What kind of summary are you trying to execute or rather what results are you looking for?

What is your elevation data? Contours etc...

How many records do you have?

deleted-user-_D_hZi4SxV2S

I am using ArcGIS Desktop tool.

I am trying to get Min Elevation, Max Elevation, Slope, Aspect etc. I typically get all these columns in a cs this tool generates.

I am trying to find elevation of water pipe lines in polyline shape.

I have 120k records.

George_Thompson

What is the exact name of the GP tool in ArcGIS Desktop that you are trying to use?

deleted-user-_D_hZi4SxV2S

SummarizeElevation.

George_Thompson

Ok, I see the tool is an ArcGIS Pro tool (Summarize Elevation—Help | ArcGIS Desktop).

You may need to write a script that only selects 1000 records at a time and then combines them back together. Due to the 1000 record limit of the tool.

MarcoBoeringa

I thought this tool was an ArcGIS Online tool.

It is, unfortunately, due to being accessed from within ArcGIS Pro, the OP may not be fully aware of this, although at least the usage of credits for this service is clearly marked with a blue warning box once you open the tool. The tool is part of the Ready To Use tools, that have their own "cloud" icon in Pro, with an ArcGIS Online reference in the tooltip:

An overview of the Ready To Use toolbox—Help | ArcGIS Desktop

DanPatterson_Retired

In arcpy, you can use FeatureClassToNumPyArray, just export the field you need to an in_memory workspace without any other fields, Here is a quick 1 minute example of getting stats for an X field in this case

<SPAN class="keyword token">import</SPAN> arcpy

fc <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"C:\Git_Dan\a_Data\arcpytools_demo.gdb\polylines_pnts"</SPAN>

f_names <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>i<SPAN class="punctuation token">.</SPAN>name <SPAN class="keyword token">for</SPAN> i <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>ListFields<SPAN class="punctuation token">(</SPAN>fc<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN>

f_names
<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'OBJECTID'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'Shape'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'ORIG_FID'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'POINT_X'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'POINT_Y'</SPAN><SPAN class="punctuation token">]</SPAN>

a <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>FeatureClassToNumPyArray<SPAN class="punctuation token">(</SPAN>fc<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'POINT_X'</SPAN><SPAN class="punctuation token">)</SPAN>

a <SPAN class="operator token">=</SPAN> a<SPAN class="punctuation token">.</SPAN>view<SPAN class="punctuation token">(</SPAN>np<SPAN class="punctuation token">.</SPAN>float64<SPAN class="punctuation token">)</SPAN>&nbsp; <SPAN class="comment token"># ---- don't ask... just to make things simpler</SPAN>

a<SPAN class="punctuation token">.</SPAN>min<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> a<SPAN class="punctuation token">.</SPAN>max<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> a<SPAN class="punctuation token">.</SPAN>std<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> a<SPAN class="punctuation token">.</SPAN>mean<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="punctuation token">(</SPAN><SPAN class="number token">300000.29069999978</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">300042.31799999997</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">13.524386947645237</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">300021.91723809525</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

An array of 100,000 is trivial in numpy

BKrzyzanowski

You can use the territory design tool + explode multi part feature to separate your data into pieces of 999. The use arc py. Although I am having trouble because the summarize by elevation tool doesn't automatically save the output to a gdb. It just gets added to the map.

arcpy.agolservices.SummarizeElevation(2, "zip5", "30m", "NO_SLOPE_ASPECT")

arcpy.agolservices.SummarizeElevation(3, "zip5", "30m", "NO_SLOPE_ASPECT")

arcpy.agolservices.SummarizeElevation(4, "zip5", "30m", "NO_SLOPE_ASPECT")

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.