topic Re: How to generate Elevation summary for huge data set in 1 go? in ArcGIS Pro Questions

How to generate Elevation summary for huge data set in 1 go?

deleted-user-_D_hZi4SxV2S — Fri, 09 Mar 2018 16:17:22 GMT

Hi,

I am trying to generate summarized elevation for water pipes using SummarizeElevaion tool. But this tool has a limitation of 1000 records at a time. I have a huge dataset that has more than a 100,000 records. When I feed my dataset to the tool, it fails. Is there any workaround to this problem?

Re: How to generate Elevation summary for huge data set in 1 go?

RobertBorchert — Fri, 09 Mar 2018 16:27:06 GMT

I suppose the tool is limited to 1000 records because it charges 1 credit per 1000 records.

This prevents users from burning through numerous credits at one time.

Any way you can get the data into a database and do it from there?

Re: How to generate Elevation summary for huge data set in 1 go?

deleted-user-_D_hZi4SxV2S — Fri, 09 Mar 2018 16:31:43 GMT

But the input to this tool should be a shape file right. I am not sure how to input the records from database to this particular tool. Thanks for the reply.

Re: How to generate Elevation summary for huge data set in 1 go?

RobertBorchert — Fri, 09 Mar 2018 16:37:35 GMT

I thought this tool was an ArcGIS Online tool.

What kind of summary are you trying to execute or rather what results are you looking for?

What is your elevation data? Contours etc...

How many records do you have?

Re: How to generate Elevation summary for huge data set in 1 go?

deleted-user-_D_hZi4SxV2S — Fri, 09 Mar 2018 16:44:19 GMT

I am using ArcGIS Desktop tool.

I am trying to get Min Elevation, Max Elevation, Slope, Aspect etc. I typically get all these columns in a cs this tool generates.

I am trying to find elevation of water pipe lines in polyline shape.

I have 120k records.

Re: How to generate Elevation summary for huge data set in 1 go?

George_Thompson — Fri, 09 Mar 2018 16:58:08 GMT

What is the exact name of the GP tool in ArcGIS Desktop that you are trying to use?

Re: How to generate Elevation summary for huge data set in 1 go?

deleted-user-_D_hZi4SxV2S — Fri, 09 Mar 2018 17:16:58 GMT

SummarizeElevation.

Re: How to generate Elevation summary for huge data set in 1 go?

George_Thompson — Fri, 09 Mar 2018 17:20:34 GMT

Ok, I see the tool is an ArcGIS Pro tool (Summarize Elevation—Help | ArcGIS Desktop).

You may need to write a script that only selects 1000 records at a time and then combines them back together. Due to the 1000 record limit of the tool.

Re: How to generate Elevation summary for huge data set in 1 go?

MarcoBoeringa — Fri, 09 Mar 2018 18:19:05 GMT

I thought this tool was an ArcGIS Online tool.

It is, unfortunately, due to being accessed from within ArcGIS Pro, the OP may not be fully aware of this, although at least the usage of credits for this service is clearly marked with a blue warning box once you open the tool. The tool is part of the Ready To Use tools, that have their own "cloud" icon in Pro, with an ArcGIS Online reference in the tooltip:

An overview of the Ready To Use toolbox—Help | ArcGIS Desktop

Re: How to generate Elevation summary for huge data set in 1 go?

DanPatterson_Retired — Sat, 11 Dec 2021 23:37:32 GMT

In arcpy, you can use FeatureClassToNumPyArray, just export the field you need to an in_memory workspace without any other fields, Here is a quick 1 minute example of getting stats for an X field in this case

import arcpy

fc = r"C:\Git_Dan\a_Data\arcpytools_demo.gdb\polylines_pnts"

f_names = [i.name for i in arcpy.ListFields(fc)]

f_names
['OBJECTID', 'Shape', 'ORIG_FID', 'POINT_X', 'POINT_Y']

a = arcpy.da.FeatureClassToNumPyArray(fc, 'POINT_X')

a = a.view(np.float64)  # ---- don't ask... just to make things simpler

a.min(), a.max(), a.std(), a.mean()
(300000.29069999978, 300042.31799999997, 13.524386947645237, 300021.91723809525)
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

An array of 100,000 is trivial in numpy

Re: How to generate Elevation summary for huge data set in 1 go?

BKrzyzanowski — Fri, 13 May 2022 15:16:07 GMT

You can use the territory design tool + explode multi part feature to separate your data into pieces of 999. The use arc py. Although I am having trouble because the summarize by elevation tool doesn't automatically save the output to a gdb. It just gets added to the map.

arcpy.agolservices.SummarizeElevation(2, "zip5", "30m", "NO_SLOPE_ASPECT")

arcpy.agolservices.SummarizeElevation(3, "zip5", "30m", "NO_SLOPE_ASPECT")

arcpy.agolservices.SummarizeElevation(4, "zip5", "30m", "NO_SLOPE_ASPECT")