Record return limit, exporting data, and related tables

1382
1
10-19-2021 11:13 AM
Labels (1)
ElijahMlawsky
New Contributor II

Hello - I've seen these topics mentioned on this board in the past, but without much in the way of discussion or resolution.

I have published a REST service from a .mxd that contains a point feature class as well as several related data tables. Roughly, there are ~10,000 points, and perhaps ~100,000 related records per table. I have pulled these into Web AppBuilder, and would like to allow app users to export the data to .csv, with any applied filters or selections persisting.

I see two options for this, each of which has its caveats:

 

Option 1) Allow exporting via the attribute table widget. This is the more intuitive option, in my opinion, however, exports are limited to the "maximum record return limit" of the service. I currently have this limit set to 1,000, and as expected, any significant increase results in a performance hit to the app.

In theory, I could upgrade the REST server hardware resources, allowing an increased record return limit. Are there other bottlenecks to be aware of (bandwidth, client browser resources, Web AppBuilder limitations - or are these insignificant)? Does anyone have hardware recommendations for this use case? I will be running some stress tests in the near future to look at the relationship between the record limit and CPU/memory usage. I'll report back here if I find anything interesting.

Oddly enough, even with the record return limit set to 1,000, all points populate on the map and in the attribute table. My guess is that the data paginate into 1,000-record batches, and continue to append to the map and table until all data been delivered.

Why then, does the "export data" option deliver a .csv with only 1,000 records? The full data are already present in the attribute table. The export function should not be referring back to the service limit at this point. This is more confusing when considering how data requests are handled by the select widget (option 2, below).

Something more: when viewing the related records in the attribute table, they do not concatenate, and are instead limited to 1,000 rows.

Is there a workaround for getting related tables to display all rows in the attribute table that does not involve making them a feature class with redundant geometry?

 

Option 2) Allow exporting via the select tool widget. I find this to be less user-intuitive, but it does seem to request data properly: the entire selection is exported to .csv regardless of the record return limit. So, limit-independent export is possible - why does the attribute table's export do things differently?

Unfortunately, there is no way to "select" from the related tables, and thus no way to export them with this option. Am I missing something here?

 

Looking forward to discussion of any questions above. The ability to export data elevates an ESRI web app from a viewing portal to an actual means of distribution, however, this functionality is severely limited as is. Thanks all for your feedback.

1 Reply
BryanSerino1
New Contributor

+1

0 Kudos