I am having issues with working with large sets of data in insights, particularly with creating a chart/graph. I have 20 or so individual features, each with a few hundred thousand records (data continues to be added to this). When attempting to create a graph, there is an error message telling me to try adding data with <250,000 records.
I'm not sure if this is a limitation of Insights, but if it is, I am wondering if there might be a better place to display/analyze large sets of data if there is not a workaround for this.
Anyone working with analyzing and displaying large datasets - do you use Insights for this? If not, what do you recommend using instead?
Solved! Go to Solution.
Hi,
The data is not remote, though we may eventually want it to be so this will be a great resource for when we eventually make that change - thanks!
Hi,
I do have a predefined filter to only show a certain dataset (water levels recordings) and Insights also defaults to showing a moving average of all of the data. Ideally, I want to show all of it over time (the span of a few years) since it's trending groundwater level data, but for some features that we have years worth of data, it's too much for the graph to handle.
Hi Ashley,
Are you using feature/map service from your organization's published services to analysis the data ?
I have learned a hard way that only hosted services works the best for insights to analysis data efficiently.
Defining a relationship in a workbooks also only works on hosted services.
We have more than 600000 parcel data and I have published hosted services with Parcel shapes and CAMA data in simple tables and then make a relationship in insights.
This setup works for us to create all type of charts and maps, though I have to refresh the service daily through a python script
Predefined filters as suggested here by Laura will also help you get better performance on any object.
Hi,
Yes, I am using a published feature service within my organization for this. That might be the issue - I may try publishing as a hosted layer to see if that works better. I've tried the predefined filters, and that still doesn't seem to work for me so I will give that a shot, thanks!
Hi Ashley,
I am wondering if your data is remote, in that it is stored outside of your organization's data store? This link may be helpful https://doc.arcgis.com/en/insights/latest/administer/configure-insights.htm
There is a limit for remote datasets but you can configure it.
Hi,
The data is not remote, though we may eventually want it to be so this will be a great resource for when we eventually make that change - thanks!