topic Re: Dynamic Dashboard/Automated Dashboard in ArcGIS Dashboards Questions

Dynamic Dashboard/Automated Dashboard

Anonymous User — Fri, 10 Sep 2021 17:11:24 GMT

Hello, We are using Enterprise 10.8.1. For dashboards we have created. we want them to automatically/dynamically update. We connected our SQL server to ArcPRO and added the SQL data to the ArcGIS Datastore. It is our understanding that the only way to dynamically update a dashboard is via the datastore. This step is now complete.

We are now working on the analyst work flow in using the data. I would like to know what the supported workflow is in using datastore data to create dashboards. This is what we have found. 1. We usually bring in the data to ArcPRO, then display x,y to map points, then publish, then create dashboard. 2. Now with the datastore, we followed the same process display x,y, create feature layer, publish to enterprise but when creating the dashboard and trying to create a serial chart we received an error that the data could not be used for statistics. We discovered that using the tool Make Event Layer vs Display x, y when mapping the data worked. In other words, we used Make Event Layer, published data, then created dashboard. The serial widget worked with the data. So my question is, is that how to create dynamic dashboards by using Make Event Layer, and you cannot use the display x,y to create a feature layer? Thank you

Re: Dynamic Dashboard/Automated Dashboard

forestknutsen1 — Fri, 10 Sep 2021 17:42:40 GMT

I am not sure how helpful this will be for you.

But we just created a dashboard that automatically refreshes x, y events. But the pattern is a little different.

The process:

1. Update a table in Oracle with latitude, longitude, and other attributes with python batch job

2. Using ArcMap create the x, y events, symbology, etc. and publish the service to portal/server

3. Create map and set refresh intervals.

4. Create dashboard

Re: Dynamic Dashboard/Automated Dashboard

Anonymous User — Fri, 10 Sep 2021 17:48:34 GMT

Hello,

Thank you for your response. Yes that is the workflow we use but use SQL.

Question when you "create the x, y events" what tool are you using?

When we use display x,y and create a feature layer, the dashboard doesn't work, but when we use the geoprocessing tool "make event layer" it does work.

Thank you,

Annette

Re: Dynamic Dashboard/Automated Dashboard

forestknutsen1 — Fri, 10 Sep 2021 18:27:18 GMT

I am just right clicking the table in the ArcMap table of contents and selecting display x,y events

then I publish it as a map service. So, on the Portal side I end up with a "Map Image Layer"

Re: Dynamic Dashboard/Automated Dashboard

jcarlson — Fri, 10 Sep 2021 18:29:15 GMT

I'd look into using the ArcGIS Python API, if that is an option for you. You can easily update your hosted layer in place, without opening desktop software or re-running any GP tools.

from arcgis import GIS from arcgis.features import GeoAccessor import pandas as pd # Log in to Portal gis = GIS('your-portal-url', 'user', 'password') # Get hosted layer; replace '0' w/ whatever the layer index is fl = gis.content.get('item-id-of-layer').layer[0] # Query out from your SQL database sql = """ some sql query string SELECT whatever FROM wherever WHERE something=1 """ conn = "some connection string" # see sqlalchemy docs for what this should be df = pd.read_sql(sql, conn) # Convert to spatial dataframe using x/y columns sdf = GeoAccessor.from_xy(df, x_column, y_column, sr=4326)

With that spatial dataframe, you can apply the resulting features to your layer quite simply.

fl.edit_features(adds=sdf.spatial.to_featureset())

If there are issues with duplicates from your query, you can truncate the hosted layer prior to adding new features.

Re: Dynamic Dashboard/Automated Dashboard

Anonymous User — Fri, 10 Sep 2021 18:47:08 GMT

That's exactly how we were doing it before. I did try to publish it as a map service and it didn't work, probably due to permissions. Good to know though that you can do it with display x,y. And I see the python script, are you using the python script as well? In the above example? thank you very much, this has been very helpful

Re: Dynamic Dashboard/Automated Dashboard

forestknutsen1 — Fri, 10 Sep 2021 18:53:04 GMT

So, I started with a similar strategy. But I was using ArcGIS Online as the gis target. It worked well over all. I had the job set to run every few minutes. The problem was that the connection would mess up (time out) every few days, creating a hole in the data updates. I found the AGS service to be far more robust. But maybe this strategy would work well for Annette.

This process for me was:

Add csv to Online by hand and create a hosted feature layer:

Then update with a python batch job.

This is my simple proof of concept script (I have the full dev one as well if you want to see it - the prototype might be faster to digest):

from arcgis import GIS from arcgis.features import FeatureLayerCollection import pickle original_pickle = 'original' in_file = open(original_pickle, 'rb') original = pickle.load(in_file) in_file.close() print(original) if original: city = 'city.csv' time = 'time.csv' else: city = 'city2.csv' time = 'time2.csv' gis = GIS(url=xxxx, username=xxxx, password=xxxx) my_content = gis.content.search(query='owner:' + gis.users.me.username, item_type='Feature Layer Collection', max_items=100) cities_item, = [x for x in my_content if x['title'] == 'city'] time_item, = [x for x in my_content if x['title'] == 'time2'] print('updating city layer to: {}'.format(city)) cities_flayer_collection = FeatureLayerCollection.fromitem(cities_item) cities_flayer_collection.manager.overwrite(city) print('updating time layer to: {}'.format(time)) cities_flayer_collection = FeatureLayerCollection.fromitem(time_item) cities_flayer_collection.manager.overwrite(time) print('done') original = not original out_file = open(original_pickle, 'wb') pickle.dump(original, out_file) out_file.close()

Re: Dynamic Dashboard/Automated Dashboard

Anonymous User — Fri, 10 Sep 2021 18:52:21 GMT

Thank you. I will see if we can try this as well.

Re: Dynamic Dashboard/Automated Dashboard

Anonymous User — Fri, 10 Sep 2021 20:39:00 GMT

Hi Forestknutsen1,

Can you show me how you publish it as a map service. I notice you are in ArcMAP, we are using ArcPRO. But I just opened ArcMAP to try it. Thank you.

Re: Dynamic Dashboard/Automated Dashboard

Anonymous User — Mon, 13 Sep 2021 16:19:08 GMT

We are federated, are you federated or stand-alone? It seems, but I am not 100% sure, that publishing a service works for stand-alone. Thank you.

Re: Dynamic Dashboard/Automated Dashboard

forestknutsen1 — Mon, 13 Sep 2021 17:33:50 GMT

We are federated

Re: Dynamic Dashboard/Automated Dashboard

IngridLuffman — Tue, 02 Sep 2025 17:37:36 GMT

Hi, I'm using a very similar python API process to update a hosted table in AGOL (which is connected to my dashboard). 1. connect and access my layers on AGOL, 2. make the feature layer into a collection, 3. overwrite the collection with a local csv file.

I am experiencing something like what you describe above: the process works fine until it times out, hangs for a day or three, and then restarts. FYI, I'm using a scheduler to run my processes and our error logs show the process starting and ending 1 min later most of the time, but sometimes it takes days (or I reboot and fix the problem for a bit).

Can you tell me more about doing this with AGS (assume this is ArcGIS Server)? Thanks!