Creating a python conversion tool to use outside of GIS

324
8
06-10-2021 07:48 AM
Daniel_Hardwick
New Contributor III

Hi all,

I am trying to work out the best way to do quite a trivial GIS task for users who don't use desktop GIS. Mainly I want to allow people to export a dataset to shapefile format, or csv, xls etc. 

For a bit of context, we store our spatial data in a SQL database. Desktop GIS users make an ODBC connection to the data, and we also publish as web services for our web GIS users. Often we publish as Map Image layers as most users only need to view the data and not edit, but this does limit some of the functionality within Enterprise for extracting data. The majority of our users view data within Enterprise, and sometimes they want to export the dataset out as a shapefile or in table format. This is very simple to do in desktop GIS, but requests are becoming more frequent. 

I have some script files setup to convert some of the frequently requested data, which is fine for myself to run, but ideally I want to allow people who aren't desktop GIS users, to self serve and select whatever data they want to convert.

I am trying to work out the best way to do this. I know I can run a geoprocessing tool, such as feature class to feature class, and share it as a webtool to Enterprise, and that would work for a specific dataset, but I was wondering if there is a better way to do this, and ideally let the user select what dataset they want to convert/extract? 

0 Kudos
8 Replies
BlakeTerhune
MVP Regular Contributor

Could you add the service to ArcGIS Online or Portal as a feature layer? That would allow users to pretty easily extract the data themselves.

Extract Data—ArcGIS Online Help | Documentation

0 Kudos
Daniel_Hardwick
New Contributor III

Not really as the way we share out data to Portal is as a 'referenced' dataset, so people can update the data in ArcGIS Pro, or in Portal, and the changes are always being applied to the source data. We didn't want duplicate data being published to Portal/AGOL. Since it is referenced, it doesn't come through as a 'hosted' feature layer and therefore doesn't allow users to extract. 

feature_layer.JPG

0 Kudos
BlakeTerhune
MVP Regular Contributor

Ah, sorry, I didn't notice that Extract Data only worked on hosted feature layers. In that case, you could set up a Hub site to share stuff internally. We are testing it out and it works great for services hosted on our internal ArcGIS Server. Not sure if you need to enable Open Data first though.

Daniel_Hardwick
New Contributor III

Thanks Blake. That could potentially be an option, but I was hoping to perhaps put together some sort of script or application to allow people to export data... perhaps even in something like a Jupyter Notebook or alike, but I am quite new to Notebook and just getting to learn it, so not sure what is possible.

 

It might be the case that there isn't an easy solution for this, and I might be best off creating a number of scripts for the most commonly requested data, and just ran them as requested, but I was hoping for a 'cleverer' way! 

0 Kudos
BlakeTerhune
MVP Regular Contributor

A script could definitely be made to export data. However, it sounded like your users are already used to using Portal so it seemed like a website offering a menu of data and download options would be easier for them to use.

Alternatively, maybe you could run a script as a nightly scheduled task that exports the most commonly used feature classes in the most commonly requested formats. Save the outputs wherever is most convenient for your users to get. At least that would reduce most of your request workload.

0 Kudos
Daniel_Hardwick
New Contributor III

That's a good suggestion, thanks. 

Am I right in thinking that if I create a script, to say export a dataset to CSV, that the user would have to have python and perhaps even ArcGIS installed (to access relevant libraries/modules)? If this is the case then allowing users to run their own scripts probably isn't the solution, as these users don't typically have desktop GIS. 

0 Kudos
BlakeTerhune
MVP Regular Contributor

Correct. At minimum, you need Python installed. If you're interacting with anything ArcPy, you'll also need ArcGIS (or the engine) installed.

0 Kudos
Daniel_Hardwick
New Contributor III

Thanks Blake. 

I think my best option is to setup a script to run overnight, or look to publish a geoprocessing tool as a web tool to our Enterprise, and allow the user to run it themselves. Thanks again for your help. 

0 Kudos