Creating a python conversion tool to use outside of GIS

[Daniel_Hardwick]

Hi all,

I am trying to work out the best way to do quite a trivial GIS task for users who don't use desktop GIS. Mainly I want to allow people to export a dataset to shapefile format, or csv, xls etc. 

For a bit of context, we store our spatial data in a SQL database. Desktop GIS users make an ODBC connection to the data, and we also publish as web services for our web GIS users. Often we publish as Map Image layers as most users only need to view the data and not edit, but this does limit some of the functionality within Enterprise for extracting data. The majority of our users view data within Enterprise, and sometimes they want to export the dataset out as a shapefile or in table format. This is very simple to do in desktop GIS, but requests are becoming more frequent. 

I have some script files setup to convert some of the frequently requested data, which is fine for myself to run, but ideally I want to allow people who aren't desktop GIS users, to self serve and select whatever data they want to convert.

I am trying to work out the best way to do this. I know I can run a geoprocessing tool, such as feature class to feature class, and share it as a webtool to Enterprise, and that would work for a specific dataset, but I was wondering if there is a better way to do this, and ideally let the user select what dataset they want to convert/extract? 

[BlakeTerhune]

Could you add the service to ArcGIS Online or Portal as a feature layer? That would allow users to pretty easily extract the data themselves.

Extract Data—ArcGIS Online Help | Documentation

[Daniel_Hardwick]

Not really as the way we share out data to Portal is as a 'referenced' dataset, so people can update the data in ArcGIS Pro, or in Portal, and the changes are always being applied to the source data. We didn't want duplicate data being published to Portal/AGOL. Since it is referenced, it doesn't come through as a 'hosted' feature layer and therefore doesn't allow users to extract. 

 

[BlakeTerhune]

Ah, sorry, I didn't notice that Extract Data only worked on hosted feature layers. In that case, you could set up a Hub site to share stuff internally. We are testing it out and it works great for services hosted on our internal ArcGIS Server. Not sure if you need to enable Open Data first though.

[Daniel_Hardwick]

Thanks Blake. That could potentially be an option, but I was hoping to perhaps put together some sort of script or application to allow people to export data... perhaps even in something like a Jupyter Notebook or alike, but I am quite new to Notebook and just getting to learn it, so not sure what is possible.

It might be the case that there isn't an easy solution for this, and I might be best off creating a number of scripts for the most commonly requested data, and just ran them as requested, but I was hoping for a 'cleverer' way! 

[BlakeTerhune]

A script could definitely be made to export data. However, it sounded like your users are already used to using Portal so it seemed like a website offering a menu of data and download options would be easier for them to use.

Alternatively, maybe you could run a script as a nightly scheduled task that exports the most commonly used feature classes in the most commonly requested formats. Save the outputs wherever is most convenient for your users to get. At least that would reduce most of your request workload.

[Daniel_Hardwick]

That's a good suggestion, thanks. 

Am I right in thinking that if I create a script, to say export a dataset to CSV, that the user would have to have python and perhaps even ArcGIS installed (to access relevant libraries/modules)? If this is the case then allowing users to run their own scripts probably isn't the solution, as these users don't typically have desktop GIS. 

[BlakeTerhune]

Correct. At minimum, you need Python installed. If you're interacting with anything ArcPy, you'll also need ArcGIS (or the engine) installed.

[Daniel_Hardwick]

Thanks Blake. 

I think my best option is to setup a script to run overnight, or look to publish a geoprocessing tool as a web tool to our Enterprise, and allow the user to run it themselves. Thanks again for your help.