How do you store your GIS projects?

Question

We are struggling with a hybrid-friendly file storage location, and I want to know how other organizations store projects and project data to use in ArcGIS Pro. We use ArcGIS Online for our data source of truth. We have 1 GIS Adminstrator (me), 12 GIS desktop users, and 1 IT technician. Users are accessing Pro on up-to-spec laptops at home or remoting into desktops in the office. Our other organizational files are on-prem, but are migrating to Sharepoint soon.

Here's what we've had:

* Originally, we used an on-prem file server for GIS. During pandemic, performance was throttled with VPN interaction. 
* Switched to using sharepoint in 2020 as a bridging solution. Snappy performance, authenticated through microsoft, but backup and file corruption issues
* Implemented cloud based Azure file storage in 2024 (standard general purpose not premium). Accessed through Azure VPN with microsoft authentication. Was super performant, easy to access. Now, running into crazy slow performance - unclear what changed. Esri support tells us that cloud file storage is unsupported (although I had used it at a previous job + on prem PCs without trouble). Azure support recommends upgrading to Premium File Share, which is not guaranteed to solve problems. I've reviewed this post  - the netapp + VM architecture is out of our budget.

Thanks in advance for sharing your setups!

RTPL_AU · Answer

@SCVOSAGIS_Program With 13 GIS people using Pro + AGOL, you could probably save 0.5 to 1 FTE per year by going local data over a fast link and have AGOL as a data product display rather than main data store.  I don't know your business so I may be 100% off on a tangent here.  With local data I mean having app & data in the same plane i.e. in local metal or AWS AppStream + AWS storage (which is non-trivial to get perfect).

I've found that with some data processes it is much faster to edit the data as a local instance and replace/append the data into AGOL, rather than trying to edit the AGOL data as a web service.

As you're already in 'I want to change things' mode, I'd suggest looking at data location as an efficiency metric and then see what you will accomplish/save if your staff are more efficient.  If you have the time run the same typical process as a local data vs AGOL comparison. Do a few edits, make a pretty map, do a few layouts, export to pdf, etc and measure the difference in time. Multiply by your number of people, cumulative over a year. When you come back from the WC after a good cry, make a few decisions. 🙂

RTPL_AU · Answer

@SCVOSAGIS_Program  Using ArcGIS Desktop (Pro or Map) with a file syncing service such as SharePoint, OneDrive. DropBox, etc may work if you are lucky or it will fail spectacularly when you least need it.

There is an option to go full cloud native - in either Azure or AWS you can host both the desktop app and file storage in the 'cloud' thereby still having your processing and data close to each other (with is always the goal). This has been a thing for almost a decade yet not spoken of often enough in my opinion.

Some Esri resellers offer this as a service where they take care of the full deployment and you only worry about your data and processes. This means you do not have to have IT skills with cloud native GIS experience in-house.

Even if you use AGOL as the main data store, there will still be the local processing, cleaning, etc requirements and the last thing you want is for users to manage data on a laptop C: drive all over the world because 'the vpn was too slow today'.  AppStream and equivalents work well on most proper internet services, even airport wifi (and as no data is being transferred just the AppStream stream that you secure appropriately, things will feel much snappier than accessing data stores over a vpn) .
With user based licensing your users can fall back to a local instance of Pro if they are fully offline due to outages, etc. but you should have that covered already with your current hybrid setup.

Personally, I run big desktops with 10GBE to local servers, with multiple backup strategies. My clients don't pay me to wait for data to load, and I don't go out much 🙂

joyoung_kerrvilletx · Answer

We still store and use ArcGIS Pro project files saved locally on each computer with the data stored in ArcGIS Enterprise. If a change needs to be made to the to the ArcGIS Pro project file, I will make the change and provide copies for other users to put on their computer. We found trying to run project files from server file shares or worse OneDrive was much slower and had issues with file locks.

This summer I will be looking into the new Portal Projects available with ArcGIS Pro 3.5 and ArcGIS Enterprise 11.4+.  Here is a link to Esri's blog post about the Portal Projects: https://www.esri.com/arcgis-blog/products/arcgis-pro/administration/work-together-in-arcgis-pro-shared-update-groups-portal-projects