Performance File Geodatabse vs Enterprise Geodatabase

JustinPence2 · ‎05-12-2022

Hello, question. We are looking to work on a project that involves parcel data involving multiple counties. The data will be used for basic geoprocessing (filtering, dissolving, merging of data), with only one or two editors, with one working on it at a time. It will just consist of feature classes, not raster information. We will not publish the data to ArcGIS Online nor ArcGIS Enterprise.

With that said, because we are dealing with hundreds of thousands of data, will performance/usability be a concern if the data is housed in a File Geodatabase, rather then an Enterprise Geodatabase. I've worked point datasets in a file geodatabase that have 5 million records or more, and generally been fine. Essentially, what I am after, are there Performance differences between File or Enterprise Geodatabases. Or, are the gains with Enterprise Geodatabases mainly Multi-User Editing, Versioning, Use in ArcGIS Enterprise, etc.

DanaNolan · ‎05-17-2022

Network data storage is often organizational policy, for many reasons, particularly in federal government. At my site, local drives are never backed up because they are only used for private data (PII) like the resumes I have to read and hold off network until a job is filled, or editing non-GIS files during scheduled network outages. We all sign an agreement to abide by this policy when we get network access.

Then there is the giant bus factor of only one person knowing where the latest copy of the data is. Or worse, they don't log on to the network for updates often enough and their computer is quarantined so no one can get to that data for days or weeks and it could get destroyed in a re-image.

So, if you can and must use local storage, back up more than once a day and in a consistent manner. But think about the problem of being locked out when you need to replace the network copy with the local copy. Even a non-editor can lock a file geodatabase or schema lock an SDE feature.

In my very security-oriented, cover your colleagues' tasks on short notice environment, we tolerate slowness of the network for most editing. It is simpler. Big edits and topology fixes and geoprocessing-heavy work, not so much. Then we work at least partly locally, backing up regularly to the network. I recommend not using the default file geodatabase in ArcMap for important work because it is stored in your profile (under Documents). Set up a scratch database or your editing database in a local folder that other people who log in to your system can see. Of course, the one time I did not do this, I had to quarantine for 2 weeks and no one could get to one of my projects in my default geodatabase.

View solution in original post

Asrujit_SenGupta · ‎05-12-2022

Hi. Considering more than 1 user is supposed to work with the data, where do you plan to store the File geodatabase?

JustinPence2 · ‎05-13-2022

Hello. It would be one user at a time working with the datasets. Other then thinning of the data, and extracting what we want out of it, it wouldn't be used for our ArcGIS Enterprise or ArcGIS Online accounts. After thinning and consolidating the data, we would just extract the Tabular Data and import to a Non Geospatial Database.

If we set up File Geodatabases, they would be stored on a Network Volume/Drive at my company's datacenter.

VinceAngelo · ‎05-15-2022

@JustinPence2 wrote:
If we set up File Geodatabases, they would be stored on a Network Volume/Drive at my company's datacenter.

Yeah, don't do that. Network drives have half the performance of local drives.

If you're single-user, copy the FGDB locally, then upload again when done.

It's a myth that relational databases are faster than local files. Relational databases have a more robust toolbox for SQL queries and data manipulation, but they're generally not faster because they have ACID overhead. Make no mistake: Having atomicity, consistency, isolation, and durability are *good* things, but they come with a price.

The real proof is in the execution. Use FGDB (locally) and see if the performance is where you need it. Then optimize what is not up to spec. Preemptive optimization can be an antipattern.

- V

DanaNolan · ‎05-17-2022

Network data storage is often organizational policy, for many reasons, particularly in federal government. At my site, local drives are never backed up because they are only used for private data (PII) like the resumes I have to read and hold off network until a job is filled, or editing non-GIS files during scheduled network outages. We all sign an agreement to abide by this policy when we get network access.

Then there is the giant bus factor of only one person knowing where the latest copy of the data is. Or worse, they don't log on to the network for updates often enough and their computer is quarantined so no one can get to that data for days or weeks and it could get destroyed in a re-image.

So, if you can and must use local storage, back up more than once a day and in a consistent manner. But think about the problem of being locked out when you need to replace the network copy with the local copy. Even a non-editor can lock a file geodatabase or schema lock an SDE feature.

In my very security-oriented, cover your colleagues' tasks on short notice environment, we tolerate slowness of the network for most editing. It is simpler. Big edits and topology fixes and geoprocessing-heavy work, not so much. Then we work at least partly locally, backing up regularly to the network. I recommend not using the default file geodatabase in ArcMap for important work because it is stored in your profile (under Documents). Set up a scratch database or your editing database in a local folder that other people who log in to your system can see. Of course, the one time I did not do this, I had to quarantine for 2 weeks and no one could get to one of my projects in my default geodatabase.