topic Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing in ArcGIS Pro Questions

huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Sat, 16 Jan 2021 17:57:22 GMT

ArcGIS Pro 2.7. Windows 10.

In the folder- C:\Users\<user>\AppData\Local\Temp

I have a huge amount (almost 40,000, 200GB) of *.geodatabase files. These are not clearing when ArcGIS Pro shuts down or when I turn off/restart my PC. I haven't noticed my internal drive fill up so quickly before, but realised my storage space depleting after installing the 2.7 update to Pro last week.

All the .geodatabase files have been created in the last week (since the 9th Jan 2021 all the way up until today, when I haven't even opened Pro!) and are anywhere between 200KB or so, to 62mb in size each.

Watching the folder now, I can see the odd .geodatabase-journal and .geodatabase-conch files pop up and disappear momentarily, and I'm not even running Pro when I see this (the only Arc related process running is ArcGISIndexingServer). Seems like these files are mobile geodatabase (new since Pro 2.7) SQLite temp files.

What are the .geodatabase files though? Are they specific to the 2.7 update? Why are they clogging my machine and not being removed automatically after sessions? Can I manually delete them safely? Will they just come back after anyway? It must be a bug of sorts.

Thanks.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

DanPatterson — Sat, 16 Jan 2021 19:53:45 GMT

I have none of that garbage. Check you windows settings as well for cleaning out temporary files.

t [install drive]:\Users\<username>\AppData\Local\ESRI\Local is for your local cache and you can clear those out from the Project tab, Options Display. I don't think those are display related.

Are working data not locally stored? That may be an issue, if you can narrow down the origin

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

Robert_LeClair — Sat, 16 Jan 2021 20:37:47 GMT

I too have none of those files in my directory. One thing to try is to do a complete uninstall of AGP 2.7 and a clean install of AGP 2.7 to see if that fixes the issue.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

MarcoBoeringa — Sun, 17 Jan 2021 12:32:21 GMT

Actually, I think you already answered most of the questions yourself...

As you state, the .geodatabase files are ESRI's new Mobile Geodatabases based on SQLite that ESRI introduced at 2.7.

I have seen similarly uniquely numbered files in TEMP in older releases, and these are likely related to some form of local caches, either for imagery or vector (tile) based services you are using in one of your projects, or alternatively, as you stated, the ArcGIS Indexing Service may be involved in the creation of these, although I doubt it would create this much of crud.

I guess ESRI substituted some other file based caches for the new format.

Of course, if these caches do eat up your entire drive, I think it is time to report this to ESRI as a potential bug. That shouldn't happen.

Anything in TEMP is by definition "scratch", and can be safely removed. If not, it is a bug in the software. Do note though, that some of these files may still be in use and locked, e.g. by the Indexing Service or whatever is using them, and thus not removable.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Sun, 17 Jan 2021 19:09:20 GMT

I've raised this as an official support case. Will report back on the progress there. But yes it must be a bug, as 200GB of temp files not clearing is certainly not an intentional process from ESRI.

I'm not sure if it was related but 'some time' over the last 7 days I was pulling in a few layers from the ArcGIS Portal, in the catalog pane in Pro. A few of the layers weren't loading and errored out. But maybe this set off a snow ball of endless caching...

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Mon, 18 Jan 2021 17:40:14 GMT

I've noticed a few things recently:

a process from the ArcGIS 'bin' folder called 'ExcelToSQLLite64.exe' is running in the background using 6% CPU and almost 500MB of RAM when I don't even have Pro open. I haven't opened it all day.

This screenshot is from today, where multiple .geodatabase files are still being created within seconds of each other (see red circle in middle of screen). Further clogging up my hard drive.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

DanPatterson — Mon, 18 Jan 2021 17:51:41 GMT

Did you stop it and see what happens? Does it affect Pro?

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Mon, 18 Jan 2021 18:28:24 GMT

OK I stopped 'ExcelToSQLLite64' at 6pm (UK time). I've monitored the C:\Users\<user>\AppData\Local\Temp folder for the last 30mins and noticed no .geodatabase files being created at all.
So perhaps the process is causing this issue. 'SQLLite' being a recent introduction into the main Pro release by ESRI in 2.6/2.7 I believe.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

DanPatterson — Mon, 18 Jan 2021 19:55:42 GMT

which would explain why some people haven't experience the behaviour

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

MarcoBoeringa — Mon, 18 Jan 2021 20:10:09 GMT

You are not by any chance an ESRI "Production Mapping" or "Defense Mapping" user? I have never used these products myself, but recently noticed in the documentation Help pages that some of the cartographic product specifications for these extensions are defined in Excel and used as input for tools. Maybe this process is related...

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Mon, 18 Jan 2021 20:21:24 GMT

some people yes, but I'd say a LOT of people have 2.6 or 2.7, so I'm surprised this issue hasn't been covered more on here. Especially considering the mess it causes. I've got the support case rolling now with ESRI so should have some answers this week I hope!

And to answer your other question, no Pro seemed to work fine on the few occasions I've used it this week.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Mon, 18 Jan 2021 20:20:04 GMT

No. Just a simple Basic Licence for ArcGIS Pro. No extensions or extra modules installed.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Mon, 18 Jan 2021 22:25:42 GMT

Please see my latest posts in this thread. When I force quit the process 'ExcelToSQLLite64', the .geodatabase files stop appearing in my temp folder.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Wed, 20 Jan 2021 12:23:35 GMT

On further investigation I can see a .log file which has logged these .geodatabase files since Jan 8th 2021 (about when this issue must have first started). See a sample entry of this log file below. There are 1000s of these, and the log file is currently 70mb which is huge for a plain text file.

File: C:\Users\<user>\AppData\Local\Temp\sde_setup.log Mode: GMT Time: 2021-01-20T12:10:41+00.00 Local Time: 2021-01-20T12:10:41+00:00 -------------------------------------------------------------------- [12:10:41.754] ================================= CONNECTING TO SQLITE DBMS ================================= [12:10:41.762] Dbms_Name : main [12:10:41.762] Dbms_Ver : 3.32.3 [12:10:41.762] Database File : file:C:\Users\<user>\AppData\Local\Temp\b6cee44cb9a3709ae03c4398c58ccdaa.geodatabase?mode=rwc [12:10:41.762] Access : Read/Write [12:10:41.762] Database Type : SQLite with no Spatial Type [12:10:41.795] VERSION table created... [12:10:41.811] LAYERS table being created... [12:10:41.825] LAYERS table created... [12:10:41.825] METADATA table being created... [12:10:41.839] METADATA table created... [12:10:41.839] TABLE_REGISTRY table being created... [12:10:41.853] TABLE_REGISTRY table created... [12:10:41.853] LOCATORS table being created... [12:10:41.868] LOCATORS table created... [12:10:41.868] COLUMN_REGISTRY table being created... [12:10:41.882] COLUMN_REGISTRY table created... [12:10:41.961] Geodatabase Schema created... [12:10:41.966] Successfully installed ArcSde.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

MarcoBoeringa — Wed, 20 Jan 2021 14:19:39 GMT

As the log clearly states, this is the log from the setting up of the SQLite "Mobile Geodatabase", with the ArcSDE repository and geodatabase system tables being created.

The initialization apparently succeeds, as the log shows no errors, so no problem there.

One thing that is mildly interesting though, is the creation of the LOCATORS table being visible. This seems to be created in all these Mobile Geodatabases, and must be associated with the geocoding facilities in ArcGIS Pro and the "Locators" option in the Pro Catalog. Are you using any of the geocoding stuff in Pro? Note though that these geodatabase system tables, as said, seem to be created by default, so this is not out of the ordinary and not necessarily related to the geocoding stuff in Pro.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Wed, 20 Jan 2021 15:36:55 GMT

I use the default geocoder to locate places in Pro (the ArcGIS World Geocoding Service).

Yes I'm not concerned with these files being made, it's just they are not being cleared at all! And they are even created when I'm not using Pro (ie. it's not even open). This leaves me with an ever growing 200GB of .geodatabase files hanging around.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

MarcoBoeringa — Wed, 20 Jan 2021 17:19:05 GMT

As you now confirmed you are using a geocoding service, and you stated you opened a support case with ESRI, I think it might be useful to add this information about your usage of geocoding to the support case, as it may be relevant to the bug. It might help ESRI finding the source of the problem.

Entirely agree your disk shouldn't fill up with so many files, all the more reason to keep ESRI posted with any potential new leads you find.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Wed, 20 Jan 2021 17:22:16 GMT

Every ArcGIS Pro user has access to the geocoder installed by default. Yes I have it too, but I've probably used it to search for a location once or twice this month.

I've sent my log to my ESRI case, so hopefully that'll help anyway.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

MarcoBoeringa — Wed, 20 Jan 2021 17:30:59 GMT

@TheodoreF wrote:
Every ArcGIS Pro user has access to the geocoder installed by default

That may be so, but not everybody uses it, so that is why I think it is relevant to explicitely add this information to your support case.

@TheodoreF wrote:
I've sent my log to my ESRI case, so hopefully that'll help anyway.

Don't know if there is any additional info in the log besides the sample entry you showed, but certainly won't hurt to send it to them.

Re: huge amount of .geodatabase files since ArcGIS Pro 2.7, not clearing

TheodoreF — Wed, 20 Jan 2021 17:33:52 GMT

I'll keep you posted on the case results. In the meantime, I need to decide whether it's safe to manually delete the 200GB of .geodatabase files or not... hmm