topic Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP% in ArcGIS Pro Questions

Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

JoshuaBixby — Mon, 18 Dec 2017 19:50:57 GMT

I recently re-imaged a PC to Windows 10 x64 Enterprise Edition. After re-imaging, I installed ArcGIS Pro 2.0.1 and no other ArcGIS Desktop components (ArcMap, ArcCatalog, etc....). Sometime after a week or so of installing Pro, I noticed thousands and thousands of *_gdal.sqlite files in my %TEMP% folder.

I cleaned up my temp files and just figured it was tied to the install. Within 48 hours, I had ~2,500 files back, and I never even launched ArcGIS Pro in that 48 hours! I tracked the files down to the ArcGISIndexingServer.exe process. I know there have been some memory-usage issues with that process, FAQ: Why does the ArcGISIndexingServer.exe use a high amount of computer memory? , but I haven't seen others talking about this issues.

The fact that this is a newly re-imaged machine with a clean install of ArcGIS Pro is what has me scratching my head. My machine is about as default as it ever will be, so I want to understand why either the process needs to create so many temp files or why it doesn't clean up after itself.

If others check their %TEMP% directories, how many *_gdal.sqlite files are there? If there are some, how many have been generated in the past day or so?

Thanks, Joshua

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

LR — Tue, 19 Dec 2017 11:30:36 GMT

I have a few (~4.8k) with ~80 generated today. It's generating a bunch hourly, with some gaps. Pro was open on the 14th last.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

JoshuaBixby — Tue, 19 Dec 2017 15:27:15 GMT

Interesting. It would be good to hear from a few more folks. My guess is that there is a certain type of geospatial content being indexed that is driving the explosion of temp files, I am just not sure what exactly. I am also working with Esri Support to see if system administrators can disabled this logging because the small files start to add up when you have hundreds of users running Pro in a virtualized environment.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

ThomasColson — Fri, 29 Dec 2017 16:56:15 GMT

Wow. Yes. Thousands of them. Agree, excessive generation of temp files is not sustainable with a large number of users.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

JoshuaBixby — Sat, 30 Dec 2017 15:51:15 GMT

Progress has been slow with Esri Support thus far. One comment I got was, "you can disable indexing." Where to even start on replying to that one. It appears not all of our users are seeing this behavior, or seeing it to the extent that I am, so I do think it is driven by a specific type of content. I plan on revisiting this and doing some process monitoring after New Year.

UPDATE: I did some quick process monitoring, and the issue appears tied to indexing of XLS and XLSX files. A couple of the directories being indexed just happen to have Excel files in them, and when the indexing service gets to those folders, the temp file explosion begins. I will work on isolating the issue next week to verify the connection to Excel files.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

ThomasColson — Sat, 30 Dec 2017 20:52:10 GMT

"One comment I got was, "you can disable indexing."

Imagine if you just bought a new car:

"The tires are out of round and it vibrates violently at highway speeds"

Dealer: "Don't drive it on the highway"

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

ThomasColson — Sat, 30 Dec 2017 21:09:01 GMT

"investigating" the SQL lite files does not reveal anything interesting, what they do, or why such an excessive number of them is generated.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

KoryKramer — Wed, 03 Jan 2018 23:43:15 GMT

Thomas and bixb0012‌ Support is working directly with development to investigate this issue. Since Joshua has the case open, remaining communication will happen through that case. You could circle back to close this thread once the case is closed.

Thank you.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

JoshuaBixby — Thu, 04 Jan 2018 00:42:28 GMT

I was planning on waiting until the Support case is closed to update this thread and close it out. I am interested to see what Development finds. At a minimum, if the files absolutely need to be generated, that the indexing service clears them out after a run is complete.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

JoshuaBixby — Fri, 05 Jan 2018 23:07:54 GMT

For those who are interested, and are being impacted by this issue, a bug has been logged with Esri Support:

#BUG-000110608: ArcGIS Pro's ArcGISIndexingServer.exe generates thousands of *_gdal.sqlite files in %TEMP% directory

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

JohnSobetzer — Fri, 19 Jan 2018 18:20:34 GMT

I just noticed hundreds of these files being generated after installing 2.1. They weren't there with 2.0.1. They keep coming even though I have turned off indexing for Pro. If I end the ArcGIS Indexing server task they stop but if I start Pro and then close it, even without doing anything the task will start up again. I renamed the indexing exe file in the bin folder and that finally seems to have stopped it for good. It does seem to involve Excel files.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

KoryKramer — Fri, 19 Jan 2018 18:23:11 GMT

The bug cited by Joshua above has not been resolved yet. Unfortunately we'll still see the behavior in 2.1.

The responsible team is aware of the issue (through the bug) and they are working on a fix.

Thank you.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

KoryKramer — Fri, 19 Jan 2018 19:41:54 GMT

John Sobetzer‌ Joshua Bixby‌ It looks like the team has this fixed in the Pro 2.2 code. We might (no guarantees) be able to get it in a patch before then so keep an eye open for Pro updates.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

JoshuaBixby — Fri, 19 Jan 2018 20:47:14 GMT

Thanks for the upates.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

KoryKramer — Fri, 19 Jan 2018 20:48:49 GMT

You're welcome.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

deleted-user-sYHsW3p_gN5o — Mon, 05 Feb 2018 16:15:37 GMT

Kory, thanks for looking into this. Do we have any idea as to when 2.2 (or a patch for 2.1) will be releasing? I also have been having this issue.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

KoryKramer — Mon, 05 Feb 2018 16:24:24 GMT

June is a reasonable estimate for the 2.2 release. I believe that there will be a second patch for Pro 2.1 in March but I am not able to say whether this fix will go into that patch.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

deleted-user-sYHsW3p_gN5o — Mon, 05 Feb 2018 16:38:13 GMT

Hmm, okay. I appreciate the response, and I know the dev teams must be working hard and have a lot to do. But is there a workaround to prevent these files being created? Because my computer spent hours deleting these files that were created in only a month or so.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

KoryKramer — Mon, 05 Feb 2018 16:42:36 GMT

Sorry, Andrew. There is not a listed workaround. It is just something that needs to be fixed. I guess since it is something that is now known, you could go to that folder and delete the files every Friday afternoon (or something like that) before there is too much buildup... Again, I apologize for the inconvenience, hopefully we can get the fix in a patch before the 2.2 release.

Re: Anyone Else Seeing an Explosion of *_gdal.sqlite Files in %TEMP%

ThomasColson — Mon, 05 Feb 2018 16:54:33 GMT

If you can get IT to let you use a domain service account, write a batch file to delete them, and run it nightly as a scheduled task. If you can't get a service account, set the task to start when you login.