Pgdata storage management

1318
12
Jump to solution
01-29-2024 02:43 AM
Arnaud_Leidgens
New Contributor III

Hello everyone,

We've been using  ArcGIS entreprise for a few years now (classic deployement, single server). Those last few months, we consumed 20 additional Gb per month, which is not really sustainable on the long run. This increase coincides more or less with the 11.1 update, but I'm not sure if this is related.

When I monitor the D drive, where everything is installed, I see that most of that storage consumption (140go) is linked to the folder ArcGISDataStore>pgdata>base>16401. All files are capped at 1go and I don't understand how they relate to my Portal's items. I understand that pgdata is useful for postgresql functionning.

We've around 2000 items, including 500 maps configured for offline mode (sync enabled), around 25 dashboards and 200 users (mostly viewers). We don't use hosted rasters. We also regularly truncate and populate several hosted feature layers with data from Microsoft SQL server (daily FME scripts). NB: Those specific items appear as having 0b size on the Portal, despite being sometimes composed of dozens of thousands of polygons. 

That storage consumption may be justified, but I need a way to manage it on our server. When I generate a report of all Portal's items, sorted by size, the total is only around 1Gb, which is far from the 140go of pgdata folder. 

NB: ESRI's support has not been able to help me on this topic since July 2023. I'm trying to find a way to better manage my storage and to identify which items are consumming the most. Is it coming from the features layers themsleves, dashboards, offline sync, ghost backup, etc. ?

Thank you in advance

Arnaud

0 Kudos
12 Replies
Scott_Tansley
MVP Regular Contributor

if it relates to the 11.1 upgrade then did you run the diskcleanup tool after the upgrade?

https://enterprise.arcgis.com/en/data-store/latest/install/windows/data-store-utility-reference.htm#... 

The upgrader will maintain the older database and copy its contents into a new 11.1 database, so it can double disk consumption.  The diskcleanup will remove the old files and potentially free up space.  

I've had several clients start to capture lots of attachments, which are stored as blob fields and rapidly fill the disk, so consider usage of attachments if disk is an issue.

Scott Tansley
https://www.linkedin.com/in/scotttansley/
Arnaud_Leidgens
New Contributor III

Thank you Scott. Yes, we run the diskcleanup tool after the upgrade, and I run it today to confirm. I may be wrong but I guess we would then see another folder  with significant size in pgdata>base, for the older DB, which is not the case here.

About attachements, we indeed capture a few of them but this is very limited. We are careful about the size of the pictures (max 1 mb) and I track the size of those few layers by saving them locally in a gdb.

Scott_Tansley
MVP Regular Contributor

Excellent.  I know a lot of people miss those things so glad you’ve got it covered. 

I would love to see a vacuum tool.  One client moved their image capture to AGOL and deleted the feature layers in their Enterprise.  The relational data store never shrank.  A vacuum would be really useful. 

Scott Tansley
https://www.linkedin.com/in/scotttansley/
0 Kudos