Is it safe to delete files in 'walarchive' from ArcGIS DataStore backups?

[WhereMatters]

ArcGIS DataStore schedules regular backups of its database every 7 days by default and retains them for 31 days.

Due to this, the disk space is getting filled up over a period of time.

I am also seeing that most of the disk space is occupied at "C:\arcgisdatastore\backup\walarchive".

I am thinking these are postgres db archive files as the name suggests.

Is it ok to clean up this directory completely from time to time?

Will this affect the ability to restore backups?

[ChrisSmith7]

Anand,

Have a look at the following:

Manage data store backups—Installation Guides (10.3 and 10.3.1) | ArcGIS for Server

Here, you can find information on adjusting retention time, which can reduce disk space utilization:

Change how long backup files are retained

The backup directory retains backup files for 31 days by default. That means if you keep the default backup frequency (every 7 days) and retention schedules (31 days), the backup directory will contain five full backups and 31 days of incremental backup files. The size of these files depends on the amount and size of your data. The machine that stores your backups must have enough disk space for all these files. If you increase the backup frequency, consider decreasing the retention period for the backup files. In the previous section, the backup frequency was increased to every 14 days. To prevent your backup directory from becoming too large, decrease the backup file retention period.

The syntax to run the updatebackupretaindays utility is as follows:

updatebackupretaindays <number of days>

In the following example, backup file retention time is changed to 27 days:

updatebackupretaindays 27

On a side note for PostgreSQL installations, I've heard of some people using PostgreSQL: Documentation: 9.5: pg_archivecleanup  for cleanup... I'm not sure if it's compatible with ArcGIS. Here's a discussion on better understanding what the tool will do:

postgresql - I need help understanding Postgres's archive cleanup functionality - Database Administr...

[JosephFrechette]

Chris,

Does this change the retain days for the incremental backups that are in the walarchive folder too? Or does making these changes only affect the full dbbackup?

[ChrisSmith7]

I'm not sure, but it seems like it would affect incremental backups. According to the documentation:

By default, the data store creates a full backup of feature layers once a week and incremental backups either when the log files are full or every 5 minutes, whichever comes first. The database controls incremental backup creation, but you can change how often the data store creates a full backup by running the updatebackupschedule utility.

...if you use default backup settings, backups will contain five full backups and 31 days of incremental backup files

The backup directory retains backup files for 31 days by default. That means if you keep the default backup frequency (every 7 days) and retention schedules (31 days), the backup directory will contain five full backups and 31 days of incremental backup files.

If you ran something like updatebackupretaindays 27 it seems that it would then only keep 27 days of incremental back-ups and 4 full backups (assuming you kept the full backup frequency of 7 days).

[JosephFrechette]

I tested this out. These are the settings that I applied:

updatebackupschedule --starttime <local server time> --frequency 1

updatebackupretaindays 1

With these settings the incremental backup kept going until I deleted the files going back over a week. I don't think that the settings are being applied to the incremental backups. That walarchive folder gets large very fast regardless of the settings.

To conserve space I have been deleting the .gz files about once a week. This has not caused any issues.

[ChrisSmith7]

It sounds like it should be applying the settings to the incremental back-ups - is this a bug, perhaps?

[TonyGegner]

Yea, the walarchive folder is an issue. Mine is making a new 7MB file every 5min which is a lot of space over time.

I didn't have a problem to delete files from it, deleted 36000 files yesterday.

Must be some way to change the interval on this as well.

[SohamShah3]

How do you manage the backups for portal? Walarchive folder in portal is also storing too many backups for me.

[JonathanQuinn]

You're using 10.5, correct?  You can run a full backup for Portal which will clear out the walarchive files.  It's a best practice to be backing up your Portal anyway.  This can be done using the DR tool, (recommended for full ArcGIS Enterprise deployments) or through Portaladmin.

[KirkChesla]

For our brand new Portal installation on RHEL7 the walarchives have completely filled up our /opt directory in just a few weeks of idle use.  And now Portal will not start up even after clear some space.  Then the webgisdr shell script will not run at all from Putty.

Its starting to look like another re-install of Portal.