Select to view content in your preferred language

Is it safe to delete files in 'walarchive' from ArcGIS DataStore backups?

12749
14
07-27-2015 05:47 AM
WhereMatters
Frequent Contributor

ArcGIS DataStore schedules regular backups of its database every 7 days by default and retains them for 31 days.

Due to this, the disk space is getting filled up over a period of time.

I am also seeing that most of the disk space is occupied at "C:\arcgisdatastore\backup\walarchive".

I am thinking these are postgres db archive files as the name suggests.

Is it ok to clean up this directory completely from time to time?

Will this affect the ability to restore backups?

14 Replies
ChrisSmith7
Honored Contributor

Anand,

Have a look at the following:

Manage data store backups—Installation Guides (10.3 and 10.3.1) | ArcGIS for Server

Here, you can find information on adjusting retention time, which can reduce disk space utilization:

Change how long backup files are retained

The backup directory retains backup files for 31 days by default. That means if you keep the default backup frequency (every 7 days) and retention schedules (31 days), the backup directory will contain five full backups and 31 days of incremental backup files. The size of these files depends on the amount and size of your data. The machine that stores your backups must have enough disk space for all these files. If you increase the backup frequency, consider decreasing the retention period for the backup files. In the previous section, the backup frequency was increased to every 14 days. To prevent your backup directory from becoming too large, decrease the backup file retention period.

The syntax to run the updatebackupretaindays utility is as follows:

updatebackupretaindays <number of days>

In the following example, backup file retention time is changed to 27 days:

updatebackupretaindays 27

On a side note for PostgreSQL installations, I've heard of some people using PostgreSQL: Documentation: 9.5: pg_archivecleanup  for cleanup... I'm not sure if it's compatible with ArcGIS. Here's a discussion on better understanding what the tool will do:

postgresql - I need help understanding Postgres's archive cleanup functionality - Database Administr...

0 Kudos
JosephFrechette
Deactivated User

Chris,

Does this change the retain days for the incremental backups that are in the walarchive folder too? Or does making these changes only affect the full dbbackup?

0 Kudos
ChrisSmith7
Honored Contributor

I'm not sure, but it seems like it would affect incremental backups. According to the documentation:

By default, the data store creates a full backup of feature layers once a week and incremental backups either when the log files are full or every 5 minutes, whichever comes first. The database controls incremental backup creation, but you can change how often the data store creates a full backup by running the updatebackupschedule utility.
...if you use default backup settings, backups will contain five full backups and 31 days of incremental backup files
The backup directory retains backup files for 31 days by default. That means if you keep the default backup frequency (every 7 days) and retention schedules (31 days), the backup directory will contain five full backups and 31 days of incremental backup files.

If you ran something like updatebackupretaindays 27 it seems that it would then only keep 27 days of incremental back-ups and 4 full backups (assuming you kept the full backup frequency of 7 days).

0 Kudos
JosephFrechette
Deactivated User

I tested this out. These are the settings that I applied:

updatebackupschedule --starttime <local server time> --frequency 1

updatebackupretaindays 1

With these settings the incremental backup kept going until I deleted the files going back over a week. I don't think that the settings are being applied to the incremental backups. That walarchive folder gets large very fast regardless of the settings.

To conserve space I have been deleting the .gz files about once a week. This has not caused any issues.

0 Kudos
ChrisSmith7
Honored Contributor

It sounds like it should be applying the settings to the incremental back-ups - is this a bug, perhaps?

0 Kudos
TonyGegner
Regular Contributor

Yea, the walarchive folder is an issue. Mine is making a new 7MB file every 5min which is a lot of space over time.

I didn't have a problem to delete files from it, deleted 36000 files yesterday.

Must be some way to change the interval on this as well.

0 Kudos
SohamShah3
New Contributor

How do you manage the backups for portal? Walarchive folder in portal is also storing too many backups for me.

0 Kudos
JonathanQuinn
Esri Notable Contributor

You're using 10.5, correct?  You can run a full backup for Portal which will clear out the walarchive files.  It's a best practice to be backing up your Portal anyway.  This can be done using the DR tool, (recommended for full ArcGIS Enterprise deployments) or through Portaladmin.

KirkChesla
Occasional Contributor

For our brand new Portal installation on RHEL7 the walarchives have completely filled up our /opt directory in just a few weeks of idle use.  And now Portal will not start up even after clear some space.  Then the webgisdr shell script will not run at all from Putty.

Its starting to look like another re-install of Portal.

0 Kudos