We wanted to synchronize services from one server (Primary) to another (Secondary). This would allow us to use Apache Web server to load balance requests between the two servers and provide us fault tolerance for the 200+ services.
We tried using an AGS Cluster; however, we had numerous problems and overall performance issues. So we decided to use this silo'd approach. Initially we tried to manage services by deploying then and updating them on each server. With multiple publishers keeping the services up to date on both servers became a management nightmare.
We are now trying a new approach. We are using Linux rsync command to copy changes from Primary to Secondary; then using a Python script (attached) to create, update, delete services. The script includes a comment with instructions for setting up rsync. We then created a Bash script than runs sync then executes the Python script (You can use the Python that comes with Linux) then added the bash script to Cron to run every few minutes.
Writing this script helped me learn more about the ArcGIS Admin API. ArcGIS Server Administrator API
The script has only been tested with Map, Image, and GP Services; however, with a tweak it might work for other service types.