Verifying backups

Documentation
1

Build Civitas has an automated backup every night to an Amazon S3 bucket. I download a copy of the backup files to my local machine daily using cron. These files are also backed up on my private Dropbox directory.

Discourse settings

Under Backups

  • enable backups => enabled (by default)
  • backup location => S3 (default is “Local Storage” meaning on the server itself)
  • automatic backups enabled=> enabled (by default)
  • backup frequency => 1 (every 7 days is the default)
  • s3 backup bucket => =bucket=
  • backup with uploads => enabled (by default)
  • include thumbnails in backups => enable (disabled by default)

Under Files:

  • enable s3 uploads => disabled (For now I don’t want to use S3 for files other than backups.)
  • s3 access key id => access key from an AWS user who has access (This is annoying and complicated to set up.)
  • s3 secret access key => secret key copied when the access key was created
  • s3 region => the region where your S3 bucket is hosted

Once everything is set, go ahead and test the backup using the button on https://beta.buildcivitas.com/admin/backups. There should be a new file in the S3 bucket under default with a name like build-civitas-beta-2024-01-09-223532-v20231222030024.tar.gz. If so, S3 backups have been set up correctly.

Daily cron

On my Mac, I’ve set up a cronjob[1] that looks like this:

$ crontab -l
@daily $HOME/bin/cron.sh
#*	*	*	*	*	$HOME/bin/cron.sh

The second line is commented out so that I can remember what the crontab looks like if I didn’t use the handy @daily shortcut. (See man -S 5 crontab for details.) My cron.sh looks like this:

#!/usr/bin/env zsh

export PATH=/usr/local/bin:/usr/bin:/bin

rsync -az --delete \
      root@meta.jlericson.com:/var/discourse/shared/standalone/backups/default/ \
      $HOME/Dropbox/meta_backups

aws s3 sync s3://=bucket= ~/Dropbox/civitas_backups --delete

I’m still using Local Storage for https://meta.jlericson.com, but I’ll probably be changing it to another S3 bucket soon. The rsync command is the way to go for that.

Be sure to test cron.sh by running it manually. You might need to run aws configure if you haven’t got the credentials set up already. If cron.sh works without error, verify the backup has been placed in the target directory.[2]

Set up a staging server

To verify this all works as expected, we need to set up a staging server that matches production as closely as possible:

Finally restore a recent backup on the test server to verify everything works. This isn’t something you need to do very often, but it is useful to test out new features in a safe environment so you don’t risk breaking production. When you start testing something new, go ahead and restore the backup first.

  1. Technically MacOS uses launchd. ↩︎

  2. ~/Dropbox/civitas_backups in my case. ↩︎