I have hosted most of my internet services from home on a motley collection of discarded or refurbished servers since 1999. Since I was soon supporting hundreds of users of email and Sacdoc (my medical records system), I needed some sort of automated, off-site backup. I did have a system of manually making backups and swapping the drives out for another set that I kept off-site, but I wanted something completely automated and geographically distant.
That’s about when Amazon started offering their Amazon Web Services Simple Storage System (AWS S3). It was shockingly cheap, and for the first couple of years just kept getting cheaper.
I’m not a big fan of “the cloud.” It’s just someone else’s computer, and you should expect that any cloud service can disappear or otherwise become inaccessible at any time and without notice. I also consider anything I send to the cloud to be published. Cloud providers can pinky swear all they want that they’ll keep my data safe, but three people can keep a secret only if two of them are dead.
That said, AWS S3 was a good fit for offsite backups. They’re backups, so if they vanish I can just create new ones elsewhere. (Though it chills to consider if the cloud service vanishes in that tiny gap between realizing you need those backup and retrieving same.) The backups are encrypted, so even if they’re made public, no harm is done. So using duplicity to back up to the AWS S3 cloud worked well for me for many years.
The price, however, did not continue to go down. In fact, for the past decade or so it seems to be steadily climbing, and now that I’m retired the cost is actually significant in spite of my greatly reduced quantity of data. Also, AWS S3 was recently found to be charging people for unsuccessful requests for your data from unauthenticated users. In other words, any internet rando who wanted to mess with an AWS S3 user need only set up a bot to make millions of bogus requests for your data, and your S3 bill can go through the roof. Amazon has since mitigated that error, but no doubt there are others. It’s time to migrate.
It turns out you can get quite capable virtual private servers (VPS) for somewhat less then $5/month from several sources. I keep a couple active for various things: a pi-hole enabled VPN, extra static IP addresses (much cheaper than my ISP), and now duplicity backups.
Here’s how I do it on my linux-based servers:
to back up server.example.com to backup.example.com:
on backup.example.com
create the user
useradd -d /home/serverbackup -s /bin/bash serverbackup
passwd serverbackup
Use a strong password and store it in your password manager.
create the home directory
mkdir /home/serverbackup
chown serverbackup.serverbackup /home/serverbackup
temporarily enable password logins on backup.example.com
In /etc/ssh/sshd_config comment out #PasswordAuthentication no, save the file, then sudo systemctl reload sshd. Keep your shell window open.
on server.example.com
set up ssh login
Copy the server public key with
ssh-copy-id serverbackup@backup.example.com
Test passwordless login: ssh serverbackup@backup.example.com then exit
On backup.example.com, re-disable passwordless login.
Re-test passwordless login: ssh serverbackup@backup.example.com then exit
create gpg default key
Create and securely store a passphrase for the gpg key pair.
gpg --full-generate-key and pretty much take the defaults. Record the key fingerprint and ID in your password manager.
set up duplicity script
touch s3bu
chmod 700 s3bu
se s3bu
Populate it with
#!/bin/bash
export PASSPHRASE=<gpg passphrase from above>
SERVER_NAME=$HOSTNAME
echo $SERVER_NAME backup...
echo
GPG_KEY=<gpg key fingerprint>
SOURCE=/
DEST=scp://${SERVER_NAME}backup@backup.example.com/$SERVER_NAME
duplicity \
--encrypt-key=${GPG_KEY} \
--sign-key=${GPG_KEY} \
--exclude=/dev \
--exclude=/lost+found \
--exclude=/media \
--exclude=/mnt \
--exclude=/proc \
--exclude=/sys \
--exclude=/tmp \
--exclude=/swap.img \
${SOURCE} ${DEST}
export PASSPHRASE=
Test with sudo ./s3bu.
Schedule periodic execution with your favorite task manager.
—2p
addendum 2024-05-18
In addition to being cheaper, it’s trivial to move your backup system among competing VPS vendors. With Amazon, you’re locked in to their protocol.