Server backups / Duplicity: verschil tussen versies
(apt install duplicity) |
k (MarkKlaauw heeft de pagina Server backups / Duplicty hernoemd naar Server backups / Duplicity zonder een doorverwijzing achter te laten) |
||
(2 tussenliggende versies door dezelfde gebruiker niet weergegeven) | |||
Regel 1: | Regel 1: | ||
+ | [[Category:Servers network & websites]] | ||
= Backup = | = Backup = | ||
Huidige versie van 27 jul 2024 om 13:35
Inhoud
Backup
There is a regular backup ran from systemd; /etc/duplicity. It starts with a full dump of all MySQL tables. It is encrypted against a public key; the private key of which is held by the Trustees of the foundation.
Initial installation
apt install duplicity
Crontab kicks off a script;
# monthly full, incrementals during the week. # 3 3 1 * * root test -x /etc/duplicity/run.sh && /etc/duplicity/run.sh full 3 3 2-31 * * root test -x /etc/duplicity/run.sh && /etc/duplicity/run.sh incremental
# Half year retention for full; 1 months for the incrementals # 1 1 * * 1 root test -x /etc/duplicity/run.sh && /etc/duplicity/run.sh remove-all-but-n-full 6 1 2 * * 1 root test -x /etc/duplicity/run.sh && /etc/duplicity/run.sh remove-all-inc-of-but-n-full 1
The script: run.sh:
#!/bin/sh set -e umask 077 HOST="xxx.backup.host" DIR=/etc/duplicity W=incremental if [ $# != 0 ]; then W=$1 shift fi T= if [ $W = full -o $W = incremental ];then T=/ mysqldump --all-databases --single-transaction --quick --lock-tables=false | gzip -9 > /var/lib/mysql-files/mysql-dump.gz fi # Verbose level 2 is errors and warnings; this way we skip # notices and quell all output if the backup is a success. # PASSPHRASE="YYYPASSWORD" \ LANG=en_US.UTF8 LC_CTYPE=C HOME=$DIR GNUPGHOME=$DIR \ PYTHONWARNINGS="ignore::DeprecationWarning" \ python3 -W ignore::DeprecationWarning /usr/bin/duplicity $W $* \ \ -v 2 \ --hidden-encrypt-key XXXXX \ --hidden-encrypt-key YYYYY \ --sign-key YYYYY \ --ssh-options="-i $DIR/backup.sftp -oUserKnownHostsFile=$DIR/knownhosts" \ --no-print-statistics \ \ --include /etc \ --include /usr/share/mediawiki \ --include /usr/share/wordpress \ --include /usr/local/makerspaceleiden-crm \ --exclude /var/lib/lxcfs \ --include /var/lib \ --include /var/www \ --include /var/log \ --exclude /dev \ --exclude /sys \ --exclude /run \ --exclude /tmp \ --exclude /snap \ --exclude /var/tmp \ --exclude /proc \ --exclude /swapfile \ --exclude /etc/duplicity/.cache \ \ $T \ sftp://msl@$HOST/backups 2>&1 |tee /var/log/last-duplcity-backup.new | grep -v DeprecationWarning | grep -v algorithm=hashes.SHA1 mv /var/log/last-duplcity-backup.new /var/log/duplicity.log mv /var/log/duplicity.log.gz /var/log/duplicity.prevlog.gz || true gzip /var/log/duplicity.log || true exit $?
Importing a new public key is done as follows
cd /etc/duplicity cp XXXX/public-key-12345.gpg . gpg --homedir . --import public-key-12345.gpg . gpg --homedir . --edit-key XXXXXX trust 5 save
And test by running it manually
sudo /etc/duplicity/run.sh incremental
Safekeeping of private key
The code at https://github.com/dirkx/gpg-offline-batch-key- can be used to keep a backup off line. It yields a printout like https://github.com/dirkx/gpg-offline-batch-key-/blob/master/sample-output.pdf. This makes sure that there are no `on line' copies of that private key; just one off-line; on paper.
Note that the backup process itself is also privileged (i.e. it can access anything on the machine) and thus has access to any and all data on the machine.
Ransomware/targeted risk
This approach is not overly resistant against a targeted delete - as the sftp user can delete/modify files (as the retention is currently done from the 'source'). This is, to some extend, mitigated by zfs snapshots -- but not sufficiently at this time.
Restoring a file
To restore a single file - there is a script in
/etc/duplicity/restore.sh
that is a wrapper for duplicity restore. Typical use is
restore.sh restore --file-to-restore etc/foo/file.txt --time 2023-01-01
and it will put this in 'restored-file'.
Updating / renewing the encryption key
Check out ttps://github.com/dirkx/gpg-offline-batch-key- and make sure you can run the 'sh gen.sh -d' as described in its README. This is to ensure you have all the dependencies (such as qrencode, libquirc and so on). This should complete with a generated PDF.
Next check that your printing subsystem works and that by default it goes to the right printer -- by doing:
echo test | lpr
If not -edit gen.sh. Now run this script `for real' (ideally with a second person present; e.g. under Four Eye principles.
The result of this should be a file, e.g. ```public-key-45557.gpg``` that contains the public key. And a printout - that contains the private key.
Once the script completes - the ONLy copy of the private key is in that paper document. It is not present anymore in digital form. The only thing left behind digital is the public key. I.e. the file which you can use to _encrypt_; but which cannot _decrypt_.
Copy the public key file into /etc/duplicity. Check that you can import it there; and extract the key identifier:
export GNUPGHOME=`pwd` gpg --import public-key-45557.gpg
This should show something such as
gpg: key 534B2DDEB431D5B4: public key "key-45557" imported gpg: Total number processed: 1 gpg: imported: 1
Take note of the key id (534B2DDEB431D5B4) shown. Next make sure your GPG trusts it:
gpg --edit-key 534B2DDEB431D5B4 trust quit
Select option 5 (ultimate) and confirm.
Then edit 'run.sh and add the key (534B2DDEB431D5B4) in this example to the list. It should be in the hidden encryption key list -- i.e where it says XXXX in the above example.
Finally run the scrip to check all is well. This will take a few minutes.
sh run.sh sh run.sh incremental
History
2022/12 -- Changed to longer incremental runs; with only monthly full's.