Backup and point-in-time restore

About 10 min

The master's scheduled task master.backup backs up two kinds of data: the MariaDB business database (hourly by default, files named gocdn-db-<UTC time>.sql.gz) and key ClickHouse tables (once a day, billing_hourly and usage_site_5m, files named gocdn-ch-*.jsonl.gz). With an encryption public key set, files end in .age.

Recent
One per day
Cleanup
NowOlderPast the keep days
NowRecent

The most recent 24 backups are all kept (hourly at a 1-hour interval)

    OlderOne per day

    Older backups keep one per day, 7 days by default

      Past the keep daysCleanup

      Deleted

        Business databaseHourly by default
        Key ClickHouse tablesOnce a day
        Offsite copy + binlogRecovery point objective of about one minute
        Recent backups are all kept, older ones keep one per day; when disk is short, the oldest go first

        Settings in the panel

        Open System Admin › System Settings, the Data Cleanup tab, "Database Backup" card:

        SettingNotes
        Scheduled backupMaster switch
        Backup intervalPresets from 10 minutes to 1 day, or custom (such as 90m); range 10 minutes–7 days; a failed run retries after 10 minutes
        Recent backups keptThe most recent N backups are all kept; default 24 (hourly at a 1-hour interval)
        Keep daysOlder backups keep one per day; default 7 days
        Minimum free diskOld backups are deleted first when free space on the backup disk falls below it; default 15%; 0 disables the check
        Backup size capTotal size cap of the backup directory; 0 means unlimited
        Backup directoryDefault /var/lib/gocdn/backup/db
        Advanced: offsite copy directory, encryption public keySee below

        Run status and errors are in System Admin › Audit Logs › Backups.

        Disk protection

        When free space is below the limit or the total exceeds the cap, the oldest backups are deleted first, but the newest of each kind is always kept. If that is still not enough, the run is skipped with an alert and retried after 10 minutes, so backups never fill the disk and take down the master.

        Encryption

        Enter an age public key (starting with age1) as the encryption public key and backups are encrypted to .age. New masters generate the private key /etc/gocdn/backup-age.key automatically; copy it to another machine. On older installs, run as root on the master:

        gocdn-master backup-keygen -out /etc/gocdn/backup-age.key

        Without the private key, backups cannot be decrypted. Unencrypted backups, especially those pushed offsite, trigger a "backups not encrypted" banner on the Backups tab.

        Offsite copy and point-in-time restore

        With an offsite copy directory set (usually a mounted NAS or another disk), every backup is copied there and the database incremental log (binlog) is pushed every minute, for a recovery point objective of about one minute.

        • The remote disk must be mounted by the operator, and before mounting, the mount point must not be writable by the gocdn user.
        • MariaDB needs binlog enabled (log_bin), and the business-database account needs the BINLOG MONITOR and REPLICATION SLAVE privileges. Otherwise only full backups are pushed and the Backups tab shows a "binlog incremental backup is not in effect" banner.

        Restore the business database

        Stop the master service first, then run on the master.

        Restore to the moment of the backup:

        gocdn-master db-restore < gocdn-db-<time>.sql.gz

        Restore to a point in time (needs the binlog from the offsite copy):

        gocdn-master db-restore -until <RFC3339 time> -binlog-dir <offsite dir>/binlog < gocdn-db-<time>.sql.gz

        Optional flags: -age-identity (private key for encrypted backups), -overwrite (overwrite existing data) and -client-defaults. The last line DB_RESTORE_DONE means success.

        Restore the ClickHouse key tables

        gocdn-master ch-restore < gocdn-ch-<time>.jsonl.gz

        The last line CH_RESTORE_DONE means success.

        Signing keys and the credential key

        • Signing keys have their own encrypted backup at /var/lib/gocdn/backup/keys.tar.age, with the passphrase file /etc/gocdn/keys-backup.pass. Restore with gocdn-keys restore -passphrase-stdin -in keys.tar.age < passphrase-file.
        • The credential key /etc/gocdn/dns-cred.key is in no backup at all; keep a copy on another machine.
        Caution If the backup directory, private key and key files all sit on the master itself, a failed master disk loses everything. Configure an offsite copy and store backup-age.key, keys-backup.pass and dns-cred.key off the machine.

        For the full procedure of moving the master to a new machine, see Uninstall and migrate.

        Get started

        Try the live demo

        Open the live demo, or contact us for a quote and the install command