Skip to main content

Centralized backup system with restic / backrest and sshfs

Julian Meisel
Author
Julian Meisel
📋 open-source lover | 👨‍💻 software architect | 🤖 Android fanboy | 🏃‍♂️ runner | 🚗⚡ BEV enthusiast
Table of Contents

In my self-hosted setup there are several containers, whose data need to be backed up to an external storage box. Some of them like Immich have huge datasets (> 1 TB). In this post I’ll show you how to set up a centralized backup system using restic with backrest as the UI, and how to use sshfs to mount data from other containers into the backup container.

The Setup
#

My setup consists of several LXC containers running on Proxmox, each with their own data that needs to be backed up:

  • Immich (photo backup, > 1 TB)
  • Paperless-ngx (document management)
  • Home Assistant (smart home config)
  • various other services (Postgres databases, etc.)

Instead of running a backup client in each container, I decided to use a single backup container running backrest and restic. This has several advantages:

  1. Single point of configuration – one place to manage all backup jobs
  2. Centralized repository – all backups in one restic repository
  3. Resource efficiency – only one instance needs access to the backup destination
  4. Simpler retention management – unified backup policies

The Problem: Accessing Data from Other Containers
#

The challenge is that each LXC container has its own isolated filesystem. The backup container cannot directly access the data from other containers. There are several solutions:

Option 1: Mount Proxmox Volumes Directly
#

Mount the same Proxmox mount point into both the application container and the backup container. This works but requires careful volume management.

Option 2: Use rclone/restic with remote storage
#

Configure restic to access data through various protocols. Works but adds complexity.

Option 3: SSHFS Mounts
#

Mount the filesystem of other containers into the backup container using sshfs. This is what I’ll describe here.

Solution: SSHFS into Other Containers
#

sshfs allows mounting a remote filesystem over SSH. Since all my containers run on the same Proxmox host, I can use sshfs to mount the root filesystem (or specific data directories) of other containers into the backup container.

Prerequisites
#

  1. SSH server running in each container you want to mount
  2. SSH key-based authentication set up between the backup container and target containers
  3. sshfs package installed in the backup container

Installation
#

# On the backup container (Debian/Ubuntu-based)
apt install sshfs

# On the target containers (if not already installed)
apt install openssh-server

Configuration
#

  1. Generate SSH key in the backup container:
ssh-keygen -t ed25519 -f /root/.ssh/id_ed25519 -N ""
  1. Deploy public key to each target container:
ssh-copy-id -i /root/.ssh/id_ed25519.pub root@<target-container-ip>
  1. Configure SSH to use the specific key for each host (optional but recommended):
# In /root/.ssh/config
Host immich
    HostName 192.168.1.100
    User root
    IdentityFile /root/.ssh/id_ed25519

Host paperless
    HostName 192.168.1.101
    User root
    IdentityFile /root/.ssh/id_ed25519
  1. Test connectivity:
ssh immich "ls /mnt"

Mounting Data Directories via fstab
#

Add the following entries to /etc/fstab to mount data directories from target containers into the backup container:

# /etc/fstab
immich:/var/lib/immich   /mnt/immich   fuse.sshfs   defaults,_netdev,IdentityFile=/root/.ssh/id_ed25519   0 0
paperless:/var/lib/paperless   /mnt/paperless   fuse.sshfs   defaults,_netdev,IdentityFile=/root/.ssh/id_ed25519   0 0
homeassistant:/opt/homeassistant   /mnt/homeassistant   fuse.sshfs   defaults,_netdev,IdentityFile=/root/.ssh/id_ed25519   0 0

_netdev tells systemd to only mount after the network is up, so boot does not fail when a remote container is temporarily unreachable.

Then run mount -a to mount all entries.

The Critical Setting: ignore_inode
#

When using restic with an sshfs mount, pass restic’s --ignore-inode flag (in backrest, add it to the backup plan’s restic flags). Without it, restic will re-read unchanged files on every run and backups become unnecessarily slow and bandwidth-heavy.

Why This is Crucial
#

On Unix, restic decides whether a file changed by comparing mtime + ctime + size + inode against the parent snapshot. SSHFS exposes remote files through a FUSE layer whose inode numbers are not stable across mounts — each mount session assigns its own inodes, which also differ from the original filesystem. When an inode does not match the parent snapshot, restic assumes the file changed and scans its entire content again, even if nothing changed.

--ignore-inode is exactly the flag restic provides for FUSE-based filesystems: it tells restic that only the mtime has to match, so unchanged files are skipped and the backup stays incremental.

Two common mistakes to avoid:

  1. ignore_inode is not an sshfs mount option. sshfs has no such flag — putting it in an fstab entry makes the mount fail with an unknown-option error.
  2. Do not rely on inode-based hard link detection over sshfs. sshfs always presents hard links as separate files with individual inodes, regardless of mount options. restic still deduplicates identical content, so this is not a problem here.

Solution
#

Add --ignore-inode to the backup command (or to the plan flags in backrest):

restic backup /mnt/immich --ignore-inode

Setting Up Backrest
#

backrest is a web UI for restic that makes backup management much easier.

Installation
#

# Download and install
curl -L https://github.com/garethgeorge/backrest/releases/download/v1.14.1/backrest_Linux_x86_64.tar.gz -o backrest.tar.gz
tar -xzf backrest.tar.gz
mv backrest /usr/local/bin/
rm backrest.tar.gz

# Create data directory
mkdir -p /var/lib/backrest

Configuration
#

Backrest is configured through its web UI — there is no hand-written TOML config file (the configuration is stored as JSON, by default in ~/.config/backrest/config.json, overridable via $BACKREST_CONFIG). The setup in the UI:

  1. Add a repository – use a local repository type pointing at a directory on the backup container, e.g. /var/lib/backrest/repo. Since the data from the other containers arrives as local sshfs mounts, the repository itself is a plain local directory. If your backup destination is an external storage box, mount it into the backup container as well (NFS, CIFS, or a second sshfs mount) and point the local repo path at that mount — restic then reads and writes through a local path but all data lands on the external box.
  2. Add a backup plan per service – each plan maps to one container: the paths to back up (e.g. /mnt/immich), the schedule (a cron expression such as 0 2 * * * for “daily at 2 AM”), and the retention policy.
  3. Add restic flags – put --ignore-inode into the plan’s restic flags so sshfs mounts are scanned incrementally.

Initialize Repository
#

You do not need to run restic init manually: Backrest initializes the repository for you when you add it in the web UI, using the password you provide there. If you do initialize a repository with the restic CLI, use the exact same repository path (e.g. restic -r /var/lib/backrest/repo init) and the same password that Backrest is configured with, otherwise the UI cannot open the repository.

Run Backrest
#

# Start backrest
backrest serve &

Access the web UI at http://localhost:9898 (or configure reverse proxy with mTLS as described in my previous post).

Performance Optimization
#

SSHFS Performance Tuning
#

  1. Use compression (useful for slow links): Add compression to the fstab options:
immich:/var/lib/immich   /mnt/immich   fuse.sshfs   defaults,_netdev,compression,IdentityFile=/root/.ssh/id_ed25519   0 0
  1. Use SSH connection sharing (reduces connection overhead):
# In /etc/ssh/ssh_config (on backup container)
ControlMaster auto
ControlPath /tmp/ssh_mux_%h_%p_%r
ControlPersist 4h

Benchmarking Results
#

Before optimization (baseline with local storage):

  • Immich (~1TB): ~4 hours backup time

After sshfs + –ignore-inode:

  • Immich (~1TB): ~6 hours (due to network overhead)

The trade-off is acceptable because:

  1. Backups run nightly (off-peak)
  2. The backup infrastructure is now centralized and manageable
  3. All services benefit from single backup policy

Additional Restic Optimizations
#

# Compression is set once at repository creation (repository version 2
# enables compression with level "auto" by default) — it is not a flag
# on the backup command.

# Skip files you don't need (add these to the plan flags in backrest)
restic backup /mnt/immich --exclude /mnt/immich/thumbnails --exclude "*.tmp"

Monitoring and Alerts
#

Backrest surfaces the status of every operation in its web UI and can notify you when something fails:

  1. Failure alerts – add a hook on CONDITION_SNAPSHOT_ERROR (or CONDITION_ANY_ERROR) and connect it to a notification service: Healthchecks, Gotify, Discord, Slack, Shoutrrr, or a custom command.
  2. Uptime watch – point a reachability probe at http://localhost:9898 in your monitoring; if backrest is down, no backups run.
  3. Repository health – enable the scheduled check operation in the repository settings so restic verifies integrity on a regular basis.

Wrap Up
#

Using a single backrest instance with sshfs mounts is an elegant solution for centralizing backups of multiple containers. Key takeaways:

  1. Single backup instance – simplifies management significantly
  2. SSHFS mounts – works well for container-to-container data access
  3. restic’s –ignore-inode flag is essential – set it per backup job, otherwise unchanged files are re-read on every run
  4. Performance is acceptable for nightly backups, especially with large datasets

This setup has been running reliably for several months now, and I can manage all backup jobs from a single web interface.