Self-Hosted Storage Replication: DRBD vs ZFS Replication vs GlusterFS Geo-Rep

Storage replication is essential for disaster recovery, high availability, and data protection across geographically distributed systems. When running self-hosted infrastructure, you need reliable replication that works independently of cloud provider APIs. In this guide, we compare three powerful open-source storage replication solutions: DRBD, ZFS Replication, and GlusterFS Geo-Replication.

Overview

What Is DRBD?

DRBD (Distributed Replicated Block Device) is a Linux kernel module that mirrors block devices across a network in real-time. Think of it as “network RAID-1” — data written to a DRBD device is simultaneously written to the local disk and transmitted to a remote node.

Key Features

• Block-level replication: Works with any filesystem (ext4, XFS, btrfs)
• Synchronous mode: Zero data loss guarantee with protocol C
• Automatic failover: Integrates with Pacemaker/Corosync for HA clusters
• Split-brain detection: Automatic detection and resolution of network partitions
• Multi-master support: DRBD 9.x supports active-active configurations
• Thin provisioning: Support for LVM thin volumes

Installation

1
2

sudo apt install drbd-utils
sudo modprobe drbd

Configuration

global_common.conf:

1
2
3
4
5
6
7
8

global { usage-count yes; }
common {
    protocol C;
    net {
        cram-hmac-alg "sha256";
        shared-secret "MySecretKey";
    }
}

Resource file (/etc/drbd.d/r0.res):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

resource r0 {
    on node1 {
        device    /dev/drbd0;
        disk      /dev/sdb1;
        address   192.168.1.10:7788;
        meta-disk internal;
    }
    on node2 {
        device    /dev/drbd0;
        disk      /dev/sdb1;
        address   192.168.1.11:7788;
        meta-disk internal;
    }
}

Initialize and Start

1
2
3
4

sudo drbdadm create-md r0
sudo systemctl start drbd
sudo drbdadm primary r0 --force
sudo drbd-overview

What Is ZFS Replication?

ZFS native replication uses ZFS snapshots to incrementally replicate datasets between systems. It leverages ZFS copy-on-write architecture for efficient, consistent transfers.

Key Features

• Incremental snapshots: Only transfers changed blocks since last snapshot
• End-to-end encryption: Data is encrypted during transit via SSH
• Compression: Optional compression during transfer
• Consistency guarantees: Each replicated snapshot is a consistent point-in-time copy
• Cross-platform: Works on any platform running OpenZFS
• Automated tools: Sanoid, Syncoid, and zrepl automate replication schedules

Automated Replication with Syncoid

1

sudo syncoid --compress=zstd pool/data user@remote:backup/data

zrepl Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

jobs:
  - name: replicate_to_remote
    type: push
    connect:
      type: ssh+stdinserver
      host: remote-server
      user: zfs-repl
    filesystems:
      "pool/<": true
    snapshot:
      type: periodic
      interval: 15m
      prefix: zrepl_
    retention:
      type: grid
      grid: 1x1h(keep=all), 24x1h, 30x1d, 6x30d
    send:
      encrypted: true
      compressed: true

What Is GlusterFS Geo-Replication?

GlusterFS Geo-Replication provides asynchronous, master-slave replication between GlusterFS volumes across geographic locations. It uses a changelog-based approach to track and replicate file-level changes.

Key Features

• File-level replication: Replicates individual files and directories
• Asynchronous mode: No write latency impact on the primary volume
• Changelog-based tracking: Efficient delta detection without full scans
• Bandwidth limiting: Built-in rate limiting to control network usage
• Cross-cluster replication: Works between independent GlusterFS clusters

Configuration

1
2
3

sudo gluster volume geo-replication master-vol root@slave-server::slave-vol create push-pem force
sudo gluster volume geo-replication master-vol root@slave-server::slave-vol start
sudo gluster volume geo-replication master-vol root@slave-server::slave-vol status

Choosing the Right Replication Solution

Choose DRBD if: You need synchronous replication with zero data loss, are building an active-passive HA cluster, and need automatic failover with Pacemaker.

Choose ZFS Replication if: You already use ZFS for storage, want point-in-time recovery with snapshot history, and need efficient incremental transfers.

Choose GlusterFS Geo-Rep if: You are already running GlusterFS, need file-level replication, and want asynchronous replication across WAN links with bandwidth control.

Disaster Recovery Best Practices

1. Test Failover Regularly

Replication is useless if failover does not work. Schedule quarterly failover drills to verify your recovery procedures.

2. Monitor Replication Lag

1
2
3

sudo drbd-overview
ssh remote zfs list -t snapshot -o name,creation backup/data | tail -5
sudo gluster volume geo-replication master-vol slave-server::slave-vol status detail

3. Encrypt Replication Traffic

All three solutions support encryption. Never replicate data over untrusted networks without TLS/SSH encryption.

Why Self-Host Your Storage Replication?

Running your own storage replication infrastructure eliminates dependency on cloud provider replication services, which often come with egress fees, vendor lock-in, and opaque SLAs. With DRBD, ZFS, or GlusterFS, you control the replication schedule, encryption, retention, and failover behavior.

For data-intensive workloads like databases, virtual machine storage, and media archives, self-hosted replication provides predictable performance and cost. You avoid the surprise bills that come from cloud provider data transfer charges during large-scale replication or disaster recovery events.

If you are building a complete high-availability infrastructure, our Kubernetes backup orchestration guide covers application-level backup, and our distributed storage comparison covers shared storage architectures. For database-level replication, check our PostgreSQL backup guide for point-in-time recovery strategies.

Storage Replication Performance Comparison

Write Latency Impact

The replication mode significantly affects write latency. DRBD Protocol C synchronous adds one network round-trip per write. On a 1 Gbps LAN with 1ms latency, each write takes approximately 2ms longer. On a WAN with 50ms latency, each write takes 100ms longer. ZFS replication asynchronous has no impact on write latency since replication happens in the background using snapshots. GlusterFS Geo-Rep asynchronous also has no impact on write latency with changes tracked via changelog and replicated in the background.

Bandwidth Requirements

DRBD synchronous replication requires bandwidth equal to your write throughput. If your application writes at 100 MB/s, you need at least 100 MB/s of network bandwidth for DRBD replication. ZFS and GlusterFS are more flexible since they can replicate at whatever bandwidth is available, accumulating changes when the network is constrained.

Disaster Recovery Planning

A complete disaster recovery plan includes more than just replication. Schedule quarterly failover testing drills to verify your recovery procedures. Maintain documentation with runbooks for failover, split-brain resolution, and data restoration. Set up monitoring alerts for replication lag, connection failures, and disk space exhaustion. Ensure sufficient bandwidth between replication sites through proper network planning. Periodically verify that replicated data matches the source using checksums for data integrity testing.

Replication Monitoring and Alerting

Set up monitoring for all three replication solutions. DRBD provides the drbd-overview command for quick status checks and integrates with Prometheus through the drbd-reactor exporter for comprehensive metrics collection. ZFS replication status can be monitored by comparing snapshot timestamps between primary and replica systems, with zrepl providing built-in status reporting. GlusterFS provides the gluster volume geo-replication status detail command for comprehensive replication health information including checkpoint progress and error reporting.

For production deployments, configure alerts for replication lag exceeding your acceptable RPO threshold, connection failures that prevent replication from occurring, and disk space exhaustion on replica systems that could halt replication entirely.

FAQ

What is the difference between DRBD, ZFS replication, and GlusterFS geo-replication?

DRBD replicates at the block device level in real-time, working with any filesystem. ZFS replication uses filesystem snapshots for efficient incremental transfers. GlusterFS geo-replication works at the file level using changelog-based tracking. DRBD provides the lowest RPO (near zero), while ZFS and GlusterFS offer higher efficiency with periodic replication.

Can DRBD work across a WAN?

Yes, but with limitations. DRBD Protocol A (asynchronous) is recommended for WAN links. Protocol C (synchronous) adds network latency to every write, which can significantly impact application performance on high-latency connections.

Does ZFS replication require the same ZFS version on both sides?

Ideally, yes. The receiving system should run the same or newer ZFS version as the sender. Cross-platform replication (Linux to FreeBSD) works but requires careful testing.

Can I use DRBD with LVM?

Yes. DRBD sits below LVM in the storage stack. You create a DRBD device, then build LVM physical volumes on top of it. This gives you the replication benefits of DRBD with the flexibility of LVM volume management.

How does GlusterFS geo-replication handle network interruptions?

GlusterFS geo-replication automatically resumes replication after network interruptions. It uses a changelog to track changes, so only modifications made during the outage need to be replicated.

What is the performance impact of synchronous replication?

DRBD Protocol C (synchronous) adds one network round-trip to every write operation. On a 1 Gbps LAN with 1ms latency, this adds approximately 2ms per write. On a WAN with 50ms latency, each write takes 100ms longer.

How do I monitor replication health?

DRBD provides drbd-overview and integrates with Prometheus. ZFS replication status can be monitored by comparing snapshot timestamps. GlusterFS provides gluster volume geo-replication ... status detail for comprehensive health information.