Self-Hosted Container Log Driver Comparison: Docker Logging Backends for Production

Every container produces logs — but where those logs go depends entirely on your Docker logging driver configuration. The default json-file driver stores logs locally on disk, which works for development but quickly becomes a liability in production. Rotated logs fill your disk, critical errors go unnoticed, and debugging across multiple containers requires SSH access to every host.

This guide compares Docker’s built-in and plugin-based logging drivers, helping you choose the right backend for centralized log collection, long-term retention, and real-time alerting in self-hosted environments.

Docker Logging Driver Overview

Docker supports two categories of logging drivers:

• Built-in drivers (included with Docker Engine): json-file, local, journald, syslog, none
• Plugin drivers (installed separately): fluentd, gcplogs, awslogs, splunk, gelf, loki, etwlogs, logentries

json-file: The Default Choice

The json-file driver writes log entries as newline-delimited JSON objects to /var/lib/docker/containers/<id>/<id>-json.log. It’s simple but requires manual or external rotation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

{
  "daemon.json": {
    "log-driver": "json-file",
    "log-opts": {
      "max-size": "100m",
      "max-file": "5",
      "labels": "com.example.environment",
      "tag": "{{.ImageName}}/{{.Name}}/{{.ID}}"
    }
  }
}

Configure rotation limits in /etc/docker/daemon.json and restart Docker:

1
2
3
4
5
6
7
8
9

sudo systemctl restart docker

# Verify the driver
docker info --format '{{.LoggingDriver}}'
# Output: json-file

# Check container log location
docker inspect --format='{{.LogPath}}' my-container
# /var/lib/docker/containers/abc123/abc123-json.log

While json-file works for small deployments, it has fundamental limitations: logs are siloed per-host, searching across containers requires SSH access to each node, and if the Docker daemon crashes, in-flight logs are lost.

syslog: Universal Protocol with Decades of Tooling

The syslog driver forwards container logs to a central syslog server using the RFC 5424 protocol. This integrates with every major log management platform — rsyslog, syslog-ng, Graylog, Splunk, and ELK can all ingest syslog.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

{
  "daemon.json": {
    "log-driver": "syslog",
    "log-opts": {
      "syslog-address": "tcp://192.168.1.100:514",
      "syslog-facility": "daemon",
      "syslog-format": "rfc5424",
      "tag": "docker/{{.Name}}",
      "labels": "com.example.service"
    }
  }
}

Per-container syslog configuration:

1
2
3
4
5
6

docker run -d \
  --log-driver syslog \
  --log-opt syslog-address=tcp://syslog.example.com:514 \
  --log-opt syslog-facility=local0 \
  --log-opt tag="nginx/{{.Name}}" \
  nginx:alpine

The syslog driver is battle-tested but comes with protocol limitations: maximum message size is typically 2048 bytes (configurable), and multiline log entries (stack traces) must be handled by the receiver. For applications producing structured logs, the RFC 5424 STRUCTURED-DATA field provides a standardized metadata channel.

journald: Systemd-Native Integration

If your Docker hosts run systemd (most Linux distributions since 2015), the journald driver stores container logs directly in the systemd journal. This unifies host and container logging under a single query interface:

1
2
3
4
5
6
7
8
9

{
  "daemon.json": {
    "log-driver": "journald",
    "log-opts": {
      "tag": "docker/{{.Name}}",
      "labels": "com.example.env,com.example.service"
    }
  }
}

Query container logs with journalctl:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# View logs for a specific container
journalctl CONTAINER_NAME=my-nginx --no-pager -n 50

# Follow logs with tag filter
journalctl -f CONTAINER_TAG=docker/my-nginx

# Filter by Docker label
journalctl CONTAINER_LABEL_COM_EXAMPLE_SERVICE=api --since "10 minutes ago"

# Export to JSON for processing
journalctl -o json CONTAINER_ID=abc123 | jq '.MESSAGE'

Journald’s binary storage format uses compression and indexing, making it more space-efficient than json-file. However, the journal is host-local by default — you’ll need systemd-journal-remote or journal-upload for centralized collection.

fluentd: Flexible Log Pipeline with Plugin Ecosystem

The Fluentd logging driver sends container logs to a Fluentd aggregator, which can then route, transform, and forward them to multiple destinations simultaneously:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

{
  "daemon.json": {
    "log-driver": "fluentd",
    "log-opts": {
      "fluentd-address": "fluentd-aggregator:24224",
      "fluentd-async": "true",
      "fluentd-buffer-limit": "8MB",
      "tag": "docker.{{.Name}}",
      "labels": "com.example.env"
    }
  }
}

Fluentd aggregator configuration example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<filter docker.**>
  @type record_transformer
  <record>
    hostname "#{Socket.gethostname}"
    collected_at "#{Time.now.iso8601}"
  </record>
</filter>

<match docker.**>
  @type copy
  <store>
    @type elasticsearch
    host elasticsearch.internal
    port 9200
    logstash_format true
    logstash_prefix docker-logs
  </store>
  <store>
    @type s3
    s3_bucket container-logs-archive
    path logs/%Y/%m/%d/
    <buffer time>
      @type file
      path /var/log/fluentd-buffer/s3
      timekey 3600
      timekey_wait 10m
    </buffer>
  </store>
</match>

Fluentd’s strength is its plugin ecosystem — over 800 plugins for input, output, filtering, and buffering. You can send logs to Elasticsearch, S3, Kafka, MongoDB, and dozens of other destinations simultaneously, with in-flight buffering to handle downstream outages.

loki: Prometheus-Native Log Aggregation

The Grafana Loki logging driver sends container logs directly to a Loki instance, where they’re indexed by labels (not full-text) for efficient storage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Install the Loki Docker plugin
docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions

# Run a container with the Loki driver
docker run -d \
  --log-driver loki \
  --log-opt loki-url="http://loki.internal:3100/loki/api/v1/push" \
  --log-opt loki-retries=5 \
  --log-opt loki-batch-size=400 \
  --log-opt loki-external-labels="environment=production,service=nginx" \
  --name my-nginx \
  nginx:alpine

Query container logs in Grafana using LogQL:

1
2
3
4

{container_name="my-nginx", environment="production"}
  |~ "error|fail|panic"
  | json
  | line_format "{{.status}} {{.method}} {{.path}}"

Loki’s label-based indexing (rather than full-text indexing) makes it significantly cheaper to operate than Elasticsearch for log storage. At typical retention volumes (30-90 days), Loki uses 1/10th the storage of a comparable ELK deployment.

Why Self-Host Your Container Logging Pipeline?

Self-hosting your log pipeline keeps sensitive application data within your network boundary. Logs often contain request payloads, error traces with internal paths, user identifiers, and configuration secrets exposed during startup. Shipping these to a SaaS log provider creates both a security risk and a potential compliance violation for SOC 2, HIPAA, or GDPR-regulated environments.

Cost predictability is another major factor. Cloud logging services charge per-gigabyte ingested, and costs balloon unpredictably during incident response when log volume spikes. A self-hosted Loki or Fluentd + Elasticsearch stack on reserved instances provides fixed-cost logging at any volume. The operational overhead is straightforward — a 3-node Loki cluster with attached storage handles billions of log lines per day with minimal maintenance.

Beyond security and cost concerns, self-hosting your logging pipeline gives you complete control over log retention policies and data lifecycle management. You decide exactly how long to keep logs, when to archive them to cold storage, and which fields to redact for compliance. This granularity is essential for organizations subject to data protection regulations. For related container infrastructure guidance, check out our container sandboxing comparison and our syslog aggregation guide.

FAQ

Can I use different logging drivers for different containers?

Yes. Docker allows setting a default logging driver in /etc/docker/daemon.json and overriding it per-container with --log-driver. For example, use json-file for development containers and fluentd or syslog for production containers. Per-container overrides take precedence over the daemon-level default.

What happens to logs when the logging destination is unavailable?

The json-file and local drivers always work because they write to local disk. For network-based drivers (syslog, fluentd, loki, gelf), Docker buffers messages in memory. If the destination is unreachable, the driver blocks the container’s stdout/stderr after the buffer fills, which can cause the container to hang. Use the fluentd-async option or configure max-buffer-size to prevent containers from blocking when the log backend is down.

How do I handle multiline logs (stack traces, exception dumps)?

The json-file and local drivers write logs line-by-line without merging multiline entries. Use the Fluentd or syslog driver and configure the receiver to handle multiline merging. Fluentd’s parser_multiline plugin or Loki’s multiline stage can reassemble stack traces based on patterns (e.g., lines not starting with a timestamp are continuations). For json-file, use a log shipper like Filebeat or Promtail that supports multiline detection.

Does Docker rotate logs automatically?

The json-file driver supports size-based rotation via max-size and max-file options. The local driver has built-in rotation that automatically removes old logs based on disk usage. For other drivers (syslog, fluentd, loki), rotation is handled by the receiving service. Journald uses a configurable disk quota in /etc/systemd/journald.conf. Always test rotation in staging — filling /var/lib/docker with unrotated logs is one of the most common production incidents.

Can I send the same container logs to multiple destinations?

Docker supports only one logging driver per container. To send logs to multiple destinations, use Fluentd as the single driver and configure the Fluentd aggregator with a copy output plugin that fans out to Elasticsearch, S3, Kafka, or any combination of backends. Alternatively, use a log shipper (Vector, Filebeat, or Promtail) to read from json-file and forward to multiple destinations — but this adds operational complexity and requires managing log file paths.

💡 想测试你的市场判断力？我用 Polymarket 做预测市场交易——这是全球最大的预测市场平台，从大选结果到 科技政策监管时间线，什么都可以押注。和赌博不同，这是真正的信息市场：你懂的信息越多，胜率越高。我靠预测 科技行业的发展趋势已经赚了不少。用我的邀请链接注册：Polymarket.com