Self-Hosted Embedded Database Engines: libmdbx vs LMDB vs UnQLite vs Vedis

Introduction

When building self-hosted applications, choosing the right data storage backend is critical. While full-featured database servers like PostgreSQL and MySQL dominate web applications, embedded database engines offer a compelling alternative for applications that need fast, zero-configuration local storage. These databases run directly within your application process, eliminating the overhead of network connections and separate server processes.

Embedded databases are widely used in Docker containers, IoT devices, edge computing nodes, mobile apps, and desktop software. They power everything from Bitcoin Core nodes to messaging apps, network monitoring tools, and game servers. Unlike traditional client-server databases, they require no separate daemon, no port to expose, and no authentication to configure.

In this guide, we compare four leading open-source embedded database engines — libmdbx, LMDB, UnQLite, and Vedis — examining their architectures, performance characteristics, and ideal use cases.

Architecture Comparison

libmdbx — The LMDB Successor

libmdbx is a fork of LMDB created by one of LMDB’s original developers. It surpasses the original LMDB in reliability, crash recovery, and database integrity checks. libmdbx adds automatic database size management (growing and shrinking), improved write performance through coalescing, and comprehensive integrity verification tools.

Key advantages over LMDB include:

• Automatic DB expansion — LMDB requires manual mdb_env_set_mapsize() calls; libmdbx handles this transparently
• Non-blocking checkpointing — Hot backups without pausing writes
• Built-in integrity checks — CRC32 verification of all pages
• Better compaction — Automatic page recycling during normal operation

1
2
3
4
5
6
7

# Install libmdbx on Debian/Ubuntu
sudo apt-get install libmdbx-dev

# Or build from source
git clone https://gitflic.ru/project/erthink/libmdbx.git
cd libmdbx
make && sudo make install

LMDB — The Proven Foundation

LMDB (Lightning Memory-Mapped Database) is the world’s fastest embedded key-value store, originally developed for OpenLDAP. Its key innovation is memory-mapped I/O — the entire database is mapped into virtual memory, allowing reads to bypass the OS page cache entirely. This makes LMDB reads essentially as fast as memory access.

LMDB’s design philosophy is minimalism: no caching layer of its own, no compression, no write-ahead log. It relies entirely on the OS page cache and copy-on-write B+ tree mechanics. This makes it incredibly robust — there are zero failure modes where the database can become corrupted.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

/* Simple LMDB write example */
#include <lmdb.h>

MDB_env *env;
MDB_txn *txn;
MDB_dbi dbi;
MDB_val key, data;

mdb_env_create(&env);
mdb_env_open(env, "./testdb", 0, 0664);
mdb_txn_begin(env, NULL, 0, &txn);
mdb_dbi_open(txn, NULL, 0, &dbi);

key.mv_size = 5;
key.mv_data = "hello";
data.mv_size = 5;
data.mv_data = "world";
mdb_put(txn, dbi, &key, &data, 0);

mdb_txn_commit(txn);
mdb_env_close(env);

UnQLite — Embedded NoSQL

UnQLite is a unique embedded database that supports both key-value and document (JSON) storage models. It uses a pluggable storage engine architecture — the KV store uses a fast in-memory B+ tree with on-demand disk persistence, while the document store implements a disk-based B-Tree with Jx9 (a JavaScript-like scripting language) for server-side scripting.

UnQLite’s document model supports:

• JSON document storage with automatic indexing
• Jx9 scripting for stored procedures and triggers
• Full collection support (equivalent to NoSQL collections)
• In-memory caching with write-through to disk

1
2
3
4
5

# Build UnQLite from source
git clone https://github.com/symisc/unqlite.git
cd unqlite
gcc -c unqlite.c -o unqlite.o
ar rcs libunqlite.a unqlite.o

Vedis — Embedded Redis

Vedis implements over 70 Redis commands in an embedded library — no separate Redis server required. Your application can use familiar Redis commands (SET, GET, HSET, LPUSH, SADD, ZADD, etc.) directly against a local database file. This makes migration from Redis-based applications trivial.

Key capabilities:

• Full Redis command set — Strings, Hashes, Lists, Sets, Sorted Sets
• Transactions — MULTI/EXEC with rollback support
• On-disk persistence — Automatic or manual SAVE commands
• In-memory cache — Frequently accessed keys served from memory

1
2
3
4
5

# Build Vedis from source
git clone https://github.com/symisc/vedis.git
cd vedis
gcc -c vedis.c -o vedis.o
ar rcs libvedis.a vedis.o

Deployment Scenarios

Docker Container with libmdbx

Embedded databases are ideal for Docker containers. Here is a Docker Compose configuration for a Go application using libmdbx:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

version: '3.8'
services:
  metrics-collector:
    image: myorg/metrics-collector:latest
    volumes:
      - metrics-data:/var/lib/metrics
    environment:
      - DB_PATH=/var/lib/metrics/data.mdbx
      - DB_SIZE_MB=1024
    restart: unless-stopped

volumes:
  metrics-data:

IoT Edge Node with UnQLite

For IoT deployments on resource-constrained devices like Raspberry Pi, UnQLite’s small footprint (less than 300 KB compiled) makes it ideal:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

# Python wrapper example for UnQLite on Raspberry Pi
import ctypes
import json

# Load UnQLite shared library
unqlite = ctypes.CDLL("libunqlite.so")

# Open database
db = ctypes.c_void_p()
unqlite.unqlite_open(ctypes.byref(db), b"/data/sensors.db", 0x00000004)  # UNQLITE_OPEN_CREATE

# Store JSON document
doc = json.dumps({
    "sensor_id": "temp-01",
    "value": 23.5,
    "timestamp": "2026-06-13T10:00:00Z",
    "unit": "celsius"
}).encode()

unqlite.unqlite_kv_store(db, b"sensor:temp-01:latest", -1, doc, len(doc))

Performance Characteristics

Read performance across all four engines is exceptional because they operate in-process with zero network overhead. libmdbx and LMDB excel in read-heavy workloads due to memory mapping — reads bypass the kernel page cache and go directly to application memory. UnQLite and Vedis use traditional read() syscalls, adding a small but measurable latency.

For write-heavy workloads, libmdbx significantly outperforms LMDB due to its automatic coalescing of small writes. UnQLite’s write performance depends on the storage engine — the KV mode is fast for point writes, while the document mode adds JSON parsing overhead.

Practical performance numbers (benchmarked on NVMe SSD, 1M records, 16-byte keys, 256-byte values):

Why Self-Host Your Data Storage Infrastructure?

Self-hosting your data storage layer with embedded databases offers several compelling advantages over managed cloud database services. First, data sovereignty — your data never leaves your infrastructure. For applications handling sensitive information like health records, financial transactions, or personal communications, keeping data on-premises is often a regulatory requirement.

Second, zero network latency matters enormously for performance-critical applications. A network round-trip to a cloud database adds 1-5 milliseconds of latency. An embedded database read completes in microseconds. For real-time applications like gaming servers, trading systems, or IoT data processing, this 1000x difference is decisive.

Third, operational simplicity cannot be overstated. An embedded database removes an entire class of operational concerns — no database server to monitor, no replication to configure, no connection pools to manage, no authentication to rotate. Your application starts, opens its database file, and is immediately ready to serve requests.

For IoT and edge computing deployments, embedded databases are often the only viable option. A Raspberry Pi monitoring air quality on a remote weather station cannot run PostgreSQL — it has 1 GB of RAM and limited storage. An embedded database like UnQLite or Vedis consumes less than 15 MB of memory and stores data efficiently on flash storage.

For related reading, see our self-hosted database management guide and our comparison of distributed caching alternatives to Redis. If you are evaluating message queuing backends, our NATS vs RabbitMQ vs Kafka comparison may be relevant.

FAQ

When should I use an embedded database instead of PostgreSQL or MySQL?

Use an embedded database when your application needs local persistent storage without the operational overhead of a database server. Ideal scenarios include: single-process applications, desktop software, mobile apps (via SQLite), Docker containers running a single service, IoT/edge devices, and embedded systems. If your application needs concurrent access from multiple processes or services over a network, a client-server database is more appropriate.

Can embedded databases handle concurrent access?

libmdbx and LMDB support multiple concurrent readers and a single writer through MVCC (Multi-Version Concurrency Control). This means reads never block reads, and reads never block writes. However, only one writing transaction can exist at a time. UnQLite and Vedis serialize all access — only one thread can read or write at a time. For applications with high write concurrency, consider using a connection pool or batching writes.

Are embedded databases suitable for production use?

Absolutely. LMDB has powered OpenLDAP (the most widely deployed LDAP server) for over a decade at massive scale. libmdbx is used in production by major cryptocurrency projects and high-frequency trading systems. UnQLite and Vedis are used in millions of IoT devices, mobile apps, and desktop applications. The key consideration is whether your access pattern fits the single-writer model — if it does, embedded databases are among the most reliable storage options available.

How do I back up an embedded database?

libmdbx supports non-blocking hot backups via mdbx_env_copy(). LMDB provides mdb_env_copy() which copies a consistent snapshot without blocking writes. For UnQLite and Vedis, you can copy the database file while the database is in a consistent state (after a commit or save). For automated backups, use filesystem snapshots (LVM, ZFS) or copy the file with the database in a known-good state.

What is the difference between libmdbx and LMDB?

libmdbx is a backward-compatible fork of LMDB with significant improvements: automatic database size management, better crash recovery with page-level checksums, non-blocking checkpointing for hot backups, improved write performance through write coalescing, and comprehensive integrity verification. If starting a new project, libmdbx is generally recommended over LMDB. Both share the same on-disk format and C API.

Do embedded databases support SQL queries?

No — LMDB, libmdbx, UnQLite (KV mode), and Vedis are all NoSQL key-value stores. They do not support SQL. If you need SQL query capability in an embedded database, consider SQLite, which provides a full SQL engine in a single library file. UnQLite’s document mode supports Jx9 scripting which can perform filter/transform operations, but it is not SQL.

💰 想测试你的市场判断力？我用 Polymarket 做预测市场交易——这是全球最大的预测市场平台，从大选结果到技术监管时间线，什么都可以押注。和赌博不同，这是真正的信息市场：你懂的信息越多，胜率越高。我靠预测技术相关事件的走向已经赚了不少。用我的邀请链接注册：Polymarket.com