Self-Hosted Binary Serialization Frameworks: bincode vs borsh vs postcard vs rkyv

Why Binary Serialization Matters for Self-Hosted Services

When building self-hosted backend services, the format you choose for data serialization directly impacts three critical metrics: throughput, latency, and memory usage. Text-based formats like JSON and YAML are human-readable but incur significant parsing overhead — every number must be parsed from decimal strings, every string must be escaped and unescaped, and the entire document must be scanned linearly. Binary serialization formats eliminate this overhead by encoding data directly in its machine representation.

For Rust-based self-hosted services — increasingly common in performance-critical infrastructure like API gateways, message brokers, and database proxies — the choice of binary serialization framework can mean the difference between 10,000 and 100,000 requests per second on the same hardware. Rust’s zero-cost abstraction model means serialization frameworks can leverage compile-time type information to generate highly optimized code with no runtime reflection overhead.

In this comparison, we examine four leading Rust binary serialization frameworks: bincode (compact general-purpose), borsh (deterministic for hashing), postcard (embedded/IoT optimized), and rkyv (zero-copy deserialization). Each targets a different trade-off in the serialization design space.

Quick Comparison

bincode: The Compact Workhorse

bincode is the most widely adopted Rust binary serialization framework, used by projects like the Bevy game engine and numerous database systems. It takes a straightforward approach: serialize data as compactly as possible using Serde’s data model, with configurable endianness and length encoding.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

use serde::{Serialize, Deserialize};
use bincode;

#[derive(Serialize, Deserialize, Debug, PartialEq)]
struct Config {
    port: u16,
    host: String,
    workers: u32,
    features: Vec<String>,
    metadata: Option<String>,
}

fn main() {
    let config = Config {
        port: 8080,
        host: "0.0.0.0".into(),
        workers: 4,
        features: vec!["tls".into(), "compression".into()],
        metadata: None,
    };

    // Serialize with default options (little-endian, variable-length)
    let encoded: Vec<u8> = bincode::serialize(&config).unwrap();
    println!("Encoded size: {} bytes", encoded.len()); // ~47 bytes

    // For deterministic/hashing-safe output, use fixed-int encoding
    let opts = bincode::config::standard()
        .with_big_endian()
        .with_fixed_int_encoding();
    let det: Vec<u8> = bincode::serde::encode_to_vec(&config, opts).unwrap();

    // Deserialize
    let decoded: Config = bincode::deserialize(&encoded).unwrap();
    assert_eq!(config, decoded);
}

bincode’s strength is its balance of simplicity and performance. For most Rust services, it’s the sensible default — serializing a struct is a single function call, and the encoded output is about 30-50% smaller than equivalent JSON.

borsh: Deterministic Serialization for Blockchain

borsh (Binary Object Representation Serializer for Hashing), developed by the NEAR Protocol team, was designed from the ground up for deterministic output — the same data always produces byte-for-byte identical output regardless of platform, compiler version, or Serde configuration. This property is essential for blockchain consensus, where different validators must compute identical state hashes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

use borsh::{BorshSerialize, BorshDeserialize};

#[derive(BorshSerialize, BorshDeserialize, Debug, PartialEq)]
struct Account {
    id: u64,
    balance: u128,
    owner: String,
    permissions: Vec<u8>,
}

fn main() {
    let account = Account {
        id: 1,
        balance: 10_000_000_000_000_000,
        owner: "alice.near".into(),
        permissions: vec![1, 2, 4, 8],
    };

    // Serialize (always deterministic)
    let mut buf = Vec::new();
    account.serialize(&mut buf).unwrap();
    println!("Encoded: {} bytes", buf.len());

    // Deserialize
    let decoded = Account::try_from_slice(&buf).unwrap();
    assert_eq!(account, decoded);

    // Verify determinism: same input always = same bytes
    let mut buf2 = Vec::new();
    account.serialize(&mut buf2).unwrap();
    assert_eq!(buf, buf2); // byte-for-byte identical
}

borsh’s deterministic property makes it ideal for any application where data must be hashed for integrity verification — API request signing, state snapshot comparison, and cryptographic audit trails. The trade-off is slightly larger output than bincode (borsh uses fixed-width encoding for numeric types), but the determinism guarantee is worth the bytes for many applications.

postcard: Serialization for Embedded Systems and IoT

postcard is designed for resource-constrained environments — microcontrollers, IoT devices, and embedded Linux systems with limited memory. It’s a no_std + serde compatible format that uses a compact, self-describing wire format similar to COBS (Consistent Overhead Byte Stuffing).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

use serde::{Serialize, Deserialize};
use postcard;

#[derive(Serialize, Deserialize, Debug, PartialEq)]
struct SensorReading {
    sensor_id: u16,
    temperature: f32,
    humidity: f32,
    timestamp: u64,
    flags: u8,
}

fn main() {
    let reading = SensorReading {
        sensor_id: 42,
        temperature: 23.5,
        humidity: 65.2,
        timestamp: 1718798400,
        flags: 0b0000_0011,
    };

    // Serialize to heapless buffer (no allocator needed!)
    let output: Result<heapless::Vec<u8, 128>, _> = postcard::to_vec(&reading);
    let encoded = output.unwrap();
    println!("Encoded: {} bytes", encoded.len()); // ~19 bytes

    // Deserialize
    let decoded: SensorReading = postcard::from_bytes(&encoded).unwrap();
    assert_eq!(reading, decoded);
}

postcard’s key advantage for self-hosted infrastructure is its ability to work without a heap allocator, making it perfect for embedded services running on ESP32 or ARM Cortex-M devices — think home automation hubs, environmental sensors, and industrial controllers that communicate with a central server over MQTT or CoAP.

rkyv: Zero-Copy Deserialization at Memory Speed

rkyv (archive) takes a fundamentally different approach to serialization: instead of encoding data into a byte format that must be parsed back, rkyv lays data out in memory in a way that’s immediately usable. Deserialization with rkyv is essentially a pointer cast — no parsing, no allocation, no copying. This makes rkyv the fastest deserialization framework by a wide margin.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

use rkyv::{Archive, Serialize, Deserialize};

#[derive(Archive, Serialize, Deserialize, Debug, PartialEq)]
#[archive(check_bytes)] // Enable validation
struct LogEntry {
    level: u8,
    message: String,
    service: String,
    timestamp: i64,
    tags: Vec<String>,
    trace_id: [u8; 16],
}

fn main() {
    let entry = LogEntry {
        level: 2,
        message: "Connection timeout after 30s".into(),
        service: "api-gateway".into(),
        timestamp: 1718798400,
        tags: vec!["network".into(), "timeout".into()],
        trace_id: [0; 16],
    };

    // Serialize (archives data in-place)
    let bytes = rkyv::to_bytes::<_, 256>(&entry).unwrap();

    // Zero-copy deserialize — no parsing, just validation
    let archived = unsafe {
        rkyv::access_unchecked::<ArchivedLogEntry>(&bytes)
    };

    // Access data directly from the archived representation
    println!("Service: {}", archived.service);  // No copy!
    println!("Level: {}", archived.level);
    println!("Message: {}", archived.message);

    // Convert back to owned type when needed
    let deserialized: LogEntry = archived.deserialize(&mut rkyv::Infallible).unwrap();
    assert_eq!(entry.message, deserialized.message);
}

rkyv excels in scenarios with large amounts of data that is written once and read many times: configuration caches, read-only databases, replay logs, and shared memory IPC between processes. For a self-hosted API gateway storing routing rules, rkyv lets you deserialize thousands of rules in microseconds rather than milliseconds.

Performance Benchmarks and Scaling Considerations

In benchmark comparisons across the Rust serialization ecosystem, the performance hierarchy is clear:

• Deserialization speed: rkyv (zero-copy, ~5 ns per struct) > bincode (~50 ns) > postcard (~80 ns) > borsh (~100 ns)
• Serialization speed: bincode ≈ borsh ≈ postcard (all within 10% of each other) — rkyv is slightly slower at serialize time due to the pointer-relative layout
• Output size: postcard (most compact, varint encoding) ≈ bincode < borsh (fixed-width ints) < rkyv (includes relative pointers, ~20% overhead)

For a typical self-hosted web service handling 50,000 requests per second, switching from JSON to bincode can reduce CPU usage by 30-40%. If you’re running a read-heavy caching layer, rkyv’s zero-copy deserialization can reduce latency by an order of magnitude compared to traditional deserialization frameworks.

When deploying self-hosted services built with these frameworks, consider the deployment context. bincode and borsh are ideal for general-purpose REST and gRPC APIs. postcard shines in edge computing and IoT gateways. rkyv is the clear winner for read-heavy caching layers and inter-process communication.

Why Self-Host Your Serialization Infrastructure?

Choosing the right serialization framework for your self-hosted services is an architectural decision that compounds across your entire system. Every byte that travels over the network between your microservices goes through serialization and deserialization — opting for an optimized binary format can reduce network bandwidth by 50% and cut request latency in half compared to JSON.

Binary serialization also provides stronger type safety guarantees than text formats. With borsh’s deterministic encoding, you can implement cryptographic verification of API responses without worrying about JSON key ordering or whitespace differences. This is critical for self-hosted services handling financial data, healthcare records, or any domain requiring audit trails.

For more on data processing performance, see our JSON parser libraries comparison. For API validation workflows, check our API schema validation guide. If you’re working with message queues, our message broker comparison provides additional context on serialization within messaging systems.

FAQ

Which serialization framework produces the smallest output for network transfer?

postcard typically produces the most compact output due to its varint encoding for integers and compact string length encoding. However, the difference from bincode is usually 5-10%. If you’re optimizing for minimal network bandwidth, consider also applying compression (zstd or lz4) on top of any of these formats.

Can I mix these serialization frameworks in the same project?

Yes, and it’s a common pattern. For example, use borsh for data that needs to be hashed (state snapshots, API signatures) and rkyv for read-heavy caching layers. Each framework operates independently on its own types — just don’t try to deserialize borsh-encoded data with bincode’s deserializer.

Is rkyv always faster than the alternatives?

For deserialization, yes — by a large margin because it bypasses parsing entirely. For serialization, rkyv is typically slightly slower than bincode or postcard because it must compute relative pointer offsets. The trade-off is worth it when data is serialized once but deserialized thousands of times.

Do any of these support schema evolution?

All four frameworks handle adding optional fields gracefully (they deserialize as None/default), but none provide built-in schema migration tools like Protobuf or Avro. For backward-compatible schema evolution, wrap potentially new fields in Option<T> and use #[serde(default)] for added fields. For more robust schema evolution, consider supplementing with Apache Avro or Protocol Buffers for cross-service boundaries.

💰 想测试你的市场判断力？我用 Polymarket 做预测市场交易——这是全球最大的预测市场平台，从大选结果到技术监管时间线，什么都可以押注。和赌博不同，这是真正的信息市场：你懂的信息越多，胜率越高。我靠预测技术相关事件的走向已经赚了不少。用我的邀请链接注册：Polymarket.com