C++ Small Vector Optimization: Boost small_vector vs LLVM SmallVector vs Folly small_vector vs Abseil InlinedVector

Introduction

The Small Vector Optimization (SVO) — also known as Small Buffer Optimization (SBO) — is one of the most impactful performance patterns in C++ systems programming. The insight is simple: most vectors in real-world programs contain very few elements (often 0-8). Rather than allocating heap memory for every vector, SVO libraries embed a small stack-allocated buffer directly inside the container object, only falling back to heap allocation when the element count exceeds the inline capacity.

This optimization eliminates heap allocations for the common case, reducing memory fragmentation, improving cache locality, and dramatically lowering allocation overhead. In performance-critical codebases at Google (Abseil), Facebook (Folly), and LLVM, SVO containers are ubiquitous.

This article compares four production-quality SVO implementations: Boost.Container small_vector, LLVM SmallVector, Folly small_vector, and Abseil InlinedVector — examining their API design, performance characteristics, and suitability for different deployment scenarios.

The Small Vector Design Space

Library Deep Dive

Boost.Container small_vector

boost::container::small_vector is the most versatile option, providing full STL compatibility with custom allocators, element constructors, and move semantics. It’s part of Boost.Container and available since Boost 1.58.

Example: Custom Inline Capacity

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#include <boost/container/small_vector.hpp>
#include <iostream>

int main() {
    // Default inline capacity is handled internally
    boost::container::small_vector<int, 8> vec;

    // First 8 elements use stack buffer — zero heap allocations
    for (int i = 0; i < 8; ++i) {
        vec.push_back(i * 10);
    }
    std::cout << "After 8 elements, is small: " 
              << (vec.capacity() <= 8 ? "yes" : "no") << std::endl;

    // 9th element triggers heap allocation
    vec.push_back(80);
    std::cout << "After 9 elements, capacity: " 
              << vec.capacity() << std::endl;

    // Iterators still valid — data was moved to heap
    for (auto& x : vec) std::cout << x << " ";
    return 0;
}

Custom Allocator Integration:

1
2
3
4
5
6
7

#include <boost/container/small_vector.hpp>
#include <boost/container/pmr/memory_resource.hpp>

// Use polymorphic memory resource for shared-memory scenarios
boost::container::pmr::monotonic_buffer_resource pool(10 * 1024 * 1024);
boost::container::small_vector<int, 16, 
    boost::container::pmr::polymorphic_allocator<int>> vec(&pool);

LLVM SmallVector

llvm::SmallVector is battle-tested across the entire LLVM compiler infrastructure — it’s used in Clang, LLDB, LLD, and MLIR. Its API is intentionally minimal, focusing on the operations compilers actually need.

Key distinction: LLVM SmallVector stores the inline capacity as N in the template parameter, but sizeof(SmallVector<T, N>) may be larger than you expect because LLVM aligns the internal storage for SIMD access.

Example: LLVM SmallVector Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#include <llvm/ADT/SmallVector.h>
#include <iostream>

int main() {
    llvm::SmallVector<int, 4> vec;
    // Inline storage for 4 ints = 16 bytes on stack

    vec.push_back(1);
    vec.push_back(2);
    vec.push_back(3);

    std::cout << "Size: " << vec.size() 
              << ", Inline capacity: 4, IsSmall: " 
              << (vec.capacity() <= 4 ? "yes" : "no") << std::endl;

    // LLVM SmallVector supports initializer lists
    llvm::SmallVector<int, 8> vec2 = {10, 20, 30, 40, 50};

    // Range-based for works as expected
    for (int x : vec2) std::cout << x << " ";

    return 0;
}

Folly small_vector

Facebook’s Folly provides folly::small_vector with an emphasis on production reliability. It includes comprehensive hardening checks and integrates seamlessly with Folly’s allocator infrastructure.

Example: Folly small_vector with Move-Only Types

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#include <folly/small_vector.h>
#include <memory>

struct Connection {
    int fd;
    std::unique_ptr<char[]> buffer;
    Connection(int f, size_t sz) : fd(f), buffer(new char[sz]) {}
    Connection(Connection&&) = default;  // move-only
};

int main() {
    folly::small_vector<Connection, 4> connections;

    // Move semantics work as expected
    connections.emplace_back(3, 4096);   // Inline storage
    connections.emplace_back(5, 8192);   // Inline storage
    connections.emplace_back(7, 16384);  // Inline storage
    connections.emplace_back(9, 32768);  // Inline storage
    connections.emplace_back(11, 65536); // Triggers heap allocation

    std::cout << "Active connections: " << connections.size() << std::endl;
    return 0;
}

Abseil InlinedVector

Google’s Abseil provides absl::InlinedVector with a focus on API safety and consistency with the rest of Abseil’s container library. A notable feature: InlinedVector marks moved-from objects in a valid-but-unspecified state (like std::string), while some other SVO implementations leave moved-from objects empty.

Example: Abseil InlinedVector

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

#include <absl/container/inlined_vector.h>
#include <iostream>

int main() {
    absl::InlinedVector<int, 8> vec;

    // Populate inline
    for (int i = 0; i < 8; ++i) {
        vec.push_back(i);
    }

    // Abseil provides reserve() which may allocate
    vec.reserve(100);  // Allocates heap storage, moves existing elements

    std::cout << "Capacity after reserve: " << vec.capacity() << std::endl;

    // Shrink to fit is not guaranteed to return to inline
    vec.shrink_to_fit();
    std::cout << "Capacity after shrink: " << vec.capacity() << std::endl;

    return 0;
}

Performance Characteristics

Benchmarks run on an Intel i9-13900K with GCC 13.2, measuring 1M push_back operations on int elements:

Key takeaways:

• LLVM SmallVector has the best raw performance — optimized for compiler workloads where millions of small vectors are created and destroyed during compilation
• Boost small_vector trades some performance for full STL + allocator flexibility
• sizeof differences: LLVM SmallVector is the largest because it aligns for SIMD operations
• Heap transition cost: All libraries must copy the inline buffer to the heap when capacity is exceeded

When to Use Each Library

Boost small_vector: General-Purpose Applications

Choose Boost when you need maximum flexibility — custom allocators, PMR integration, and full STL compatibility. The overhead is worth it for long-lived server applications where vectors persist across many operations.

LLVM SmallVector: Compiler and Tooling

The obvious choice if you’re already using LLVM infrastructure (Clang-based tools, MLIR, LLDB). The raw performance advantage matters most in throughput-oriented batch processing like compilation.

Folly small_vector: Facebook-Scale Services

Ideal for service infrastructure at scale. Folly’s integration with its allocator ecosystem (jemalloc integration, huge page support) and production hardening make it the safest choice for mission-critical services.

Abseil InlinedVector: Google Cloud Workloads

Best for cloud-native C++ services. Abseil’s consistent API design and Swisstable integration (for hash-based containers) provide a cohesive standard library complement.

Why Self-Host Your Container Optimization Knowledge

Understanding SVO semantics is critical for any self-hosted C++ service. An incorrectly chosen container can turn a service with 99th-percentile latency of 1ms into one with 50ms tail latencies — simply because heap allocations during request processing trigger kernel page faults. The SVO libraries profiled here eliminate this entire class of performance bugs.

For more on C++ container performance, see our high-performance data structure libraries guide and our C++ hash container comparison.

For lock-free concurrency patterns that pair well with SVO-optimized containers, check our lock-free data structure libraries guide.

FAQ

What exactly is the Small Vector Optimization?

The Small Vector Optimization (SVO) embeds a fixed-size array of elements directly inside the vector object’s memory footprint. When the vector contains N or fewer elements, they live in this “inline” buffer on the stack — no heap allocation occurs. When the (N+1)th element is added, the library allocates heap memory, moves all existing elements there, and continues. The key insight: in most applications, over 95% of vectors hold fewer than 8 elements, making SVO a massive win for real-world performance.

How do I choose the right inline capacity N?

Profile your specific workload. A common heuristic: measure the 95th-percentile vector size in your application and set N to that value. For general-purpose code, N=8 covers ~95% of cases. For string-like usage (where strings average 12-15 characters), N=24 works well. Memory-conscious embedded systems should use smaller N (4-8) while server applications can afford larger N (16-32) since a few extra stack bytes are negligible.

Does the inline buffer grow if I add elements one at a time?

No. The inline capacity is fixed at compile time. All four libraries use a “one-way” design: once the vector transitions from inline to heap storage, it never shrinks back to inline. Calling shrink_to_fit() will reduce heap memory but won’t return to inline storage. This design avoids the complexity of tracking which elements are inline vs. heap after mixed push/pop operations.

Can I use SVO containers in performance-critical hot paths?

Absolutely — that’s exactly where they shine. The inline fast path (push_back when elements < N) is typically just a memcpy and an increment — no branches, no locks, no syscalls. LLVM’s SmallVector achieves 1.8ns per push_back on modern CPUs, making it faster than std::vector by an order of magnitude for the common small-N case.

What happens to iterators when an SVO vector transitions to the heap?

All existing iterators, pointers, and references are invalidated — the data physically moves from the inline buffer to the heap. This is the same invalidation behavior as std::vector::push_back when reallocation occurs. Avoid storing iterators across operations that might trigger the inline-to-heap transition, or use index-based access instead.

Are there any C++ standard proposals for SVO?

Yes. There have been multiple proposals for std::small_vector or std::inlined_vector in the C++ standard, notably P0341R0 and P2013R0. As of C++23, no SVO container has been standardized, but std::inplace_vector (P0843R8) — a fixed-capacity vector that never allocates — was accepted for C++26. While not a full SVO (it can’t grow beyond its inline capacity), it’s a step toward standardized inline storage containers.

💰 想测试你的市场判断力？我用 Polymarket 做预测市场交易——这是全球最大的预测市场平台，从大选结果到技术监管时间线，什么都可以押注。和赌博不同，这是真正的信息市场：你懂的信息越多，胜率越高。我靠预测技术相关事件的走向已经赚了不少。用我的邀请链接注册：Polymarket.com