llama.cpp KV Cache Heap Buffer Overflow

Overview

CVE-2026-43629 is a heap buffer overflow vulnerability in llama.cpp, the widely used C++ runtime for running large language models locally and in production inference stacks. The flaw resides in the KV cache state restore path — specifically in the state_read_data() function — where a cell_count multiplication used to compute a write size is performed without integer overflow checking or bounds validation against the underlying tensor buffer allocation.

The vulnerability affects builds b4882 through b9058, a range spanning several months of active development during which llama.cpp saw rapid adoption in enterprise inference pipelines, edge deployments, and consumer AI applications. Any deployment that persists and later restores KV cache state (a common optimization for session continuity and speculative decoding workflows) and reads state files from a directory writable by an untrusted party is directly exploitable.

Security researchers identified the issue while auditing the session save and restore subsystem, which had received substantial refactoring over the affected build range. The attack surface is meaningful: an attacker who can place a crafted .bin state file in the slot_save_path directory — reachable through a misconfigured file share, a compromised sidecar service, or a multi-tenant deployment with insufficient path isolation — can trigger the overflow the next time any llama.cpp instance loads that slot.

Technical Analysis

The root cause is a classic integer overflow preceding a heap write. When state_read_data() deserializes a saved KV cache, it reads a cell_count value directly from the state file and uses it to compute the byte size of the data that follows. The multiplication is performed in a 32-bit (or otherwise width-insufficient) context before being used as the length argument to a memcpy-class operation targeting a tensor buffer whose size was allocated based on the model’s actual context configuration — not the attacker-controlled value from the file.

A simplified representation of the vulnerable pattern:

// VULNERABLE — build range b4882–b9058
bool state_read_data(struct llama_context * ctx, llama_data_source & source) {
    uint32_t cell_count;
    source.read(&cell_count, sizeof(cell_count));   // fully attacker-controlled

    // Integer overflow: if cell_count > (UINT32_MAX / sizeof(llama_kv_cell))
    // the result wraps, producing a small apparent size.
    size_t read_size = cell_count * sizeof(llama_kv_cell);

    // tensor_buf was allocated for ctx->kv_self.size entries — not cell_count.
    // When cell_count <= ctx->kv_self.size but the per-cell payload is large,
    // OR when the overflow wraps to a non-zero value that passes a naive check,
    // the copy destination overflows the heap allocation.
    void * tensor_buf = ctx->kv_self.buf.data;
    source.read(tensor_buf, read_size);             // heap buffer overflow
    // ...
}

There are two distinct overflow primitives here. First, if cell_count is large enough that cell_count * sizeof(llama_kv_cell) wraps to a small value on a 32-bit intermediate, the read_size check (if any exists) passes, yet the subsequent loop writing individual cell fields may still walk past the buffer. Second, and more directly, if cell_count legitimately exceeds the allocated KV cache slot count, the copy simply writes past the end of the heap allocation with no wrap-around required.

The tensor buffers in llama.cpp’s ggml backend are allocated as contiguous heap regions. Adjacent allocations typically include other model tensors, ggml graph nodes, or glibc heap metadata. Depending on heap layout — which an attacker can influence through controlled allocation patterns in the preamble of the state file — writing attacker-controlled bytes past the tensor buffer boundary can corrupt: adjacent model weight tensors (producing misclassification or poisoned outputs), ggml operator function pointers stored in graph node structs, or glibc malloc heap metadata chunks, all of which are classical primitives for achieving arbitrary code execution.

The CVSS 8.1 score (High) reflects the Adjacent/Network attack vector combined with the High confidentiality, integrity, and availability impact — appropriate given that exploitation requires file-write access to the save path rather than direct network reach to the inference endpoint.

Impact

An attacker with write access to the slot_save_path directory can achieve the following:

• Arbitrary code execution on the host running llama.cpp inference by overwriting a ggml function pointer or glibc heap control structure, then triggering its invocation during subsequent graph execution.
• Model weight corruption causing silent misbehavior — misclassified outputs, backdoored responses — without immediately crashing the process, making this useful for persistent poisoning attacks against AI-as-a-service deployments.
• Denial of service via heap metadata corruption causing abort() in the allocator, crashing the inference server and interrupting availability.

Deployments most at risk include multi-tenant inference APIs where different users or services share a filesystem, Kubernetes pods with overly broad volume mounts, and CI/CD pipelines that cache KV state across job boundaries in shared storage.

How to Fix It

The fix requires two cooperating controls: a safe multiplication that detects integer overflow before the value is used, and a bounds check that rejects any cell_count exceeding the context’s actual allocated KV cache size.

// FIXED — validate cell_count before computing read_size
bool state_read_data(struct llama_context * ctx, llama_data_source & source) {
    uint32_t cell_count;
    source.read(&cell_count, sizeof(cell_count));

    // Reject any cell_count that exceeds the model's configured KV cache size.
    if (cell_count > ctx->kv_self.size) {
        LLAMA_LOG_ERROR("%s: cell_count %u exceeds kv cache size %u\n",
                        __func__, cell_count, ctx->kv_self.size);
        return false;
    }

    // Use a checked multiplication — abort or return error on overflow.
    size_t read_size;
    if (__builtin_mul_overflow((size_t)cell_count, sizeof(llama_kv_cell), &read_size)) {
        LLAMA_LOG_ERROR("%s: cell_count multiplication overflowed\n", __func__);
        return false;
    }

    void * tensor_buf = ctx->kv_self.buf.data;
    source.read(tensor_buf, read_size);
    // ...
    return true;
}

For users building from source, update to a build newer than b9058 once the patch is merged from the reference patch repository. If you are consuming llama.cpp as a vendored dependency, apply the patch manually and rebuild. There is no package manager distribution for llama.cpp itself, but downstream wrappers should be updated as vendors integrate the fix:

# If using the official repo directly:
git pull origin master
cmake -B build && cmake --build build --config Release

# For Python bindings (llama-cpp-python) — update once upstream is patched:
pip install --upgrade llama-cpp-python

As an interim hardening measure, restrict filesystem permissions on slot_save_path so that only the inference process owner can write to it, and validate state files with a checksum or signature before loading them in environments where the save path is shared.

Our Take

Heap buffer overflows driven by attacker-controlled length fields in deserialization paths are one of the most reliably exploitable vulnerability classes in systems code, and they keep appearing in AI inference runtimes for a straightforward reason: these codebases grew rapidly from research prototypes into production infrastructure without the security-review cadence applied to more mature server software. The state_read_data() path in particular is exactly the kind of “internal” utility function that developers assume will only ever receive trusted input — an assumption that breaks the moment the feature is exposed to multi-user or multi-service deployments.

For enterprises deploying llama.cpp in production — whether on-premise for data-privacy reasons or in cloud inference clusters — this vulnerability is a reminder that the model file and the session state file are both attack surfaces. Defense-in-depth means treating state files with the same suspicion as network input: validate lengths, enforce maximums, and use OS-level isolation to prevent unauthorized writes to state directories.

Detection with SAST

This vulnerability falls under CWE-122 (Heap-based Buffer Overflow) and CWE-190 (Integer Overflow or Wraparound). Offensive360’s SAST engine detects this class of issue through dataflow taint analysis that tracks values read from external sources (files, network sockets, shared memory) to arithmetic operations used as size arguments in memory-write primitives (memcpy, memmove, read(), fread(), and their ggml/llama.cpp wrappers).

Specifically, the engine flags:

• Tainted length arithmetic: any multiplication of a file-sourced integer value by sizeof(T) before use as a copy length, without an interposing overflow check or upper-bound assertion.
• Missing allocation-size comparisons: memcpy/read calls where the length argument is not statically or dynamically bounded by the size of the destination allocation.
• Unsafe deserialization patterns: sequential source.read() calls where a count field controls subsequent data reads without validation against a known-safe maximum.

These rules are enforced at both the interprocedural and cross-translation-unit level, which is necessary here because the allocation (ggml_backend_buffer_alloc) and the overflow (state_read_data) occur in different compilation units.

References

• https://github.com/Vladimir-tokarev-cyera/llama-cpp-security-patches