libssh2 AES-GCM Underflow DoS

Overview

libssh2 is one of the most widely deployed client-side SSH libraries in the C ecosystem, embedded in everything from cURL builds to embedded IoT firmware and enterprise automation tooling. CVE-2026-66033 is a pre-authentication integer underflow in ssh2_cipher_crypt() inside src/openssl.c that surfaces whenever a connecting client negotiates an AES-GCM cipher suite during the SSH handshake. Because the flaw is triggered during cipher negotiation — before any credential exchange — an attacker controlling or impersonating an SSH server can crash any libssh2-based client that connects to it, without needing a valid account or any knowledge of the target environment.

The vulnerability was identified through code review of the cipher decryption path and reported via the libssh2 project’s GitHub issue tracker. The fix was merged as commit a2ed82d and is tracked publicly in pull request #2401. Any application that links against libssh2 ≤ 1.11.1 and allows connections to untrusted or user-supplied SSH endpoints is exposed, making the attack surface substantially broader than it might appear from a library-level advisory.

Given how frequently libssh2 is statically linked into third-party binaries and language extension modules (Python’s paramiko-adjacent bindings, PHP’s ssh2 extension, Ruby’s net-ssh C layer, and numerous vendor SDKs), the real-world blast radius of this denial-of-service is considerably larger than the CVSS 7.5 score alone communicates.

Technical Analysis

The root cause lives in the length arithmetic performed inside ssh2_cipher_crypt() when the negotiated cipher is an AES-GCM variant (e.g., [email protected] or [email protected]). AES-GCM is an Authenticated Encryption with Associated Data (AEAD) scheme: a portion of the SSH packet header is treated as Additional Authenticated Data (AAD/AAD length: aadlen), and a 16-byte authentication tag is appended to the ciphertext. The plaintext payload length is therefore:

payload_len = blocksize - aadlen - tag_len

The problem is that all three quantities — blocksize, aadlen, and tag_len — are derived from values supplied or influenced by the server during handshake, and the subtraction is performed using unsigned arithmetic on size_t operands without any prior bounds check. When the server provides values such that aadlen + tag_len > blocksize, the result wraps around to a value near SIZE_MAX rather than producing a negative number or a runtime error.

A representative approximation of the vulnerable code pattern (simplified for clarity) is:

/* src/openssl.c — vulnerable pattern, libssh2 <= 1.11.1 */
static int
ssh2_cipher_crypt(LIBSSH2_SESSION *session,
                  const LIBSSH2_CRYPT_METHOD *method,
                  int encrypt,
                  unsigned char *block,
                  size_t blocksize,
                  void *abstract)
{
    size_t aadlen  = method->aadlen;          /* server-influenced */
    size_t taglen  = method->auth_tag_len;    /* fixed at 16 for GCM */

    /* BUG: no check that blocksize > aadlen + taglen before subtraction */
    size_t paylen  = blocksize - aadlen - taglen;  /* wraps to ~SIZE_MAX */

    unsigned char *payload = block + aadlen;

    /* paylen is now ~SIZE_MAX — memcpy reads far out of bounds */
    memcpy(session->remote.crypt_abstract, payload, paylen);

    /* ... EVP_DecryptUpdate / EVP_AEAD_CTX_open call follows */
}

Two consequences cascade from the wrapped paylen value:

1. Out-of-bounds read: payload points into the legitimate packet buffer, but paylen of roughly 0xFFFFFFFFFFFFFFFF instructs the subsequent memcpy (or equivalent buffer operation) to read far beyond any allocated region.
2. Process crash: On virtually every modern OS and allocator, accessing memory dozens of gigabytes past the stack or heap triggers a segmentation fault or access violation, killing the process immediately.

Because this happens during the key-exchange / cipher-negotiation phase, it occurs before libssh2_userauth_* is ever called. The client process terminates with no opportunity to log a meaningful error, no authentication challenge is issued, and no credentials are involved. A malicious SSH server — or a network adversary performing a machine-in-the-middle attack on an unverified connection — can reliably reproduce the crash on every connection attempt.

Impact

The direct impact is unauthenticated denial of service against any process that uses libssh2 ≤ 1.11.1 to initiate an SSH connection. The CVSS 7.5 HIGH rating (Network / Low complexity / No privileges required / No user interaction / High availability impact) accurately captures the reliability and ease of exploitation.

Concrete scenarios include:

• CI/CD pipeline disruption: Build agents that clone from or deploy to SSH-exposed Git servers can be crashed mid-pipeline, blocking deployments and potentially leaving infrastructure in a partially updated state.
• Automated backup and transfer tools: rsync-over-SSH wrappers, sftp automation scripts, and similar tooling that call into libssh2 will terminate on first contact with a malicious endpoint.
• Embedded and IoT devices: Firmware that statically links libssh2 for configuration management cannot be patched without a full firmware update cycle, extending exposure windows significantly.
• Supply-chain amplification: Language-level SSH extensions (PHP ssh2, Ruby FFI bindings, and similar) expose the crash to higher-level applications whose authors may be entirely unaware they link against libssh2 at all.

There is no evidence of confidentiality or integrity impact from this specific vulnerability; however, repeated crashes can facilitate secondary attacks by disrupting monitoring and alerting pipelines that themselves rely on SSH connectivity.

How to Fix It

Upgrade libssh2 to a version that includes commit a2ed82d. Verify your installed version with:

# Linux (dpkg-based)
dpkg -l libssh2-1

# macOS (Homebrew)
brew info libssh2

# From source
pkg-config --modversion libssh2

The fix introduces a pre-subtraction guard that validates the relationship between blocksize, aadlen, and taglen before performing any arithmetic:

/* src/openssl.c — corrected pattern after commit a2ed82d */
static int
ssh2_cipher_crypt(LIBSSH2_SESSION *session,
                  const LIBSSH2_CRYPT_METHOD *method,
                  int encrypt,
                  unsigned char *block,
                  size_t blocksize,
                  void *abstract)
{
    size_t aadlen = method->aadlen;
    size_t taglen = method->auth_tag_len;

    /* Guard: reject malformed packets before unsigned arithmetic */
    if(blocksize < aadlen || (blocksize - aadlen) < taglen) {
        return LIBSSH2_ERROR_DECRYPT;
    }

    size_t paylen = blocksize - aadlen - taglen;   /* now safe */

    unsigned char *payload = block + aadlen;
    memcpy(session->remote.crypt_abstract, payload, paylen);

    /* ... normal AEAD decryption proceeds */
}

For package manager users:

# Debian / Ubuntu
sudo apt-get update && sudo apt-get install --only-upgrade libssh2-1

# RHEL / Fedora
sudo dnf upgrade libssh2

# Alpine
apk upgrade libssh2

# macOS
brew upgrade libssh2

# vcpkg
vcpkg upgrade libssh2

Applications that statically link libssh2 must recompile against the patched source. Confirm the fix is present by checking that ssh2_cipher_crypt in src/openssl.c contains the bounds check prior to the subtraction.

Our Take

Integer underflows in length arithmetic are a textbook C vulnerability class, yet they keep appearing in mature, well-scrutinised libraries because the C type system treats signed and unsigned overflow differently, and size_t subtraction produces silent wraparound with no compiler warning under default build flags. The combination of server-controlled inputs and a client-side AEAD framing function is an especially dangerous pairing: the attacker has direct leverage over the exact values fed into the vulnerable arithmetic.

For enterprises, the lesson is not simply “update libssh2.” It is that transitive C library dependencies are opaque to most runtime monitoring. A Python script, a PHP web application, or a Ruby deployment tool may all carry this vulnerability without any indication in their own dependency manifests. SBOM practices that capture native library linkage — not just language-level packages — are essential for identifying true exposure.

From a threat-model perspective, pre-authentication crashes deserve additional weight: they require no foothold, no credential knowledge, and no target cooperation. Any environment that allows clients to connect to externally influenced SSH endpoints (cloud provider APIs, customer-supplied jump hosts, third-party Git remotes) should treat this as an active risk until patched.

Detection with SAST

This vulnerability maps to CWE-191: Integer Underflow (Wrap or Wraparound), with a secondary classification under CWE-125: Out-of-Bounds Read and CWE-20: Improper Input Validation.

Offensive360’s SAST engine detects this pattern through a combination of:

• Unsigned arithmetic taint tracking: Any subtraction where one or more operands are derived from external (network/server-controlled) data and the result feeds into a memory-copy length argument is flagged as a potential underflow sink.
• Pre-condition absence checks: The analysis identifies subtraction expressions on size_t or unsigned types that lack a preceding conditional guard confirming the minuend is greater than or equal to the subtrahend.
• AEAD framing pattern recognition: Specific rules target the blocksize - aadlen - taglen idiom and its structural variants in SSH, TLS, and similar protocol implementations.
• Dangerous sink propagation: memcpy, memmove, memset, and related functions are treated as high-severity sinks; any tainted, unvalidated length argument reaching these calls triggers a HIGH-confidence finding.

DAST coverage requires a custom SSH server fixture that negotiates AES-GCM and then sends a crafted packet with mismatched length fields; Offensive360’s protocol fuzzing module includes this as a standard probe for SSH client targets.

References

• Patch commit a2ed82d — libssh2/libssh2
• [Pull Request #2401 — libssh2/libssh2](https://github.com/