Description
An architectural security boundary bypass in inline-detach and MessageInspector allows remote, unauthenticated attackers to force unbounded recursive decompression routines. By supplying a specially crafted OpenPGP message containing deeply nested compressed packet structures, an attacker can completely bypass the library's built-in DoS protections (MAX_LAYER_DEPTH = 16), leading to resource exhaustion, thread blocking, and potential Out-of-Memory (OOM) application crashes.
Details
The pg-painless core processing pipeline implements a defensive threshold constraint to prevent decompression bomb sequences (Zip Bombs) and arbitrary resource consumption:
Enforcement Location: pgpainless-main/pgpainless-core/src/main/kotlin/org/pgpainless/decryption_verification/MessageMetadata.kt
Threshold Constant: MAX_LAYER_DEPTH = 16
However, a code architecture review revealed two specific packet-walking implementations that unpack and step through PGPCompressedData packets manually, entirely failing to track or increment a layer depth index counter against the static compliance limits:
In InlineDetachImpl.kt (Line ~78-93): The logic loops over data packets via compressedData.dataStream. Because it processes the incoming data block-by-block without leveraging or passing constraints down to the guarded OpenPgpMessageInputStream tracking mechanism, it accepts infinite nesting.
In MessageInspector.kt (Line ~95): The inspector utility descends recursively down nested compressed packet layouts to ascertain inner content classification markers (e.g., determining encryption states) without referencing or respecting maximum layer depth boundaries.
PoC
The vulnerability can be verified by constructing an inline-signed binary message wrapped recursively inside multiple layers of compression packets:
Generate a standard, valid OpenPGP public/private keypair.
Sign a plaintext document sample utilizing binary inline-signing configurations (.noArmor()).
Programmatically loop 64 times over the output payload. In each iteration, encapsulate the previous binary output into a new PGPCompressedDataGenerator context configured with CompressionAlgorithmTags.ZIP.
Pass the final 64-layer nested byte array stream directly to the SOP interface via sop.inlineDetach().message(nestedBytes).toByteArrayAndResult().
Observed Result: The library completely parses the highly nested structures, returning a successful outcome without raising any execution constraints. The code fails to throw the expected SOPGPException.BadData, proving that the 16-layer boundary constraint was completely bypassed during the streaming packet loop.
Impact
Who is impacted: Any downstream application, gateway, or mail client (such as FlowCrypt or any framework integrating the library's SOP wrapper interface/MessageInspector modules) that handles untrusted, incoming external user files or signed attachment streams. An attacker can use this behavior to systematically freeze application threads, exhaust target heap allocation metrics, and disrupt backend infrastructure services.
Affected versions
fixed in 1.7.8
From 2.0.0 · fixed in 2.0.4