hickory-proto: NSEC3 closest-encloser proof validation enters unbounded loop on cross-zone responses

The NSEC3 closest-encloser proof validation in hickory-proto's (0.25.0-alpha.3 ... 0.25.2) and hickory-net's (0.26.0-alpha.1 .. 0.26.0) DnssecDnsHandle walks from the QNAME up to the SOA owner name, building a list of candidate encloser names. The iterator used assumes the QNAME is a descendant of the SOA owner, terminating only when the current candidate equals the SOA name. When the SOA in a response's authority section is not an ancestor of the QNAME, the loop stalls at the DNS root and never terminates, repeatedly calling Name::base_name() and pushing newly allocated Name and hashed-name entries into the candidate Vec.

The bug is reachable by any caller of DnssecDnsHandle, including the resolver, recursor, and client, when built with the dnssec-ring or dnssec-aws-lc-rs feature and configured to perform DNSSEC validation. It is triggered while validating a NoData or NXDomain response whose authority section contains an SOA record from a zone other than an ancestor of the QNAME, on a code path that requires NSEC3 closest-encloser proof. In practice this can be reached through an insecure CNAME chain that crosses zone boundaries into a DNSSEC-signed zone returning NoData, but the minimum condition is just a mismatched SOA owner on a response requiring NSEC3 validation.

A debug_assert_ne!(name, Name::root()) guards the loop body, so debug builds abort with a panic on the first iteration past the root. Release builds compile the assertion out and run the loop unbounded, allocating until the process exhausts available memory. A reachable upstream attacker who can return such a response can therefore crash a debug build or exhaust memory on a release build, for the affected configurations.

The affected code was migrated from hickory-proto to hickory-net as part of the 0.26.0 release. Hickory DNS recommends that all affected users update to hickory-net 0.26.1 for the fix.

Reporter

David Cook, ISRG