Skip to main content
Menu
Advisories

Quadratic time unique_items list validation in Pydantic V1

GHSA-49G6-96JJ-F7R9Published Updated GitHub

Description

Validating an untrusted list into a field constrained with unique_items=True with Pydantic V1 takes quadratic time: an input of a few hundred kilobytes takes seconds to validate.

Details

The ConstrainedList.unique_items_validator validator (in pydantic/types.py) checks each item against all the following ones:

@classmethod
def unique_items_validator(cls, v: 'Optional[List[T]]') -> 'Optional[List[T]]':
    if v is None:
        return None

    for i, value in enumerate(v, start=1):
        if value in v[i:]:
            raise errors.ListUniqueItemsError()

    return v

For a list with no duplicates (the common, valid case), this results in a quadratic number of comparisons.

The issue was fixed in commit github.com/pydantic/pydantic/commit/6ec75e57f277a820ec583e601167f25fe8dc7c87.

If you are relying on the pydantic.v1 namespace in Pydantic V2, upgrade to V2.14.2 or later.

Impact

Denial of service through validation of untrusted input. Affected are applications validating untrusted lists into fields constrained with unique_items=True (using conlist() or Field()).

The fix only applies to lists of builtin scalar types (int, float, str, bytes, bool and None), for which duplicates can be detected in linear time. Lists of other types (e.g. models or dictionaries) are still checked in quadratic time, as their items aren't guaranteed to be hashable: bound their size using the max_items constraint.

Pydantic V2 models are not affected.

Affected versions

References

GitHub · pydantic/pydantic