Quadratic time unique_items list validation in Pydantic V1
Description
Validating an untrusted list into a field constrained with unique_items=True with Pydantic V1 takes quadratic time: an input of a few hundred kilobytes takes seconds to validate.
Details
The ConstrainedList.unique_items_validator validator (in pydantic/types.py) checks each item against all the following ones:
@classmethod
def unique_items_validator(cls, v: 'Optional[List[T]]') -> 'Optional[List[T]]':
if v is None:
return None
for i, value in enumerate(v, start=1):
if value in v[i:]:
raise errors.ListUniqueItemsError()
return vFor a list with no duplicates (the common, valid case), this results in a quadratic number of comparisons.
The issue was fixed in commit github.com/
If you are relying on the pydantic.v1 namespace in Pydantic V2, upgrade to V2.14.2 or later.
Impact
Denial of service through validation of untrusted input. Affected are applications validating untrusted lists into fields constrained with unique_items=True (using conlist() or Field()).
The fix only applies to lists of builtin scalar types (int, float, str, bytes, bool and None), for which duplicates can be detected in linear time. Lists of other types (e.g. models or dictionaries) are still checked in quadratic time, as their items aren't guaranteed to be hashable: bound their size using the max_items constraint.
Pydantic V2 models are not affected.