vLLM: Remote DoS in vLLM via Invalid Recovered Token Reinjection
- Published
- Last modified
- Where the record stands: published, rejected by its publisher, or superseded by another id.What do these mean? →
- published
- Whether CISA lists it in its Known Exploited Vulnerabilities catalog: proof that attackers are using it.What do these mean? →
- Not listed in CISA KEV
- Fix
- Fixed in 0.24.0
- Whether working attack code is public (Exploit-DB, Metasploit, Nuclei or GitHub). Public code makes attacks easier.What do these mean? →
- None on Exploit-DB, Metasploit or GitHub
- Exploitation: poc
- Automatable: yes
- Technical impact: partial
Summary
vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs. Prior to 0.24.0, a frontend-legal multi-request speculative decoding workload can cause the rejection sampler to produce a recovered token equal to the model vocabulary size boundary value, which is then converted to negative one when the engine selects the next live token for a request and is written back into the drafter's input ids; that out-of-vocabulary value is later consumed by the model's embedding and attention path and crashes the engine worker with a GPU device-side assertion. The same triggering request sequence is reachable through the public gRPC Generate and Abort endpoints, so a remote client that can send generation requests can crash the shared engine worker, aborting concurrent requests and causing a service-wide denial of service for other clients of the deployment until the worker is restarted. This issue is fixed in version 0.24.0.
Timeline
- CVE published
- CISA SSVC assessment
Exploitation: poc · Automatable: yes · Technical impact: partial
- Record last updated
Affected versions
- vllm-project vllm
fixed in 0.24.0
- vllmPyPI
From 0.17.1 · fixed in 0.24.0