net/rds: acquire the fastpath locks in rds_conn_shutdown()
- Published
- Last modified
- Where the record stands: published, rejected by its publisher, or superseded by another id.What do these mean? →
- published
- Whether CISA lists it in its Known Exploited Vulnerabilities catalog: proof that attackers are using it.What do these mean? →
- Not listed in CISA KEV
- Fix
- Fixed in 5.10.271, 5.15.222, 6.1.189, 6.6.158, 6.12.111, and 2 more
- Whether working attack code is public (Exploit-DB, Metasploit, Nuclei or GitHub). Public code makes attacks easier.What do these mean? →
- None on Exploit-DB, Metasploit or GitHub
Summary
In the Linux kernel, the following vulnerability has been resolved: net/rds: acquire the fastpath locks in rds_conn_shutdown() rds_conn_shutdown() quiesces the transmit and receive-refill paths by waiting for RDS_IN_XMIT and RDS_RECV_REFILL to be sampled clear, and then runs the transport shutdown and rds_conn_path_reset(). Sampling the bits clear is not the same as owning them: the moment after the wait_event() returns, rds_send_xmit() can re-acquire RDS_IN_XMIT (or rds_ib_recv_refill() can re-acquire RDS_RECV_REFILL) and run concurrently with the teardown. The sender does recheck the connection state after taking the lock, but that recheck is a classic store-buffering pattern: teardown writes the state and reads the bit while the sender writes the bit and reads the state. acquire_in_xmit() is only an acquire operation, so on weakly ordered architectures both sides can miss each other's write, and the transmit path then runs while the transport zeroes its rings (e.g. rds_ib_ring_init()) and rds_send_path_reset() rewrites the transmit state under it. Oracle UEK fixed the same class of crashes - a 14-year tail of BUG_ON()s in rds_ib_sub_signaled(), unexpected op-codes and NULL dereferences in rds_ib_send_cqe_handler() during failover testing - by making the teardown path acquire the fastpath bit locks instead of testing them ("rds: Make sure transmit path and connection tear-down does not run concurrently"). Ownership of a single word is decided by RMW atomicity, so no cross-variable ordering is needed. Do the same here: take both locks before calling the transport shutdown, hold them across rds_conn_path_reset(), and release them explicitly with a wake-up afterwards. Both are released with clear_bit_unlock(), so that the ring re-initialization done by the transport shutdown and the transmit state rewritten by rds_send_path_reset() are ordered before either bit is seen clear by the next acquire_in_xmit() or acquire_refill(). The fastpath users of these bits - rds_send_xmit() and rds_ib_recv_refill() - are trylock style and back off while teardown owns the locks, so no new lock dependency is introduced for them. rds_tcp_reset_callbacks() is different: since the previous patch it acquires RDS_IN_XMIT as well, and it blocks doing so, so its wait now spans the teardown instead of at most one send batch. That waiter runs from rds_tcp_accept_one() on the single-threaded krdsd workqueue and holds rds_tcp_accept_lock and t_conn_path_lock while it waits, so a duelling SYN accepted while its path is being torn down parks accept processing for the duration of the teardown - for TCP bounded by the (up to 5 s) drain loop in rds_tcp_conn_path_shutdown(). An IB path's drain in rds_ib_conn_path_shutdown() has no round cap, but no blocking waiter either: rds_tcp_reset_callbacks() is the only blocking acquirer of these bits and waits only on its own TCP path, and the fastpaths are trylock-and-back-off on both transports, so a long IB drain lengthens only that path's own quiesce. The window is narrow: the accept-side state check has to pass before the teardown moves the path to RDS_CONN_DISCONNECTING. Because krdsd is a single global workqueue, everything else queued there - accept processing for other connections and network namespaces, and the flush_workqueue(rds_wq) in rds_tcp_listen_stop() during namespace teardown - waits behind the parked accept worker for that time. It cannot deadlock, although the waits do point at each other: the teardown blocks until the bit's holder releases it, and the holder may be that krdsd accept worker. The holder finishes without needing anything the teardown owns: the sync cancels rds_tcp_reset_callbacks() issues target cp_send_w and cp_recv_w on the path's ordered cp_wq, whose only execution slot is occupied by the blocked cp_down_w itself, so they are pending at most and cancel without flushing - a reliance on cp_wq being ordered that is now noted next to those cancels (on ---truncated---
Timeline
- CVE published
- Record last updated
Affected versions
- Linux Linux
From 0f4b1c7e89e699f588807a914ec6e6396c851a72 · fixed in 7f4f21e4439df30d9aca9fb3bac38191a2f34729; From 0f4b1c7e89e699f588807a914ec6e6396c851a72 · fixed in 1f900f585cbb0fbe6aa00c31c9f5bb63ee614456; From 0f4b1c7e89e699f588807a914ec6e6396c851a72 · fixed in 32e21bcf4b5b11454bf57c4bb3bf019b71512ace; From 0f4b1c7e89e699f588807a914ec6e6396c851a72 · fixed in 2e46248e375a516301884719f08b6015c79a6456; From 0f4b1c7e89e699f588807a914ec6e6396c851a72 · fixed in 7febb113795d5de5b690b208f4b0e64a5fad1201; From 0f4b1c7e89e699f588807a914ec6e6396c851a72 · fixed in 900e96c9749a06833801f60c393aa1d405ea226c; From 0f4b1c7e89e699f588807a914ec6e6396c851a72 · fixed in 1fe627e5db5c3f53a9f9f9c8a66671755d306955; From 0f4b1c7e89e699f588807a914ec6e6396c851a72 · fixed in 813f3582ac7ae9f60f917937d54660e0952d5f2d
- Linux Linux
2.6.37
- KernelLinux
From 2.6.37 · fixed in 5.10.271; From 5.11.0 · fixed in 5.15.222; From 5.16.0 · fixed in 6.1.189; From 6.2.0 · fixed in 6.6.158; From 6.7.0 · fixed in 6.12.111; From 6.13.0 · fixed in 6.18.53; From 6.19.0 · fixed in 7.2.7
References 8
git.kernel.org
- stable/c/7f4f21e4439df30d9aca9fb3bac38191a2f34729
- stable/c/1f900f585cbb0fbe6aa00c31c9f5bb63ee614456
- stable/c/32e21bcf4b5b11454bf57c4bb3bf019b71512ace
- stable/c/2e46248e375a516301884719f08b6015c79a6456
- stable/c/7febb113795d5de5b690b208f4b0e64a5fad1201
- stable/c/900e96c9749a06833801f60c393aa1d405ea226c
- stable/c/1fe627e5db5c3f53a9f9f9c8a66671755d306955
- stable/c/813f3582ac7ae9f60f917937d54660e0952d5f2d