Uncontrolled Resource Consumption in io.netty:netty-codec-http3

Summary

Netty's HTTP/3 codec versions from 4.2.0.Final through 4.2.17.Final contain an uncontrolled resource consumption vulnerability in the QPACK encoder-stream instruction decoder ("QpackEncoderHandler", installed on the peer-initiated unidirectional QPACK encoder stream, type 0x02). The handler accepts an attacker-declared string-literal length of up to "Integer.MAX_VALUE" (approximately 2 GiB) for the "Name Length" and "Value Length" fields of the "Insert With Literal Name" instruction (RFC 9204 4.3.3), with no per-instruction or per-literal length cap and no cumulation-size limit; the existing HTTP/3 limits ("maxHeaderListSize", "maxUnknownFramePayloadLength", "DEFAULT_MAX_FIELD_SECTION_SIZE") are not applied to this handler. A remote, unauthenticated peer with an established HTTP/3 connection to a default Netty HTTP/3 server can declare a very large literal length and then trickle fewer bytes than declared, causing the "ByteToMessageDecoder" MERGE cumulator to retain and grow the per-connection buffer, and ultimately triggering a large byte-array allocation. This leads to unbounded per-connection heap growth and "OutOfMemoryError", resulting in Denial-of-Service (Dos).