seanmonstar

2025 in review

Tue, 27 Jan 2026 11:32:00 +0000

Come along with me as I review the past year. Heh, I often start these kinds of posts right at the start of the year, but it takes a few weeks longer than I ever expect to think them through.¹

Two years of being independent

After a second year of operating as an independent open source maintainer, it’s starting to feel more normal.

In terms of personal execution, it felt pretty fantastic, actually. Thanks to high-touch conversations from my retainers, I knew what was needed; there was an underlying trend. And I was able to spec out a grant that made a project out of that trend. All while managing to do the necessary maintenance work that the ecosystem requires. Granted, it did occasionally feel like a conflict of priorities, but that’s life.

Honestly, though, I wasn’t so sure when trying to plan this all out initially.

Maintaining hyper

Perhaps the biggest deal for hyper this year was launching our first user survey. I’ve thought of doing it a few times over the years, but finally remembered in Q4 to launch it. Thanks to all who answered! I’ve looked through the results, and I think this will be extremely useful. Some stats real quick: 96% of respondents have upgraded to hyper v1.x, most commonly combine it with Tokio (99%) and rustls (92%).² A proper analysis coming soon!

katelyn martin joined us as a collaborator, and has continued to be a multiplier with kind reviews and maintenance glue. And general maintenance doesn’t stop, including growing security reports (more below).³

Besides all that, I took on a larger project for the year. You see, after updating the roadmap at the end of the previous year, I started to focus on one of the four defined areas: improved hyper-util. This lined up with what many have been asking for.

I did that by modularizing parts out of reqwest.

Modularizing reqwest

Most of my year was spent on modularizing reqwest. Or, from another angle, giving back the building blocks that reqwest has accumulated over the years. A lot of functionality that people rely on in reqwest started life as internal glue, and this was the year I finally pulled many of those pieces out into places where the rest of the ecosystem could use them too. Between reqwest and hyper‑util, that work ended up producing quite a few releases: 14 for reqwest itself, and 8 for hyper‑util.

Ages ago, I added a bunch of features that you expect any client to have directly into reqwest. Later, as tower-http grew, it copied some of those same features. Meanwhile, reqwest was used in weirder and weirder places, so we hardened those features, and tossed in some tests to check for the weird. But tower-http never saw any of that.

This year, we completely tossed the redirect and decompression code from reqwest, depended on the tower-http pieces, and then allowed the test suite to find the difference. The tower-http layers got those fixes backported, and now everyone benefits.

We also created new things, but still modular.

Easier retries were added to reqwest, making use of the lower-level pieces in tower. I’m still interested in ways to improve the feature, so more people can use retries more safely.

reqwest has grown extensive support for connection proxies. But an increasingly common pattern was people using reqwest only for the proxy support; they didn’t need any other feature. So I extracted proxy matchers and proxy connectors (tunnel, socks) into hyper-util.

The largest piece was designing and implementing composable pools for hyper-util. In many ways, this was my highlight of the year. It’s a problem I’ve been thinking about since … 2018? I’d done a lot of research throughout the years, and never found anything quite like it. Now, it’s not quite “done”, but it’s a base that allows a lot of new layers and compositions to be explored.

To end the year, we released v0.13 with rustls as default. It’s a big improvement for most people. But. I am not currently happy with how difficult it is to build the defaults on some other targets (Windows, Cranelift, cross-compiling). I want that fixed. Maybe that’s improvements to upstream aws-lc-rs; it looks like it’s already been improved to not need cmake. Or maybe we use a different default crypto provider on some targets.

The value of deadlines

The work on composable pools was hard. The reason it had taken me years to finally try was that I wasn’t sure about some of the design. After staring hard at it during the summer, I did solve some of the questions. But there was one problem towards the end that consumed another month or so of staring. And this time, I couldn’t stop staring.

With a hard deadline set, however, there was no possibility of waiting longer. Instead, I had to settle with shipping what I had, and accepting that it can always be better.

And that’s also the beauty of deadlines: they keep you user-driven. As long as I’m staring hard at a problem, holding back shipping, users have nothing. But software doesn’t need to be shipped all at once. It’s a lesson I’ve learned before, and yet it pops up to, uh, delight me over and over.

I feel like I go through waves: I hate setting a deadline, and many times feel disappointed at not shipping all the glory that was in my head. But I always appreciate that at least they got something.

Security takes time

We take security seriously, and the amount of reports we receive is slowly increasing. This past year, we had a 8 in total, including our first AI slop report (yay!).

The biggest one resulted in a large coordinated disclosure event. But hyper was just fine.

That didn’t stop it from being stressful trying to handle reports while simultaneously sticking to feature deadlines.

It is a reminder, though, that this is often urgent and important work that must be handled, but that traditional pay-for-features doesn’t support. Sponsorships and retainers make this sort of maintenance much more sustainable.

Talks

On the 10th anniversary of Rust 1.0, I gave a talk for the Rust for Lunch meetup. It was sort of ‘lessons using Rust for 10 years’, but also ‘why you should consider Rust’.

And I did a podcast episode on Netstack.FM, discussing the history of Rust’s networking ecosystem.

Pondering

Last year, I liked just sharing some questions I’m thinking about. It wasn’t a promise to work on them actively, but I look at them from time to time to see if there’s something that I can tackle soon.

Here’s just a few things I’m thinking about at the start of 2026:

How do I balance keeping up with LLM advances while keeping my mind and skills sharp?
How far can one reasonably go with typestate builders, considering ergonomics and correctness?

I wonder.

See some previous yearly reviews: 2024, 2023, 2022, 2021. ↩
Though a significant amount also use other runtimes and TLS libraries. As mentioned, more coming soon. ↩
There were three feature releases of hyper and two of http, bringing support for things like on_informational and constants for EARLY_HINTS. There was also five bug fix releases of h2, hardening the HTTP/2 story for Rust. ↩

reqwest v0.13 - rustls by default

Tue, 30 Dec 2025 10:48:00 +0000

To end out the year, here comes a new major release of reqwest, the opinionated higher-level HTTP client for Rust.

We don’t really need major breaking versions to keep providing value. Improvements keep coming all the time. But we did need one to make one particular big adjustment, and we’ve taken the opportunity to clean up other things too. At the same time, we strove make it disrupt as little as possible, especially if you stick to the defaults.

reqwest v0.13.0 is out now! Read on for why.

rustls is now the default TLS backend

The biggest deal is that reqwest now sets its default TLS feature to use rustls, instead of native-tls.

Granted, native-tls has its place. It provides a unified library that uses the “native” TLS implementation on each target. SecureTransport on macOS, schannel on Windows, and OpenSSL on Linux (as the most likely already installed). It was the right choice several years, while rustls was young. But rustls is now safer and faster than most choices.¹ Seems like an obvious improvement. But would people want the better choice?

A recent hyper user survey found that 93% of respondants already use rustls. 30% said they also use native-tls at times, so we will continue to provide that option. And while a survey is already biased, it’s also most certainly true that the vast majority of users just allow the default options, and it works for them. That will continue to be true for most everyone.

So, if you use the default options, things just got better for you.

Certificate verification features are consolidated

Previously, reqwest had many crate features to enable various ways to load sources of root certificates. Too many. It’s a bit of a mess.

We consolidate the options into just a few simple choices:

Defaults to using the native platform verifier.
If the target supports it, tls_certs_merge() can add additional certificates. If it doesn’t, a builder error is returned.
If you really need a specific certificate, use tls_certs_only(), which will always work, and doesn’t include native verification.

These basic options should allow for any use case, while reducing complexity within reqwest. For instance, if an application really needs to use the webpki-roots, they can be configured with tls_certs_only(your_roots).

Soft-deprecation to improve option naming

While in there, most ClientBuilder methods were given better names. The previous methods have been soft-deprecated. That means that you can keep using the old names, without warnings, but they are documented as deprecated, and they will eventually be removed in a later breaking change.

But in most cases, the improved name is mostly to help with understanding. It doesn’t help to trigger a bunch of additional warnings when you upgrade. Perhaps we’ll add a deprecations crate feature for those who want to clean up.

The upside is that all the methods now start with tls_. They are grouped together in the docs, but it will also help exploration with autocomplete.

Other crate feature adjustments

As this is a breaking change version, we did include one making the RequestBuilder::query() and form() methods optional features, disabled by default. With these as optional, it is now possible to build reqwest without serde.

We also made native-tls imply ALPN automatically. This will mean most people who don’t think about it will now get HTTP/2 upgrades. The reason it was opt-in previously is because older native libraries might not have the symbols required. They are quite old at this point. If you still need the previous behavior, there is now native-tls-no-alpn.

Thanks!

Thanks to all who contribute, use, sponsor, fix, complain, and help reqwest be what it is! Here’s v0.13.0.

This does drop i686-pc-windows-gnu support for the default features. The reality is that a lot of things don’t work for that target. Even Rust itself has it as a Tier 2 target. We do still support it with native-tls, though. ↩

hyper-util Composable Pools

Wed, 03 Dec 2025 08:48:00 +0000

I’m so excited to announce hyper’s new composable pool layers!¹

As part of making reqwest more modular, we’ve designed a new connection pool, and made the pieces available in hyper_util::client::pool. But this is more than just a “hey, we have a Pool, it moved over there.” We’ve literally pulled apart the pool, in a way I haven’t found elsewhere.

Building a purpose‑specific pool is now straightforward. Add the features you want, even custom ones, and skip the bloat, no forks required.

Read on to see what exactly we solved, how, and what comes next. If you just want to use them, here’s the docs. Everyone else, let’s dive in.

We started with the users

We started with the users, looking back over past issues filed, common questions in chat, and private conversations explaining what they needed to do. Boiled down, that got us to these requirements:

A full-featured pool, like the one in legacy, must be possible.
Microservices shouldn’t have to handle multiple protocols or hostnames.
Some clients need custom keys for the pool.
Others need to limit new connections made at a time.
Or cap the total number of connections.
Customize connection expiration based on idle time, max lifetime, or even poisoning.
And importantly, allow custom logic not already thought of.

From past experience combining middleware, I had a strong feeling the pool requirements could be broken up into tower layers. But what would that even look like? Would it be horrible to use?

To answer that, we took the requirements and considered the developer experience of using layers. It had to feel nice. Not just to write, but also to come back to and read.

I then sketched out several of these layers to make sure they could actually work. Once most of it was working, the proposal was ready.

The initial 4 working pools

No plan survives contact with the enemy. We originally proposed five pool types, but launch with just the following four: singleton, cache, negotiate, map.

The singleton pool wraps a connector² that should only produce a single active connection. It bundles all concurrent calls so only one connection is made. All calls to the singleton will return a clone of the inner service once established. This fits the HTTP/2 case well.

The cache pool maintains a list of cached services produced by a connector. Calling the cache returns either an existing service, or makes a new one. When dropped, the cached service is returned to the cache if possible. Importantly for performance, the cache supports connection racing, just like the legacy pool.

The negotiate pool allows for a service that can decide between two service types based on an intermediate return value. Unlike typical routing, it makes decisions based on the response (the connection) rather than the request. The main use case is supporting ALPN upgrades to HTTP/2, with a fallback to HTTP/1. And its design allows combining two different pooling strategies.

The map pool isn’t a typical service like the other pools, but rather is a stand-alone type that maps requests to keys and connectors. As a kind of router, it cannot determine which inner service to check for backpressure until the request is made. The map implementation allows customization of extracting a key, and how to construct a connector for that key.

Ineffably unstable

I knew this work would land in hyper-util first, because it’s not stable yet. Being so freshly designed, changes are expected after some more real-world usage. Still, I wanted to shield earlier adopters from breaking changes. At the same time, valuing performance and flexibility, I wanted to push as much as reasonably possible into the type system.

When initially tinkering during the summer, I had one of those thoughts. The kind that clangs like a giant lock snapping open: what about type-state builders and unnameable types? I took a side quest, and tackled the warp v0.4 upgrade, to test out this API design. That post explains it a bit more.

The various threads were all coming together.

With each pool concept a tower service, once composed, a user shouldn’t care what it is beyond being some impl Service. I tested this out in reqwest, and yea, I don’t need to name the types. While I did need a type, I was able to store a dyn Service, and inference handled the rest.

Real world usage: in reqwest

Once those main pieces seemed ready, I needed a real example to test drive them. Tool-makers that don’t use their tools make bad tools, after all.

I started by replacing the legacy pool inside reqwest. Part of the larger diff in reqwest is handling all of reqwest’s different pool configuration options.

But, putting the default case together is pretty self-explanatory:

// Note: some noise has been trimmed
let http1 = (
    pool::cache(exec),
    util::http1_request_target(),
    util::http1_set_host(),
    util::meta(MyMetaIdleAt::new),
    conn::http1(),
);

let http2 = (
    pool::singleton(),
    conn::http2(),
);

let pool_layers = tower::layer::layer_fn(move |svc| {
    pool::negotiate::builder()
        .fallback(http1.clone())
        .upgrade(http2.clone())
        .inspect(|conn| conn.is_negotiated_h2())
        .connect(svc)
        .build()
});

let pool_map = pool::map::builder::<http::Uri>()
    .keys(|dst| scheme_and_auth(dst))
    .values(move |_dst| {
        pool_layers.layer(connector.clone())
    })
    .build();

And it works! Making the full-featured pool was one of the requirements: check. But, the next part was even more important.

As I mentioned before, I punted one of the proposed types: expire. Expiration is a necessary concept to a pool. But try as I might to fit the various generic shapes, it just wasn’t happening. Thankfully, this work had a hard deadline. And deadlines keep you user-driven: let them have something now, it can always be better later.

To prove the general design allowed expiration, I implemented a specific version of it directly in reqwest.

tokio::spawn(async move {
    loop {
        tokio::time::sleep(idle_dur).await;
        let now = Instant::now();
        let Some(pool) = pool_map.upgrade() else { return };
        
        pool.lock().unwrap().retain(|_key, svc| {
            svc.fallback_mut().retain(|svc| {
                if svc.inner().inner().inner().is_closed() {
                    return false;
                }

                if let Some(idle_at) = svc.meta().idle_at {
                    return now > idle_at + idle_dur;
                }
                true
            });
            svc.upgrade_mut().retain(|svc| {
                !svc.is_closed()
            });
            !svc.fallback_mut().is_empty() || !svc.upgrade_mut().is_empty()
        });
    }
});

The ease of adding it helped solidify to me that this was definitely the right design. I was able to slot in a meta layer tracking idle time, and then use that to retain services. I placed that layer in right next to some of the other HTTP/1-specific layers. Easy!

Being modular opens up customization

With the ability to build a stack for your pool, consider an example of how we can start to solve other requirements listed earlier.

let svc = ServiceBuilder::new()
    // cached connections are unaware of the limit
    .layer(pool::cache())
    // in-flight handshakes are limited
    .concurrency_limit(5)    
    .layer(conn::http1())
    .service(connect::tcp());

It also allows adding in layers we don’t currently have, such as per-host connection semaphores, or a few layers up over all hosts. Adding new functionality isn’t blocked on us, and no one has to “pay” for features they don’t need.

I can’t wait to see what else is done with the design!

Pools ready

The hyper_util::client::pool module is now available in v0.1.19. Go check the docs, and try to build cool things. Please file issues if parts are missing, we’ll keep iterating.

I’ve been working on this feature set for long time. It’s something I started thinking about years ago, and after months of work this year, it feels awesome to finally be able to release it.

Thanks to my sponsors, retainers, and grants for making this all possible!

I mean, who isn’t excited to announce anything? /s ↩
All “connectors” are actually MakeServices, which are jsut a Service that produces a Service. It doesn’t have to create a connection, but it reads better when talking about pools. ↩

hyper User Survey 2025

Tue, 18 Nov 2025 10:20:00 +0000

I’m excited to announce the inaugural hyper user survey! (Update: it ended.)

Part of being user-driven is knowing what our users want, what they’re trying to do, and then we can figure out how best to help.

This survey should only take about 5 minutes, no questions are required, and responses are anonymous. The survey will be open for a month, until December 17th, and then we’ll analyze the results and share with you all what we find.

Thanks for taking the time to help make hyper better!

~~[Take the hyper user survey for 2025.~~

Podcast: Netstack.fm, story of Rust's networking with hyper

Tue, 02 Sep 2025 09:24:00 +0000

Last week I was a guest on the Netstack podcast. We talked abit about how I got into Rust, how async Rust developed, and the story behind hyper and its surrounding ecoystem.

We started (and ended) with my goal of better software:

On your about page, you say that “Rust is the least bad option.” Can you elaborate a bit on that?

Yeah, I love Rust. I think Rust solves a ton of problems, but I also don’t want to be stuck when something better comes along, which it inevitably will. Then yeah, I’ll move to that. To me, all these tools are the means to an end, which is to make better software.

The rest of the conversation was really fun to talk about:

7:54: beginning of hyper, async Rust
13:20: hyper as something bigger
15:36: splitting off hyper-util
17:35: the http and headers crates
22:51: motivation behind warp
29:00: reqwest as the opinionated layer
30:17: open source independence
31:31: HTTP/3 in hyper and reqwest
39:40: hyper in 5 years, contributors, ownership

hyper HTTP/2 (Didn't) MadeYouReset

Wed, 13 Aug 2025 08:25:00 +0000

A new HTTP/2 attack vector was disclosed today called MadeYouReset. hyper’s h2 is negligably affected, weathering the attack well. But, we have provided patches just in case. We published patches weeks ago, so if you’ve been keeping up-to-date, you’re fine! If not, you’re most likely fine, but you can upgrade now. No CVE or security advisory is included with this.

This sort of work is supported exclusively through retainers, get in contact to support.

Follow along for the background information, if you’re curious!

What is the attack

The attack is quite similar to the previous Rapid Reset attack from a couple years ago. That is, it starts new requests up to the limit imposed by the server. And then instead of sending explicit RST_FRAMES, it sends malformed frames that force the server to reset the stream.

So far, the described behavior is normal HTTP/2 flow. The problem occurs in some implementations that don’t respond well to quick cancelation.

As the attack describes, in some implementations it allows for more stream concurrency than might have been configured. See more in VU#767506.

What hyper does

A couple years ago, we added a defense mechanism to h2 that keeps count of how many times a frame from the remote causes a local reset, and once a configurable limit is reached, the connection is abruptly closed. This mechanism caught nearly all variants of the attack.

We had forgotten one code path dealing with WINDOW_UPDATE frames. But what happens when new streams are created and then a bad window frame is received?

In h2, the reset is sent to user code. Check the request body will notice the error. Additionally, hyper’s server code using h2 is essentially waiting simultaneously on the user’s service to reply, and for any reset to be received. Once the reset is received, the service future is canceled.¹

So, not much. When a simple hello world server example was tested against the attack, the only thing that happened was increased CPU usage. Which makes sense, it’s receiving and canceling more streams.

Stream concurrency is enforced, since all futures are canceled immediately, and a signal is sent to anything listening for longer.

All of this before any additional patch.

What we did

We received a report² about the attack from security researchers³ on May 23, 2025. The attack includes many variants, so we set out trying to determine if hyper was vulnerable. We knew that hyper already added a defense mechanism for this exact sort of abuse. Only one of the variants did not get cut off, but saw increased CPU usage, so we dug in. It turns out the original feature forgot to check the limit when WINDOW_UPDATE frames were bad.

We determined that while the variants was not caught, it also did not break any guarantees. The increase in CPU usage was obvious, since it was necessarily handling more requests in general. Also important to our determination: the attack is observable by user code, so any user code can react however they wish.

The researchers did a great job of explaining the concept behind the attack, providing a proof-of-concept script that we could test with, and conversed respectfully and thoughtfully as we went through the investigation.

Because the code of the aforementioned mechanism was already public, and the damage quite benign, we decided to release a patch weeks before the general embargo. It was just a refactor, converting it from logic-based to utilizing the type system more, so no other cases are “forgotten”. That patch was released on June 30, 2025.

For similar reasons, we also determined there was no need for a CVE or security advisory.

The researchers came back a little afterwards showing that in some cases, servers using h2 under this attack could be forced to crash. They even brough a new reproducible proof-of-concept. After analyzing it, we determined that the problem was not that of hyper/h2, but that of user code which might have unbounded queues and does not react to cancelation.

We decided to stick with our original determination.

What you can do

After reading all that, what more can you do? Again, you’re likely fine! But here’s some good ideas, regardless:

If you’ve been staying up-to-date, you’re already done!
If you have a server running hyper, it would be wise to make sure your server handlers really can cancel easily.
You can run cargo update -p h2 to upgrade to the latest versions (at least 0.4.11 and 0.3.27).
If you’re running hyper v0.14 (so h2 v0.3.x), it’s time to upgrade to v1. After nearly 2 years, patches likely won’t be backported anymore.

There’s one more thing you can consider: set up a retainer to make sure security reports get taken care of. This was a considerable amount of important work. This was several months of higher-stress but important work which doesn’t look like shipping a new feature.⁴

Besides funding that important work, I’m also happy to provide pre-disclosure and hands-on guidance to those with a retainer.

This difference from Rapid Reset is also important. In Rapid Reset, the resets were not observable by user code, so they could not react to the wasting of resources. With MadeYouReset, however, users can if they need to. ↩
I just realized today that I cannot publish the GHSA as an informational, since it does not include a severity or CVE. GitHub, please? ↩
The original reporters were Gal Bar Nahum, Anat Bremler-Barr, and Yaniv Harel of Tel Aviv University. I primarily interacted with Gal, which was a 5/5 experience, would repeat. ↩
On top of what has been outlined here, there’s a lot of coordination work involved, dealing with VINCE and other vendors. ↩

warp v0.4

Wed, 06 Aug 2025 11:05:00 +0000

Yesterday, warp v0.4 was released. warp is a Rust web server framework, with a focus on functional programming and type system routing.

Upgrading is likely pretty simple, the main API stayed very similar. The biggest deal is that it should be easier for you to stay up-to-date on dependencies. You could stop there. But if you want to know what else happened, off we go!

Filters remain the focus

The biggest raison d’être of warp is its Filter system. It was why I made a different web framework originally, and it remains the reason to consider warp even now.

Should you use warp? That depends. If you just want a standard, super fast, featureful Rust server framework, one that looks like ways you’ve coded servers before, you probably want Axum. But, if you like functional programming, and (ab)using the type system, I think warp is pretty cool.

Consider an example:

let update = warp::path!("todos" / u32)
    .and(warp::header("accept"))
    .and(warp::body::json())
    .map(|id, todo: Todo| {
       // ... 
    });

Every filter extracts something from the request (even if that’s just ()). They can all be .and()ed together in any order, it will result in natural, typed arguments you can map.

You can build up layers of filters by just putting another .and() on anything. You can make a group of filters with .or().

Once you have combined all into a master filter, you can convert them into a tower Service, which you can then add any other middleware from other libraries, and serve with an HTTP implementation, such as hyper.

Upgraded to hyper v1

The underlying HTTP dependencies, those which are public, have been upgraded to hyper v1.

This should improve interoperability if used with other libraries such as reqwest or tower-http. Their latest versions depend on v1 of hyper and http. So adding in more middleware, or using a higher-level HTTP client should mean less dependencies compiled, and no conversion required between types.

It also means warp users will be able to stay on maintained versions of hyper and its dependencies.¹

Crate features on a diet

In previous versions, a few heavy features were enabled by default. Now, the default features are much slimmer.

Keeping with filters being the focus, the default features only include the Filter system, and a way to convert them into an impl Service. You can take that service and use it with your own build of hyper, with HTTP/1 or 2 or whatever enabled, and warp won’t care.

If you want the simple server runner, just enable the server feature, and then warp::serve(filters) is back, along with all the hyper implementation to run it.

Likewise, the warp::test helpers have been put behind a test feature. You can enable it in your dev-dependencies for testing, and while keeping that code out when building for release.

The tls feature was dropped completely. It existed previously as a simple way to add TLS to your server, but it’s not strictly required in a server framework, and it adds a bit of maintenance churn to stay up-to-date and safe. Anyone wanting TLS with their warp server can make an accept loop using the TLS implementation of their choice, and use warp’s filters as a Service.

Finally, the multipart and websocket features were removed from default, but are still available to enable. This was a planned change once they became features in the first place, since they enable a lot of code that you might not need.

Iterating on the Server builder

The Server API was, uh, iterated on.

Before, it had a whole bunch of bind_*, run_*, and serve_* methods that slowly grew as people wanted more options to configure the server. Now, most all of them are gone.

Part of the reason is because it’s not the focus of warp. The Filter system is. I figured the Server is meant to be a simple way to get your filters serving requests. If you wanted more customization, make your own.

But there’s another reason, too.

I’ve been noticing more and more places where I wish the API of some crate was at the same time simpler, and also more powerful, putting options into the type system. The warp Server was an excellent place to iterate on this.

It is now an unnameable builder making use of expandable type state.

It encodes as generics into the Server<F, A, R, Etc...> various parts of how the server works. The filter, the acceptor, the runner loop. Depending on what methods you call, new ones get unlocked. For instance, bind() will make the acceptor a TcpListener, but once one is set, it doesn’t make sense to be able to call it again.

The type is rendered in the docs, but otherwise not publicly exported, so you can’t name it. There’s a benefit to this, as the API designer and maintainer. I can add more options, needing more generics, without it being a breaking change. As long as add a default that keeps it all compiling, there are now instances of Server<F, A, R> that need to be updated to Server<F, A, R, E>.

And the reality is that the name of the type is not generally interesting to you as a user. You just need a fluid way to add options, and once it’s all combined away, you just await a future.

I’m hopeful this pattern carries over well to other builders as well.

I’ve backported important bug fixes to hyper v0.14 (and dependencies like h2), but I think that’s about done. It’s been almost 2 years, time to focus on v1. (If you need help, reach out to me!) ↩

reqwest retries

Tue, 15 Jul 2025 10:39:00 +0000

One part of working on a more modular reqwest was adding support for retries.

Intermittent failures are a fact of life, and a simple retry could have made things so much better. It’s a somewhat frequent feature request for reqwest. It’s also a very common pattern to implement manually, and far too easy to do incorrectly. Since reqwest’s internals are shifting more and more to a stack of services, it felt like an ideal time to explore this.

While skeleton pieces do exist in tower::retry, they require Expert Mode. Doing things right is tricky. We can do better.

Policy composition

A retry policy is several things at once. But each of those things often is only one of a few choices. So what if we made it easier to compose your choices? That is, we can provide the common choices, and a builder to combine them.

Budgets by default

Retry counts alone are insufficient, but absurdly common. Please don’t do this. We can educate and document and warn and wave our hands, but we can only plead so much.

Budgets will be default on. Sure, there’s an escape hatch if you absolutely must. But this way we can help you fall into the pit of success.

And to help users better understand what the budget results in, configuration looks like .max_extra_load(0.3), instead of implementation parameters of the windowed bucket.

Scoped retries

Budgets are basically a must. But budgets also don’t make sense across different targets. Just because requests to frobnica.te start failing doesn’t mean retries should also be disabled to storali.ze. This dichotomy perturbed me quite a bit, actually, and so I put off trying to solve the whole thing for a few months. But after letting my mind rest, looking at the exact same problem revealed an obvious solution: scopes.¹

I believe the most likely use of scopes is based on hostname. So that’s the easiest constructor to start a builder: retry::for_host(host). But it’s actually a sealed trait, to allow continued exploration. Maybe a closure is needed by some people? We’ll find out!

Cloning requests

In many languages, this isn’t even a consideration. You can just share a pointer, and wave hands mostly reuse it.

In Rust, with ownership, we need to keep a clone of the request for a potential retry.

At the hyper level, we don’t know enough about the request body to automatically clone it. At reqwest’s level, we do know enough: we can absolutely clone when it’s a simple non-streaming body. (Even possibly more, discussed below.)

Classifiers

You might have a very custom combination of routes and responses. Or an API where every idempotent request has standard status codes. Or you might need a deeper understanding of specific connection or protocol errors.

So to start, there’s a simpler classify_fn option, where you provide a closure to determine whether to retry or not.

.classify_fn(|req_rep| {
    match (req_rep.method(), req_rep.status()) {
        (&GET, Some(SERVICE_UNAVAILABLE)) => req_rep.retryable(),
        // or check req_rep.error()?
        _ => req_rep.success(),
    }
})

Another possible avenue to explore is the composition of classifiers. For example, what if you combined something like protocol_nacks.or(idempotent.and(service_unavailable))? Is this a good idea? Would people want this? If you want to use something like that, file an issue or PR.

Probable enhancements

Besides the above, there’s a few other things we could add to reqwest::retry:

Backoffs: They’re pretty easy to add, though the benefit they bring is only really noticeable in specific situations. This is why they aren’t included yet or would not even be enabled by default.
Replayable Body: Retrying needs to clone the request, and a streaming body is not freely able to be cloned and restarted. But it’s desirable by users, so we could make it an easy configurable option, such as .max_replay_body_bytes(). We could likely learn a bit from how Linkerd implemented it.
Per-Request Retries: More and more, users ask for configuration for a specific request that differs from the Client. We’ve been working on general per-request config system.

Exploring and upstreaming

We’re exploring these ideas in reqwest, with the goal of upstreaming the more generic ideas to tower or tower-http.

Maybe that’s a builder:

tower::retry::builder()
    .clone_reqs(|req| Some(req.clone()))
    .matcher(|req, res| res.status() == RETRY_AFTER)
    .max_extra_load(0.3)
    .build();

Or maybe something closer to middleware, a trait to compose pieces, with and() or .or():

tower_http::retry::clone_replayable(8192)
    .and(idempotent)
    .and(server_error)
    .or(protocol_nack)
    .and(budget(0.3))

Maybe a combination! Who knows?

Check it out

There’s a pull request implementing some of these ideas, and your feedback and usage will help us to keep refining it.

The two-pronged goal is to allow you to easily get retries in reqwest, and to also be able to get that easier configuration in any tower stack.

Give a whirl!

Scoping budgets wouldn’t be something to care about if the client were already built for a specific endpoint. That’s a common way to build a stack of tower layers. But reqwest is open-ended, you can send requests to anywhere with an address. So, scopes felt required. ↩

Body::poll_progress

Tue, 22 Apr 2025 10:01:00 +0000

This describes a proposal for a cancelation problem with hyper’s request and response bodies. hyper is an HTTP library for the Rust language.

Background: what is the `Body` trait?

The Body trait used by hyper is meant to represent a potentially streaming (asynchronous) body of a request or response. It sorta looks like Stream or AsyncIterator.

The biggest reason for a different trait was because we needed a stable trait for hyper v1, and Stream/AsyncIterator were not going to be (and still aren’t) stable. With it being a separate trait, though, we also added some HTTP-body-specific methods to it.

Its similarity to Stream/AsyncIterator means we run into a problem with forwarding into a sink. But that we own the trait also lends to a decent solution, which this post outlines.

First though, piping woes.

Problem: backpressured cancelation

Piping stream-like things into sink-like destinations feels natural, and looks quite simple. Elegant even. They can be piped together, and backpressure occurs naturally. However, they lack a mechanism to completely propagate cancelation. More specifically, cancelation while backpressure is currently applied.

Consider an example:

while let Some(frame) = body.frame().await? {
    // what if body (^) cancels while we wait here?
    dst.send(frame).await?;
}

This simple loop is piping a Body into some sort of sink. The way most streams and channels work, this mostly propagates backpressure. As long as the dst is not able to send a frame, we don’t poll for another one, and whatever is the source of the body will back up.

However, it has a flaw. Those familiar with writing proxies may notice it immediately. The destination might not have space, and so we’ll wait for it to get more. But the body could give up during that time. Since the task is only waiting on when the dst is ready, it could wait a significant amount of time and never notice that the body (source) has canceled.

Why not just timeouts?

One initial question was “why not just timeouts”. Like, why not just add body_write_timeout() or something to hyper’s connection builders. Fair. That would work in some cases, for sure.

The thing is, sometimes timeouts are inappropriate.

For example, a transparent proxy may not want to force timeouts where they didn’t exist before. They’re willing to wait nearly forever. They just want to cancel the forwarding if the sender gives up.

Additionally, a chain of potential timeouts means that cancelation propagation can be delayed longer and longer, as each hop in the chain has to wait its own timeout.

More generally, yet not

This issue can also exist for Stream/AsyncIterator. A poll_progress idea was outlined by withoutboats, though it was solving a different problem.

I do think there’s some crossover that may affect each other. But it’s also worth considering separately.

I’m not talking about a concept that some for await syntax could magically care for. I don’t want that. Like at all. In the above example, it’d be impossible for a compiler to determine which action I want to take. Maybe the source ended cleanly. I don’t want that to cancel sending the last item.

Also, and this is a biggie: Body will need to solve this problem quicker.

Solution: `Body::poll_progress()`

We propose to add a method to the Body trait:

pub trait Body {
    // existing methods ...
    
    fn poll_progress(
        self: Pin<&mut Self>,
        cx: &mut Context<'_>,
    ) -> Result<(), Self::Error> {
        Ok(())
    }
}

The purpose of the method is allow for it to make any “progress” that might result in cancelation, while not producing another frame. The details:

It’s a poll_* function, so it allows a body to poll something else async, like a timeout, or perhaps some shutdown channel.
Returning Ready(Ok(())) immediately is fine, it simply means there’s nothing that the body would do before another frame could be made available.
However, returning an Err would indicate that the body is canceled, and a piping task can interpret that to shutdown the forwarding operation.

We could provide some utilities to aid use of this function, such as http_body_util::BodyExt::progress().

Consider an example, something that could be even be used in hyper internals, but could also occur elsewhere:

loop {
    // produce a frame
    let frame = select! {
        Some(frame) = body.frame() => frame,
        dst.hup() => {
            // unexpected EOF
            return;
        }
    };

    // send it
    select! {
        dst.write(frame) => {
            // continue
        },
        Err(e) = body.progress() => {
            dst.abort();
            return;
        }
    }
}

Alternatives: not `poll_closed`

An alternative we considered was making this method be a poll_closed. It had the welcome effect of meaning we could await body.closed(), and that is fairly self-documenting. However, it ran into several problems.

Being an addition to an existing trait, it needs a default implementation. But a default for poll_closed would likely end up being confusing. If it returns closed by default, then existing impl Bodys will suddenly start aborting early. If it returns not-closed, then any naive task that might await body.closed() would hang forever.

The return value also felt confusing however we put it. Does closed return a Result? Both Ok and Err would still mean the body had closed.

Request for Comments

I wrote this up because it felt like a big enough change to a fundamental mechanism in the ecosystem that it’d could benefit from more eyes and comments. It kinda-sorta looks like AsyncIterator::poll_progress, but it’s also not. Still, are there things we should prepare for? Would wg-async have thoughts? Others who are using hyper deeply?

Could it be simpler? Is it flexible enough?¹

Comments on the pull request are most welcome!²

Calls for completely changing hyper’s design to not need such a function aren’t helpful. hyper is stable at v1. ↩
Thanks to Steven Fackler and Oliver Gould for helping iterate on this design. ↩

A More Modular reqwest

Tue, 04 Mar 2025 13:05:00 +0000

reqwest came out 8 years ago. It was meant to be a higher-level HTTP client, with batteries and opinions included. And it continues to fulfill that role very well.

To do so, reqwest combined or implemented a lot of features that weren’t easily accessible elsewhere in Rust. Things like redirect handling, connection proxies, and compression.

People love it. It’s the most popular HTTP client used in Rust code. As of today, it gets downloaded 10 million times per month from crates.io (which doesn’t include any internal mirrors). It helps engineers all around the world.

People also wish they could better customize it. They want to add in custom logic somewhere. This makes sense. And the ecosystem has gotten even better at reuse. It’s time for reqwest to help be more reusable.

First, what is the goal?

Easy and modular

Top priority is that reqwest remain easy to use. It should be the best default for most people. If you don’t know what customizations you want, you shouldn’t have to care.

It should also be possible to customize. To tinker. To grab just a single piece. To re-order the pieces. To add in completely new pieces.

Some concrete work

So, how do we get there? Well, here’s a start.

The connection pool can be changed from a single Pool type into a series of Services that allow better composition, such as a racing cache pool, singleton pool for HTTP/2, an Either pool to combine them. This allows people to use less complicated pools than one that includes connection racing. It makes way for using tower’s load balancing services directly within reqwest, even.

The HTTP Proxy internals can be refactored to be another Service which fits into the series of “connector” services reqwest uses. These include loading proxy data from environment variables, and providing HTTP/HTTPS/SOCKS proxies and tunneling. Other applications that only use reqwest for proxying can depend on only those utilities.

reqwest supports some basic redirect policies, and allows for custom ones. Eventually, tower-http added middleware to do that too. We can now use redirects from tower-http. It’d be better to update reqwest to depend on that middleware internally, allowing reqwest’s test suite to find any pieces lacking in the port.

We can add easy request retries. reqwest doesn’t really have retries, other than a very specific case: special HTTP/2 error cases. And conceptually, retries aren’t that different from redirects. Retrying requests is pretty common thing to ask for, and it’s pretty easy to do them wrong. There’s middleware in tower::retry, but it’s not easy. We could make it as easy as calling .retries(reqwest::retry::idempotent()) on a client builder. Or a few other common policies.

We can use decompression from tower-http. reqwest has automatic response body decompression. Eventually, it was added as middleware in tower-http. Instead of having the feature essentially duplicated, we can just depend on the modular version, and the test suite will keep the quality up. Doing it this way improves the middleware for everyone. For instance, when one user swapped from reqwest’s decompression to using tower-http, performance got worse. Eventually, a reused allocation was backported, bringing it back in line.

Better default TLS

Initially, reqwest made use of native-tls for HTTPS support. Eventually, we added optional support for rustls. But we’ve kept native-tls as the default. The reason was because it worked for more users, and more sites. However, I think it’s time to change the defaults.

At this point, reqwest’s defaults actually target the worst common denominator. It not longer feels right to make people opt-in to the better choice. As I said at the beginning, reqwest bundles opinions too.

Our opinion is that reqwest should have the best defaults for the most people. Those that need something different can opt-in to that.¹

So, reqwest will be changing it’s default-tls feature to rely on rustls. Likely with aws-lc-rs as the crypto backend. Optional support for native-tls will remain, for now.

The default-tls feature theoretically was designed to allow for such a change, since nothing should be exposed with it that isn’t supported by rustls. But they do have different build requirements, so it won’t be 100% smooth.

But it will be better for most people.

Getting to work

That’s the work. Keeping reqwest easy. Allowing more customization. Even letting people use fewer parts of it. While having strong opinions. Making it better for most people.

Want to be part of it? Help contribute code or documentation. Here’s the issue plan.

You can also sponsor the work. Roadmaps and priorities like these come about from conversations with companies that keep retainers.

All of it helps! To making HTTP requests better!

This could also be applied to other features too. I think there’s an argument to be made for switching the default DNS resolver to use hickory-dns instead getaddrinfo. ↩