seanmonstar

warp

Over the past several months, I’ve been working a web framework in Rust. I wanted to make use of the new hyper 0.12 changes, so the framework is just as fast, is asynchronous, and benefits from all the improvements found powering Linkerd. More importantly, I wanted there to be a reason for making a new framework; it couldn’t just be yet another framework with the only difference being I’ve written it. Instead, the way this framework is used is quite different than many that exist. In doing so, it expresses a strong opinion, which might not match your previous experiences, but I believe it manages to do something really special.

I’m super excited to reveal warp, a joint project with @carllerche.

Background

What makes warp different?

I’ve been working on web servers for years. Before coming to Rust, I did several things in PHP, moved over to Python, and then shifted again to Nodejs. I’ve tried many frameworks. I found that I often times need to configure predicates, like certain headers required, query parameters needed, etc, and sometimes, I need to configure that a set of routes should be “mounted” at a different path, and possibly want certain predicates there too. I noticed the concept of mounting or sub-routes or sub-resources or whatever the framework calls them didn’t feel… natural, at least to me. It frequently felt like a secondary concept, occasionally not having all the power that a standard route does.

I’ve also been working in Rust for several years now, and what kept using the language was its powerful type system¹. The more I wrote Rust, and learned how amazing the “fearless refactoring” is, the more I hated working in dynamic languages (in my case, it was a large Nodejs server), as trying to refactor pieces inevitably would remind us (in production) that our supposedly comprehensive test suite still had holes in it. I wanted app-specific types to save me from shipping bugs.

A few months ago, I found the Finch library in Scala, and shortly after, Akka, both of which instead just treat everything as a sort of function converting from input to output, and from there, you just chain together these different pieces, and they compose and reuse really well. Scala also has a powerful type system, and those frameworks embrace converting information from HTTP messages into app-specific types. I fell in love.

The thing that makes warp special is its Filter system.

Filters

A Filter in warp is essentially a function that can operate on some input, either something from a request, or something from a previous Filter, and returns some output, which could be some app-specific type you wish to pass around, or can be some reply to send back as an HTTP response. That might sound simple, but the exciting part is the combinators that exist on the Filter trait. These allow composing smaller Filters into larger ones, allowing you modularize, and reuse any part of your web server.

Let me show you what I mean. Suppose you need to piece together data from several different places of a request before your have your domain object. Maybe an ID is a path segment, some verification is in a header, and other data is in the body.

let id = warp::path::param();
let verify = warp::header("my-app-header");
let body = warp::body::json();

Each of these is a single Filter. We can combine them together with and, and then map the combined result to get a really natural feeling handler:

let route = id
    .and(verify)
    .and(body)
    .map(|id: u64, ver: MyVerification, body: MyAppThingy| {
        // ...
    });

The above route is a new Filter. It has combined the results of the others, and provided their results naturally to the supplied function for map. Additionally, the types are enforced, cause well yea, this is Rust! If you were to change around one of the filters such that it returned a different type, the compiler would let you know that you need to adjust for that change.

This combining of results is smart : it is able to automatically toss results that are nothing (well, unit, so ()), instead of passing worthless unit arguments to your handlers. So if you needed to combine a new Filter into this route that only checks some request values to determine if the request is valid, and otherwise returns nothing, your handler doesn’t need to change.

Besides dropping units, did you notice how even though multiple results were combined together, the map closure received each as individual arguments? This greatly improves development, since that means that id.and(verify).and(body) is actually exactly the same as id.and(verify.and(body)), but using just tuples would have changed around the signature of the results. The routing documentation shows more ways this is useful.

This concept powers everything in warp. Once you know you can match a single path segment via warp::path("foo"), then the idea of mounting doesn’t need to be something special. You just have your filter chain for a set of endpoints, and simply “and” it with a new path filter. If your “mount” location needs to also gate on headers, or something else, you can just and those Filters as well.

Built-in functionality

As awesome as the Filter system is, if warp didn’t provide common web server features, it’d still be annoying to work with. Thus, warp provides a bunch of built-in Filters, allowing you compose the functionality you need to descibe each route or resource or sub-whatever.

• Path routing and parameter extraction
• Header requirements and extraction
• Query string deserialization
• JSON and Form bodies
• Static Files and Directories
• Websockets
• Access logging
• And others, and more being added.

The docs explains how to use each, and the examples go more in-depth on how to combine them to make actual web servers.

tower-web

A few months ago, there was mention of a web framework, tower-web, that’d be coming soon. The concept behind it is to provide a web framework built around tower’s Service trait. That is still coming. warp is being released right now for a couple reasons:

1. The Filter system is really awesome, as touched on above.
2. To explore some ideas before solidifying tower and tower-web. We’d like for warp to be able to make use of all the great tower middleware that already exists.

Expect to hear more about it, and how it fits with warp, soon!

warp

This is warp v0.1. It’s awesome. It’s fast. It’s safe. It’s correct. There’s documentation, and examples, and an issue tracker to file bugs and track progress of new Filters that are coming (CORS almost ready). I want to thank those of you who tried warp out privately and sent feedback in, it was super valuable!