Chris Morgan’s blog posts tagged Rust

U+: pretty Unicode code point literals for Rust

2022-04-01T00:00:00+00:00

Stop worrying about whether char literal syntax uses '\u{1234}', "\u1234", \x1E\x88\xB4 or something else, and use the True Unicode Syntax of U+1234!

I was reading https://uncyclopedia.com/wiki/Rust_(programming_language) for some reason, and read let U = 0; U = U + 1;.

Suddenly my mind was awhirl with a Concept. I implemented it at once.

The problem

Unicode expresses its code points in syntax like U+1234 (full range U+0000–U+10FFFF).

But then when you want to transfer it to a programming language, you have to learn another syntax. Will it be '\u1234', '\u{1234}', "\x1E\x88\xB4", \341\210\264, something else?

And then astral plane characters make it even worse: "\U0001F631", '\u{1F631}', \xF0\x9F\x98\xB1, "\uD83D\uDE31" (with all the associated pain the abomination UTF-16 entails, especially that your char type may simply not be able to represent this), something else?

And so here is this crate that lets you use the True Unicode Syntax in Rust:

use u_plus::U;

assert_eq!(U+1234, '\u{1234}');

So forget about \u{…} syntax and use U+… literals!

(Caution: there are some limitations with this approach, see KNOWN_ISSUES.md for details.)

Links

The u-plus crate on crates.io (or see on lib.rs).
The source code is on my Git server [Pity about that u-plus in the URL. I wanted to do U+, but gitweb wasn’t coping very well with plusses in the path, and I figured fixing that bug would take far too much effort, especially given the era that Perl code comes from. I’ve already patched it enough, I should just write something from scratch.]; it may be easiest to view the entire thing as a patch.

`#[cfg_attr(…, path = …)]` for platform-specific module implementations

2021-06-18T00:00:01+00:00

I wrote about #[cfg_attr] and some interesting use cases for it some years ago. Time for another dose!

I have seen people write things like this often:

#[cfg(unix)]
mod unix;
#[cfg(unix)]
pub use self::unix::*;

#[cfg(windows)]
mod windows;
#[cfg(windows)]
pub use self::windows::*;

… and commonly even more variations …

Figure 1: multiple mod and use statements in order to switch between platform‐specific implementations.

This can get unwieldy.

There are two features that you can combine to make this simpler and smaller:

The #[cfg_attr] attribute, which lets you apply an attribute conditionally, and
The #[path] attribute on mod statements, which lets you specify the path to the module rather than using Rust’s usual lookup (where for mod foo; it tries foo.rs and foo/mod.rs).

Here’s how it goes when combined:

#[cfg_attr(unix, path = "unix.rs")]
#[cfg_attr(windows, path = "windows/mod.rs")]
… and perhaps even more variations …
mod platform;
pub use self::platform::*;

Figure 2: just one mod statement decorated by multiple attributes in order to switch between implementations, and one use statement.

And so in this example, on Unix platforms it’ll load unix.rs as the platform module, while on Windows it’ll load windows/mod.rs [FYI, if you went for windows/mod.rs because windows contained other modules inside it, know that from Rust 1.30 onwards you can actually have a file named windows.rs beside that windows/ subdirectory, instead of it having to be windows/mod.rs. The Rust Reference even recommends switching to this new style, though I’m not sold on it.] as the platform module, and so in either case it can subsequently do a reexport if desired.

Mind you, for complex enough specifications, even the single mod version can still be messy:

#[cfg_attr(
	all(
		target_arch = "wasm32",
		any(target_os = "emscripten", target_os = "unknown"),
	),
	path = "wasm32/mod.rs"
)]
#[cfg_attr(windows, path = "windows/mod.rs")]
#[cfg_attr(
	not(any(
		all(
			target_arch = "wasm32",
			any(target_os = "emscripten", target_os = "unknown"),
		),
		windows,
	)),
	path = "common/mod.rs"
)]
mod imp;

Figure 3: a very slightly unwieldy situation seen in os_str_bytes.

(While thinking about os_str_bytes, I wonder what the chances are of ever convincing the Rust core and libs teams to make cross‐platform OsStr → [u8] and OsString → Vec<u8> conversions (ideally plus checked and unchecked conversions for the other way). As it stands, OsStr::to_str forces them to be roughly UTF-8 already, and the current implementation depends on compatibility, which I can’t honestly imagine ever changing, but the docs stubbornly insist on being mysterious, and the source code stubbornly insists of OsStr, OsString, Path and PathBuf that their “representation and layout are considered implementation details, are not documented and must not be relied upon”. But such musings as these don’t fit in a figure caption. I can already just tell that half the comments about this article are going to be about this little side point.)

And truly they can get much worse than that; the worst cases I’ve seen might even be enough for me to reach for cfg-if (which I am loath to do—I reckon the significant majority of its uses are unwarranted) and separate mod statements, rather than using #[cfg_attr].

`<_>::v::<_>`

2019-08-20T00:00:00+00:00

A fun little piece of Rust artwork.

Following on from someone’s discovery that <_>::method() is valid, I got thinking about the left‐swimming and right‐swimming turbofishes, and decided that making an owl’s face out of it, <_>::v::<_>, should be possible. Here’s what I came up with:

<_>::v::<_>, the owl is sleeping soundly. Playground link

type O = u8;

trait V {
	fn v<T: Default>() -> (T, Self);
}

impl V for O {
	fn v<T: Default>() -> (T, Self) {
		(T::default(), 0)
	}
}

fn main() {
	let owl = <_>::v::<_>;

	println!("{:?}", owl() as (O, O));
}

When you call the owl, it wakes up.

(0, 0)

(Fun fact: the artwork of the awakened owl happened by accident. For inference to work, I needed both the Self type and one generic type to be in the return type, so (Self, T) was the obvious type to use. Then, integers seemed the natural types to implement things on, and zero is the easiest number to use. And so I ended up with (0, 0), which I then realised was a picture in itself! The O type alias is then just to mix things up a bit. Empty tuples would work as well, yielding ((), ()), but that didn’t feel quite as nice.)

Changelog

2019-08-20: code simplified a tad from this, as demonstrated by /u/TROPAFLIGHT2. (Look at /u/ThomasdH’s variant too, it’s fun.)

Tween: a middleware library experiment

2016-06-01T00:00:00+00:00

This article is intended for people who are familiar with how web frameworks such as Django and Iron work; if you’re not familiar with such things you probably won’t get much out of this particular post. I’m not going to be explaining everything you might need to know.

Personal update (for those interested)

I’m steadily getting back into R&D for a Rust web framework. I’ve been mostly off the radar for most of the last year, but I haven’t left Rust! I’ve been using it in my spare time, I just haven’t had much of that to dedicate to it, and I haven’t been doing much research that would be directly useful to others. Now at last I am, so here it is.

After this, one of the next things I’m intending to tackle is a templating language inspired by ASP.NET Razor—or Play Framework’s Twirl, which is much the same thing for Scala. I like statically typing all the things and am not fond of the handlebars and mustache templating engines for that reason, however much I like my own moustache. Lots of nice correctness and performance wins to be had. We can talk about that at some point, and if anyone wants to help out there, it’d be nice—I haven’t worked with rustc much before, this will end up a fairly heavy-weight compiler plugin with lots of parsing and all to do. Should be fun.

One of these days it’ll get to the stage where I finally start making the web framework that I came to Rust in the first place a few years back to make, rather than just making components for it. I’m hoping we’ll have a viable compile-to-JS story by then so I can proceed to experiment with an end-to-end Rust web app.

The existing state of middleware libraries in Rust

I’ll use iron::middleware as an example of the current state of middleware in Rust:

The request and response objects have slots for adding data to them: Request.extensions and Response.extensions, which are TypeMaps;
Middleware can inject things into the request and/or response objects, or not, as the fancy takes you, and read from it to get data from previous middleware or the main handler in post-processing;
There is no type-level dependency tracking, so you can’t say “such and such a piece of middleware must have been executed first”;
Putting more data into the extension location requires allocating a new trait object each time, and accessing it is dynamic dispatch. Not slow, especially compared with what a dynamic language would be doing, but not as fast as if you wrote the whole stack by hand.

There are two main problems I wish to outline with all of this:

Confidence. Lack of dependency tracking means that you end up with all sorts of places where you might end up panicking, and that it’s easy to make mistakes. Debugging can also be difficult, because your extension data is approximately opaque and so you can’t easily examine or print out its contents. (Admittedly, Debug could be added as a bound in the TypeMap, MOPA-style, which would reduce its opacity to debugging.)
Performance. Heap memory allocations and indirection everywhere. Imagine if you just had a simple struct containing all of this stuff, and so no heap allocations or complex lookups—just straight field lookup. This turns something from being fairly fast to being fast.

Encoding middleware dependencies into the type system is something that we (people such as myself and Jonathan Reem) tried two years back, but the Rust type system wasn’t quite good enough back then: we collectively came to the conclusion that it just wasn’t possible, and probably wouldn’t be until the language gets HKT.

This has annoyed me from time to time. Sure, they’re not necessarily serious problems, but I do want to solve them.

I came up with a new approach last week or the week before, and decided to have one last try. Well, that particular approach didn’t work (I think—I’ve actually forgotten most of the details of what it was now), but a part of it did and it headed me in another direction and I wound up with something that got steadily more and more complex until it worked, and then I steadily pulled bits out of it until I wound up with a drastically simplified core that also worked. Then I just had to continue cutting the macros down (Quxxy helped me with the subcontexts one, thanks!) until the whole thing of a middleware chain was just one macro with only one extraneous piece of information per middleware in the chain (and when we get eager macro expansion, there will be no extraneous information in the definitions).

The end result: Tween

The end result is a proof-of-concept middleware library for web frameworks. I’ve called it Tween, because of a certain fondness I have for Pyramid, which calls ’em that. (Aside: using traversal for URL routing instead of dispatch is really interesting. If you’re not familiar with it, you should try it purely to broaden your horizons. I don’t care whether you actually end up using it or not.)

Tween’s key features are robustness (no cause for panicking) and performance (no need for heap memory), and static tween dependency declaration.

The repository is a work in progress, a proof of concept. It has proven its concept, I believe.

What to do

An explanation of how it works is there in that README.

This is a proof of concept. For the sake of performance it plays fast and loose with memory safety internally, and although I believe it handles everything appropriately even in case of panicking, it’d be good to have review by others. I could easily have missed something. There could also be better ways of doing some parts of it.

If you want to help me by giving feedback—good things and bad things about the design, issues I haven’t thought of, ideas of ways of improving it, &c.—please comment in /r/rust, file an issue in the tween repository, or drop me a line at me@chrismorgan.info.

And no, I haven’t written any benchmarks yet.

Rust ownership, the hard way

2015-05-12T00:00:00+00:00

In order to truly understand Rust, one needs to understand its crucial differentiating feature: ownership. Let’s go through it in detail the hard way.

This article is an introduction to Rust’s concept of ownership. It’s designed for someone who is already a programmer but who is not especially familiar (maybe even not at all familiar) with Rust. It doesn’t attempt to explain all the concepts it deals with, but for the most part it should be clear enough. If you’re a beginner you may also find that you don’t understand various parts of it; in such a case, you might like to hold the article while you go and learn more about them elsewhere; you’ll find more value in coming back to this article after.

This concept of ownership is the pivotal part of Rust; it’s the part that makes its combination of efficiency and safety possible. While the rest of Rust is pretty similar to what you’ll find in mainstream languages [OK, so pattern matching and algebraic data types aren’t widespread in C‐style languages, but they’re conceptually simple.], these concepts of ownership and lifetimes are different from anything in any mainstream language, so it’s likely to be the part that you’ll spend the longest trying to grok; if you don’t give up but persist, then when it all clicks, working in Rust will be a joy, and you’ll have unlocked a marvellous ability to reason about code. You’ll also probably wish you could transplant a lot of the aspects of Rust’s ownership model to other languages you deal with. [Seriously. The ownership model makes reasoning about many things so much easier, and I really do miss it when working in other languages.]

This article deals with the concepts and theory of ownership and lifetimes; it doesn’t provide much in the way of practical usage examples; those you can find elsewhere. But it does explain all the rules that go to make Rust’s ownership model what it is. (It is called “the hard way” for a reason.)

Well, on with the first of our four rules:

Each object can be used exactly once. When you use an object it is moved to the new location and is no longer usable in the old.

Later on we’ll deal with a couple of features that make this model more palatable, but for now we’ll skip them, considering it at the most basic level.

struct A {}

fn main() {
    let a = A {};
    let b = a;
    let c = a;
}

error[E0382]: use of moved value: `a`
 --> <anon>:6:13
  |
4 |     let a = A {};
  |         - move occurs because `a` has type `A`, which does not implement the `Copy` trait
5 |     let b = a;
  |         - value moved here
6 |     let c = a;
  |         ^ value used here after move

error: aborting due to previous error

The A instance placed in the slot a is moved to b, rendering a unusable.

When an object passes out of scope, it is destroyed and is no longer usable.

Blocks, delimited by curly braces [But not to be confused with struct declarations like struct A {} and struct literals like A {}, as seen in this next example.], introduce a new level of scope, and anything declared inside a block (e.g. the binding x in let x = y;) will live until the end of that block. [This was strictly true when I first wrote this article, but non‐lexical lifetimes landed in Rust 1.31 in 2018: in certain situations objects can now be destroyed before the end of their containing lexical scope; but I don’t think this is the right article to delve into detail about it. Suffice it to say that although it makes Rust conceptutally more complex, it makes it do what you want (rather than producing a compiler error) more of the time.]

struct A {}

fn main() {
    {
        let a = A {};
    }
    let b = a;
}

This example won’t work, because a has fallen out of scope and been destroyed by the time we try to assign it to b:

error[E0425]: cannot find value `a` in this scope
 --> <anon>:7:13
  |
7 |     let b = a;
  |             ^ not found in this scope

error: aborting due to previous error

When an object passes out of scope and is destroyed, if the type implements Drop, that destructor will be run. I won’t get into the details of destructors here. Types like &T and &mut T (immutable and mutable references) do not have destructors.

Blocks can produce a value which goes up one level of scope.

As a language feature, this is typically known as expression orientation, as distinct from statement orientation. It’s a fairly simple concept and will seem perfectly natural to users of languages like Ruby, though to users of languages like C++ and Python it may seem a little odd. (I came originally from a Python background; at first I thought it a gimmick useful only for skipping the word return on the last statement of a function, but I rapidly discovered it isn’t a gimmick at all; it’s a very useful feature, for all that it wouldn’t suit a language like Python.)

The main rule is simple: the last expression in a block is the value that the block produces. (A block is thus an expression too.) There are a couple of other rules to deal with the corner cases:

An empty block produces () (unit).
If a block’s contents ends with a semicolon and the end of the block is reachable, the block produces ().

Here’s a simple example:

let a = {
    // … do anything we like …
    1
};

Here, a ends up storing the value 1. Given these rules, you can see that putting braces around an expression changes nothing, for at each added level the block contains only a single expression, which is then its own value. Hence, let a = { { { { 1 } } } }; and let a = 1; are equivalent. [They’re actually not quite equivalent in general, for a block expression is an rvalue; thus string[..].is_empty() works, while { string[..] }.is_empty() doesn’t.]

In languages without this feature, the only type of block that produces a value (if you’ll allow my sloppy terminology here) is a function; there, they have the return keyword to fill in this blank. (Rust also has the return keyword to make early return more convenient, but the language would work fine without it—it’d just be harder to write some sorts of code.)

fn foo_the_c_way() -> i32 {
    return 1;
}

fn foo_the_rust_way() -> i32 {
    1
}

Because of this paradigm of expression orientation, the concept of special ternary expressions [condition ? a : b in most languages; a if condition else b in Python.] is not necessary in Rust, for an if expression can do the job just fine:

let x = if a { b } else { c };

let x = if a {
    // … do things …
    b
} else {
    // … do things …
    c
};

All objects have a lifetime which constrains which scopes they may be moved out of.

Now we’re getting into the harder stuff, the biggest thing that’s unusual about Rust: lifetimes. Syntactically, a lifetime in Rust is any identifier with the prefix of a single quotation mark, e.g. 'a. Any name will do for a lifetime, but there is one special lifetime, 'static, which means that an object contains no non‐static references. Most of the primitive types (i32, bool, str, [T], &c.) are static; those that are not are:

Arrays ([T; n]) and slices ([T]) of non‐static types;
Tuples with non‐static members;
Non‐static references, viz. any &T or &mut T except &'static T and &'static mut T.

Lifetime positions in types

There are four places where a lifetime can appear in a type:

&'a Type: the lifetime of an immutable reference;
&'a mut Type: the lifetime of a mutable reference;
Type<'a>: a generic lifetime parameter on a type;
dyn Trait + 'a [Before Rust 1.27, this was spelled without the dyn keyword, called a bare trait object; that syntax was subsequently deprecated and is removed altogether in the 2021 edition.]: a trait object’s lifetime, as with generic bounds (dealt with below).

The first two are, I believe, fairly obvious; it’s clearly unsafe to have a reference to an object that has been freed. (By baking this into the language, we avoid the problem of dangling pointers that languages like C have.)

The third is much the same, as all generic lifetime parameters will, somewhere down the way, be of one of the other types, and so it’s just a way of passing that constraint through the types, like this:

struct Ref<'a, T: 'a>(&'a T);

As you can see, this Ref type is basically just a wrapper around an immutable reference; as the contained field is of that lifetime, clearly the containing structure may not live any longer than it and so it must have that lifetime also. (The T: 'a part is a type bound, saying that the generic type T must live for at least 'a; without this it would not work, for as already discussed it clearly wouldn’t make sense to have a reference living longer than the object it refers to.)

The fourth and final of these (dyn Trait + 'a) is, I think, the most interesting, and it’s worth spending a short time on trait objects, because the way in which they fit into this arrangement is slightly different from elsewhere.

Trait objects are Rust’s form of safe and convenient dynamic dispatch (that’s what the dyn stands for); they allow you to store arbitrary types that satisfy a trait in the one type. Because of the potential difference in size of the types implementing a trait, trait objects are only usable through a reference of some form; the owned form is thus Box<dyn Trait>, and for references you can have &dyn Trait and &mut dyn Trait. This is a very brief and wholly lacking explanation of trait objects; you can find documentation of them elsewhere; a detailed explanation is out of scope for this article.

As stated, a trait object may be of a variety of different types; what, then, is the lifetime of a trait object? The answer is that we must specify it, like Box<dyn Trait + 'static>, indicating that the contained object must be 'static, and &'a (dyn Trait + 'a), indicating that the contained object’s lifetime must be at least 'a. Now as it happens, some of these common cases like + 'static on a boxed trait object and the duplication of the 'a in the reference are taken care of as default trait bounds—Box<dyn Trait> is normally equivalent to Box<dyn Trait + 'static> and &'a dyn Trait to &'a (dyn Trait + 'a). RFC 599, on default object bounds, treats the current rules on this subject more precisely. [RFC 599 was written before the invention of the dyn keyword, so you’ll see the old‐style bare trait object syntax in that document.]

(Note that just as the lifetime on a trait object can be omitted due to default object bounds, lifetimes on the other three cases can also often be omitted; the current rules of lifetime elision are covered in RFC 141.)

But still you may wonder: why do trait object need a lifetime? I think it’s easiest to just give an example of something that would be bad if it were allowed to compile (which it isn’t):

fn main() {
    let trait_object: &dyn AsRef<str> = {
        let string = "I am a String".to_owned();
        &string
    };
    println!("The string is {:?}", trait_object.as_ref());
}

By the time execution gets to the println! statement, string has been freed, so if this were allowed to compile, trait_object, which points to the contents of string, would also thus be pointing to freed memory, which is emphatically a Bad Thing™. To maintain memory safety, the boxed trait object cannot live longer than the string it contains a reference to.

Lifetime positions in generic bounds

As shown very basically above, a lifetime can also appear in type parameter bounds and lifetime parameter bounds. Here are some more examples:

struct Ref<'a, T: SomeTrait + AnotherTrait + 'a>(&'a T);

// Suppose we have something that is reading values from a buffer and storing
// the decoded values, which still contain references to parts of the original
// buffer, into a vector. For maximal generality, we have *two* lifetimes,
// though normally one would suffice (this is a poor example since it doesn’t
// benefit from taking two lifetimes). The reference to the vector must not
// outlive the vector and all its items, which must not outlive the buffer.
struct DecodeResult<'buf: 'decoded, 'decoded, T: 'buf> {
	buffer: &'buf [u8],
	decoded: &'decoded mut Vec<T>,
}

By type parameter bounds we mean the T: 'buf, stating that the lifetime of the type T must be at least 'buf (it must live at least as long as the buffer).

By lifetime parameter bounds we mean the 'buf: 'decoded, stating that the lifetime 'buf must be at least as long as (must satisfy the constraint) 'decoded. This is extremely rare and is typically only of any use when subtyping and variance gets involved.

Really, these sorts of bounds are very similar to the trait object bounds: because generics can be of arbitrary types, you will often need to stipulate a minimal lifetime in order to work with a type. In the DecodeResult example above, T’s lifetime must be constrained to be at least 'decoded, or else the reference in the decoded field would not be valid. Rust could infer this sort of thing, but it would lead to surprises in various areas, so it refrains from the attempt.

Exceptions to “each object can be used exactly once”

There; we’ve covered the four basic rules. There remains still one point to explain. Earlier on I posited the simple rule that each object can be used exactly once; as I also said, this isn’t actually universally true. There are two exceptions to this rule.

Copy and move semantics

Objects of types with move semantics can only be used once, but this is not true of types with copy semantics.

An object has copy semantics if it implements Copy. This means that a value of that type can be duplicated simply by copying the bytes of memory. i32, for example, is Copy because it’s just an arbitrary four bytes of data, entirely self‐contained, while Box<T> is not because it involves a heap allocation that it must own and so a simple copy of the bytes would cause two objects to own the data at the same time, breaking all sorts of invariants and probably eating your laundry.

In the first code example’s error, we saw the explanatory note “move occurs because a has type A, which does not implement the Copy trait”. A has move semantics; suppose we now implement the Copy trait, like this:

#[derive(Clone, Copy)]
struct A {}

fn main() {
    let a = A {};
    let b = a;
    let c = a;
}

#[derive(X)] means “hey compiler, I want to implement X for this type, I just want the obvious implementation, so can you please do it for me?”

The Rust documentation explains deriving Clone and Copy.

With the addition of the Copy implementation for A, it’s changed from having move semantics to having copy semantics, and so the code now compiles.

There is one final matter in this exception to the rule worth mentioning: generics. If you work with a generic type, it will have move semantics unless you specify Copy in its bounds. In cases where you wish to copy the value, you might wish to use Clone instead, as it is more general (and all types that implement Copy implement Clone). Don’t worry about efficiency of cloning versus copying, either; unless the writer of the type explicitly went out of their way to do so (which they should never do), cloning and copying will perform identically when optimised.

For more information on this topic, see the Copy docs.

Reference reborrowing

A particularly notable example of a type with copy semantics is &T; you can have as many immutable references to an object alive as you like, so long as there are no mutable references to it at the same time. &mut T, however, has move semantics, for you’re not allowed to have more than one mutable reference in scope at a time.

So then, why does this example work?

fn take_ref<T>(_: &T) { }
fn take_mut<T>(_: &mut T) { }
fn main() {
    let mut a = 6;
    {
        let a_ref = &a;
        take_ref(a_ref);
        // Naturally this will work, for &T is Copy.
        take_ref(a_ref);
    }
    {
        let a_mut = &mut a;
        take_mut(a_mut);
        // Surely a_mut should have been consumed by the line above?
        take_mut(a_mut);
    }
}

There is another exception in play here: reference reborrowing. Where we pass a_mut to a function, it effectively becomes &mut *a_mut. This does mean that for a time two mutable references to the same thing exist (one in the caller and one in the function called), but at no point are both accessible, so it’s OK.

Note that this exception only applies in situations where the parameter is of the form &mut T; if it is a generic taken by value (e.g. fn<T>(x: T)) then a mutable reference you pass in will not be reborrowed automatically. You can still use the &mut *x incantation to manually reborrow it, however.

Summary

Let’s list the main points once again:

Each object can be used exactly once. When you use an object it is moved to the new location and is no longer usable in the old. (Copy and automatic reference reborrowing are the two exceptions that make this more palatable.)
When an object passes out of scope, it is destroyed and is no longer usable.
Blocks can produce a value which goes up one level of scope. (As a language feature, this is known as expression orientation.)
All objects have a lifetime which constrains which scopes they may be moved out of.

These points are all that you need to understand all of Rust’s ownership model. The rules are surprisingly simple, though you can quickly get to fairly complex interactions which make things harder to reason about. Don’t worry if you haven’t understood all of this; if you are trying to learn Rust, go and do some other things for a while and come back to the article again later and you’ll probably find it makes more sense and you can understand how ownership and lifetimes work. When you finally get to that point, you’ll love Rust’s unusual ownership model.

Quick tip: the `#[cfg_attr]` attribute

2015-04-22T00:00:00+00:00

Today I want to remind people of #[cfg_attr]: it’s a really handy way of reducing code duplication in some situations, and not enough people know about.

What is `#[cfg_attr]`?

It’s just like #[cfg], but for attributes. Just as #[cfg(condition)] allows you to only compile the thing it decorates if the condition is true, #[cfg_attr(condition, attribute)] allows you to only add the attribute to the thing it decorates if the condition is true. That is, if the condition is true, it’s equivalent to #[attribute], and if the condition is false, it vanishes into thin air.

Real‐world examples

Enabling nightly features in certain situations

In anymap, I have a nightly Cargo feature which, if enabled, implements some additional features and performance optimisations (like using a more efficient hasher on the hash map) that are not yet possible on stable rustc. Those require a couple of extra gated features, but I only want the #![feature(core, std_misc)] crate attribute to exist if the nightly Cargo feature is enabled. I can do that:

#![cfg_attr(feature = "nightly", feature(core, std_misc))]

This might also be handy for things like tests:

#![cfg_attr(test, feature(test, anything_else_only_relevant_for_tests))]

Special attributes for bootstrapping rustc

A number of times I have noticed duplicated items, one #[cfg(stage0)] and one #[cfg(not(stage0))], differing only in attributes; this is not necessary. I’ll take a current example:

Before: duplication just for one attribute’s difference.

/// `PhantomFn` is a deprecated marker trait that is no longer needed.
#[lang="phantom_fn"]
#[stable(feature = "rust1", since = "1.0.0")]
#[deprecated(since = "1.0.0", reason = "No longer needed")]
#[cfg(stage0)]
pub trait PhantomFn<A:?Sized,R:?Sized=()> {
}

/// `PhantomFn` is a deprecated marker trait that is no longer needed.
#[stable(feature = "rust1", since = "1.0.0")]
#[deprecated(since = "1.0.0", reason = "No longer needed")]
#[cfg(not(stage0))]
pub trait PhantomFn<A:?Sized,R:?Sized=()> {
}

After: yay! no duplicating the entire thing!

/// `PhantomFn` is a deprecated marker trait that is no longer needed.
#[cfg_attr(stage0, lang = "phantom_fn")]
#[stable(feature = "rust1", since = "1.0.0")]
#[deprecated(since = "1.0.0", reason = "No longer needed")]
#[cfg(stage0)]
pub trait PhantomFn<A:?Sized,R:?Sized=()> {
}

Bonus: doc comments are truly sugar

Dynamic doc comments is a non‐obvious area in which #[cfg_attr] can be used.

First of all you must understand that these two are precisely the same in all ways that matter, because the former desugars to the latter during parsing:

/// This is the first line of the documentation.
///
/// This is another line.

#[doc = " This is the first line of the documentation."]
#[doc = ""]
#[doc = " This is another line."]

(Multiple #[doc] attributes are fine; they are joined together with line breaks between to form the final doc comment.)

This can be combined with #[cfg_attr] to allow conditional documentation, like adding a part to the documentation if a feature is enabled:

/// This trait does all sorts of cool things.
/// If the crate is built with the `foo` feature enabled,
/// it also does this other whizzy thing!
///
/// This documentation was built with the `foo` feature
#[cfg_attr(feature = "foo", doc = " **enabled**.")]
#[cfg_attr(not(feature = "foo"), doc = " **disabled** (tough luck).")]
///
/// Now we go onto some more documentation.
///
#[cfg_attr(feature = "bar", doc = "\
 # Barness

 You also have barness available. This is really cool stuff that lets you go
 whizzety bang pop. In real life I’d be dedicating a few paragraphs and a code
 sample or two to this. Because this isn’t real life, I’ll just ask that you
 please don’t confuse *barness* with *baroness*, because that would make me sad
 because you would be wrong.")]
#[cfg_attr(not(feature = "bar"), doc = "\
 You know, if you compiled with the `bar` feature enabled, you’d have a lot
 more fun. See [this helpful documentationy link that I will have] for details
 on how to accomplish that.")]
trait Traitor { }

When built, the documentation will then announce whether it was built with the foo feature enabled or disabled, which might be handy. And a new section will be added if the bar feature is enabled, or a note on what you’re missing out on and maybe how to enable it if you don’t have it enabled. There, four versions of the documentation without the need to duplicate even a single vast swathe of code!

Note also how when we manually write #[doc] attributes we put a single leading space at the start of the string; this is because /// Foo desugars to #[doc = " Foo"]; if all the lines start with a space, it will be trimmed from them all, but if not then such lines will have a leading space, which might mess up your Markdown formatting of things like headings marked with leading hashes.

I will admit that dynamic doc comments isn’t going to be completely useful everywhere, for useful applications of it aren’t going to be very common and if people use only one centralised documentation source they’ll miss out on getting the special version that just suits their use case, but as we get better tooling I think it might become more appropriate. And even if it’s not so very useful (hey, it’s bonus material, it doesn’t need to be so very useful), it’s definitely fun.

Super bonus: adding macros into the mix!

Imagine that the trait bounds need to be changed if the bar feature is enabled. Argh! you say. Back to the drawing board! Not so—you can save all the duplication with a simple macro that, given an item and appropriate attributes, dumps them out with the documentation attributes added. Here’s a simple example:

macro_rules! define_traitor {
    (#[$m:meta] $t:item) => {
        /// All the doc comment lines and
        #[cfg_attr(feature = "bar", doc = "cfg_attr lines and")]
        /// so forth.
        #[$m] $t
    }
}

define_traitor! {
    #[cfg(not(feature = "bar"))]
    trait Traitor { }
}

define_traitor! {
    #[cfg(feature = "bar")]
    trait Traitor: BarOfChocolate { }
    // Yeah, a traitor can depend on a bar of chocolate, why not?
}

Voilà! Whichever version of the Traitor trait we get, it has the right documentation on it.

Why your first FizzBuzz implementation may not work: an exploration into some initially surprising but great parts of Rust (though you still may not like them)

2014-10-03T00:00:00+00:00

FizzBuzz is intended as a simple task, but in Rust there are a few pitfalls for the novice to be aware of. These aren’t problems with Rust but rather differences from what people are commonly familiar with, restrictions that may seem arduous at first but ultimately give you more power, at a small cost.

A simple functional implementation

OK, I said in the title that your first FizzBuzz implementation might not work—then again, it might. You might write it like this which follows, for example, a way that works fine.

Figure 1: a simple FizzBuzz implementation with separate print statements: easy and functional.

(For brevity of the code examples, assume that the Rust code is always wrapped in fn main() { … }.)

Rust

for i in 1..101 {
    if i % 15 == 0 {
        println!("FizzBuzz");
    } else if i % 5 == 0 {
        println!("Buzz");
    } else if i % 3 == 0 {
        println!("Fizz");
    } else {
        println!("{}", i);
    }
}

Python

for i in range(1, 101):
    if i % 15 == 0:
        print('FizzBuzz')
    elif i % 5 == 0:
        print('Buzz')
    elif i % 3 == 0:
        print('Fizz')
    else:
        print(i)

These programs will both produce the desired output and they’re evidently very similar visually. The main thing to note about the Rust version is that println!() needs as its first argument a string literal, a formatting string; in Python it’d be equivalent to print('{}'.format(i)).

But how about if we decided that we wanted only one print call in the code base? Here’s how it might go:

Figure 2: storing the result in a variable, so as to use only one print statement.

if block as an expression; the whole result variable assignment is actually unnecessary, we could just put the if block straight where it’s subsequently used. Ruby programmers may recognise this as expression orientation. As a Python programmer when I came to Rust, I thought it just a gimmick to let you omit return; but I rapidly discovered that it’s a marvellous paradigm. Rust totally ruined Python for me.

Rust (won’t compile)

for i in 1..101 {
    let result = if i % 15 == 0 {
        "FizzBuzz"
    } else if i % 5 == 0 {
        "Buzz"
    } else if i % 3 == 0 {
        "Fizz"
    } else {
        i
    };
    println!("{}", result);
}

Python

for i in range(1, 101):
    if i % 15 == 0:
        result = 'FizzBuzz'
    elif i % 5 == 0:
        result = 'Buzz'
    elif i % 3 == 0:
        result = 'Fizz'
    else:
        result = i

    print(result)

This Rust code may seem all very well, but it’s actually not, owing to its strict typing rules. For what is the type of the variable result? The first three branches of the if‐expression produce strings and the fourth an integer:

Figure 2a: compiler output for Figure 2.

{integer} identifies a partially‐resolved type and is used for integer literals, when the compiler hasn’t yet determined what actual type it will be. An integer literal can be of type u8, i8, u16, i16, u32, i32, u64, i64, u128, i128, usize or isize. If the compiler doesn’t get any hints as to what it should be, it’ll end up choosing i32.

error[E0308]: if and else have incompatible types
  --> <anon>:10:9
   |
7  |       } else if i % 3 == 0 {
   |  ____________-
8  | |         "Fizz"
   | |         ------ expected because of this
9  | |     } else {
10 | |         i
   | |         ^ expected `&str`, found integer
11 | |     };
   | |_____- `if` and `else` have incompatible types

error: aborting due to previous error

This clearly won’t do. How about we turn it into a string first?

Figure 3: adding .to_string() to the code from Figure 2.

for i in 1..101 {
    let result = if i % 15 == 0 {
        "FizzBuzz"
    } else if i % 5 == 0 {
        "Buzz"
    } else if i % 3 == 0 {
        "Fizz"
    } else {
        i.to_string()
    };
    println!("{}", result);
}

This is where we stray from the familiar parts of computer science that most people are familiar with to a part that far fewer people will relate to. Because this doesn’t work.

Figure 3a: compiler output for Figure 3.

Note that from Rust 1.58 this produces a spurious second error related to integer type inference; I’ve reported this, hopefully it’ll be resolved soon.

error[E0308]: if and else have incompatible types
  --> <anon>:10:9
   |
7  |       } else if i % 3 == 0 {
   |  ____________-
8  | |         "Fizz"
   | |         ------ expected because of this
9  | |     } else {
10 | |         i.to_string()
   | |         ^^^^^^^^^^^^^ expected `&str`, found struct `String`
11 | |     };
   | |_____- `if` and `else` have incompatible types

error: aborting due to previous error

“What?” I hear you saying; “aren’t they both strings? What’s the deal with this &str (how do I even pronounce it? [I can’t even decide whether to spell its indefinite article “a” or “an”. Sometimes I say “an ampersand str”, sometimes “a str”, sometimes “a str reference” or “a str slice” (inverted order). &'static str I call “a static str”. &'a str I mostly say something like “a str slice, lifetime a”.]) and String?” Now indeed I need to be more specific than I was in the previous analysis of the types of the branches: the first three branches didn’t just produce “strings”, they each produced a &str; the fourth branch produced a generic integer, which might be of a number of types (though in the absence of any further information it would become a i32, the default type of integer). Languages like Python, Ruby and JavaScript unify their integer types (actually, JavaScript unifies its number types altogether), while languages like C♯ and Java and Go have a variety of integral types of different sizes, so there being multiple integer types is familiar to many. But even C♯, Java and Go each have just one type of string.

Rust doesn’t. It has two.

Two types of strings? What is this?

We could look at a simple explanation and continue on our way, but we’ve come so far down this pathway that we might as well go the rest of the way—understanding specifically what’s going on is definitely worthwhile. Why can C♯ and Java and Go get away with one String type and Rust can’t? To answer this, we have to step down to the level of memory management.

C♯, Java and Go are all managed languages (also known as garbage collected languages). That is, they have a runtime which takes care of the allocation and freeing of memory at the appropriate times; when nothing is using the string any more, it can be freed. They can thus return references to a string and not need to worry about liveness issues: parts of a string that are in use will not be freed.

There is a trade‐off here for them; as a general rule, such languages have immutable strings—if you concatenate two strings, it involves a completely new allocation. (This means that a solution that modifies a string by adding “Fizz” if it’s divisible by three and then “Buzz” if it’s divisible by five and then the number if the string is still empty may well involve two allocations for multiples of fifteen, though something called string interning which is commonly employed may reduce or fix that, depending on how it’s done and how clever any optimiser is.) This is, I presume, because the alternatives are worse: copying for any slicing will lead to a very significant increase in memory usage for most programs; and allowing mutation of one string to potentially affect others derived from it is terrible for correctness, and it could lead to nasty data race conditions in what is for practical purposes a primitive type, and it could also, for anything using UTF‐16 or UTF‐8, lead to illegal code points in a subslice (substring); sure, most of these problems arise in other places, but having it in their string types would be much worse. (I said that they only have one string type, but that also is typically not quite true—there are enough cases that need more efficient mutable strings that they tend to have such a type. For example, Java and .NET have things called StringBuilder.)

Rust’s model is somewhat different, not being a garbage collected language, being centred around its language-wide model of ownership, where each object is owned in one place at a time, though other places may safely borrow references to it.

String is an owned type. That is, it has exclusive ownership of the contents of the string; and when it passes out of scope, the memory for the contents of the string will be freed immediately. For this reason, any substring can’t be of the type String, for there will be no connection between the two, and so when one passed out of scope, the other would become invalid, leading to a loss of memory safety. And so instead it is that slices (substrings) use a type which is a reference to the contents that something else owns—&str. Rust, through its concept of lifetimes, is able to guarantee that no slice outlives the actual String, and so memory safety is ensured.

Lifetimes have syntax, 'like_this, with one special lifetime 'static that indicates the life of the program. But when there’s only one possible value or an obvious way things should be hooked up, you can normally skip writing the lifetime; this is called lifetime elision. We’ll encounter lifetimes a bit later in this article, but so far they’ve all been elided. Just to prepare you: a string slice of lifetime a is spelled &'a str, and string literals (being stored in the binary) are &'static str.

If you’re interested in learning more about lifetimes at this point, the Rust Book has a chapter on them.

Back to the fizzing and buzzing

So then, the problem is that one of them is an owned string while the other three are string slices (references to statically defined strings). How shall we resolve it? How about we try making them all string slices (that is, of type &str):

for i in 1..101 {
    let result = if i % 15 == 0 {
        "FizzBuzz"
    } else if i % 5 == 0 {
        "Buzz"
    } else if i % 3 == 0 {
        "Fizz"
    } else {
        &*i.to_string()
    };
    println!("{}", result);
}

Figure 4: adding &* to the code from Figure 3 to convert the String to &str.

String dereferences to str—viz., it implements Deref<Target = str>—so &*string produces a string slice, type &str, pointing to the contents of the String.

That seems like a good idea, right? Sorry, that won’t do:

Figure 4a: compiler output for Figure 4.

error[E0716]: temporary value dropped while borrowed
  --> <anon>:10:11
   |
7  |       } else if i % 3 == 0 {
   |  ____________-
8  | |         "Fizz"
9  | |     } else {
10 | |         &*i.to_string()
   | |           ^^^^^^^^^^^^^ creates a temporary which is freed while still in use
11 | |     };
   | |     -
   | |     |
   | |_____temporary value is freed at the end of this statement
   |       borrow later used here
   |
   = note: consider using a `let` binding to create a longer lived value

error: aborting due to previous error

Here we’re running into a lifetime issue; the String created by i.to_string() is not being stored anywhere, and so it is freed at the end of the else block. Thus the reference to it can’t escape that block, like we were trying to make happen. This is a memory safety bug that the Rust compiler caught; in some languages it would wind up as a dangling pointer, which is a Very Bad Thing.

Now in this case, the compiler gave us a suggestion: “consider using a `let` binding to create a longer lived value”. Not all of the compiler’s suggestions will be workable, but I think this one always is if it makes it. We can indeed lift the String variable to outside the else block; the compiler was only caring that the object being referenced needed to be valid for the body of the for loop. Here’s how that looks:

for i in 1..101 {
    let x;
    let result = if i % 15 == 0 {
        "FizzBuzz"
    } else if i % 5 == 0 {
        "Buzz"
    } else if i % 3 == 0 {
        "Fizz"
    } else {
        x = i.to_string();
        &*x
    };
    println!("{}", result);
}

Figure 5: taking a reference as in Figure 4, but storing the owned string in the outer scope: this works.

This way, the String lives until the end of the for‐loop, which is longer than result needs to be valid for, and so result is allowed to refer to it.

How about we make it a `String`?

Another direction that we could go is to make all of the branches return owned strings:

for i in 1..101 {
    let result = if i % 15 == 0 {
        "FizzBuzz".to_string()
    } else if i % 5 == 0 {
        "Buzz".to_string()
    } else if i % 3 == 0 {
        "Fizz".to_string()
    } else {
        i.to_string()
    };
    println!("{}", result);
}

Figure 6: String all the way: this works, at a runtime cost.

This will work fine, but it has meant that a new chunk of memory is allocated for all values of i, rather than just for the ones which aren’t factors of three or five.

Making it a function

We’ve gone as far as we can in this direction without things becoming absurd; how about if we alter the parameters of the problem so that we’re not printing the result, but rather returning it from a function?

Here’s what we might start with:

Figure 7: the fizz_buzz function, with Strings.

Rust

fn fizz_buzz(i: i32) -> String {
    if i % 15 == 0 {
        "FizzBuzz".to_string()
    } else if i % 5 == 0 {
        "Buzz".to_string()
    } else if i % 3 == 0 {
        "Fizz".to_string()
    } else {
        i.to_string()
    }
}

for i in 1..101 {
    println!("{}", fizz_buzz(i));
}

Python

def fizz_buzz(i):
    if i % 15 == 0:
        return 'FizzBuzz'
    elif i % 5 == 0:
        return 'Buzz'
    elif i % 3 == 0:
        return 'Fizz'
    else:
        return i



for i in range(1, 101):
    print(fizz_buzz(i))

We now have an extra layer of encapsulation around it which demonstrates clearly that the solution when we were producing a &str of hoisting the String variable declaration x to the loop won’t work, because that variable would be going out of scope also, being contained inside the function. (Try it if you’re not sure; the return type is unrepresentable in Rust’s type system, as there is no suitable lifetime—x won’t live for 'static (the duration of the process) and there’s nothing else to tie it to.)

And so, simply because we’ve put it in a function, we have a little inefficiency—we’re allocating new strings for multiples of three or five when it shouldn’t be necessary.

Enter `Cow<str>`

Fortunately, Rust has algebraic data types (known as enums in the language), and there’s a convenient type defined in the standard library as something that is either a string slice or an owned string.

Here are the relevant definitions (without any of the methods that make it more useful):

pub enum Cow<'a, B: ?Sized + 'a> where B: ToOwned {
    Borrowed(&'a B),
    Owned(<B as ToOwned>::Owned)
}

impl ToOwned for str {
    type Owned = String;

    fn to_owned(&self) -> String {
        …
    }
}

Figure 8: the std::borrow::Cow definition and how Cow<'a, str> comes from that.

The ToOwned implementation is defined in library/alloc/src/str.rs in the Rust repository, if you’re interested; but it doesn’t really matter.

And lo! Lifetime syntax makes an appearance.

I’ll admit that to be somewhat opaque to the inexperienced, so let’s fill in the generics to get an idea of approximately what it ends up for real life:

pub enum Cow<'a, str> {
    Borrowed(&'a str),
    Owned(String),
}

Figure 9: an approximation of Cow<str>.

(This is for demonstration purposes only; the grammar is not correct.)

Now the particular type that we wish to pay attention to is Cow<'static, str>, which is either a &'static str or a String, and which dereferences to str [Cow<B> implements Deref<Target = B>; put another way, given a Cow<str> x, &*x produces a &str and you can mostly just treat a Cow<'a, str> as a &'a str.]). Why is this type so interesting for us? Because it is completely owned data, containing no non‐'static lifetimes. In other words, if we create one of these inside a function, it is entirely self‐contained, not having any references to things inside the function and we can return it without any trouble. Expressed even more sloppily: this is a type that can contain a string literal, or an owned string that you have constructed.

For assistance in creating these objects, there are implementations of From for Cow<str>; thanks to these, if you have a &str or a String, you can write Cow::from(x) or, where it can be inferred that the result must be Cow<str>, x.into(), instead of having to write the specific enum variant, Cow::Borrowed(x) or Cow::Owned(x).

As mentioned, Cow<str> dereferences to str, allowing us to very often treat it as though it were a &str. A prime example of this is the manner in which we can use it directly in string formatting, using that type’s std::fmt::Display implementation with the {} format string (fmt::Display is the direct parallel of __str__() in Python’s string formatting, or to_s(), toString(), &c. in various other languages, though it works directly with a writer allowing you to skip the writing to an intermediate string if you want; the to_string() method on any type implementing fmt::Display uses this mechanism to produce a String).

Here’s what it looks like:

use std::borrow::Cow;

fn fizz_buzz(i: i32) -> Cow<'static, str> {
    if i % 15 == 0 {
        "FizzBuzz".into()
    } else if i % 5 == 0 {
        "Buzz".into()
    } else if i % 3 == 0 {
        "Fizz".into()
    } else {
        i.to_string().into()
    }
}

for i in 1..101 {
    println!("{}", fizz_buzz(i));
}

Figure 10: a FizzBuzz function returning Cow<'static, str>: this works.

Great! We’ve reduced the work the computer needs to do and made our admittedly trivial case faster.

But can we go further?

Writing our own enum and implementing `std::fmt::Display` efficiently

Of course, what we’re representing for each term in the sequence isn’t actually “a static string slice or an owned string”, it’s “‘Fizz’, ‘Buzz’, ‘FizzBuzz’ or a number”. We just converted all the choices to strings eagerly; we could just as easily make it lazy, avoiding the extra allocation where possible (and in fact they are all avoidable).

Let’s make an enum of our own, which we’ll call Term.

While we’re at it, we’ll implement std::fmt::Display, which means that we can print to stdout without even needing to create the intermediate String. We’ll also implement std::fmt::Debug, which answers to the format string {:?} and is for giving a developer-oriented representation of the object, akin to Python’s __repr__() and Ruby’s inspect; it can be just the same thing in this case.

Figure 11: using a dedicated data type to represent the real possibilities efficiently.

The suggested Python code is thoroughly contrived, as is going to be the case in all languages without algebraic data types; I provide it for the sake of users not familiar with algebraic data types to at least give an impression of how it works.

Rust

use std::fmt;

enum Term {
    Fizz,
    Buzz,
    FizzBuzz,
    Number(i32),
}

impl fmt::Display for Term {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        match *self {
            Term::Fizz => f.write_str("Fizz"),
            Term::Buzz => f.write_str("Buzz"),
            Term::FizzBuzz => f.write_str("FizzBuzz"),
            Term::Number(num) => write!(f, "{}", num),
        }
    }
}

impl fmt::Debug for Term {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        fmt::Display::fmt(self, f)
    }
}

fn fizz_buzz(i: i32) -> Term {
    if i % 15 == 0 {
        Term::FizzBuzz
    } else if i % 5 == 0 {
        Term::Buzz
    } else if i % 3 == 0 {
        Term::Fizz
    } else {
        Term::Number(i)
    }
}

for i in 1..101 {
    println!("{}", fizz_buzz(i));
}

Python analogue (thoroughly contrived)

class Term:

    def __init__(self, value):
        self._value = value

    def __str__(self):
        if self is Term.Fizz:
            return "Fizz"
        elif self is Term.Buzz:
            return "Buzz"
        elif self is Term.FizzBuzz:
            return "FizzBuzz"
        else:
            return str(self._value)

    def __repr__(self):
        return str(self)

    @staticmethod
    def Number(number):
        return Term(number)

# Pretend that these are opaque
Term.Fizz = Term(object())
Term.Buzz = Term(object())
Term.FizzBuzz = Term(object())

def fizz_buzz(i):
    if i % 15 == 0:
        return Term.FizzBuzz
    elif i % 5 == 0:
        return Term.Buzz
    elif i % 3 == 0:
        return Term.Fizz
    else:
        return Term.Number(i)

for i in range(1, 101):
    print(fizz_buzz(i))

Observe also that this is doing a really good job of actually representing the data contained, though for this trivial case it wouldn’t matter much if it didn’t. We could just as easily have replaced the first three variants of that enum with a single Word(&'static str) variant, using Word("FizzBuzz") et al. for the values. (In fact, that was actually the first version I wrote of this step of the process—see, even I was fixed on using strings where it’s simply unnecessary!)

We could go further, showing how to write a dedicated iterator, but with how Rust’s iterator chaining works, this isn’t really necessary—you can just write (1..101).map(fizz_buzz). That gives a lot more flexibility; as soon as you have something implementing Iterator<Item = i32>, you can just tack .map(fizz_buzz) onto the end of it and you have a type implementing Iterator<Item = Term>.

That loop could be rewritten in this style, incidentally:

Figure 12: mapping an integer iterator to the fizz_buzz function.

Rust

for f in (1..101).map(fizz_buzz) {
    println!("{}", f);
}

Python

for f in map(fizz_buzz, range(1, 101)):
    print(f)

Whichever of these sorts of ways we choose to do it, we end up with the good ol’ FizzBuzz lines:

1
2
Fizz
4
Buzz
Fizz
7
8
Fizz
Buzz
11
Fizz
13
14
FizzBuzz
16
17
Fizz
19
Buzz
Fizz
22
23
Fizz
Buzz
26
Fizz
28
29
FizzBuzz
31
32
Fizz
34
Buzz
Fizz
37
38
Fizz
Buzz
41
Fizz
43
44
FizzBuzz
46
47
Fizz
49
Buzz
Fizz
52
53
Fizz
Buzz
56
Fizz
58
59
FizzBuzz
61
62
Fizz
64
Buzz
Fizz
67
68
Fizz
Buzz
71
Fizz
73
74
FizzBuzz
76
77
Fizz
79
Buzz
Fizz
82
83
Fizz
Buzz
86
Fizz
88
89
FizzBuzz
91
92
Fizz
94
Buzz
Fizz
97
98
Fizz
Buzz

Conclusion

So now you have it: why your first FizzBuzz implementation in Rust might not work. Some of the difficult parts of it are typical of statically typed languages, and some more of them are more specific to Rust. (As a matter of fact, the situation would be similar in C++, but C++ lets you do all sorts of silly things and doesn’t give much in the way of memory safety. Don’t bother arguing with me on that point though, I’m just going off the word of others—I don’t know C++ especially well.)

We’ve covered how Rust’s ownership model can prevent you from writing things in certain ways that you might be used to, and why (though not the concrete benefits of doing this); also how Rust is able to express more efficient concepts; we’ve seen how enums (algebraic data types) can be used to model data more precisely and efficiently.

Hopefully you’ve seen the power of this and it interests you.

Is this an increased conceptual burden? Yes.

Is it a nuisance? Occasionally. (My experience is that it saves trouble at least as often as it causes it.)

Does it allow you to improve your program’s efficiency? Certainly, and with complete confidence. While in the past these things have required a loss of certainty about safety and correctness properties, in Rust you don’t need to make it a tradeoff.

Does it make code easier to reason about? In simple cases like this you won’t see that, but these general attributes of the ownership model and algebraic data types really do help in more complex cases. (I really miss these things when working in Python.)

These matters end up both a good thing and a bad thing about Rust; sometimes you will love them, and occasionally you will hate them. But I at least don’t hate them often at all.

Should you use Rust? Well, I would suggest that you at least try it if you haven’t. You may well find it not ready or fit for your purposes. Because of its focus on systems development, it is more cumbersome for many higher‐level tasks, though even on those it can be surprisingly effective.

Finally, if you’re not familiar with Rust or got lost at any point, I suggest you go through the official documentation; the Rust book has sections dealing with lifetimes and ownership, enums and a whole lot more. I also wrote Rust ownership the hard way, an article that you may find helpful in understanding this crucial defining feature of Rust. If you still have questions, places like #rust-beginners on irc.mozilla.org are great. (Pretty much everyone agrees that the Rust community is superb. You will get good answers there.)

Appendix: other FizzBuzz techniques

This section is by no means comprehensive, but includes a few extras that have been pointed out over time.

1. `match` instead of `if`

People often suggest replacing the chained conditionals with a match block, which incidentally lets you skip the “superfluous” i % 15:

for i in 1..101 {
    match (i % 3, i % 5) {
        (0, 0) => println!("FizzBuzz"),
        (_, 0) => println!("Buzz"),
        (0, _) => println!("Fizz"),
        (_, _) => println!("{}", i),
    }
}

Figure 13: Figure 1, but with match instead of chained if.

Application of this technique to other examples is left as an exercise for the reader.

Opinions differ on which is nicer. In this particular case I personally prefer the if version, but if you introduced a third word I’d switch to the match version.

2. `std::fmt::Display` trait objects

This possibility was brought to my attention in September 2019, sitting somewhere between the code in Figure 2 and the code in Figure 11, by using a dynamic trait object:

for i in 1..101 {
    let result: &dyn std::fmt::Display = if i % 15 == 0 {
        &"FizzBuzz"
    } else if i % 5 == 0 {
        &"Buzz"
    } else if i % 3 == 0 {
        &"Fizz"
    } else {
        &i
    };
    println!("{}", result);
}

Figure 14: making Figure 2 work by the use of std::fmt::Display trait objects.

Application of this technique to other examples is again left as an exercise for the reader.

This is a worthwhile technique to be aware of: if there exists a trait that covers the required functionality shared between the types you wish to store in the one variable, a dynamic trait object can be a viable option instead of making your own enum. If you wanted to be able to inspect the values to determine what they semantically were, this would be insufficient, and something like the Term enum would be called for, but so long as all you want to do is format them for display, a std::fmt::Display trait object is fine.

3. And if you really like fiddling with FizzBuzz optimisation…

Feel free. Here’s the final conclusion of that lot, updated to compile now, plus making OUT a byte string for improved legibility (!?):

#![no_std]
#![feature(lang_items, start, rustc_private)]

extern crate libc;

static OUT: [u8; 413] = *b"\
    1\n2\nFizz\n4\nBuzz\nFizz\n7\n8\nFizz\nBuzz\n11\nFizz\n13\n14\nFizzBuzz\n\
    16\n17\nFizz\n19\nBuzz\nFizz\n22\n23\nFizz\nBuzz\n26\nFizz\n28\n29\nFizzBuzz\n\
    31\n32\nFizz\n34\nBuzz\nFizz\n37\n38\nFizz\nBuzz\n41\nFizz\n43\n44\nFizzBuzz\n\
    46\n47\nFizz\n49\nBuzz\nFizz\n52\n53\nFizz\nBuzz\n56\nFizz\n58\n59\nFizzBuzz\n\
    61\n62\nFizz\n64\nBuzz\nFizz\n67\n68\nFizz\nBuzz\n71\nFizz\n73\n74\nFizzBuzz\n\
    76\n77\nFizz\n79\nBuzz\nFizz\n82\n83\nFizz\nBuzz\n86\nFizz\n88\n89\nFizzBuzz\n\
    91\n92\nFizz\n94\nBuzz\nFizz\n97\n98\nFizz\nBuzz\n";

#[start]
fn start(_argc: isize, _argv: *const *const u8) -> isize {
    unsafe {
        core::arch::asm!(
            "syscall",
            in("rax") 1, // syscall number = write
            in("rdi") 1, // fd = stdout
            in("rsi") OUT.as_ptr(),
            in("rdx") OUT.len(),
            out("rcx") _,
            out("r11") _,
            lateout("rax") _,
        );
    }
    0
}

#[lang = "eh_personality"] extern fn eh_personality() {}
#[panic_handler] extern fn panic_handler(_: &core::panic::PanicInfo) -> ! { loop {} }

Figure 15: optimising the FizzBuzz problem roughly as far as humanly possible.

Applying this technique should be left unexercised by the reader: it’s provided for demonstration purposes only, and should not be attempted at home or at work.

(This code depends on Linux syscall conventions, so it won’t work on other platforms.)

Teepee design: the HTTP method

2014-07-08T00:00:00+00:00

Now we come to the Method type. While there are a few improvements to be made, this is one of the things that probably can’t be significantly improved on over rust-http.

This article is part of the series on the rust-http redesign, Teepee.

An apology

Teepee has stood with nothing much happening for a while; this is my fault and I’m sorry to have caused it. Things are starting moving again and will continue moving now; I have the low‐level design of the HTTP/1 message reading parts of Teepee almost done and expect to post this within a few days. But I don’t blame you if you don’t trust my “few days”; I wouldn’t either.

There are also quite a number of people that have emailed me asking how you can help on Teepee—sorry, it’s a little hard to figure out ways just at present, for the project is not really mature enough for that yet. It will come, though. If you really want fun, start looking at HTTP/2 and figure out a sensible implementation strategy, especially for its stream dependencies and prioritisation…

Unless stated otherwise, all grammar comes from RFC 7230.

Relevant definitions

Here’s what rust-http has:

pub enum Method {
    Options,
    Get,
    Head,
    Post,
    Put,
    Delete,
    Trace,
    Connect,
    Patch,  // RFC 5789
    ExtensionMethod(String),
}

Here is what RFC 2616 had:

   The Method  token indicates the method to be performed on the
   resource identified by the Request-URI. The method is case-sensitive.

       Method         = "OPTIONS"                ; Section 9.2
                      | "GET"                    ; Section 9.3
                      | "HEAD"                   ; Section 9.4
                      | "POST"                   ; Section 9.5
                      | "PUT"                    ; Section 9.6
                      | "DELETE"                 ; Section 9.7
                      | "TRACE"                  ; Section 9.8
                      | "CONNECT"                ; Section 9.9
                      | extension-method
       extension-method = token

Here is what RFC 7230 has:

   The method token indicates the request method to be performed on the
   target resource.  The request method is case-sensitive.

     method         = token

   The request methods defined by this specification can be found in
   Section 4 of [RFC7231], along with information regarding the HTTP
   method registry and considerations for defining new methods.

The contents of RFC 7231 are largely immaterial here, except insofar as the common properties that are defined for methods are specified (in section 8.1.1): these are safety and idempotency. Representing these properties seem to me desirable.

The `token` type

By the way, if you’re wondering what characters are valid in a method, method = token; token = 1*tchar; tchar = "!" / "#" / "$" / "%" / "&" / "'" / "*" / "+" / "-" / "." / "^" / "_" / "`" / "|" / "~" / DIGIT / ALPHA.

In other words, a method (or any other place we get token in the grammar) is a sequence of letters, numbers or any of those other fancy characters—yes, !#$%&'*+-.^_`|~ is a valid HTTP method name.

This is the sort of thing that should probably be encoded in the type system, using a separate token type instead of using String or Vec<u8>, &str or &[u8]. Given this much, we will need both owned and borrowed versions of this, for I am imposing a rule in the low‐level API that it must not allocate, while causing the token to be owned is the only sensible route for the higher‐level API.

The solution is an equivalent to std::str::MaybeOwned<'a>, which applies to strings and is an owned string or a string slice (possibly even of static duration): then the owned form can be MaybeOwnedToken<'static>, and the unowned form MaybeOwnedToken<'a>. (Incidentally, we really need a general solution to this problem.)

On second thoughts, I doubt that the owned and slice tokens will be considered of value by themselves, so we might just eliminate them altogether, putting them straight into the enum.

Here’s the gist of what it’ll end up as:

pub enum Token<'a> {
    Owned {
        #[doc(hidden)]
        pub _bytes: Vec<u8>,
    },
    Slice {
        #[doc(hidden)]
        pub _bytes: &'a [u8],
    },
}

impl<'a> Token<'a> {
    fn bytes<'b>(&'b self) -> &'b [u8];
    fn into_owned(self) -> Token<'static>;
    fn as_slice<'b>(&'b self) -> Token<'b>;
}

(The #[doc(hidden)] pub part is to allow statics to be defined outside the defining module. You know, we really need a general solution to this problem also.)

While a little cumbersome, unifying these imposes certain constraints on the solution space for handling methods.

Design one: a struct and statics

Nothing like a bit of code.

pub struct Method<'a> {
    name: Token<'a>,
    safe: bool,
    idempotent: bool,
}

pub static GET: Method<'static> = Method {
    name: token::Slice { _bytes: b"GET", },
    safe: true,
    idempotent: true,
};
// Et cetera.

Method::from_token would then produce one of those statics where possible, thus automatically filling in the details with regards to safety and idempotency, which are important to model. (I don’t think I clarified this earlier; well, I have seen too much code comparing method names in an ad‐hoc way, where what they really wanted to be examining was whether the method was safe or idempotent.)

This looks very well, doesn’t it? Unfortunately, it simply won’t do because of its ergonomics. Because of how we unified the token type, Method<'static> is no longer Copy, and something like request.method = GET will not work—can’t move out of static. We are left with request.method = GET.clone(), and the hypothetical request(url, GET.clone()), a construct which is all in all rather an untidy thing. Is this a reasonable trade‐off to make? I am not convinced it is.

Design two: back to the enum

The second option is a mere extension of what is in rust-http: simple variants (which avoids the Copy problem) plus one at the end for custom things. Something like this:

macro_rules! method_enum {
    ($(
        $ident:ident
        $bytes:expr
        $safe:ident
        $idempotent:ident
        #[$doc:meta];
    )*) => {
        /// …
        pub enum Method<'a> {
            $(#[$doc] pub $ident,)*
            /// A method not in the IANA HTTP method registry.
            pub UnregisteredMethod {
                token: Token<'a>,
                safe: bool,
                idempotent: bool,
            },
        }

        impl Method {
            pub fn token(&self) -> Token<'a> {
                match *self {
                    $($ident => SliceToken { _bytes: $bytes },)*
                    UnregisteredMethod => token.as_slice(),
                }
            }
            // &c. for safe and idempotent
        }
    }
}

method_enum! {
    // Variant name   method name          safe  idempotent
    Acl               b"ACL"               false true  #[doc = "`ACL`, defined in RFC3744, Section 8.1"];
    BaselineControl   b"BASELINE-CONTROL"  false true  #[doc = "`BASELINE-CONTROL`, defined in RFC3253, Section 12.6"];
    Bind              b"BIND"              false true  #[doc = "`BIND`, defined in RFC5842, Section 4"];
    Checkin           b"CHECKIN"           false true  #[doc = "`CHECKIN`, defined in RFC3253, Section 4.4, Section 9.4"];
    Checkout          b"CHECKOUT"          false true  #[doc = "`CHECKOUT`, defined in RFC3253, Section 4.3, Section 8.8"];
    Connect           b"CONNECT"           false false #[doc = "`CONNECT`, defined in RFC7231, Section 4.3.6"];
    Copy              b"COPY"              false true  #[doc = "`COPY`, defined in RFC4918, Section 9.8"];
    Delete            b"DELETE"            false true  #[doc = "`DELETE`, defined in RFC7231, Section 4.3.5"];
    Get               b"GET"               true  true  #[doc = "`GET`, defined in RFC7231, Section 4.3.1"];
    Head              b"HEAD"              true  true  #[doc = "`HEAD`, defined in RFC7231, Section 4.3.2"];
    Label             b"LABEL"             false true  #[doc = "`LABEL`, defined in RFC3253, Section 8.2"];
    Link              b"LINK"              false true  #[doc = "`LINK`, defined in RFC2068, Section 19.6.1.2"];
    Lock              b"LOCK"              false false #[doc = "`LOCK`, defined in RFC4918, Section 9.10"];
    Merge             b"MERGE"             false true  #[doc = "`MERGE`, defined in RFC3253, Section 11.2"];
    MkActivity        b"MKACTIVITY"        false true  #[doc = "`MKACTIVITY`, defined in RFC3253, Section 13.5"];
    MkCalendar        b"MKCALENDAR"        false true  #[doc = "`MKCALENDAR`, defined in RFC4791, Section 5.3.1"];
    MkCol             b"MKCOL"             false true  #[doc = "`MKCOL`, defined in RFC4918, Section 9.3"];
    MkRedirectRef     b"MKREDIRECTREF"     false true  #[doc = "`MKREDIRECTREF`, defined in RFC4437, Section 6"];
    MkWorkspace       b"MKWORKSPACE"       false true  #[doc = "`MKWORKSPACE`, defined in RFC3253, Section 6.3"];
    Move              b"MOVE"              false true  #[doc = "`MOVE`, defined in RFC4918, Section 9.9"];
    Options           b"OPTIONS"           true  true  #[doc = "`OPTIONS`, defined in RFC7231, Section 4.3.7"];
    OrderPatch        b"ORDERPATCH"        false true  #[doc = "`ORDERPATCH`, defined in RFC3648, Section 7"];
    Patch             b"PATCH"             false false #[doc = "`PATCH`, defined in RFC5789, Section 2"];
    Post              b"POST"              false false #[doc = "`POST`, defined in RFC7231, Section 4.3.3"];
    PropFind          b"PROPFIND"          true  true  #[doc = "`PROPFIND`, defined in RFC4918, Section 9.1"];
    PropPatch         b"PROPPATCH"         false true  #[doc = "`PROPPATCH`, defined in RFC4918, Section 9.2"];
    Put               b"PUT"               false true  #[doc = "`PUT`, defined in RFC7231, Section 4.3.4"];
    Rebind            b"REBIND"            false true  #[doc = "`REBIND`, defined in RFC5842, Section 6"];
    Report            b"REPORT"            true  true  #[doc = "`REPORT`, defined in RFC3253, Section 3.6"];
    Search            b"SEARCH"            true  true  #[doc = "`SEARCH`, defined in RFC5323, Section 2"];
    Trace             b"TRACE"             true  true  #[doc = "`TRACE`, defined in RFC7231, Section 4.3.8"];
    Unbind            b"UNBIND"            false true  #[doc = "`UNBIND`, defined in RFC5842, Section 5"];
    Uncheckout        b"UNCHECKOUT"        false true  #[doc = "`UNCHECKOUT`, defined in RFC3253, Section 4.5"];
    Unlink            b"UNLINK"            false true  #[doc = "`UNLINK`, defined in RFC2068, Section 19.6.1.3"];
    Unlock            b"UNLOCK"            false true  #[doc = "`UNLOCK`, defined in RFC4918, Section 9.11"];
    Update            b"UPDATE"            false true  #[doc = "`UPDATE`, defined in RFC3253, Section 7.1"];
    UpdateRedirectRef b"UPDATEREDIRECTREF" false true  #[doc = "`UPDATEREDIRECTREF`, defined in RFC4437, Section 7"];
    VersionControl    b"VERSION-CONTROL"   false true  #[doc = "`VERSION-CONTROL`, defined in RFC3253, Section 3.5"];
}

Incidentally, Teepee will support all the methods in the IANA HTTP method registry.

This has in its favour that it is something that has good ergonomics for the standard case. It does, however, have a slight weakness in that when a new method is added to the registry, code that was already using it may break, depending on how things were done. Pattern matching may break, for example. But this is a rare thing, so I’m not terribly distressed about it. It does, however, stand against it—strict semver would require a new major release to add a new item to the method enum.

Summary

In the end, I think that what rust-http does at present is sound; it just needs to be expanded to cover the entire HTTP method registry, and expanded to also keep track of whether a method is safe and/or idempotent.

Pending a suitable outcome to discussion on Reddit, the second approach is what I intend to implement.

Teepee design: header representation

2014-05-10T00:00:00+00:00

Header representation is a critical matter to Teepee’s design: it is uncompromisingly strongly typed, but there must be tradeoffs. After trying quite a few different schemes at length, I have settled upon quite a novel scheme which I believe to optimally balance all considerations.

This article is part of the series on the rust-http redesign, Teepee.

The current situation

Here’s how rust-http currently handles headers:

struct HeaderCollection {
    // … plenty of strongly typed headers …
    connection: Option<Vec<Connection>>,
    content_type: Option<MediaType>,
    // … plenty more s.t. headers … and then at the end:
    extensions: HashMap<StrBuf, StrBuf>
}

… and then the header collection is placed into the request or response struct as the headers field. Thus the value of the Content-Length header of a response is response.headers.content_length and is of type Option<uint>. Note also that the request and response have different header collections, as there are various headers appropriate only to a request or response.

At present, headers are parsed proactively; valid headers are placed in a Some value and invalid or missing values are stored as None.

Criteria

There are a few criteria that I considered for header design in Teepee:

Performance must be “good”.
Ergonomics for using standardised headers must be “good”.
Ergonomics for using extension headers must be “good”.
Extension headers should be strongly typed.
Badly formatted headers must be accessible in some way.
A shared access technique for both standardised and extension headers is desirable (for backwards compatibility as more headers are added to the Teepee libraries), especially if it allows headers to be used as multiple types.

A sample of schemes tried

Bad header field value handling

For various reasons, it is necessary that invalid headers be able to be accessed, though normally an illegal value should be dropped. This was not done at all in rust-http, but is essential in Teepee. (For more fun, there are certain things to be aware of in parsing headers like that there may be multiple fields with the same name, but only if they would be equivalent to a single comma-separated list, with the difference that any illegal items should be dropped, or that in some cases part of a field may be invalid and be dropped while the rest is interpreted.)

Use a Option<Result<T, Vec<u8>>> (or similar) type instead of Option<T>. Rejected outright for (a) bad ergonomics, for almost no one can do anything with malformed values, so normal code must not be burdened with it; and (b) because partially bad headers would be lost.
Expose the unparsed values through callbacks that can be injected, or a conditions‐like scheme (minus the fail‐by‐default behaviour). Rejected because while the raw values can be accessed, doing so will be excessively difficult, leading to complaints when people do actually want it, as the layer or framework they are using may not have exposed it in an accessible way.
Store raw headers alongside the main place; for example, given request.headers, have request.raw_headers or request.headers.raw as a HashMap<Vec<u8>, Vec<Vec<u8>>> (that is, a mapping of field names to the values for that field) or a Vec<(Vec<u8>, Vec<u8>)> (that is, all of the header fields, completely raw). (Try saying “vec” ten times rapidly.) This isn’t ideal for the overhead, as it requires two heap allocations for each header field (the lowest level will still operate without allocations there; this is purely a feature of the high level) and must still store each header in raw form.

Of the options shown here, the last is probably the best. I have one further objection to it, though: it makes it too easy to access those raw values. I’m afraid that if I put that there, people might use it (!), and I don’t want that. (Yes, I’m stubborn about all this; I want to give people what’s good for them, not necessarily what they want. We’ll find out if I’m right in the years to come.) This can be remedied by having the raw fields private and only exposed through unsafe functions.

There is one other scheme, building upon the concepts of the third option, where the handling of bad header fields falls quite naturally out of the solution for header storage; I omit that here is it will be expanded upon later.

Header storage

I shall not bore you with the full details, though I have plenty more details should you be interested; here is a smattering of some of the things I’ve considered:

Fields for standard headers, a Map<StrBuf, StrBuf> for the rest: (a.k.a. “leave it as it is”) great performance, great ergonomics for standard headers, but extension headers are very much second class citizens, being weakly typed.
Fields for known headers, drop (or Map) unknown headers, multiple inheritance edition: very cool and perfect in almost every way… except the wait for a feature that will probably never be implemented.
Fields for known headers, drop (or Map) unknown headers, say‐what‐you‐want edition: (require people to declare the headers they expect to deal with and their types, producing a struct for each case). Great performance, poor ergonomics of declaration (especially for its unfamiliarity), useless for code reuse.
The other thing I’m about to talk about: good enough performance, good ergonomics for all headers, treats all headers alike; flawless with regards to backwards compatibility; allows read and write access to raw headers where necessary.
A combination of fields for standard headers with that thing I’m about to talk about for the rest: gets better performance for standard headers, but at the cost of consistency, future‐proofing and raw access to standard headers. (In short, a cute idea but not worth it.)

The scheme I have settled on

This scheme is not perfect, but weighing up the advantages and disadvantages I am fairly confident that it is fairly optimal for Rust.

Here are some examples of the API we end up with. We get nice and easy typed access:

headers.get(CONNECTION); // -> Option<Vec<Connection>>
headers.get_ref(LOCATION); // -> Option<&url::Url>
headers.get_mut_ref(DATE); // -> Option<&mut time::Tm>
headers.set(LOCATION, url /* url::Url */);

(get() necessarily takes &mut self, so you can’t borrow more than one header at a time—that’s a large part of why get is there, which clones the object before returning it.)

It would be possible to expose raw values via unsafe functions…

unsafe {
    headers.get_raw("connection"); // -> Option<&[Vec<u8>]>
    headers.set_raw("connection",
                    vec!(Vec::from_slice(bytes!("close"))));
}

But that’s actually not necessary! We can use get and set with a different argument and, from that, a different return type:

headers.get(raw("connection")); // -> Option<Vec<Vec<u8>>>
headers.set(raw("connection"),
            vec!(Vec::from_slice(bytes!("close"))));

(This will be slower than exposing raw values directly, as it is treating Vec<Vec<u8>> as a strong header type and so incurring an extra conversion; therefore in practice extra unsafe methods will probably be provided for raw access. But it demonstrates the principle of the thing.)

And the good part—it’s in there, so now we can access it as another strong type:

assert_eq!(headers.get(CONNECTION), Some(vec!(Close)));

How is this done? Internally it maintains a hash map of values which may be either raw or Box<Header> (a trait object); initially everything is raw, but as you access them they are parsed into the appropriate header type and stored; if you try to access a value as a different type, it is converted to the HTTP wire format and then parsed as the new type, and stored if that succeeded.

All this is pretty magic, and something that you couldn’t easily achieve in most languages, especially with regards to the type inference—but it’s something that can be achieved in Rust in a very convenient and completely safe way! (OK, so it takes just a smidgeon of unsafe code, implementing the Any*Ext traits—just a straight copy of the implementations for &Any et al. But the interface it exposes is absolutely solid and crash‐proof.)

A proof‐of‐concept implementation

I could yammer on and on about how it works, but I don’t think it’s necessary. Here’s my proof of concept in all its half‐baked glory:


/// The data type of an HTTP header for encoding and decoding.
pub trait Header: Any {
    fn parse_header(s: &[Vec<u8>]) -> Option<Self>;

    /// Introducing an `Err` value that does *not* come from the writer is incorrect behaviour and
    /// may lead to task failure in certain situations. (The primary case where this will happen is
    /// accessing a cached Box<Header> object as a different type; then, it is shoved into a buffer
    /// through fmt_header and then back into the new type through parse_header. Should the
    /// fmt_header call have return an Err, it will fail.
    fn fmt_header(&self, f: &mut fmt::Formatter) -> fmt::Result;
}

// I'd prefer this as another trait with an implementation of Header for something satisfying that
// trait, but I can't do that because of those pesky "conflicting implementations" things.
macro_rules! require_single_field {
    ($field_values:expr) => ({
        let mut iter = $field_values.iter();
        match (iter.next(), iter.next()) {
            (Some(ref field_value), None) => field_value.as_slice(),
            _ => return None,
        }
    })
}

// impl copied from std::any. Not especially nice, sorry :-(
impl<'a> AnyRefExt<'a> for &'a Header {
    #[inline]
    fn is<T: 'static>(self) -> bool {
        use std::intrinsics::TypeId;
        // Get TypeId of the type this function is instantiated with
        let t = TypeId::of::<T>();

        // Get TypeId of the type in the trait object
        let boxed = self.get_type_id();

        // Compare both TypeIds on equality
        t == boxed
    }

    #[inline]
    fn as_ref<T: 'static>(self) -> Option<&'a T> {
        if self.is::<T>() {
            Some(unsafe { self.as_ref_unchecked() })
        } else {
            None
        }
    }
}

trait UncheckedAnyRefExt<'a> {
    /// Returns a reference to the boxed value, assuming that it is of type `T`. This should only be
    /// called if you are ABSOLUTELY CERTAIN of `T` as you will get really wacky output if it’s not.
    unsafe fn as_ref_unchecked<T: 'static>(self) -> &'a T;
}

// Actually, I want to move this upstream. I have a patch ready which adds as_ref_unchecked,
// as_mut_unchecked and move_unchecked. What do you reckon? Opinions solicited. Of course, because
// this is Header rather than Any, I can't actually *use* such a thing myself if it's on Any...
impl<'a> UncheckedAnyRefExt<'a> for &'a Header {
    #[inline]
    unsafe fn as_ref_unchecked<T: 'static>(self) -> &'a T {
        use std::raw::TraitObject;
        use std::cast::{transmute, transmute_copy};

        // Get the raw representation of the trait object
        let to: TraitObject = transmute_copy(&self);

        // Extract the data pointer
        transmute(to.data)
    }
}

trait UncheckedAnyMutRefExt<'a> {
    /// Returns a reference to the boxed value, assuming that it is of type `T`. This should only be
    /// called if you are ABSOLUTELY CERTAIN of `T` as you will get really wacky output if it’s not.
    unsafe fn as_mut_unchecked<T: 'static>(self) -> &'a mut T;
}

impl<'a> UncheckedAnyMutRefExt<'a> for &'a mut Header {
    #[inline]
    unsafe fn as_mut_unchecked<T: 'static>(self) -> &'a mut T {
        use std::raw::TraitObject;
        use std::cast::{transmute, transmute_copy};

        // Get the raw representation of the trait object
        let to: TraitObject = transmute_copy(&self);

        // Extract the data pointer
        transmute(to.data)
    }
}

/// Standard usage of this is a marker type, like this:
///
/// ```rust
/// // The header data type
/// pub struct Foo {
///     ...
/// }
///
/// impl Header for Foo {
///     ...
/// }
///
/// // The marker type for accessing the header and specifying the name
/// pub struct FOO;
///
/// impl HeaderMarker<Foo> for FOO {
///     fn header_name(&self) -> SendStr {
///         Slice("foo")
///     }
/// }
/// ```
///
/// Then, accessing the header is done like this:
///
/// ```rust
/// let foo = request.headers.get(FOO);
/// request.headers.set(FOO, foo);
/// ```
///
/// And lo! `foo` is a `Foo` object corresponding to the `foo` header in the request.
///
/// Authors are strongly advised that they should not implement `HeaderMarker<T>` for more than one
/// `T` on the same type.
pub trait HeaderMarker<OutputType: Header + 'static> {
    /// The name of the header that shall be used for retreiving and setting.
    ///
    /// Normally this will be a static string, but occasionally it may be necessary to define it at
    /// runtime, for dynamic header handling.
    fn header_name(&self) -> SendStr;
}

/// The data type for the ``expires`` header.
#[deriving(Clone, Eq, Show)]
pub enum Expires {
    /// The Expires header had an invalid format, which MUST be interpreted as “in the past”.
    Past,
    /// A valid Expires header date.
    ExpiresDate(Tm),
}

impl Header for Expires {
    fn parse_header(raw: &[Vec<u8>]) -> Option<Expires> {
        require_single_field!(raw);
        match Header::parse_header(raw) {
            Some(tm) => Some(ExpiresDate(tm)),
            None => Some(Past),
        }
    }

    fn fmt_header(&self, f: &mut fmt::Formatter) -> fmt::Result {
        match *self {
            Past => write!(f.buf, "0"),
            ExpiresDate(ref tm) => tm.fmt_header(f),
        }
    }
}

impl Header for Tm {
    fn parse_header(raw: &[Vec<u8>]) -> Option<Tm> {
        let raw = require_single_field!(raw);
        let raw = match std::str::from_utf8(raw) {
            Some(raw) => raw,
            None => return None,
        };
        // XXX: %Z actually ignores any timezone other than UTC. Probably not a good idea?
        match strptime(raw, "%a, %d %b %Y %T %Z") {  // RFC 822, updated by RFC 1123
            Ok(time) => return Some(time),
            Err(_) => ()
        }

        match strptime(raw, "%A, %d-%b-%y %T %Z") {  // RFC 850, obsoleted by RFC 1036
            Ok(time) => return Some(time),
            Err(_) => ()
        }

        match strptime(raw, "%c") {  // ANSI C's asctime() format
            Ok(time) => Some(time),
            Err(_) => None,
        }
    }

    fn fmt_header(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f.buf, "{}", self.to_utc().strftime("%a, %d %b %Y %T GMT"))
    }
}

impl Header for Box<Header> {
    fn parse_header(_raw: &[Vec<u8>]) -> Option<Box<Header>> {
        // Dummy impl; XXX: split to ToHeader/FromHeader?
        None
    }

    fn fmt_header(&self, f: &mut fmt::Formatter) -> fmt::Result {
        self.fmt_header(f)
    }
}

impl<'a> Header for &'a Header {
    fn parse_header(_raw: &[Vec<u8>]) -> Option<&'a Header> {
        // Dummy impl; XXX: split to ToHeader/FromHeader?
        None
    }

    fn fmt_header(&self, f: &mut fmt::Formatter) -> fmt::Result {
        self.fmt_header(f)
    }
}

macro_rules! impl_header_from_fromstr {
    ($ty:ty) => (impl Header for $ty {
        fn parse_header(raw: &[Vec<u8>]) -> Option<$ty> {
            std::str::from_utf8(raw).and_then(from_str)
        }

        fn fmt_header(&self, f: &mut fmt::Formatter) -> fmt::Result {
            write!(f.buf, "{}", self)
        }
    })
}

macro_rules! header {
    ($struct_ident:ident, $header_name:expr, $output_type:ty) => (
        pub struct $struct_ident;

        impl HeaderMarker<$output_type> for $struct_ident {
            fn header_name(&self) -> SendStr {
                Slice($header_name)
            }
        }
    )
}

header!(EXPIRES, "expires", Expires)
header!(DATE, "date", Tm)

pub struct RawHeaderMarker {
    pub name: SendStr,
}

/// This grants you access to raw header values:
///
/// ```rust
/// # headers = Headers::new();
/// headers.set(raw("expires"), "bar".as_bytes().iter().map(|&x| x).collect());
/// assert_eq!(headers.get(raw("expires")).to_slice(), bytes!("bar"));
/// ```
///
/// That is, the header collection values are interpreted as `Vec<u8>`.
///
/// You should largely avoid doing this.
pub fn raw<S: IntoMaybeOwned<'static>>(s: S) -> RawHeaderMarker {
    RawHeaderMarker {
        name: s.into_maybe_owned()
    }
}

impl HeaderMarker<Vec<Vec<u8>>> for RawHeaderMarker {
    fn header_name(&self) -> SendStr {
        self.name.clone()
    }
}

impl Header for Vec<Vec<u8>> {
    fn parse_header(raw: &[Vec<u8>]) -> Option<Vec<Vec<u8>>> {
        Some(raw.iter().map(|x| x.clone()).collect())
    }

    fn fmt_header(&self, f: &mut fmt::Formatter) -> fmt::Result {
        let mut first = true;
        for field in self.iter() {
            try!(f.buf.write(field.as_slice()));
            if !first {
                try!(f.buf.write([',' as u8, ' ' as u8]));
            }
            first = false;
        }
        Ok(())
    }
}

enum Item {
    /// A raw, unparsed header. Uses the ISO-8859-1 character set, but could contain things in other
    /// character sets according to the rules of RFC 2047, e.g. in a *TEXT rule (RFC 2616 grammar).
    Raw(Vec<Vec<u8>>),

    /// A strongly typed header which has been parsed from the raw value.
    Typed(Box<Header>:'static),
}

/// A collection of HTTP headers.
struct Headers {
    data: HashMap<SendStr, Item>,
}

impl Headers {
    fn new() -> Headers {
        Headers {
            data: HashMap::new(),
        }
    }
}

/* I think I'll skip the Index implementation, though it works OK.
impl<H: Header + Clone + 'static, M: HeaderMarker<H>> Index<M, Option<H>> for Headers {
    fn index(&self, header_marker: &M) -> Option<H> {
        let mut_self = unsafe { std::cast::transmute_mut(self) };
        match mut_self.mostly_get(header_marker) {
            Some(&Typed(ref h)) => Some(unsafe { h.as_ref_unchecked::<H>() }.clone()),
            _ => None,
        }
    }
}
*/

impl<H: Header + Clone + 'static, M: HeaderMarker<H>> Headers {
    pub fn get<'a>(&'a mut self, header_marker: M) -> Option<H> {
        // At this point, we know that item is None, or Some(&mut Typed(h)) and that h.is::<H>().
        // On that basis, we can use as_ref_unchecked instead of as_ref, to save a virtual call.
        match self.mostly_get(&header_marker) {
            Some(&Typed(ref h)) => Some(unsafe { h.as_ref_unchecked::<H>() }.clone()),
            _ => None,
        }
    }
}

impl<H: Header + 'static, M: HeaderMarker<H>> Headers {
    fn mostly_get<'a>(&'a mut self, header_marker: &M) -> Option<&'a mut Item> {
        let name = header_marker.header_name();
        let item = match self.data.find_mut(&name) {
            // Yes, there's a header… or something… by that name
            Some(v) => v,
            // Header? There is no header!
            None => return None,
        };

        let (insert_parsed, parsed): (bool, Option<H>) = match *item {
            // We've parsed this header before, as some type or other.
            // Question is: is it the right type?

            // Yes, it's the right type, so we can immediately return it.
            // Well, we could, except that that makes the borrow checker keep the item borrow alive,
            // preventing the insert in the other cases. So we must refactor it to return at the
            // end instead.
            Typed(ref h) if h.is::<H>() => {
                (false, None)
            },

            // No, it was parsed as a different type.
            // Very well then, we will turn it back into a string first.
            Typed(ref h) => {
                let raw = fmt_header(h);
                (true, Header::parse_header([raw]))
            },

            // We haven't parsed it before, so let's have a go at that.
            Raw(ref raw) => (true, Header::parse_header(raw.as_slice())),
        };

        if insert_parsed {
            match parsed {
                // It parsed. Let's store the new value (replacing the raw one)
                Some(v) => *item = Typed(box v),
                // If the header doesn't parse, that's the same as it being absent.
                None => return None,
            }
        }
        Some(item)
    }

    pub fn get_ref<'a>(&'a mut self, header_marker: M) -> Option<&'a H> {
        match self.mostly_get(&header_marker) {
            Some(&Typed(ref h)) => Some(unsafe { h.as_ref_unchecked::<H>() }),
            _ => None,
        }
    }

    pub fn get_mut_ref<'a>(&'a mut self, header_marker: M) -> Option<&'a mut H> {
        match self.mostly_get(&header_marker) {
            Some(&Typed(ref mut h)) => Some(unsafe { h.as_mut_unchecked::<H>() }),
            _ => None,
        }
    }

    /// Set the named header to the given value.
    pub fn set(&mut self, header_marker: M, value: H) {
        self.data.insert(header_marker.header_name(), Typed(box value));
    }

    /// Remove a header from the collection.
    /// Returns true if the named header was present.
    pub fn remove(&mut self, header_marker: &M) {
        self.data.remove(&header_marker.header_name());
    }
}

/// An adapter which provides `std::fmt::Show` as equivalent to `Header.fmt_header`, so that you can
/// actually *use* the thing.
struct HeaderShowAdapter<'a, H>(pub &'a H);

impl<'a, H: Header> fmt::Show for HeaderShowAdapter<'a, H> {
    #[inline]
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        let HeaderShowAdapter(h) = *self;
        h.fmt_header(f)
    }
}

#[inline]
pub fn fmt_header<H: Header>(h: &H) -> Vec<u8> {
    format_args!(format_but_not_utf8, "{}", HeaderShowAdapter(h))
}

// Parallel to ::std::fmt::{format, format_unsafe}, but returning Vec<u8> rather than Box<str>.
#[inline]
#[doc(hidden)]
pub fn format_but_not_utf8(args: &fmt::Arguments) -> Vec<u8> {
    let mut output = std::io::MemWriter::new();
    fmt::write(&mut output as &mut Writer, args).unwrap();
    output.unwrap()
}

fn main() {
    let mut headers = Headers::new();
    fn expect<H: Header + std::fmt::Show + Eq>(h: Option<H>, h_expected: H, raw: &[u8]) {
        let h = h.unwrap();
        assert_eq!(fmt_header(&h).as_slice(), raw);
        assert_eq!(h, h_expected);
    }
    fn expect_none<H: Header>(h: Option<H>) {
        assert!(h.is_none());
    }

    expect_none(headers.get(EXPIRES));
    headers.set(EXPIRES, Past);
    expect(headers.get(EXPIRES), Past, bytes!("0"));
    expect(headers.get(raw("expires")), vec!(vec!('0' as u8)), bytes!("0"));
    expect(headers.get(EXPIRES), Past, bytes!("0"));
    headers.remove(&EXPIRES);
    expect_none(headers.get(EXPIRES));

    expect_none(headers.get(DATE));
    let now = time::now();
    let now_raw = fmt_header(&now);
    headers.set(DATE, now.clone());
    expect(headers.get(DATE), now.clone(), now_raw.as_slice());
}

Assessment

Here’s how this goes on the criteria listed:

Performance must be “good”.
The good news: headers are parsed lazily. The bad news: two or three allocations per header field, plus an allocation and a deallocation for every type change (including that lazy parsing). Performance is, I think, the weakest part of this scheme, but I don’t believe anything faster is possible while still ticking all the other boxes.
Ergonomics for using standardised headers must be “good”.
There’s a certain perfection about field access… but this will do dandy.
Ergonomics for using extension headers must be “good”.
As above.
Extension headers should be strongly typed.
Got it.
Badly formatted headers must be accessible in some way.
Readily accessible, though accessing a header as a strongly typed value will destroy the raw stuff, so anyone caring about such things will need to be careful.
A shared access technique for both standardised and extension headers is desirable (for backwards compatibility as more headers are added to the Teepee libraries), especially if it allows headers to be used as multiple types.
This solution absolutely shines at this: if two libraries want to access the same header as a different type, it will happily convert between them, just sacrificing a little bit of performance for the benefit.

Unless anyone finds any significant problems with this scheme, or a better scheme, Teepee will be using this type of header collection.

Remaining questions

Should access to the raw values be granted via unsafe functions, or should typed access be genuinely the only interface exposed, leaving an implementation for Vec<Vec<u8>> as the way of accessing raw values? (Side note: without that, the library will accept multiple header fields with the same name, but will merge them separated by a comma when writing them. This is perfectly spec‐compliant behaviour, but it is conceivable that someone may wish to genuinely write multiple header fields with the same name and not merge them. I guess I probably just talked myself into answering this question. Still, without such behaviour, the header collections for writing could skip the possibility of a raw value and just use a Box<Header> directly instead of Item, which would produce moderately trivial performance improvements. So I guess the quesiton is still open. Can anyone think of a way that one could use the std::fmt interface and yet still allow the writing of multiple header fields from the one value?)
Is jumping on the tails of std::fmt a good idea? (Bear in mind that std::fmt is UTF-8‐centric while HTTP is ISO-8859-1‐centric, though I do not really know if that influences thing.)
At present, the header collections for requests and responses have different fields; should the header markers use phantom types to specify that they are only legal for requests or responses (or both)? That would make it so that request.headers.set(REFERER, …) would work while response.headers.set(REFERER, …) would fail to compile.

I now invite you to join in the discussion at /r/rust.

Teepee design: `Status-Line`, take two

2014-04-25T00:00:00+00:00

My first look at the Status-Line kept largely to what rust-http had done; some great discussion came up and a conceptual flaw in my models was revealed. Now I present some better options.

This article is part of the series on the rust-http redesign, Teepee.

Of the first article, most details are superseded here; the most important part that is still definitely in place is the status code class part.

The criticism and responses to it

The main point of criticism levelled at the designs I posted is that it was still treating the Reason-Phrase too importantly.

Some argue in favour of dropping the reason phrase altogether. That I am not willing to do, citing the 451 example for justification. No, the reason phrase must remain around, somewhere.

A common suggestion is splitting the status code and reason phrase apart. This is looking like a healthy option. It is one that had not occurred to me; I had become too tied up in their correlation, that being what I had been working with in the past in rust-http.

Library designers, beware—your own perspective will become warped over time so that you often see only slight variations from what you are doing. Getting fresh eyes on a solution is marvellous; I am very glad that I wrote about this matter. Thank you, everyone, for the feedback: Teepee will be the stronger for it.

The new design

I now present a new design, influenced strongly by the feedback received.

At the lowest level, rather than giving a single Status object, the code will work with the pair (StatusCode, ReasonPhrase).

Responses received by the client will have both of these:

struct Response {
    status_code: StatusCode,
    reason_phrase: ~str,
    ...
}

Responses a server is writing will have both fields, but with the reason phrase optional:

struct Response {
    status_code: StatusCode,
    reason_phrase: Option<~str>,
    ...
}

Normally people will just set the status code. If the reason phrase is None, the appropriate value will be inferred, or for unknown statuses, something like “<no reason phrase given>”.

impl StatusCode {
    fn default_reason(&self) -> Option<&'static str> {
        match self.to_u16() {
            100 => Some("Continue"),
            101 => Some("Switching Protocols"),
            102 => Some("Processing"),
            200 => Some("OK"),
            ...,
            _ => None,
        }
    }
}

I am satisfied with this design. Now the only question remaining is how StatusCode itself is represented.

The `StatusCode` part: a few designs

There are a few possibilities for the Status-Code part; I present here a few, each with its benefits and drawbacks. Their ordering is not significant.

#1: an enum with an extension code variant

pub enum StatusCode {
    Continue,
    SwitchingProtocols,
    Ok,
    ...,
    ExtensionCode(u16),
}

This is similar to what is done in rust-http. It takes four bytes, while the others take two bytes.

This maps extremely well onto the RFC 2616 definition of Status-Code:

      Status-Code    =
            "100"  ; Section 10.1.1: Continue
          | "101"  ; Section 10.1.2: Switching Protocols
          | "200"  ; Section 10.2.1: OK
[many fields omitted for brevity]
          | extension-code

      extension-code = 3DIGIT

However, this has the backwards‐compatibility problem (one I identified earlier but forgot to write down in my previous article) that when a new status 103 Newly Registered Value is registered, things will change from using ExtensionCode(103) to using NewlyRegisteredValue. It’s a very rare case, but very significant.

#2: a 500‐variant C‐style enum

pub enum StatusCode {
    Continue = 100,
    SwitchingProtocols = 101,
    Processing = 102,
    Code103 = 103,
    Code104 = 104,
    ...,
    Ok = 200,
    ...,
    Code599 = 599,
}

This scheme has the slight weakness that when a status becomes registered, its variant will change; this is not, however, a severe backwards‐compatibility problem, for we would leave static aliases behind (e.g. pub static Code103: StatusCode = NewlyRegisteredValue). That could also be used to take care of cases like 451.

#3: a simple wrapping struct with associated statics

I handle this as a separate struct rather than just as u16, because there are only 500 valid values (100–599). Just as ~str applies checks to maintain the invariant that it is valid UTF-8, I will ensure that an illegal StatusCode can never exist. (That applies to the ExtensionCode variant of the enum solution too, by the way.)

pub struct StatusCode {
    code: u16,
}

// 1xx Informational
pub static CONTINUE: Status = Status { code: 100 };
pub static SWITCHING_PROTOCOLS: Status = Status { code: 101 };
pub static PROCESSING: Status = Status { code: 101 };

// 2xx Success
pub static OK: Status = Status { code: 200 };
...

The biggest thing this has going against it is that pattern matching won’t work.

#4: just a number

A variant of #3, one can cause pattern matching to work by using a number type directly:

pub type StatusCode = u16;

I am not in favour of this solution because of semantic mismatch. A status is not just a number—it has 500 valid values, as mentioned above, and I wish to maintain the invariant. Also things like + simply don’t make sense for a status. Sure, it’s shown as a number, but it is opaque data.

Conclusion

I am now satisfied with the overall design of the Status-Line handling. As to the representation of StatusCode, my current order of preferences, from favourite to least favourite, is #2, #3, #1, #4.

I now invite you to join in the discussion at /r/rust.

Update: the HTTP/2.0 situation

Something I neglected to do all along was refer to the HTTP/2.0 specification. (I looked at that part a few months ago, but had forgotten about it. I’ve been focusing more on the multiplexing aspects of HTTP/2.0, as that’s the main change and the main area of implementation difficulty for Teepee.) It has a couple of notable changes:

The reason phrase is gone altogether (Section 8.1.3.2, Response Header Fields). This confirms that separating the code and the reason is the correct thing to do.
1xx Informational class statuses are removed (Section 8.1.1, Informational Responses). This suggests that restricting the values structurally in the type system might not be quite so valuable, as only a subset of those values are actually valid in certain situations.

Teepee design: a careful look at the HTTP/1.1 `Status-Line`

2014-04-25T00:00:00+00:00

rust-http has a nice Status enum which provides good, strong typing of statuses, but alas, it is not without its issues. Let’s take a look at the Status-Line as it is defined.

This article is part of the series on the rust-http redesign, Teepee.

I say Status-Line, but actually I don’t care about part of it; Status-Line is defined in RFC 2616 (HTTP/1.1) as HTTP-Version SP Status-Code SP Reason-Phrase CRLF, but all I care about in this article is the Status-Code (e.g. 200) and its corresponding Reason-Phrase (e.g. OK).

The current position

At present, rust-http has http::status::Status. Here’s the implementation, distilled:

#[deriving(Eq, Clone)]
pub enum Status {
    // 1xx Informational
    Continue,
    SwitchingProtocols,
    Processing,  // WebDAV; RFC 2518

    // 2xx Success
    Ok,
    Created,
    ...,

    // 3xx Redirection
    MultipleChoices,
    MovedPermanently,
    ...,

    // 4xx Client Error
    BadRequest,
    Unauthorized,
    ...,

    // 5xx Server Error
    InternalServerError,
    NotImplemented,
    ...,

    UnregisteredStatus(u16, ~str),
}

impl Status {
    /// Get the status code
    pub fn code(&self) -> u16 {
        match *self {
            Continue => 100,
            ...,

            UnregisteredStatus(code, _)   => code,
        }
    }

    /// Get the reason phrase
    pub fn reason(&self) -> StrBuf {
        match *self {
            Continue => StrBuf::from_str("Continue"),
            ...,

            UnregisteredStatus(_, ref reason) => (*reason).clone(),
        }
    }

    /// Get a status from the code and reason
    pub fn from_code_and_reason(status: u16, reason: ~str) -> Status {
        let reason_lower = reason.to_ascii_lower();
        match (status, reason_lower.as_slice()) {
            (100, "continue") => Continue,
            ...,

            (_, _) => UnregisteredStatus(status, reason),
        }
    }
}

impl FromPrimitive for Status {
    /// Get a *registered* status code from the number of its status code.
    ///
    /// This will return None if an unknown code is passed.
    ///
    /// For example, `from_u64(200)` will return `OK`.
    fn from_u64(n: u64) -> Option<Status> {
        Some(match n {
            100 => Continue,
            ...,

            _   => { return None }
        })
    }
}

On the surface this approach looks very well; unfortunately, it turns out that it has some problems. For now, the main thing to note is that it treats 404 Not Found as the same as 404 not found but as distinct from 404 File Not Found.

The specification

Later on we’ll look at the practical applications of these things; for now let’s take a look at what the spec says. RFC 2616 (HTTP/1.1), section 6.1.1:

6.1.1 Status Code and Reason Phrase

   The Status-Code element is a 3-digit integer result code of the
   attempt to understand and satisfy the request. These codes are fully
   defined in section 10. The Reason-Phrase is intended to give a short
   textual description of the Status-Code. The Status-Code is intended
   for use by automata and the Reason-Phrase is intended for the human
   user. The client is not required to examine or display the Reason-
   Phrase.

   The first digit of the Status-Code defines the class of response. The
   last two digits do not have any categorization role. There are 5
   values for the first digit:

      - 1xx: Informational - Request received, continuing process

      - 2xx: Success - The action was successfully received,
        understood, and accepted

      - 3xx: Redirection - Further action must be taken in order to
        complete the request

      - 4xx: Client Error - The request contains bad syntax or cannot
        be fulfilled

      - 5xx: Server Error - The server failed to fulfill an apparently
        valid request

   The individual values of the numeric status codes defined for
   HTTP/1.1, and an example set of corresponding Reason-Phrase's, are
   presented below. The reason phrases listed here are only
   recommendations -- they MAY be replaced by local equivalents without
   affecting the protocol.

      Status-Code    =
            "100"  ; Section 10.1.1: Continue
          | "101"  ; Section 10.1.2: Switching Protocols
          | "200"  ; Section 10.2.1: OK
[many fields omitted for brevity]
          | extension-code

      extension-code = 3DIGIT
      Reason-Phrase  = *<TEXT, excluding CR, LF>

   HTTP status codes are extensible. HTTP applications are not required
   to understand the meaning of all registered status codes, though such
   understanding is obviously desirable. However, applications MUST
   understand the class of any status code, as indicated by the first
   digit, and treat any unrecognized response as being equivalent to the
   x00 status code of that class, with the exception that an
   unrecognized response MUST NOT be cached. For example, if an
   unrecognized status code of 431 is received by the client, it can
   safely assume that there was something wrong with its request and
   treat the response as if it had received a 400 status code. In such
   cases, user agents SHOULD present to the user the entity returned
   with the response, since that entity is likely to include human-
   readable information which will explain the unusual status.

The problem

I have marked the key phrases which the current implementation does not take properly into account. Simply, they boil down to this: the reason phrase is absolutely meaningless (sigh). I didn’t notice or didn’t pay attention to this first time around.

I also treated the field as case insensitive, having observed things written in lowercase sometimes, and having looked at how something else, I forget what, did it. This is incorrect; while in various places the spec defines things as case insensitive, the Reason-Phrase does not fall under such a category. The Reason-Phrase, in all its uselessness, must thus be considered case sensitive.

Well then—can we just drop the reason phrase?

The answer is both yes and no.

Yes: the integrity of the protocol itself is not affected if the reason phrase is altered; the semantics of the protocol lie purely in the Status-Code.

No: at the lowest level, we must provide the Reason-Phrase intact, because it may be meaningful. For example, someone might be writing an HTTP inspection tool for which they wish to report this value, or someone else might be writing a proxy, where changing the value would be a highly suspect move.

No: although the specification declares the number to be all that matters to a machine, there are cases (mostly with unregistered status codes) where people have used the same status code with multiple distinct meanings, and one may need to figure out what that meaning is. For example, the status code 451 has been used by Microsoft as “Redirect” in Exchange ActiveSync (niche, granted), and as “Unavailable for Legal Reasons” now.

So then, how do we reconcile this?

The possible consequences

By the way, this is a genuine problem and must be fixed; as it is, it will lead to people comparing statuses and suddenly finding bugs appearing when servers use different reason codes and all of a sudden their comparisons are not working. It might also get people comparing codes, as status.code() == 200 if they know of this deficiency. I do not want either of these to happen.

Some solutions

One solution to the primary symptoms is to change equality checking on a Status (the Eq implementation) to just compare the Status-Code and not the Reason-Phrase. Another method can be provided to check strict equality, inclusive of Reason-Phrase:

impl Eq for Status {
    fn eq(&self, other: &Status) -> bool {
        self.code() == other.code()
    }
}

impl Status {
    fn strict_eq(&self, other: &Status) -> bool {
        self.code() == other.code() && self.reason() == other.reason()
    }
}

This doesn’t sit especially well with me (in Rust, people expect equality comparison to check everything), but it would serve the purpose.

Another not incompatible solution is to make it so that unless the user needs the reason-phrase, a known status is normalised so that 200 All Good will come through as Ok rather than as UnregisteredStatus(200, ~"All Good"). This could be arranged with a response object from a request containing two fields, status: Status containing the normalised status and raw_status: Option<Status> containing, if it differed, the unnormalised status. At the lowest level, of course, the status code will not be normalised.

Either of these solutions will take care of the majority of cases, and each leaves a gap of potentially surprising (hence undesirable) behaviour. I am mildly inclined at present to go with both, but I would like opinions on the matter.

The status code class

This part is still relevant in take two.

One other thing I am going to add is better support for the class of a status. I don’t want people to be writing status.code() >= 400 && status.code() < 500; they should instead be able to write status.class() == ClientError.

The representation technique: enum or struct?

At present Status is an enum. It could also be represented as a simple struct with plenty of constants:

pub struct Status {
    code: u16,
    reason: std::str::SendStr,
}

// 1xx Informational
pub static CONTINUE: Status = Status { code: 100, reason: Slice("Continue"), };
pub static SWITCHING_PROTOCOLS: Status = Status { code: 101, reason: Slice("Switching Protocols"), };
pub static PROCESSING: Status = Status { code: 102, reason: Slice("Processing"), };

// 2xx Success
pub static OK: Status = Status { code: 200, reason: Slice("OK"), };
...

Is this feasible? Sure. Is it better? I don’t know.

Using a struct and constants would remove the clash on the name Ok which the Result type uses (it would become OK).
An enum uses less memory and is faster to compare on.
Using a struct and constants would allow slightly better ergonomics for others using unregistered statuses (they could create their own statics, rather than needing to have a function that generated it, owing to the ~str contained in the current model, though that could also be changed to use SendStr), leveling the field a little on what is an extremely rare case.
Pattern matching doesn’t work on struct statics (or for that matter, non‐C‐style enums). (Actually, it complains “unsupported constant expr” at the static site, rather than at the match! If you’re careful, you can probably find an ICE nearby—pnkfelix did. Bad.)

At present I’m mildly in favour of the status quo. Either way, if using SendStr, the built‐in statics would still be special if I applied the normalisation technique, unless I were also to retain a mapping of IDs to statuses, ([Option<&'static Status>, ..500], it would be, I suppose. Nasty global state.)

Bear in mind also that the method currently uses the same technique, and should, I think, use the same technique in general.

But really, I’m open to being swayed. Do you have an opinion? Same goes for the earlier questions.

Feel free to chip in to the discussion at /r/rust.

Introducing Teepee: the next step for rust-http

2014-04-23T00:00:00+00:00

rust-http was but an experiment, an essay in the craft. Here, at last, is the real thing: the Teepee project, a properly engineered HTTP toolkit.

This post has been sitting 90% complete for two months, and I’ve been promisng Teepee for around three months; I apologise to all those whose hopes and plans I have been delaying. Well, at least it’s here now, and it’ll be progressing faster again now.

Background

What is currently rust-http was from the beginning an experimental project. I wrote it for a combination of reasons:

I wanted to use an HTTP library with a strong typing discipline, where things like status codes would actually be status codes, not merely numbers and/or strings.
Having primarily used Django in the past, I had formed a vision of the web framework that I wanted. Shortly after writing down a first draft of that vision, I came across Rust again and assessed it more closely; I quickly decided that it was the first really suitable language for my vision.
… but Rust had no proper HTTP library—only a couple of toy libraries already just about to become obsolete, for these were the days when the new runtime was being finished up and the old runtime was still the default.

As a proof of concept (especially in the extensive usage of Rust’s type system), rust-http has been successful. It’s even—despite its limitations—a useful library, as is evidenced by (a) people using it, and (b) no one writing any direct competitor since.

Various things have changed since the initial experimentation. Most notably:

I understand the requirements for high-performance HTTP/1.1 and HTTP/2.0 better (I only understood HTTP/1.1 well before, and I only had general notions of what the significant differences were in HTTP/2.0).
I actually know Rust well now (rust-http was the first non-trivial Rust code I wrote—it shows).
The old runtime is now completely supplanted by the new runtime. (This happened some time ago and is, truth to tell, scarcely of note as concerns rust-http.)
I/O errors are now explicitly handled with things returning IoResult.
A TcpStream can now be cloned, allowing for storing the TcpStream in both the request and the response for a server (important for squeezing optimal performance out of an HTTP/1.1 server) and also the ability to work with the same stream across multiple tasks (which is crucial for HTTP/2.0).

What comes next for rust-http?

Having assessed the situation and the code extant at length, I have come to the following conclusion: rust-http must be completely rewritten.

It is expedient for this to happen immediately rather than later, to minimise technical debt.

The alternative was to keep a fair amount of the code base intact and alter it gradually into the new, deliberately designed form. This was my preferred approach until recently when I took a more careful look at how the existing structures would map onto what I was designing. When I did that I decided that at the very least the transfer model—currently one layer due to what TcpStream used to be and because I was at the time writing the buffered stream support myself and made the (quite natural) error of putting too much HTTP logic into it—this transfer model needs to be two distinct layers (the stream buffer, applicable to the entire connection, and the transfer encoding, applicable to only one request) to ensure it is maintainable, plus a third for the data structures (again, that is presently more or less blended in with the rest). I am convinced that replacing this part at least wholesale is the only sensible plan.

Beyond that, there are various other things which I would like to take this opportunity to redesign properly, such as the headers system. In the end, there is very little that I don’t want to do at least something to. (I believe I’m content with the Status struct, but even it will be redone with a macro rather than source code generation, for added coolness.)

So, then, I believe a complete rewrite is in order. Parts will be used as reference material, but no code will simply be copied over. It is safer thus. I will also be imposing the rule that all code that is written must include meaningful doc comments and must be tested. (Some of the key parts of rust-http are not tested; and bugs in an HTTP implementation can be a nightmare to pinpoint and fix.)

I recognise that in this I am vulnerable to second system syndrome. This is part of the reason why I want my plans reviewed by others before building it all.

It’s becoming a broader project

Now that I’ve become thoroughly acquainted with Rust, I’m more sure of my footing: I want to work more with Rust in the future and I believe it will make a very good platform for web-related work; and, what’s more, I intend to be in the forefront of this brave new world where writing fast, safe and correct websites is actually popular.

I have been mulling over what to do with rust-http. It’s got to the stage where I think separating the client and server into separate crates will be a good plan—they have some shared things (certain traits and utility methods and all the header types), but beyond that they are largely different. I think that splitting it into httpc (client) and httpd (server) crates, with a common crate containing their shared dependencies and data types (perhaps httpcommon), is probably the way forward (this is one of the points I’m least confident about).

In the end, this project will end up quite a bit like Spray is for Scala, and I think that having an overarching project which manages it all is probably a good idea.

My first desire was http.rs with its eponymous domain name, but alas! RNIDS (the operator of the .rs registry) has reserved http.rs, presumably to avoid any form of confusion.

The name I ended up settling upon was the only one which I came up with that I really liked: Teepee. Teepees are those American Indian tent thingies (also spelled tipi or tepee) but is absolutely nothing to do with Apache HTTP server (that connection was suggested by a couple of people independently afterwards—it was not my intent in any way). The real derivation of the name was the latter two letters of HTTP. The project will be based at teepee.rs.

Aside: registering .rs domains

A .rs domain is a nice-looking thing for a Rust project, but it’s not without its caveats. I went for the cheapest registrar I could find. Here are some of the problems I found:

Although registration is not limited to citizens of the Republic of Serbia (~~RNIDS General T&C, article 3, clause 2: “registrant can be domestic or foreign, a physical person or a legal entity”~~ [2017-01-01: link broken; the latest version doesn’t seem to make it as clear, at the very least]), in practice a national personal identification number is required of the registrant (~~RNIDS General T&C, article 8, clause 2~~) and for this registrar at least their form required such a thing. 0000000 satisfied it and citing article 3 I’m confident that I am in fact permitted to register a .rs domain myself.
Back to this particular registrar: their registration process did not use HTTPS. (Sure, the payment part did as they outsource that to 2checkout, but the part on their site did not.)
And then they emailed me my password (one hundred random characters, as is my practise) in plain text.

I’m not sure that I trust them much.

What are the important new features?

Proper multiple HTTP version support. The library will have good support for all HTTP versions: 0.9 (yes! really!), 1.0, 1.1 and 2.0.
Various “bad things” will be able to be dealt with, rather than being dropped or otherwise inaccessible—e.g. bad headers,
System boundaries will be more accurately segregated, abstracting away HTTP’s transfer details more completely than it is presently done and leaving a more sound API.
The tools for working with HTTP at a lower level will be more solid, and the higher level interface will sit cleanly on top of that lower level interface.
In consequence, extending it will be easier; WebSocket, for example, with its CONNECT upgrading, could only be implemented as a kludge in rust-http; such things will have proper support under Teepee.
Unified, pure‐Rust compilation model (using custom syntax extensions in place of source code generation).
Partially in consequence of all these things, Teepee’s HTTP libraries will be much easier to maintain. (I have decreed it so, and so it shall be!)
Under the umbrella of the Teepee project will also exist implementations of other related specifications like WebSocket and Server-Sent Events.

The new designs

I am still making some of the designs for Teepee. I will post them here as I complete them (or a little staggered); I will also be posting discussions of some of the matters that I think could do with particular discussion.

At this point, everything can be questioned and is subject to change. Comments and criticisms are welcome as I publish the designs.

If you wish to discuss the design, please join #teepee on irc.mozilla.org.

Are you interested?

In order to succeed, Teepee is going to need collaboration. There are a few classes of people that will be particularly valuable at this stage:

Servo developers: Teepee’s HTTP client needs to meet Servo’s requirements.
HTTP implementers: for both the client and the server, to ensure everything (client and server) is implemented in a reasonable manner.
Web app developers: HTTP server design—is it appropriate for the forms of servers that you want to write? (I’ve also got to be careful about the web framework story, that things are built in a way that doesn’t end up with frameworks reimplementing most things too much, or that it forces too much on you. That’s part of the reason for having a more layered approach this time round.)

If you have any feedback on rust-http as it is at present and things you like about it, don’t like about it, can’t stand about it, couldn’t bear to see changed about it—please speak up.

If you’re interested in getting involved with Teepee or have opinions to express, please join #teepee on irc.mozilla.org or drop me a line.

A little bit of personal plugging

In the long term, I hope the Teepee project as a whole to be both directly and indirectly a source of income for me and others in many and various ways (e.g. consulting, support, sponsorship of features). Whether or not this will pan out remains to be seen. At present I’m operating as a freelancer, spending most of my time working with a certain web agency (working mostly with Django), but I would love to spend more time working in/on Rust, rust-http and Teepee. If you should happen to want any development done in Rust or with rust-http, or should out of some extraordinary and inexplicable generosity desire to sponsor work on Teepee, I’m ready!

Actively soliciting donations in this way is not something that I have observed in the Rust community yet, so I’ll make myself a test case of what the response is and whether anything happens. (Even if it’s not quite a representative case, for by my observations good HTTP support is the biggest thing people want in Rust, beyond the language itself.)

If you want to hire or contract me for anything, please email me@chrismorgan.info.

If you want to donate to me directly to work on these things:

Via PayPal
Bitcoin: 14NcYrmxFCN9wAHhxCHkyt4PU9bqnveBUj
Dogecoin (well, why not?): DRfMMHVrnEdCmC5UaLFMp4DfMCLXMKLeSe
Gittip: chris-morgan

Finally: yes, this article is rather light on actual details, especially of the design. These will come in the coming weeks.

A broad vision for the Rust docs stack

2013-12-19T00:00:00+00:00

A presentation that I made to the San Fransisco Bay Area Rust meetup about the implementation side of Rust docs, and what I think should be used.

Yesterday (Tuesday evening in America, Wednesday afternoon in Australia) I gave a presentation to the San Fransisco Bay Area Rust meetup on the topic of the Rust documentation; Steve Klabnik, who spoke before me, dealt mostly with the contents of the documentation and I dealt much more with the tech stack which I thought would best achieve this goal (we agreed pretty well on the broad goals of the documentation).

I give especial thanks to Erick Tryzelaar for going to the trouble of arranging for this presentation to occur and for ironing out all the issues that arose in SF.

The slides

A video recording of the presentation is available on Air Mozilla; my talk begins at about 34:30 and continues until 55:00. The slides below are sufficient to get the significant majority of the important stuff from the presentation if you prefer.

View the slides here (SVG; you’ll need a “modern” browser to view them, or you could ask me to make a PDF of them for you)

The slides. Click, tap, arrow keys—whatever you like.

These slides are best viewed in Firefox. (I just tried Chromium and it renders the SVG incorrectly in a number of places—collapsing multiple sequential spaces into one in text, selecting the wrong font weight if you have the Ubuntu font installed locally, getting manual character offsets on the wrong characters in one of the slides… not good.)

If you’re interested in the tech that went into the slides themselves, I’ve written an article on that, too.

Now onto some of the other things I wanted to add more about in the languages section.

The Java documentation

Try to find the official Java documentation. Some places you might start:

https://www.java.com/: … well, if you dig deep enough you’ll find something (start with that tiny “developers” link in the footer).
https://duckduckgo.com/?q=java+docs: showing mostly API documentation.
https://encrypted.google.com/search?q=java%20docs: ditto.

As I observed, hunt hard enough and you’ll find https://docs.oracle.com/javase/. It doesn’t try particularly hard to head you in the right direction, but if you’re persistent you’ll find there are three main things it’s got.

The root of the docs: https://docs.oracle.com/javase/7/docs/index.html
OK, so… now where do I go? Oh! The table cells are clickable. How about we try Input/Output? Now what do we have?
- A few links to things in different places—the tutorials, for the odd topic a tech guide (often about actually implementing the feature itself rather than using it)
- API documentation links
- If you’re lucky, an example or two
- And a bit of changelog
And this is all of the Java Platform Standard Edition 7 Documentation.
The official tutorials: https://docs.oracle.com/javase/tutorial/
They’re decent for learning something, but frankly, I don’t have time to learn that thing. I already know Java, right? I just want to find out how to do this one particular thing. The tutorials also make dubious reference material, because they’re not geared to minimal examples; rather they’re designed for linear progression; the right one can also be difficult to find, assuming it even exists.
They’re simply not designed for discovery. Everyone already knows Java or they’re learning it at University (also known as “college” or “school”) from other sources, and so won’t need them.
The API documentation: https://docs.oracle.com/javase/7/docs/api/index.html
This is the heavy hitter of the lot, and the heavy hitter in more ways than one.
This is somehow the panacea for everything; you want to know what something can do? Here! Here’s what it can do!
Now read this forty page document which explains what each method does and gain enlightenment.

I come away not particularly impressed. Yes, the Java community can produce fairly good API documentation as they go, but it’s not designed for learning, which is a problem.

An example

I’ll draw this all together with an example; in this past semester for a Computer Graphics unit (“course” for Americans) I was doing some OpenGL stuff in Java and I wanted a spinner (you know, the number entry field with arrows) for the window chrome; I wanted to find something in the official docs about them.

I found a tutorial about them, but I really had no time to learn it that way. I had my project set up the way it was set up and setting up a new project for that stuff and going through it in a linear fashion simply couldn’t cut it—that would, all up, have taken hours.

Well, how about the JSpinner API documentation?

… that was even worse.

That’s why I conclude that pragmatism is what the Java docs are lacking.

(P.S. Remember that javadoc was once a pretty radical thing; Java has played a significant part in improving documentation standards, and the existence of javadoc means that Java projects do tend to have at least basic documentation. It’s really isn’t all bad.)

And as for that quote I used on pragmatism, here’s the source:

What sold me on Rust is how insanely practical it is…
— /u/Jurily on Reddit

The Python docs

Start at https://python.org/. Look in the navigation. Ah! There’s “documentation”.

https://python.org/doc/. That’s a nice and easy‐to‐read thing but pretty much everything you could want is there and clear.

Let’s try the Python 3 documentation. https://docs.python.org/3/. Again, everything is nice and clear.

Do I want to look at the standard library reference? https://docs.python.org/3/library/index.html

This is no auto‐generated javadoc; this is curated. Look through it all; I don’t mind what you look at there because I know it’s all pretty good.

There’s a general pattern of things that I want to point out:

good explanation at the top of the concepts and functionality;
genuine usage example—as in, bits of code that you can copy and verify to be doing what you want;
directing you to other good third party sources—even going so far as to say things like “this is a common use case, but it’s not in the standard library; here’s the technique most people are using” (Java, by comparison, is fairly leery of third‐party links);
oh yes, actually documenting the API as well (of course).

A few interesting examples (but don’t just look at these ones!):

subprocess uses sound categorisation of things rather than alphabetical order or mere order in source. It also gives good examples, comparisons with what other people may be used to, security considerations, &c.
itertools uses tables to concisely demonstrate the information that most people will come for, before then proceeding to more traditional documentation, followed at the end by a large number of recipes. (Don’t constrain yourself to text! Something that would be useful in places in the Python docs is images. Yes, you can use images in these sorts of documentation; for comparison, here is one I prepared a few years ago, when using Sphinx in the PortableApps.com Launcher documentation; a flowchart augments the textual description of the same thing.
collections explains the purposes, gives usage examples, gives alternative data structures and implementations, provides links for implementations in other languages or for old versions of Python, explains briefly where it’s good or bad to use this, and so on and so forth.

A couple of other important things it shows are the module index (we can make whatever indexes we desire) and search.

Chris Morgan’s blog posts tagged Rust

U+: pretty Unicode code point literals for Rust

The problem

Links

#[cfg_attr(…, path = …)] for platform-specific module implementations

<_>::v::<_>

Changelog

Tween: a middleware library experiment

The existing state of middleware libraries in Rust

The end result: Tween

What to do

Rust ownership, the hard way

Each object can be used exactly once. When you use an object it is moved to the new location and is no longer usable in the old.

When an object passes out of scope, it is destroyed and is no longer usable.

Blocks can produce a value which goes up one level of scope.

All objects have a lifetime which constrains which scopes they may be moved out of.

Lifetime positions in types

Lifetime positions in generic bounds

Exceptions to “each object can be used exactly once”

Copy and move semantics

Reference reborrowing

Summary

Quick tip: the #[cfg_attr] attribute

What is #[cfg_attr]?

Real‐world examples

Enabling nightly features in certain situations

Special attributes for bootstrapping rustc

Bonus: doc comments are truly sugar

Super bonus: adding macros into the mix!

Why your first FizzBuzz implementation may not work: an exploration into some initially surprising but great parts of Rust (though you still may not like them)

A simple functional implementation

Two types of strings? What is this?

Back to the fizzing and buzzing

How about we make it a String?

Making it a function

Enter Cow<str>

Writing our own enum and implementing std::fmt::Display efficiently

Conclusion

Appendix: other FizzBuzz techniques

1. match instead of if

2. std::fmt::Display trait objects

3. And if you really like fiddling with FizzBuzz optimisation…

Teepee design: the HTTP method

An apology

Relevant definitions

The token type

Design one: a struct and statics

Design two: back to the enum

Summary

Teepee design: header representation

The current situation

Criteria

A sample of schemes tried

Bad header field value handling

Header storage

The scheme I have settled on

A proof‐of‐concept implementation

Assessment

Remaining questions

Teepee design: Status-Line, take two

The criticism and responses to it

The new design

The StatusCode part: a few designs

#1: an enum with an extension code variant

#2: a 500‐variant C‐style enum

#3: a simple wrapping struct with associated statics

#4: just a number

Conclusion

Update: the HTTP/2.0 situation

Teepee design: a careful look at the HTTP/1.1 Status-Line

The current position

The specification

The problem

The possible consequences

Some solutions

The status code class

The representation technique: enum or struct?

Introducing Teepee: the next step for rust-http

Background

What comes next for rust-http?

`#[cfg_attr(…, path = …)]` for platform-specific module implementations

`<_>::v::<_>`

Quick tip: the `#[cfg_attr]` attribute

What is `#[cfg_attr]`?

How about we make it a `String`?

Enter `Cow<str>`

Writing our own enum and implementing `std::fmt::Display` efficiently

1. `match` instead of `if`

2. `std::fmt::Display` trait objects

The `token` type

Teepee design: `Status-Line`, take two

The `StatusCode` part: a few designs

Teepee design: a careful look at the HTTP/1.1 `Status-Line`