Skipper is the open-source HTTP router we’ve created to help us rebuild the infrastructure behind Zalando’s customer-facing Fashion Store (“the Shop”). Developed with Go, it’s “go get” compatible and serves as a common entry point in front of the Shop’s service components. Skipper can be useful for non-Zalando teams who are trying to deploy a microservices architecture and want (or need) to decouple routing from service logic.

Skipper’s role is similar to the in-process router component in a typical MVC web application. It handles incoming requests by first selecting a route based on the request attributes — typically the method and the path — and then executing the associated controller action. What's different about Skipper is that the "controller" layer is moved to a set of independent services.

Why We Created Skipper

One of our goals is to establish an effective development and deployment cycle for multiple engineering teams working on the same product. Our approach is to run independently maintained and deployed services for web pages, so that we can compose them into a single website. Skipper is a supporting component of this architecture.

Initially, we sought an existing router solution that would fulfill our three main requirements:

• Support detailed request matching when selecting routes. This not only includes choosing routes by request path, but other request attributes like method and headers.
• Provide fair performance at scale. This covers the high traffic that periodically hits the Fashion Store, as well as the high number of routes we need to support our feature-rich website.
• Be able to continuously reconfigure the routing table for new settings without downtime or temporary performance penalty.

Why Not Vulcand?

Before building Skipper, we evaluated a few existing solutions. A strong candidate was Vulcand, a great project by Mailgun that we used in our earliest rounds of prototyping. We really liked how we could extend it with our own logic in the form of small pieces of middleware. Here, we apply the term “middleware” as it’s used in some well-known web MVC frameworks: as software logic applied to a general request flow or subset of routes (to change or augment requests and responses); or to execute related tasks. Think of these middleware as filters in signal processing, but applied to HTTP traffic.

A simplified, schematic representation might look like this:

route1: request type 1 -> filterA -> filterB -> http://service1.example.org
route2: request type 2 -> filterA -> fitlerC -> http://service2.example.org

For a few weeks we ran Vulcand in our prototyping setup while we focused on other Fashion Store components. In the end, we realized that we needed a better mechanism for streaming and dynamically updating the routing configuration. That’s when we decided to create our own version of the router by taking the best parts of Vulcand’s design and implementing the features we needed. This is how Skipper came to be.

Request Matching

In the core of Skipper's mechanism is a simple reverse proxy that copies incoming requests to different target service endpoints. The attributes of a particular HTTP request are used to decide the correct endpoint for that request and find it in the configured routing table. This lookup is the single most expensive logic that affects any route.

In trying different solutions for request matching, we found that path-based radix tree implementations produced the best results in terms of CPU time and memory footprint. This matched our requirements well, because most of our routes are identified primarily by path. The best-performing, off-the-shelf library we tried was httprouter, which in the Go world is a widely used router in web applications. It's an awesome piece of code, but fell short in meeting all our requirements.

The next candidate was httptreemux: an open-source, embeddable router package with an easy-to-integrate interface. It satisfied our requirements, but at the cost of somewhat higher memory usage. We only needed its tree lookup implementation, so we forked it, stripped off its unnecessary wrapping, added the logic to match the rest of the HTTP request attributes (method, headers, etc.), and continued using it in Skipper. We call this solution Pathmux.

Innkeeper: Route Configuration

Our Fashion Store features many promotions and custom pages, so frequent updates of our custom routes are necessary. Manually maintaining the data for all of these updates would be too laborious for a single team, so we created Innkeeper: a service that offers an integration point for Skipper and for any other services that also need to deal with Fashion Store routes. Innkeeper provides an easy-to-use JSON/HTTP API as well as OAuth2-based permission management for both humans and programs, and relies on our company-wide authentication system.

As an alternative route storage solution, Skipper also supports etcd clusters for simpler scenarios. For the very simplest cases, it can read configuration from the file system.

Conditioning Traffic with Filters

Skipper filters are the middleware in the request-processing pipeline. While most of the request handling is supposed to take place in the services behind Skipper, filters condition or augment the requests and responses for all routes and subsets of routes. This functionality covers generating session IDs, custom logging, XSRF protection, securing cookies and modifying the path, among other things. The mechanism is simple:

• the object representing the incoming request is passed to each filter of the route in the order of definition;
• then the proxy request is made to the backend service;
• the received response is passed to each filter in reverse order;
• and, finally, the response is returned to the original client.

One special case is when a filter handles the request on its own, and no proxy request is made to the backend. Filters form the main extension point of Skipper for adding any custom logic. One needs only to implement a simple interface for filter specifications in Go, deploy it with their custom-built version of Skipper, and run it.

Eskip: Our Own Routing Language

Earlier we showed a schematic description of a simple routing table, including the request matching, filtering and the backend service endpoints. During Skipper's development, we discovered that we could easily turn this notation into a formal syntax, and use it as a more readable way of describing routes. To support the syntax we created Eskip: a tool and language that parses, displays and update routes. Eskip is more DevOps-friendly than manual setups as a supplement to the default JSON interfaces, which are primarily targeting programmatic clients.

A simple routing table example in Eskip format:

apiRoute: Path("/api/*_") -> flowId() -> "https://api.example.org";
uiRoute: Path("/ui/*_") -> flowId() -> auth() -> "https://ui.example.org";

When installing the Skipper packages, also install the cmd/eskip subdirectory. Then try running `eskip -help` for hints.

What’s Next

In the upcoming months we'll go live gradually with the new infrastructure. Skipper will get hit by a fair amount of traffic soon, and we’ll let you know how it holds up!

We're hiring! Do you like working in an ever evolving organization such as Zalando? Consider joining our teams as a Frontend Engineer!