ninefive/vulcain

Use HTTP/2 Server Push to create fast and idiomatic client-driven REST APIs

Vulcain is a brand new protocol using HTTP/2 Server Push to create fast and idiomatic client-driven REST APIs.

An open source gateway server which you can put on top of any existing web API to instantly turn it into a Vulcain-compatible one is also provided!

It supports hypermedia APIs but also any "legacy" API by documenting its relations using OpenAPI.

Grab What You Need... Burn The REST!

• Introduction
  • Pushing Relations
  • Filtering Resources
• Gateway Server
  • Install
  • Configuration
  • Mapping a Non-Hypermedia API using OpenAPI
• Comparison with GraphQL and Other API Formats
• Using GraphQL as Query Language for Vulcain
• Demo API
• Cache Considerations
• Formal Specification
• Getting Help

The protocol has been published as an Internet Draft that is maintained in this repository.

A reference, production-grade, implementation gateway server is also available in this repository. It's free software (AGPL) written in Go. A Docker image is provided.

Introduction

Over the years, several formats have been created to fix performance bottlenecks impacting web APIs: over fetching, under fetching, the n+1 problem...

Current solutions for these problems (GraphQL, JSON:API's embedded resources and sparse fieldsets, ...) are smart network hacks for HTTP/1. But these hacks come with (too) many drawbacks when it comes to HTTP cache, logs and even security.

Fortunately, thanks to the new features introduced in HTTP/2, it's now possible to create true REST APIs fixing these problems with ease and class! Here comes Vulcain!

See also the comparison between Vulcain and GraphQL and other API formats.

Pushing Relations

Considering the following resources:

/books

{
    "member": [
        "/books/1",
        "/books/2"
    ]
}

/books/1

{
    "title": "1984",
    "author": "/authors/1"
}

/books/2

{
    "title": "Homage to Catalonia",
    "author": "/authors/1"
}

/authors/1

{
    "givenName": "George",
    "familyName": "Orwell"
}

The Preload HTTP header introduced by Vulcain can be used to ask the server to immediately push resources related to the requested one using HTTP/2 Server Push:

GET /books/ HTTP/2
Preload: /member/*/author

In addition to /books, a Vulcain server will use HTTP/2 Server Push to push the /books/1, /books/2 and /authors/1 resources!

Example in JavaScript:

const bookResp = await fetch("/books/1", { headers: { Preload: "/author" } });
const bookJSON = await bookResp.json();

// Returns immediately, the resource has been pushed and is already in the push cache
const authorResp = await fetch(bookJSON.author);
// ...

Full example, including collections, see also use GraphQL as query language for Vulcain.

Thanks to HTTP/2 multiplexing, pushed responses will be sent in parallel.

When the client will follow the links and issue a new HTTP request (for instance using fetch()), the corresponding response will already be in cache, and will be used instantly!

For non-hypermedia APIs (when the identifier of the related resource is a simple string or int), use an OpenAPI specification to configure links between resources. Tip: the easiest way to create a hypermedia API is to use the API Platform framework (by the same author as Vulcain).

More than 90% of users have devices supporting HTTP/2. However, for the remaining 10%, and for cases where using HTTP/2 Server Push isn't allowed such as when resources are served by different authorities, Vulcain allows to gracefully fallback to preload links, which can be used together with the 103 status code.

Query Parameter

Alternatively to HTTP headers, the preload query parameter can be used:

Filtering Resources

The Fields HTTP header allows the client to ask the server to return only the specified fields of the requested resource, and of the preloaded related resources.

Multiple Fields HTTP headers can be passed. All fields matching at least one of these headers will be returned. Other fields of the resource will be omitted.

Considering the following resources:

/books/1

{
    "title": "1984",
    "genre": "novel",
    "author": "/authors/1"
}

/authors/1

{
    "givenName": "George",
    "familyName": "Orwell"
}

And the following HTTP request:

GET /books/1 HTTP/2
Preload: /author
Fields: /author/familyName
Fields: /genre

A Vulcain server will return a response containing the following JSON document:

{
    "genre": "novel",
    "author": "/authors/1"
}

It will also push the following filtered /authors/1 resource:

{
    "familyName": "Orwell"
}

Query Parameter

Alternatively to HTTP headers, the fields query parameter can be used to filter resources:

See Also

• Mapping a non-hypermedia API using OpenAPI
• Cache considerations
• Using GraphQL with Vulcain
• Using other selectors such as XPath and CSS selectors for non-JSON documents (only JSON Pointer is currently supported by the Gateway Server)

License and Copyright

tl;dr:

• proprietary software can implement the Vulcain specification
• proprietary software can be used behind the Vulcain Gateway Server without having to share their sources
• modifications made to the Vulcain Gateway Server must be shared
• alternatively, a commercial license is available for the Vulcain Gateway Server

The specification is available under the IETF copyright policy. The Vulcain specification can be implemented by any software, including proprietary software.

The Vulcain Gateway Server is licensed under AGPL-3.0. This license implies that if you modify the Vulcain Gateway Server, you must share those modifications. However, the AGPL-3.0 license applies only to the gateway server itself, not to software used behind the gateway.

For companies not wanting, or not able to use AGPL-3.0 licensed software, commercial licenses are also available. Contact us for more information.

Credits

Created by Kévin Dunglas. Sponsored by Les-Tilleuls.coop.

Some ideas and code used in Vulcain's reference implementation have been taken from Hades by Gabe Sullice, an HTTP/2 reverse proxy for JSON:API backend.

See also the prior arts.