/specref

An open-source, community-maintained database of Web standards & related references.

Primary LanguageJavaScriptApache License 2.0Apache-2.0

Specref API Build Status

Specref is an open-source, community-maintained database of Web standards & related references.

API

The API to the service is very simple. It supports four operations which are:

  1. Get a set of bibliographic references:

    GET https://specref.herokuapp.com/bibrefs?refs=FileAPI,rfc2119

    parameters:

    refs=comma-separated,list,of,reference,IDs
callback=nameOfCallbackFunction

    returns: a JSON object indexed by IDs

    {
    "FileAPI": {
        "authors": [
            "Arun Ranganathan",
            "Jonas Sicking"
        ],
        "date": "12 September 2013",
        "deliveredBy": [
            {
                "shortname": "webapps",
                "url": "http://www.w3.org/2008/webapps/"
            }
        ],
        "edDraft": "http://dev.w3.org/2006/webapi/FileAPI/",
        "href": "http://www.w3.org/TR/FileAPI/",
        "id": "FileAPI",
        "publisher": "W3C",
        "status": "LCWD",
        "title": "File API"
    },
    "rfc2119": {
        "authors": [
            "S. Bradner"
        ],
        "date": "March 1997",
        "href": "http://www.ietf.org/rfc/rfc2119.txt",
        "id": "rfc2119",
        "publisher": "IETF",
        "status": "Best Current Practice",
        "title": "Key words for use in RFCs to Indicate Requirement Levels"
    }
}

  2. Search bibliographic references

    GET https://specref.herokuapp.com/search-refs?q=coffee

    parameters:

    q=search%20term
callback=nameOfCallbackFunction

    returns: a JSON object indexed by IDs

    {
    "rfc2324": {
        "authors": [
            "L. Masinter"
        ],
        "date": "1 April 1998",
        "href": "http://www.ietf.org/rfc/rfc2324.txt",
        "id": "rfc2324",
        "publisher": "IETF",
        "status": "Informational",
        "title": "Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0)"
    },
    "rfc7168": {
        "authors": [
            "I. Nazar"
        ],
        "date": "1 April 2014",
        "href": "http://www.ietf.org/rfc/rfc7168.txt",
        "id": "rfc7168",
        "publisher": "IETF",
        "status": "Informational",
        "title": "The Hyper Text Coffee Pot Control Protocol for Tea Efflux Appliances (HTCPCP-TEA)"
    }
}

    Used to get a set of bibliographic references that include the search term in any of their attributes. This is usefull to find specs related to a given area of study, specs by a given editor, etc.

  3. Reverse Lookup

    GET https://specref.herokuapp.com/reverse-lookup?urls=http://www.w3.org/TR/2012/WD-FileAPI-20121025/

    parameters:

    urls=comma-separated,list,of,reference,URLs.
callback=nameOfCallbackFunction

    returns: a JSON object indexed by URLs

    {
    "http://www.w3.org/TR/2012/WD-FileAPI-20121025/": {
        "authors": [
            "Arun Ranganathan",
            "Jonas Sicking"
        ],
        "date": "12 September 2013",
        "deliveredBy": [
            {
                "shortname": "webapps",
                "url": "http://www.w3.org/2008/webapps/"
            }
        ],
        "edDraft": "http://dev.w3.org/2006/webapi/FileAPI/",
        "href": "http://www.w3.org/TR/FileAPI/",
        "id": "FileAPI",
        "publisher": "W3C",
        "status": "LCWD",
        "title": "File API"
    }
}

    Notice this finds you the canonical version of a spec and not the precise version the URL points to. This is by design.

Aliases

Because of legacy references, case sensivity issues and taste, many entries have multiple identifiers. Thus an aliasing system was put in place. It isn't that complicated really: an identifier either points directly to the reference object or to another identifier (through the aliasOf property), recursively. All aliases are resolved (there are tests for that) and when you query the API for a reference you always get all the objects necessary to resolve it in the same response. So for example, https://specref.herokuapp.com/bibrefs?refs=rfc7230 responds with:

{
    "rfc7230": {
        "authors": [
            "R. Fielding, Ed.",
            "J. Reschke, Ed."
        ],
        "date": "June 2014",
        "href": "https://tools.ietf.org/html/rfc7230",
        "id": "rfc7230",
        "publisher": "IETF",
        "status": "Proposed Standard",
        "title": "Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing"
    }
}

while https://specref.herokuapp.com/bibrefs?refs=HTTP11 gives you:

{
    "HTTP11": {
        "aliasOf": "RFC7230",
        "id": "HTTP11"
    },
    "RFC7230": {
        "aliasOf": "rfc7230",
        "id": "RFC7230"
    },
    "rfc7230": {
        "authors": [
            "R. Fielding, Ed.",
            "J. Reschke, Ed."
        ],
        "date": "June 2014",
        "href": "https://tools.ietf.org/html/rfc7230",
        "id": "rfc7230",
        "publisher": "IETF",
        "status": "Proposed Standard",
        "title": "Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing"
    }
}

Which let's you get to the data by using a simple while loop over the response. The contract guaranteed by the API is to always let you resolve aliases.

Now whether you decide to display the result as [HTTP1], [rfc7230], [RFC7230], or even [1] is up to you. Of course, it's silly to reference both [HTTP1] and [rfc7230] in the same specification, but that's something for the editors and/or their tools to avoid.

CORS

CORS is enabled for all origins. By default the service returns JSON data, which is great but not convenient for browsers that do not support CORS yet. For those, simply adding the callback parameter with the name of the callback function you want will switch the response to JSON-P.

Examples

Some examples should help:

// get references for SVG, REX, and DAHUT
GET https://specref.herokuapp.com/bibrefs?refs=SVG,REX,DAHUT

// the same as JSON-P
GET https://specref.herokuapp.com/bibrefs?refs=SVG,REX,DAHUT&callback=yourFunctionName

If you need to find a reference ID (for either bibliographic or cross-references) you need to look for it on specref.org.

Updating & Adding

Daily Auto-Updating

There are scripts that pull fresh data from IETF, W3C, and WHATWG, and update their relevant files in the refs directory. These are now run daily. Their output is tested, comitted and deployed without human intervention. Content should now always be up to date.

Manual Changes

You can make modifications to the databases simply by editing either refs/biblio.json in the GitHub repository.

In order to do so you can fork the project and make a pull request to update it, or you can ask to be added as a project collaborator (we're pretty open about that) in which case you'll be able to push changes directly.

In both cases, all changes are automatically tested using travis. If all tests pass, the changes are immediately (and automatically) deployed.

Some rules to observe when editing the database files:

  • If you have commit rights, don't commit to master directly. Commit to a seperate branch (preferably to your fork) and send a pull request. Only merge the pull request to master once travis is green.
  • Don't remove entries unless you are 100% certain that no one is using it. Typically that only applies to cases in which you have just added a reference and want to remove it.
  • Don't duplicate entries. Make sure that what you want to add is not in the DB. If it is, add an alias.
  • Please use structured objects instead of raw strings as much as you possibly can.
  • The format for structured objects is described in JSON-schema. The schema is used to test new entries, so you better abide by it. :) (Note I'm still looking for a tool to turn the JSON schema into something more easily consumed by human beings. Let me know if you have an idea, or better yet, send a pull request.)
  • When you want to update an existing reference, if you see that it uses the old string style, please convert it to a structured object.
  • References in this database are expected to be to the “latest and greatest” version of a given specification. In some cases this may be the draft residing in the editor's repository, or it may be the latest snapshot as published by a Working Group into TR — this choice is left to your appreciation. If you really, really want to have a reference to a dated version, then use the versions property like so:
{
    "REFID": {
        "versions": {
            "YYYYMMDD": {
                "href": "http://..."
            }
        }
    }, //...
}

Licenses