merkledag datastructure DSL

[jbenet]

~~ VERY MUCH WIP ~~

It would be very convenient to construct a simple DSL for merkledag datastructures. We have two options here, constructing a DSL that's nice to type (which we compile to an IPFS-LD schema), or just using JSON-LD syntax, with the IPFS-LD subset (simple conversion at that point).

A Merkledag DSL based on Go

Let's take the Go syntax. let's define things like Keys

rough DSL syntax

// NODE_TYPE is a type of node
// DATA_TYPE is a primitive data type
// LINK_TYPE is a merkle-link to a given NODE_TYPE: link<NODE_TYPE>
// "any" is _any_ LINK_TYPE
// NAME is a name token

// defining a new NODE_TYPE
type NAME struct {
  // the links, first class.
  NAME LINK_TYPE // COMMENT
  ...

  // the data segment, and what it contains
  data struct {
    NAME DATA_TYPE
  }
}

And here is an example, with explicit "Link<NODE_TYPE>" to make it clear

// version with explicit "link<TYPE>" to make it clear
type commit struct {
  parents   []link<commit>     // named "parent0" ... "parentN"
  author    link<authorship>   // link to an Authorship
  committer link<authorship>   // link to an Authorship
  object    link<any>          // what we version ("tree" in git)

  data struct {
    comment string // describes the commit
  }
}

The Link<NODE_TYPE> is a bit ugly. without it:

// version with implicit "Link<NODE_TYPE>"
type commit struct {
  parent    []commit     // "parent0" ... "parentN"
  author    authorship   // link to an Authorship
  committer authorship   // link to an Authorship
  object    any          // what we version ("tree" in git)

  data struct {
    comment string // describes the commit
  }
}

A Merkledag DSL based on YAML

# NODE_TYPE is a type of node
# DATA_TYPE is a primitive data type
# LINK_TYPE is a merkle-link to an object of a given NODE_TYPE
# "any" is _any_ LINK_TYPE
# NAME is a name token

# defining a new NODE_TYPE
NAME:
  NAME: NODE_TYPE
  data: 
    NAME: DATA_TYPE

Example:

commit:
  parentN: commit
  author: authorship
  committer: authorship
  object: any
  data:
    comment: string

this feels way too flexible. i wouldnt even know what to do.

A Merkledag DSL in BNF

maybe just use BNF? (here is git's)

<commit> ::= <parents> <author> <committer> <object> <data> ;

<parents> ::= <parent> | <parents> <parent> ;

<author> ::= "author" <authorship> ;

<committer> ::= "committer" <authorship> ;

<object> ::= "object" <any> ;

<data> ::= <byte-sequence> ;

A Merkledag DSL in JSON-LD

Assume that /ipfs/<hash-of-merkledag-schema/context.jsonld is valid context, such as:

{
  "@context": {
    "node": "/ipfs/<hash-of-merkledag-node-schema>/node",
    "link": "/ipfs/<hash-of-merkledag-link-schema>/link",
    "any": "/ipfs/<hash-of-merkledag-link-schema>/link",
  }
}

cat /ipfs/<hash-of-commit-schema>/commit | ipfsld-to-jsonld

// "any", "string", and "node" are defined in .../mdag
{
  "@context": "/ipfs/<hash-of-merkledag-schema>/mdag",
  "@type": "node",
  "@id": "commit",

  "parentN": { "@type": "commit" },
  "author": { "@type": "/ipfs/<hash-of-authorship-schema>/authorship" },
  "commiter": { "@type" : "/ipfs/<hash-of-authorship-schema>/authorship" },
  "object": { "@type": "any" },

  "data": {
    "comment": { "@type": "string" },
  }
}

wait, maybe any JSON-LD!?

A more general idea. JSON-LD is so powerful, and so simple, that we could actually just give users completely free reign over IPFS data... Let them make the datatype of a merkledag node be ANY JSON-LD document they want, and provide a special context file and types for {"node", "any", and "link"}. Links marked with our types (from the merkledag @context) will be able to be traversed + understood by IPFS. this is a bit mind blowing. and perhaps way too flexible. but it does make IPFS way easier to use for arbitrary things...