IPFS DSL

A declarative embedded language for building compositional programs and protocols over the InterPlanetary File System.

Language example (docs to come)

The following lazy computation expression yields an AST that captures the notion of adding a in-memory stream to the local IPFS node, which can be reasoned about and transformed.

    // preliminaries, types you would write to model the boundary between your code and mine
    type StartContext =
        | AddStream of Stream * fn:string * fo:AddFileOptions

    type EndContext =
        | Finished of IFileSystemNode

    type Ctx =
        | Start of StartContext
        | End of EndContext
        
    // a program that uses the file system subDSL
    let addStream (client:IpfsClient) (continuation:FileSystemDSLResultContext<'a>) =
        FileSystemProcedure(
            Effects.constant (FileSystemDSL.addStream client),
            // a context reader
            (fun ctx ->
                match ctx with
                | Start(sctx) ->
                // we don't match on the other case to make the program crash fast,
                // the DSL assumes the context to be consistent
                
                    match sctx with
                    | AddStream(s, fn, fo) ->
                        FileSystemDSLArgs.prepareAddStream s fn fo Cancellation.dontUse),
                        
            continuation) |> liftFreer
            
    // context variable
    let mutable ctx = monad {
        File.WriteAllBytes("testFile.bin", [|24uy;55uy;22uy;66uy;0uy;99uy;|])
        use fs = File.OpenRead("testFile.bin")
        return! Start(AddStream(fs, "testFile.streamed.bin", AddFileOptions()))
    }

    let finish node = ctx <- End(Finished(node))

    // a continuation that receives the result
    let retrieveCid :FileSystemDSLResultContext<Async<unit>> = 
        fun result ->
            match result with
            | AddStreamResult(afsnode) -> async {
                let! node = afsnode
                return finish node }

            | _ -> async {return ()}

To run this abstract fragment, you would

let client = IpfsClient()
let r = Async.RunSynchronously (IpfsDSL.run ctx (addStream client retrieveCid))

What's in the algebra

Main DSL	Sub DSL	Args	Result	Low-level API docs
BitswapProcedure	BitswapDSL	BitswapDSLArgs	BitswapDSLResult	read
BlockProcedure	BlockDSL	BlockDSLArgs	BlockDSLResult	read
BootstrapProcedure	BootstrapDSL	BootstrapDSLArgs	BootstrapDSLResult	read
ConfigProcedure	ConfigDSL	ConfigDSLArgs	ConfigDSLResult	read
DagProcedure	DagDSL<'Out>	DagDSLArgs	DagDSLResult<'Out>	read
DhtProcedure	DhtDSL	DhtDSLArgs	DhtDSLResult	read
DnsProcedure	DnsDSL	DnsDSLArgs	DnsDSLResult	read
FileSystemProcedure	FileSystemDSL	FileSystemDSLArgs	FileSystemDSLResult	read
GenericProcedure	GenericDSL	GenericDSLArgs	GenericDSLResult	read
KeyProcedure	KeyDSL	KeyDSLArgs	KeyDSLResult	read
NameProcedure	NameDSL	NameDSLArgs	NameDSLResult	read
ObjectProcedure	ObjectDSL	ObjectDSLArgs	ObjectDSLResult	read
PinProcedure	PinDSL	PinDSLArgs	PinDSLResult	read
PubSubProcedure	PubSubDSL	PubSubDSLArgs	PubSubDSLResult	read
SwarmProcedure	SwarmDSL	SwarmDSLArgs	SwarmDSLResult	read

All of these correspond to calling the lower-level IPFS API methods. Some names may be changed slightly.

Generally the IPFS DSL wraps around the lower-level API in the following way:

each API gets it's own DSL InameAPI -> nameDSL
the parameters of methods of the API get a constructor case in the type nameDSLArgs
however, those are private, you construct the args by calling the static prepare methods
the return types of the methods get wraped in Async or AsyncSeq
the return types of the methods get a constructor case in the type => nameDSLResult
you use those to pattern match when deciding what to do with results

Typeparams and free

Next - is the type of the value returned when the program terminates.

Context - is the type that the effectful branching is parameterized by. When a IpfsClientProgram branches, the path it takes depends on the state of the Context available.

Produces - the type that is read from the Context when deciding how to branch.

DagOut - DagDSL includes a method that deserializes a JSON object to a native .NET object. That method gets parameterized with the DagOut type parameter even if you don't use the DagDSL in your program because it's subsumed to the main IpfsDSL. If you do use it, remember that you can flatMapR to change the return type when constructing ASTs.

// the "freer monad", also called a program, this little recursive structure models all possible
// execution scenarios of using the IPFS API with the embedded langauge, more precisely,
// the following three can occur at each evaluation step:
// (1) the program steps by performing an instruction of the embedded language
// (2) the program branches by producing a visible effect (where you plug in your context),
//     and promises to step after getting the result of the effect
// (3) the program reaches a value of type 'Next, returns it and terminates
type IpfsClientProgram<'Next, 'Context, 'Produces, 'DagOut> =

        // an expression in the embedded language about the next step in the program
        | Step of program:IpfsDSL<IpfsClientProgram<'Next,'Context,'Produces,'DagOut>,'Context,'DagOut>
    
        // a branching step depending on the result of the effectful computation
        | Branch of visible:Effect<'Context,'Produces> * program:('Produces -> IpfsDSL<IpfsClientProgram<'Next,'Context,'Produces,'DagOut>,'Context,'DagOut>)

        // the final step, produces a value (or often, a new program!)
        | Return of value:'Next

Acknowledgements

Built over the net-ipfs-api and net-ipfs-core by Richard Schneider. Many thanks!

MarneeDear/fsharp-ipfs-dsl

IPFS DSL

Language example (docs to come)

What's in the algebra

Typeparams and free

Acknowledgements

MIT License