/gecko

A programming language for radically distributed applications.

Primary LanguageGo

Gecko

Gecko is an experimental functional programming language and interpreter for radically distributed applications. Every local node in a Gecko program can be transparently replaced with a remote node. Remote nodes are accessed via RPC over the XMTP protocol.

XMTP is an end-to-end-encrypted messaging protocol which uses Ethereum addresses as identities. By using these decentralized, open protocols for message passing (in the Smalltalk sense) we may be able to bootstrap a computing environment with unheard-of levels of collaboration and composability.

Gecko is heavily influenced by Scheme, Smalltalk, Ruby, and Erlang/Elixir.

This project and documentation are both under heavy development. To see how things are going, please see the roadmap.

👋 Say hi!

Thanks for checking out the project! If you think it's interesting I'd love to hear from you (I'd love to hear criticism too, actually). The best ways to reach out are probably Discord, Twitter, or an issue.

Table of contents

Why?

A few reasons (one intuitive, one practical, and one speculative):

  1. The intuitive reason is that decentralized, end-to-end-encrypted messaging for message passing simply feels incredibly powerful.
  2. The practical reason is that service discovery via public keys makes for an extremely simple software distribution mechanism, at least for "toy" software. No installation, no imports, no account creation, no DNS, no hosting, just function calls. In certain cases you don't even need a web server because XMTP can run in the browser. This ease-of-use could be game-changing in certain situations e.g. in the classroom.
  3. We're heading towards a world where a large fraction (all, eventually) of software is encoded by, or generated on the fly by, an AI model. In that world the API may overtake the library as the most natural method for sharing software.

Syntax and semantics, overview

Gecko is a dynamically-typed, lexically-scoped, expression-oriented, interpreted, functional programming language with a Ruby-like syntax.

Some design goals

Other than the primary goal of exploring message passing via an open network, we have:

  • Beginner friendly
  • Uncluttered syntax (inspired by Ruby)
  • Semantic simplicity (inspired by Go)
  • Application-oriented (less "general purpose" than, say, Python)

Expressions

In Gecko everything is an expression that returns a value. The kinds of expressions are:

  • variable definition
  • function definition
  • function call
  • block
  • predicate
  • literal

Values and variables

Values can be bound to variables. Variables are referenced by name. All values are immutable but variables can be re-bound to new values. Gecko includes the following primitive value types:

  • string
  • number
  • boolean
  • nil
  • array
  • dict
  • function

Some examples:


# string

"I'm a string"

# number

10.0

# boolean

true

# nil

nil

# array

array
  1
  2
  "test"
end

# dict

dict
  "a" => 10
  "b" => array 1 2 3 end
  "c" => "why?"
end

# function

(n: Number d: Number -> Number)
  return n / d 
end

Functions

Every function has a signature. A signature is a list of named parameters, their schemas (optional), and the function's return schema (optional). All language built-ins are functions, a new function is defined by writing its signature and body. For example:

(car: Car to_speed: Mph -> Mph)
  if
    test
      to_speed > 50
    end

    then
      # crash
    end

    else
      # set the car's speed to the new speed
    end
  end

  return to_speed
end

Functions are anonymous and must be bound to a variable if we want to call it. For example:

def
  name
    "accelerate"
  end

  value
    (car: Car to_speed: Mph -> Mph)
      if to_speed > 50
        # crash
      else
        # set the car's speed to the new speed
      end

      return to_speed
    end
  end
end

After a function is defined it can be called by naming it and assigning values to its parameters. For example:

accelerate
  car
    my_moms_car
  end

  to_speed
    100
  end
end

If we want to reference a function without calling it we can prefix its name with an &. For example:

def
  name
    "double"
  end

  value
    (n: Number -> Number)
      return 2 * n
    end
  end
end

for
  array
    my_values
  end

  do
    &double
  end
end

A note on schemas

A schema in Gecko is a function used to validate values, especially values which are the arguments to or return from a function. If you're a TypeScript programmer and have used zod they should look familiar.

When a function is called, each named argument is passed to its corresponding type. Each type is a parser function that validates the argument or throws. If a parameter's type is not specified, then Identity is used, which always succeeds. When a function returns a value to the caller, the function's return type is used to first parse the value.

Dataflow in a Gecko function call

Comments

Comments begin with a # and continue until the end of the line. Whitespace is ignored (except as token separators).

Evaluation

All expressions, with a few exceptions, in a Gecko program tree are evaluated according to a basic depth-first tree walk. The children of the conditional expressions and, or, if, when, and switch may be (in certain obvious cases) skipped. Additionally, Gecko provides a single mechanism for parallel execution of children expressions via the parallel keyword.

Syntax and semantics, reference

Please note that this section has gone stale. For current examples of syntax and usage please see the examples or tests.

Blocks

A block is a sequence of expressions delimited by a keyword and end. A keyword determines its block's behavior or semantics. Most of the language's keywords will be described throughout the rest of this section but you can also find a comprehensive, runnable example in examples.core.fly.

The simplest block is the do block:

do expression* end

The expressions are evaluated in order and the value of the last expression is returned.

do
  puts "hey" end

  2

  do
    3 + 4
  end
end

Variables

A Gecko variable is an expression that resolves to a value by referencing it. A variable is defined using a def block and re-defined using a let block. After a variable is defined it can be referenced in any expression.

def identifier expression end

Defines a variable with the given identifier. The variable resolves to the value of the expression. Variables are lexically scoped. If the variable is already defined in the local scope, it is an error. If the variable is defined in an outer scope, it will be shadowed in the local scope.

def surname "smith" end

let identifier expression end

Re-defines an existing variable with the given identifier. The variable resolves to the value of the expression. If the variable does not already exist, it is an error.

def val "hi" end
let val "goodbye" end
  • Namespace declaration and resolution.

Values

Every value is a string, number, array, map, lambda, or nil.

A string is created by enclosing characters in quotes.

"I am string"

A number is created by writing it out in decimal notation. All numbers are represented as floats internally.

1
0.1
10.0

There is no boolean type in Gecko. All "boolean" operators take number operands and treat 0 as false-y and any other number as truth-y. All other values cause errors when used as a boolean.

An array is created using the array block and is a number-indexed list of values. See the arrays section for more details on arrays.

A map is created using the map block and is a string-keyed dictionary of values. See the maps section for more details on maps.

A lambda is created using the fn block and can be thought of as a parameterized do block or "anonymous function". See the lambdas section for more details on lambdas.

Lambdas, parameters, and arguments

A lambda is a "parameterized block" that is not evaluated until each time it is called. A lambda can have zero or more parameters. A parameter is a name that is defined each time the lambda is called. Parameters are declared between | characters. If the lambda takes zero parameters, the | characters must be omitted. The arguments to the lambda are the values of the expressions in the calling block (using the . keyword) bound to the lambda's parameters.

fn (|identifier+|)? expression end

When the lambda expression is evaluated, it creates a lambda. The key difference between a lambda expression and other expressions is that its subexpressions are evaluated only when the lambda is called. The lambda can take zero or more parameters. If the lambda takes zero parameters, the | characters must be omitted.

. expression* end

Calls the lambda expression. Each subexpression is evaluated and bound to the lambda's parameters. The lambda is then evaluated, returning the value of its last subexpression.

def add
  # parameters are a and b
  fn |a b|
    a + b
  end
end

.add
  # arguments are 8 and 3, bound to a and b
  2 * 4
  3
end

map
  array 1 2 3 end

  fn |n i|
    n + i
  end
end

Predicates, operators, and literals

A predicate is an expression involving an operator and operands. See the operators section for more details on each operator. An operand is either a predicate or a literal. A literal is an expression without subexpressions (string, number, boolean, variable). A predicate evaluates to a number (because an operator evaluates to a number).

Because predicates cannot include blocks they cannot include function calls. This is somewhat cumbersome to us human programmers, forcing us to write many instances of trivial indirection, but I think we'll see strong benefits for code generation and program synthesis because it will make parse trees simpler. Maybe not, we'll see.

# Not predicates.

fn
  std.write "hi" end
end

def val "hi" end

# Predicates.

val

val == "goodbye"

10 > 0 # => 1

100 / 20 # => 5

!val

Branching

The key difference between branching expressions and other expressions is that their subexpression are evaluated conditionally. The specific behavior of which subexpressions are evaluated depends on the keyword.

Note that branching expressions are not predicates, they may return any value.

if number expression expression end

If the number is truth-y, the first expression is evaluated. Otherwise, the second expression is evaluated. The value of the last evaluated expression is returned.

and (number expression)+ end

For each pair of subexpressions, if the first evaluates to a truth-y value, the second is evaluated. If any of the subexpressions evaluate to a false-y value, nil is returned. Otherwise, the value of the last subexpression is returned.

or (number expression)+ end

For each pair of subexpressions, if the first evaluates to a truth-y value, the second is evaluated and returned. If none of the subexpressions evaluate to a truth-y value, nil is returned.

while number expression+ end

While the first expression evaluates to a truth-y value, the rest of the expressions are evaluated. The value of the last subexpression is returned.

Arrays

array expression* end

Creates an array whose values are the values of the subexpressions. The array is returned.

array.read array number end

The value of the array at the index of the number is returned.

array.write array number expression end

Clones the array and sets the value at the index of the number to the value of the expression. The cloned array is returned.

array.for array lambda end

For each value in the array, the lambda is called with the value bound to the lambda's first parameter and the index bound to the lambda's second parameter. The value of the last evaluated lambda is returned.

array.map array lambda end

For each value in the array, the lambda is called with the value bound to the lambda's first parameter and the index bound to the lambda's second parameter. An array whose values are the result of each lambda call is returned.

array.filter array lambda end

For each value in the array, the lambda is called with the value bound to the lambda's first parameter and the index bound to the lambda's second parameter. An array whose values are the values for which the lambda call returned a truth-y value is returned.

array.reduce array expression lambda end

For each value in the array, the lambda is called with the value bound to the lambda's second parameter and the index bound to the lambda's third parameter. When the lambda is called for the first value in the array, the first parameter is bound to the value of expression. For each subsequent value in the array, the first parameter is bound to the value returned by the previous lambda call. The value of the last evaluated lambda is returned.

array.push array expression end

Clones the array and appends the value of the expression to the cloned array. The cloned array is returned.

array.pop array end

Clones the array and removes the last value from the cloned array. The cloned array is returned.

array.unshift array expression end

Clones the array and prepends the value of the expression to the cloned array. The cloned array is returned.

array.shift array end

Clones the array and removes the first value from the cloned array. The cloned array is returned.

array.reverse array end

Clones the array and reverses the order of the values in the cloned array. The cloned array is returned.

array.sort array lambda end

Clones the array and sorts the values in the cloned array according to the value returned by the lambda. The lambda takes two parameters, the values of which are the values in the array. The lambda returns a negative number if the first value should be sorted before the second, a positive number if the first value should be sorted after the second, and 0 if the values are equal. The cloned (sorted) array is returned.

array.segment array number number end

Clones the array and returns a new array whose values are the values of the cloned array between the first index and the second index (exclusive). The cloned array is returned.

array.splice array number array end

Clones the first array and divides it in half at the index of the number. It appends the values of the second array to the first half, and then appends the second half to the result. The result is returned.

Maps

map (string expression)* end

Creates a map whose keys are the strings and whose values are the values of the expressions. The map is returned.

map.read map string end

The value of the map at the key of the string is returned.

map.write map string expression end

Clones the map and sets the value at the key of the string to the value of the expression. The cloned map is returned.

map.delete map array end

The array is an array of strings. Clones the map and deletes the keys of the strings from the cloned map. The cloned map is returned.

map.extract map array end

The array is an array of strings. Returns a map whose keys are the keys of the strings and whose values are the values of the keys of the strings in the map. The new map is returned.

map.merge map map end

Clones the first map and then for each kv pair in the second map, sets the value of the cloned map at the key of the kv pair to the value of the kv pair. Returns the cloned map.

map.keys map end

An array whose values are the keys of the map is returned.

map.values map end

An array whose values are the values of the map is returned.

Strings

split string end

Returns an array whose values are the characters in the string.

concat string+ end

Returns a string whose value is the concatenation of the values of the strings.

substring string number number end

Returns a string whose value is the substring of the string between the first index and the second index (exclusive).

Input and Output

Signals and exceptions

TODO

Run a Gecko script

Requires go 1.21 or higher. Learn how to install go here.

go run . <path to Gecko source>

Try running the examples:

for file in examples/*.fly; do
  go run . $file
done

Testing

You can run the tests with:

./test.sh

The goal is to have test coverage commensurate to the maturity of the project and its components. The near term goal is to have something like 100% coverage for the core language keywords. Basically, this means "all of the keywords and operators". We'll do this incrementally, in phases.

Roadmap

Phase 1, minimal language core

  • design and implement the architecture
    • lex
    • parse
    • eval
  • keywords
    • for arrays
    • for strings
    • for dicts
    • for control flow
    • for functions
    • for variables
    • for predicates
    • for io
      • fs
      • env
  • errors

Phase 2, the rest of the core

  • namespaces
    • export
    • import
  • syntax highlighting

Phase 3, XMTP RPC

  • golang XMTP client port
  • brpc implementation using XMTP client

Phase 4, nice to haves

  • language server protocol implementation
  • repl
  • tail call optimization
  • basic static analysis
    • auto format
    • linter
    • schema -> typechecking?