Generating plain OCaml code / Benchmarking

[marzipankaiser]

When trying to use the version built from current main (65a4cec190591c43ff75779f68dbcdc966689349) for the benchmarks at https://github.com/effect-handlers/effect-handlers-bench , all OCaml builds fail with type errors.

Is there a more recent version than the OOPSLA2021 Artifact that can be used to build and run those benchmarks (or different ones)?
Alternatively, how could one go about reasonably benchmarking the current version?

More details on the type errors

in text:

File "benchmarks/001_nqueens/generated_001_nqueens.ml", line 73, characters 18-33:
73 |                   _safe_50 _x_105 >>= fun _b_176 ->
                       ^^^^^^^^^^^^^^^
Error (warning 5 [ignored-partial-application]): this function application is partial,
maybe some arguments are missing.
File "benchmarks/001_nqueens/generated_001_nqueens.ml", line 73, characters 18-33:
73 |                   _safe_50 _x_105 >>= fun _b_176 ->
                       ^^^^^^^^^^^^^^^
Error: This expression has type int -> 'a
       but an expression was expected of type 'b computation
File "benchmarks/002_generator/generated_002_generator.ml", line 87, characters 5-65:
87 |      (fun (_x_85 : unit -> generator computation) -> Value _x_85))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: This expression should not be a function, the expected type is
       unit OcamlHeader.computation
File "benchmarks/003_tree_explore/generated_003_tree_explore.ml", line 174, characters 13-80:
174 |              (fun (_x_161 : int -> (int * intlist) computation) -> Value _x_161))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: This expression should not be a function, the expected type is
       int OcamlHeader.computation
File "benchmarks/004_triples/generated_004_triples.ml", line 143, characters 5-41:
143 |      (fun (_x_156 : int) -> Value _x_156))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: This expression should not be a function, the expected type is
       int OcamlHeader.computation
File "benchmarks/007_simple_counter/generated_007_simple_counter.ml", line 86, characters 13-49:
86 |              (fun (_x_175 : int) -> Value _x_175))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: This expression should not be a function, the expected type is
       int OcamlHeader.computation

Simpler example

That is, eff runs successfully, but the resulting OCaml code does not compile.
This also happens with any example I try to write myself, as soon as it uses any effects, e.g.:

effect Emit : int -> unit

let main = 
    handle
      perform (Emit 3); 0
    with
        | effect (Emit i) k -> k ()

with wrapper code

let _ = Printf.printf "%d\n" (Generated_eff_primes.main) 

and using the Build structure from https://github.com/effect-handlers/effect-handlers-bench also fails with

File "primes_eff/generated_eff_primes.ml", line 42, characters 5-39:
42 |      (fun (_x_53 : int) -> Value _x_53))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: This expression should not be a function, the expected type is
       int OcamlHeader.computation

[matijapretnar]

Hmm, this is strange. I do not get an error when trying to run it. Can you please attach the compiled version of your example and of ocamlHeader.ml?

[matijapretnar]

No need :) I see that the header you are using is outdated. When I put it in, I get the same error as you. If you use the updated version from our repo it should work.

[marzipankaiser]

Thank you very much :)
It works with the updated header.

[marzipankaiser]

It is significantly slower than the OOPSLA2021 Artifact version for me, though. 🤔 Is that to be expected or possibly another mistake on my side?

[matijapretnar]

Yes, this is probably because the generated code is now polymorphic and passes around explicit coercion witnesses. @jO-Osko and I are working on optimizations right now that should remove most of those witnesses and get back to the OOPSLA speeds.

[marzipankaiser]

Makes sense. Thanks!
I'm looking forward to reading about those.