Skip to content

Fast native toplevel using JIT #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 140 additions & 0 deletions rfcs/ocamlnat.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Fast native toplevel using JIT

# Overview

We (Jane Street + OCL/Tarides) would like to make the native toplevel
faster and more self-contained.

At the moment, the native toplevel works by calling the assembler and
linker for each phrase. This makes it slow and dependent on an
external toolchain which is not great for deployment.

To reach this goal, we would simply like to bring [this work][paper]
up to date and merge it in the compiler.

# Motivation

This work would provide a simple way to compile and execute OCaml code
at runtime, which would unlock a lot of new possibilities to develop
great new tools.

Coupled with the fact that we can already embed cmi files into an
executable, this work would make it possible to distribute a
self-contained binary that can evaluate OCaml code at runtime. This
would make it simple and straightforward to use OCaml as an extension
language.

## Verified examples in documentation comment

We are particularly interested in this feature for the [mdx][mdx]
tool. More precisely, we are currently working on a feature allowing
verified toplevel snippets in mli files. For instance:

```ocaml
(** [chop_prefix s ~prefix] returns [s] without the leading [prefix]. *)

{[
# chop_prefix "abc" ~prefix:"a";
- : string option = Some "bc"
# chop_prefix "abc" ~prefix:"x";
- : string option = None
]}
*)
val chop_prefix : string -> prefix:string -> string
```

In the above example, the `{[ ... ]}` would be kept up to date by
`mdx` to ensure that the document stays in sync when the code
changes. In fact, the user would initially only write the `#` lines
and `mdx` would insert the results just as with expectation tests.

# The change in detail

This change would add JIT code generation for x86 architectures as
described in [the paper][paper]. For other architectures, we would
still rely on the portable method of calling the assembler and linker.
The main additions to the compiler code base would be:

- some code in the backend to do the assembly in process
- a few more C functions to glue things together

The paper mentions that it adds 2300 lines of OCaml+C code to the
compiler code base.

One detail to mention: IIUC the [JIT ocamlnat][ocamlnat] from [the
paper][paper] goes directly from the linear form to binary
assembly. Now that we have a symbolic representation of x86 assembly
we could also go from the symbolic assembly in order to share more
logic between normal compilation and JIT.

We discussed with @alainfrisch and @nojb since LexiFi has been using
an in-memory assembler in production for a while. They mentioned that
they would be happy to open-source the code if they can, which means
that we could be using code that has been running in production for a
long time and is likely to be well tested and correct.

LexiFi's binary emitter is about 1800 lines of code including comments
and newlines. This looks a bit smaller than the JIT part of the JIT
ocamlnat, so we would still be adding approximately the same amount of
code if we went this way.

# Drawback

This is one more feature to maintain in the compiler and it comes with
a non-negligible amount of code. However, and especially if we can
reuse LexiFi in-memory assembler, most of the additions would come from
well tested code. @alainfrisch and @nojb also mentioned that this code
was very low-maintenance and had pretty much not changed in 5 years.

# Alternatives

For the mdx case, we considered a few alternatives.

## Using a bytecode toplevel

Mdx currently uses a bytecode toplevel where everything is compiled
and executed in byte-code. This includes:

- code coming from user libraries
- the full compilation of the toplevel phrases

as a result, mdx is currently very slow and the round-trip time
between the user saving a file and seeing the result easily climbs in
the tens of seconds.

In the case of Jane Street, we have one more difficulty with this
method: a lot of our code doesn't work at all in bytecode because we
never use bytecode programs.

## Staging the build

Given that mdx is a build tool, one alternative is to redesign the
interaction between mdx and the build system. For instance, it could
done in stages with a first step where mdx generates some code that is
then compiled and executed normally by the build system. This is how
the [cinaps][cinaps] tool works for instance.

However, it is difficult to faithfully reproduce the behavior of the
toplevel with this method. What is more, such a design is tedious and
requires complex collaboration between the tool and the build system.

Going through this amount of complexity for every build tool that wants
to compile OCaml code on the fly doesn't feel right.

## Using a mixed native/bytecode mode

One idea we considered is using a mixed mode where a native
application can execute bytecode. This would work well for us as the
snippets of code we evaluate on the fly are always small and fast.

However, it is completely new work while the native JIT has already
been done. What is more, while it would work for us it might not work
for people who care about the performance of the code evaluated on the
fly.

A native JIT would likely benefit more people.

[mdx]: https://github.com/realworldocaml/mdx
[paper]: https://arxiv.org/pdf/1110.1029.pdf
[ocamlnat]: https://github.com/bmeurer/ocaml-experimental/
[cinaps]: https://github.com/ocaml-ppx/cinaps