Structured diagnostics for the compiler #13766

Octachron · 2025-01-31T16:00:57Z

See also ocaml/RFCs#45 and https://icfp24.sigplan.org/details/ocaml-2024-papers/16/Structured-diagnostics-for-the-OCaml-compiler for previous discussions .

A limitation of the current compiler diagnostics (error messages, warnings, time profile information and other compiler developer debugging output) is that it cannot be parsed easily and reliably by developer tools. This creates an impedance mismatch between the compiler and those tools that tend to resort to trying to parse manually compiler error message.

This PR proposes to fix this problem by adding to the compiler a machinery for building structured compiler diagnostics before displaying them in various formats.

This PR is probably best reviewed commit by commit:

the first eight commits slowly introduce the new machinery part by part
commit 9 to 14 converts the current compiler diagnostics to this new machinery (going through the error messages, the debugging outputs, then the toplevel, and finally the compiler configuration)
commit 15 adds a new tools for instrospecting the compiler diagnostics, and the following commit uses this new tools
to export json-schema and adds the corresponding tests in the testsuite
the last commits 4 commits update the manual and the man pages.

Octachron · 2025-01-31T18:33:20Z

Note that I have tested some quick prototypes for deserializers and a bridge between the proposed compiler diagnostic and the grace library at https://github.com/Octachron/sdiag-prototypes .

johnyob · 2025-01-31T20:06:15Z

Note that I have tested some quick prototypes for deserializers and a bridge between the proposed compiler diagnostic and the grace library at https://github.com/Octachron/sdiag-prototypes .

Its lovely to see grace being used here :) I look forward to playing with this

Octachron · 2025-02-18T15:48:06Z

I have pushed a commit that hide the change of API to the toplevel library beyond a compatibility layer.

gasche

I did a first round of looking at the code, rather shallow as a review.

I did not look at the manual yet, as I read in commit order; this was a mistake and I'll start there on my next round.
I did not look at the new modules (diagnostics, diagnostics_history) in depth, only quickly at the interface (assessment: too hard to follow for me, I need to read the manual first). It's a lot of non-critical code and I plan to declare that it is reasonable and go by the tests mostly, instead of trying to review everything line-by-line.
I looked at the code that adapts the compiler codebase. This is as I expected, invasive but reasonable.

gasche · 2025-05-11T15:19:23Z

utils/diagnostic.mli

+
+  | List: 'a typ -> 'a list typ (** Combinators *)
+  | Pair: 'a typ * 'b typ -> ('a * 'b) typ
+  (** Specialize (2,3,4)-tuples to avoid defining tuples as heterogeneous


Nitpick: clearly(?!) this should be {2,3,4}-tuples rather than (2,3,4)-tuples.

gasche · 2025-05-11T15:22:52Z

utils/diagnostic.mli

+      record at update [u]. The field is optional if [opt] is [Some true] *)
+
+  val new_field_opt: vl update  -> string -> 'a typ -> 'a field
+  (** [new_field_opt] is a short-hand for [new_field ~opt:true] *)


I would expect either two functions or an optional argument, do we really want to offer both?

gasche · 2025-05-11T15:25:56Z

utils/diagnostic_history.mli

+  type point =
+    | Inception
+     (** constructor only: derived constructor are incepted before being
+         published.*)


I'm not sure what "inception" mean.

(Also: derived constructors are incepted ...)

gasche · 2025-05-11T15:28:44Z

utils/diagnostic_history.mli

+(** Last version present in the history *)
+val current_version: 'a t -> version
+
+(** An ['id update] is a version registered in the history ['id History.t].*)


['id Diagnostic_history.t], I suppose?

gasche · 2025-05-11T15:32:25Z

utils/diagnostic_history.mli

+(** An ['id update] is a version registered in the history ['id History.t].*)
+type 'a update
+val new_version: 'a t -> version -> 'a update
+(** [new_version h v] unconditionnally creates an update but register an error


gasche · 2025-05-13T07:03:31Z

toplevel/topcommon.ml

@@ -425,8 +452,8 @@ let loading_hint_printer ppf cu =
  let find_with_ext ext =
    try Some (Load_path.find_normalized (cu ^ ext)) with Not_found -> None
  in
-  fprintf ppf
-    "@.Hint: @[\
+    fprintf ppf


the extra spacing of fprintf looks unintentional.

gasche · 2025-05-13T07:14:41Z

utils/config_diagnostic.ml

+
+let print log =
+  log_variables log;
+  Log.flush log


I tested the new -config outputs on my machine and I observe a bug:

ocamlc -config works exactly as before

ocamlc -log-format {json,sexp} -config work as expected (nice!)
(nitpick: the sexp format will sometimes fit several fields on the same line, I would rather have a vertical printing with at most one field per line)

ocamlc -log-format stdout -config prints only the values and not the keys/variables, so that output is incorrect

@Octachron points out that in fact stdout is not supposed to be the same as not setting anything, currently there is stdout and stdout-with-keys and the latter is not exposed. It would be nice if both options were exposed, and maybe they could be named stdout-light and stdout-full, or maybe stdout-values and just stdout.

gasche · 2025-05-13T07:46:08Z

tools/diaginfo.mli

+(*   the GNU Lesser General Public License version 2.1, with the          *)
+(*   special exception on linking described in the file LICENSE.          *)
+(*                                                                        *)
+(**************************************************************************)


The .mli would be an okay place to include some documentation of how to use diaginfo.

Help output

The documentation of -schema-format is weird.

-schema-format {json|adt} <name> print the schema <name> in <name> format

I don't understand what it means. (Does this command expect two parameters, or just one that is either json or adt? I assume the latter.)

How to get a list of schemas?

On a first try using the tool, I failed to understand how to get a list of schemas. I think that there should be an explicit option, for example --list, to list all schemas. When I printed the --help output to write the comment about I noticed the part about *, but this does not seem to work easily on my machine:

$ ./runtime/ocamlrun ./tools/ocamldiaginfo -schema * Unknown schema name: aclocal.m4 $ ./runtime/ocamlrun ./tools/ocamldiaginfo -schema "*"

The first comes from the fact that my shell interpolates the *, and the second produces empty output.

History output

A trailing newline seems to be missing.

$ ./runtime/ocamlrun ./tools/ocamldiaginfo -history | tail -n 5 New label backtrace, ?l structured_text New label compiler, ?compiler New label errors, ?l l structured_text New label trace, ?l l structured_text Seal[gasche@framawork github-trunk (structured_diagnostics_full)]$

gasche · 2025-05-13T07:48:57Z

diagnostic_schemes/config_scheme.json

+  "properties" :
+    {
+      "metadata" : { "$ref" : "#/$defs/metadata"},
+      "linear_magic_number" : { "type" : "string"},


Nitpick: it's slightly weird that there is a space before the key ({ "type") but not after the value ("string"}).

gasche · 2025-05-13T07:53:12Z

toplevel/topdirs.ml

+  Fun.protect (fun () -> f log x)
+  ~finally:(fun () -> Log.flush log)
+
+  let dir_load = with_log V2.dir_load


The indentation here is confusing

Octachron force-pushed the structured_diagnostics_full branch from 8edb2d8 to 8e07bd5 Compare February 3, 2025 12:17

gasche self-assigned this Feb 5, 2025

Octachron added 21 commits February 17, 2025 14:37

structured diagnostic: definitions

12f2469

structured diagnostics: validation

aa6b544

structured diagnostics: log

65556a8

structured diagnostic: color handling refactorization

3045459

Format doc: tree representation

905d08c

structured diagnostic: compiler diagnostic definition

2bc846c

structured diagnostics: json and s-expression backends

4092cd9

fully switch otype printer to Format_doc

cdba025

structured diagnostics: switch to error compiler to log

9c49550

structure diagnostics: timing profile

a04657d

structured diagnostics: compiler debugging output

eabc0cd

structured diagnostic: toplevel response

136da09

structured diagnostics: config

8c404a4

structured diagnostics: diaginfo tool

0d9093d

structured diagnostic: testsuite

023a40b

structured diagnostics: manual

eaeeeda

diagnostic: add description to fields and constructors

406fb96

Update man pages

ee2b4d1

Manual, version 2

b005b5a

Rename Config_diagnostic to avoid the config prefix

e33915c

toplevel: backward compatibility for toplevel API

355143d

Octachron force-pushed the structured_diagnostics_full branch from a3ede5d to 355143d Compare February 18, 2025 15:46

johnyob mentioned this pull request Apr 15, 2025

More multi-location error messages #12633

Open

gasche mentioned this pull request May 11, 2025

[minor] use a record in Compenv.process_deferred_actions #14026

Merged

gasche reviewed May 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Structured diagnostics for the compiler #13766

Structured diagnostics for the compiler #13766

Uh oh!

Octachron commented Jan 31, 2025

Uh oh!

Octachron commented Jan 31, 2025

Uh oh!

johnyob commented Jan 31, 2025

Uh oh!

Octachron commented Feb 18, 2025

Uh oh!

gasche left a comment

Uh oh!

gasche May 11, 2025

Uh oh!

gasche May 11, 2025

Uh oh!

gasche May 11, 2025

Uh oh!

gasche May 11, 2025

Uh oh!

gasche May 11, 2025

Uh oh!

gasche May 13, 2025

Uh oh!

gasche May 13, 2025

Uh oh!

gasche May 14, 2025

Uh oh!

gasche May 13, 2025

Uh oh!

gasche May 13, 2025

Uh oh!

gasche May 13, 2025

Uh oh!

Uh oh!

Structured diagnostics for the compiler #13766

Are you sure you want to change the base?

Structured diagnostics for the compiler #13766

Uh oh!

Conversation

Octachron commented Jan 31, 2025

Uh oh!

Octachron commented Jan 31, 2025

Uh oh!

johnyob commented Jan 31, 2025

Uh oh!

Octachron commented Feb 18, 2025

Uh oh!

gasche left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Help output

How to get a list of schemas?

History output

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!