feat: add ability to rewrite test files output

[sdht0]

Currently if the output of a query changes, we've to manually fix the output in test files.

We should add a --rewrite mode in the test runner which switches from checking the output to rewriting the output.

Prior work:

• Sqlite "completion mode": https://www.sqlite.org/sqllogictest/doc/trunk/about.wiki
• Apache Datafusion: apache/datafusion@04d095d
• Materialize: https://github.com/MaterializeInc/materialize/blob/main/doc/developer/sqllogictest.md#useful-tips

[sdht0]

cc @ray6080

[sdht0]

While we're at it, we should also remove explicit counts I think. The test runner should implicitly read all output until a blank line or EOF, instead of the user having to manually specify and update output counts.

[acquamarin]

While we're at it, we should also remove explicit counts I think. The test runner should implicitly read all output until a blank line or EOF, instead of the user having to manually specify and update output counts.

I don't think we can remove the number of output counts.
What is a query returns 4 lines?
-STATEMENT unwind [null,null,null,null] as t return t
Newline
Newline
Newline
Newline
Newline
-STATEMENT OTHER
How can the test framework know whether the expected result is 4 lines or 5lines?

[sdht0]

How can the test framework know whether the expected result is 4 lines or 5lines?

Yeah this will need to be handled. Perhaps the count can be optional.

However I think in the general case it adds a bit more friction when iterating on test cases. Although the automatic rewriter will solve this problem too.