[WIP] Implement rewrite validator

[serenity4]

Closes #47.

This PR is based on #54, for a clean view of the changes see the comparison against that PR.

This is still largely a work in progress, I am just putting it here for discussion.

[serenity4]

@Keno does the code in test/validation.jl look aligned with your expectations?

[serenity4]

I briefly mentioned it in our last call, but I'm not sure I understand how we should work around that assuming that the overall approach is correct. Codegen looks at the past du for the first equation but then looks at its expected value (i.e. its RHS) for the last equation, even though we are using ddt(x1) in both expressions.

[Keno]

For linear equations, you can't compare the residuals directly, since ssrm rewrites them. However, you can reconstitute them uniquely from the state vector, since the total_incidence has the coefficients. Sorry, I think I forgot to mention that detail in the issue.

[serenity4]

I see, so that tells us how to reconstruct the residual for the linear equation x₃ - x₁.
However, the equations I wrote about in #55 (comment) were ddt(x₁) - x₁ * x₂ and x₄ * x₄ - ddt(x₁), which are each using a different value for ddt(x₁) after scheduling. Would you have any pointers on how to deal with that?

[Keno]

The values should be numerically equal up to tolerance (in an on-manifold state vector), so it shouldn't matter.

[serenity4]

Do we want to require the u/du we provide for testing to be on-manifold? That sounds like a reasonable requirement, but perhaps it might be good to also check that outside of this condition the optimized version behaves similarly to the unoptimized version, unless that doesn't affect the correctness of our solutions.

[Keno]

I need to read it in more detail, some initial comments are that I don't think we should be using RHSSpec for this, since the ABI is different. The other is that in this mode, we should probably also not do the IPO/argument transform and then in particular, the ABI needs to retain all the original arguments (in addition to the vectors that you have). Because of those two points then, there should also not be a TornCacheKey for this, since there's always only one specialization for this (for each method instance). In general, I think you're on the right track though.

[serenity4]

That makes sense, thanks. IIUC, RHSSpec should then be replaced by a singleton struct for the CodeInstance's owner, right?

we should probably also not do the IPO/argument transform and then in particular, the ABI needs to retain all the original arguments

To make sure I understand: by IPO/argument transform, do you mean the flattening transform, or/and another one? Then my understanding is that because we don't flatten, we must pass around all arguments because the callees will need them whole (which is a consequence of not performing SICM). Is that correct?

[Keno]

That makes sense, thanks. IIUC, RHSSpec should then be replaced by a singleton struct for the CodeInstance's owner, right?

Yes

To make sure I understand: by IPO/argument transform, do you mean the flattening transform, or/and another one?

Yes, the flattening and the nonlinear replacement rewrite. Although it's a little bit tricky, because the latter introduces extra equations, so you may want to run it anyway and just copy the IR before it finishes (and then the introduced extra equations just don't show up in the IR and end up as 0).

Then my understanding is that because we don't flatten, we must pass around all arguments because the callees will need them whole (which is a consequence of not performing SICM). Is that correct?

Yes, the code data flow (and code itself, frankly) is exactly the same as the original code, except that all the functions have a few extra arguments to magically synthesize the correct return value for variable.