(Long-running problem) Expose the models here for AD package developers to test

[penelopeysm]

Title. We would like a way for a given AD package to run all the models in here as an integration test.

Having this, plus maybe a couple more convenience functions in DynamicPPL, would essentially, finally, solve TuringLang/Turing.jl#2411

I think the role of this repo is only to collate models. It shouldn't be concerned with VarInfo types and other things, that should all be handled in DynamicPPL test utils.

We should differentiate between models that are important to test on, and models that are nice to look at in the table, but realistically don't add anything beyond.

[penelopeysm]

This will probably mean registering this as a package. Ah well. Unless, say, the models are defined in DynamicPPL. But then that messes with the model definition thing. So.... maybe duplicate them?!

One possible idea: Make each model a file, so that get_model_definition can just wholesale read the file contents instead of parsing the Julia source code. Then for the models imported from DPPL, the definition could just point to DPPL source code.

[penelopeysm]

Alternatively, the other option would be to just let this be its own thing. We can duplicate models if needed, but philosophically DPPL testing should be an entirely separate issue from the table, and if there is overlap in terms of models then so be it. That's not ideal because it seems very likely that there will be overlap, although I'm hardly enthused by any of the other options either.

[yebai]

Does DIT provide the functionality for this purpose now that JuliaDiff/DifferentiationInterface.jl#771 is fixed? If DIT is adequate, we should revisit whether test models in DynamicPPL can be exported as DIT Scenarios.

cc @gdalle

[gdalle]

I think DIT still does a bit too much for Turing's needs, but it would be awesome to have someone take a good long look at the design and figure out if we can mutualize anything. Mutualization was the whole point of DI in the first place.

[penelopeysm]

@yebai FYI: @mhauru and I discussed this and we finally think the best solution is to adopt the DynamicPPL monorepo and have the models be exported from a sister package. If you're ok with that, I'll put in a PR to move this over when I can. (Our second choice is to stick all of these into DynamicPPL itself, but we think that it's better from a software engineering perspective to keep these separate)

[yebai]

I trust your judgment on this.