Including VarNames in sampler states, etc.

[penelopeysm]

Many things in the Turing ecosystem assume that parameters have a form of Vector{<:Real}. We currently tend to do a lot of vi[:] and vi = unflatten(vi, params) to convert between this vector-of-real and a VarInfo.

However, this representation leads to information loss in many places. See e.g.

TuringLang/DynamicPPL.jl#833

Sometimes we attempt to use a NamedTuple, but even that is not fully safe because NamedTuples are indexed by Symbols and not VarNames. This leads to issues like

TuringLang/DynamicPPL.jl#814
TuringLang/MCMCChains.jl#470
TuringLang/DynamicPPL.jl#702

To this end I've been thinking for a while now about changes to AbstractMCMC. It's also come up in discussions with @yebai and @mhauru. This is quite tough because it's one of the lowest-level packages in TuringLang and everything directly or indirectly depends on it.

I'm opening this issue here to collect some thoughts on it and will edit it slowly. Please bear with me, it's hard to formulate sensible thoughts on this topic, and even harder to write them down in a clear way.

Related:

TuringLang/AbstractMCMC.jl#156