Description
In #2217, @carl-mastrangelo had the idea to cache high order objects in HPACK's dynamic table. Carl mentioned the example of instead of parsing a jwt token on every request, we could cache an object that already contains the parsed representation, reducing garbage creation and parsing overhead.
Additionally, since Metadata uses String for names, we could for header names cache the String representation.
This would mostly require changes to Netty's HPACK implementation. We need some way to attach additional headers. I suggested to modify the StaticTable, Decoder and HuffmanDecoder to take a HeaderLiteralFactory as an optional ctor argument.
interface HeaderLiteralFactory {
CharSequence newKey(byte[] bytes);
CharSequence newValue(CharSequence key, byte[] bytes);
}Netty's default implementation would simply return AsciiString objects. The gRPC implementation, could use String objects for keys. For values, we could special case certain header names and return holder objects e.g.
class JwtHolder implements CharSequence {
JwtObject jwt;
byte[] bytes;
}Risks
The decoder has no power over what gets cached and what doesn't - it's up to the encoder. We might introduce overhead for headers that don't get cached. Also, a (name, value) is treated as one header entry in HPACK's dynamic table. Say if a users use custom headers, with lots of different values, then caching in the dynamic table might not be very effective, as entries might get evicted before they can be reused (if ever).