Per type instance metadata

[ikonst]

I've been working on pynamodb-mypy (started a while ago and coming back to it now...). The first task I took up is to implement attribute optionality, i.e.

from pynamodb.models import Model
from pynamodb.attributes import NumberAttribute

class MyModel(Model):
  my_number = NumberAttribute()
  my_optional_number = NumberAttribute(null=True)

reveal_type(MyModel().my_number)  # note: Revealed type is 'int'
reveal_type(MyModel().my_optional_number)  # note: Revealed type is 'Optional[int]'

I've been implementing:

• get_function_hook to handle NumberAttribute initialization and parsing the 'null' parameter
• get_attribute_hook to wrap the return type in optionality as needed

This required storing state between the function hook (parsing the arguments) and the attribute hook. One approach I've successfully tried is to append a "sentinel" type arg to the instance's args, i.e. from NumberAttribute it turns into NumberAttribute[None], and generally from SomeAttribute[...] to SomeAttribute[..., None] (I'm using a NoneType as the sentinel). Then in get_attribute_hook I'm looking for the sentinel.

You can see my current implementation here.

I was wondering what would be your thoughts on storing extra metadata on a type instance. Type args appear to be the only sanctioned "extras" as they're effectively part of the type. The new Annotated comes to mind, e.g. have my function hook return something like Annotated[AttrT, 'nullable_attr'], but I got the impression that it doesn't quite work this way -- that Annotateds should be converted to something else during semantic analysis rather than keeping them around for later stages to use.

[ikonst]

@JukkaL In #8750 we discussed what Annotated could be useful for — this "PynamoDB" use case is another practical example.

Generally I was thinking about tagging an expression's type with extra information so that at least within the same statement (or function) we could apply extra logic — like

• knowing that a datetime is tz-aware
• knowing that an integer is positive

etc. There's probably a name this concept (e.g. in Haskel?) that I just don't know. Maybe a NewType is all I need here?