Improve Hugging Face integration

[NielsRogge]

Hi there,

Thanks for making all perception models available on 🤗 .

Some things we could improve:

• wondering how easy it is to make the Perception Encoder models compatible with the Transformers, Timm and/or OpenCLIP libraries.
• regarding the original checkpoints, I see you're achieving download stats by adding an empty config.yaml file. That's clever ;) I've opened Improve HF integration #3 in case you are interested in adoption of safetensors weights along with a config.json which stores all the hyperparameters of the model's architecture.

Kind regards,

Niels

[berniebear]

Hello @NielsRogge,
Thank you for the great suggestion and sorry for the late response. We are working on integrating perception encoder into timm. Thanks!

[berniebear]

Hello PE is now integrated in timm (HF). Thank you for your support!

import timm
model = timm.create_model('vit_pe_core_gigantic_patch14_448', pretrained=True)