https://huggingface.co/hf-internal-testing tiny random models need to be converted to safetensors

[sfc-gh-sbekman]

The problem is that many transformers CI tests rely on these tiny models and they are mostly pytorch_model.bin format, e.g. see: https://huggingface.co/hf-internal-testing/tiny-random-T5Model/tree/main
But modeling_utils has massively changed recently and uses a different code pass for safetensor model files - and most modern models use that.

Which means transformers CI isn't testing the code properly.

For example a recent Deepspeed integration was broken because the tests use https://huggingface.co/patrickvonplaten/t5-tiny-random/tree/main but the code was doing something different for pytorch_model.bin files and thus a massive breakage introduced in #36963 was missed. I looked at replacing it with https://huggingface.co/patrickvonplaten/t5-tiny-random/tree/main but it has the same issue - doesn't have a .safetensor file.

You can see why the massive testing model update is needed if we want the tests to actually test:

transformers/tests/deepspeed/test_model_zoo.py

Lines 56 to 90 in 0ef339f

	# *** Working Models ***
	ALBERT_TINY = "hf-internal-testing/tiny-albert"
	BART_TINY = "sshleifer/bart-tiny-random"
	BERT_TINY = "hf-internal-testing/tiny-bert"
	BIGBIRD_PEGASUS_TINY = "hf-internal-testing/tiny-random-bigbird_pegasus"
	BIG_BIRD_TINY = "hf-internal-testing/tiny-random-big_bird"
	BLENDERBOT_TINY = "hf-internal-testing/tiny-random-blenderbot"
	BLOOM_TINY = "bigscience/bigscience-small-testing"
	DEBERTA_TINY = "hf-internal-testing/tiny-random-deberta"
	DEBERTA_V2_TINY = "hf-internal-testing/tiny-random-deberta-v2"
	DISTILBERT_TINY = "sshleifer/tiny-distilbert-base-cased"
	ELECTRA_TINY = "hf-internal-testing/tiny-electra"
	FLAUBERT_TINY = "hf-internal-testing/tiny-random-flaubert"
	FSMT_TINY = "stas/tiny-wmt19-en-de"
	FUNNEL_TINY = "hf-internal-testing/tiny-random-funnel"
	GPT2_TINY = "sshleifer/tiny-gpt2"
	GPTJ_TINY = "hf-internal-testing/tiny-random-gptj"
	GPT_NEO_TINY = "hf-internal-testing/tiny-random-gpt_neo"
	LAYOUTLM_TINY = "hf-internal-testing/tiny-layoutlm"
	LED_TINY = "hf-internal-testing/tiny-random-led"
	LONGFORMER_TINY = "hf-internal-testing/tiny-random-longformer"
	M2M_100_TINY = "stas/tiny-m2m_100" # hf tiny model is unsuitable
	MARIAN_TINY = "sshleifer/tiny-marian-en-de"
	MBART_TINY = "sshleifer/tiny-mbart"
	MOBILEBERT_TINY = "hf-internal-testing/tiny-random-mobilebert"
	MPNET_TINY = "hf-internal-testing/tiny-random-mpnet"
	PEGASUS_TINY = "stas/pegasus-cnn_dailymail-tiny-random"
	PROPHETNET_TINY = "hf-internal-testing/tiny-random-prophetnet"
	ROBERTA_TINY = "sshleifer/tiny-distilroberta-base"
	SQUEEZEBERT_TINY = "hf-internal-testing/tiny-random-squeezebert"
	T5_TINY = "patrickvonplaten/t5-tiny-random"
	T5_V1_TINY = "hf-internal-testing/tiny-random-t5-v1.1"
	VIT_TINY = "hf-internal-testing/tiny-random-vit"
	XLM_ROBERTA_TINY = "hf-internal-testing/tiny-xlm-roberta"
	XLNET_TINY = "sshleifer/tiny-xlnet-base-cased"

cc: @ydshieh, @LysandreJik

[ydshieh]

Hi!

I will update those repositories in transformers/tests/deepspeed/test_model_zoo.py so they have safetensors files.

[ydshieh]

I updated 22 repositories in transformers/tests/deepspeed/test_model_zoo.py that is in hf-internal-testing.

For example: https://huggingface.co/hf-internal-testing/tiny-random-t5-v1.1/tree/main

Those repositories contained files with old format that I couldn't detect what their architecture were, and I only load them with AutoModel.

[sfc-gh-sbekman]

Thank you very much, @ydshieh - that's great!

besides deepspeed tests I'd imagine many other transformers tests are impacted by this as well

[ydshieh]

Yeah True. At some point, I made a script to create tiny model for pipeline testing in a more systematic way , and most of them contain safetensors

https://huggingface.co/hf-internal-testing/tiny-random-Gemma3ForConditionalGeneration/tree/main

Some earlier ones don't have safetensor, like

https://huggingface.co/hf-internal-testing/tiny-random-BertModel/tree/main

I will find the time to update the scripts, push safetensors, and use them to replace (at least some) tests that are still using the legacy repository that are not really maintained by me.

[sfc-gh-sbekman]

Thanks a lot, @ydshieh!