FP8 tensors not saved correctly

[Rocketknight1]

I tried making a "mini-Deepseek" for testing but encountered some issues. This works fine:

from transformers import AutoConfig, AutoModelForCausalLM

config = AutoConfig.from_pretrained("deepseek-ai/DeepSeek-V3-0324")
config.num_hidden_layers = 1
config.intermediate_size = 1024

model = AutoModelForCausalLM.from_config(config)
model.save_pretrained("test_save")

However, when I try to reload the model, I get the following:

>>> AutoModelForCausalLM.from_pretrained("test_save")
  File "/home/matt/PycharmProjects/transformers/src/transformers/modeling_utils.py", line 806, in _load_state_dict_into_meta_model
    not hf_quantizer.check_quantized_param(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matt/PycharmProjects/transformers/src/transformers/quantizers/quantizer_finegrained_fp8.py", line 155, in check_quantized_param
    raise ValueError("Expect quantized weights but got an unquantized weight")
ValueError: Expect quantized weights but got an unquantized weight

It seems like even though we support FP8 loading after #36828, we may not be saving it correctly? cc @kylesayrs

[Rocketknight1]

cc @MekkCyber who worked on #36026 as well

[MekkCyber]

I think it's related to some changes that were made for the deepseek v3 integration ! will look into that