qwen3 tokenizer convert fail

[pass-lin]

I'm adding Qwen3's implementation to Keras_hub, and now I'm encountering a problem.

The tokenizer of qwen3 and qwen2 is the same. But we can't convert it directly with keras_hub. What's the reason for this? How can we solve it?

from keras_hub.tokenizers import Qwen2Tokenizer
Qwen2Tokenizer.from_preset("hf://Qwen/Qwen2.5-0.5B")
’‘’
<QwenTokenizer name=qwen_tokenizer_5, built=False>
‘’‘
Qwen2Tokenizer.from_preset("hf://Qwen/Qwen2.5-0.6B")
'''
InvalidArgumentError: {{function_node __wrapped__LookupTableImportV2_device_/job:localhost/replica:0/task:0/device:CPU:0}} Expected shape [151387,2] for value, got [151387] [Op:LookupTableImportV2] name:
'''

We can't achieve qwen3 without solving this problem.