TabbyML plugin did not start when using LM Studio and qwen-2.5-coder-7b model

[ashm650]

Tabby VS code plugin giving error Server start failed when trying to use local model by starting lm studio, it is not able to connect LM studio server.

[wsxiaoys]

Hi - can you share your config.toml to help debug the issue?

[tranghaviet]

I got same issue. My sample config

[model.chat.http]
kind = "openai/chat"
model_name = "qwen2.5-coder-7b-instruct"  # Example model
api_endpoint = "http://127.0.0.1:1234/v1"    # LM Studio server endpoint with /v1 path
api_key = ""                                 # No API key required for local deployment

[model.completion.http]
kind = "openai/completion"
model_name = "qwen2.5-coder-7b-instruct"                 # Example code completion model
api_endpoint = "http://127.0.0.1:1234/v1"
api_key = ""
prompt_template = "<PRE> {prefix} <SUF>{suffix} <MID>"  # Example prompt template for CodeLlama models

[ashm650]

@wsxiaoys

disable = true # set to true to disable

[model.chat.http]
kind = "openai/chat"
model_name = "qwen2.5-coder-7b-instruct"  # Example model
api_endpoint = "http://localhost:1234/v1"    # LM Studio server endpoint with /v1 path
api_key = ""                                 # No API key required for local deployment

[model.completion.http]
kind = "openai/completion"
model_name = "qwen2.5-coder-7b-instruct"                 # Example code completion model
api_endpoint = "http://localhost:1234/v1"
api_key = ""
prompt_template = "<PRE> {prefix} <SUF>{suffix} <MID>"  # Example prompt template for CodeLlama models

[model.embedding.http]
kind = "openai/embedding"
model_name = "qwen2.5-coder-7b-instruct"
api_endpoint = "http://localhost:1234/v1"
api_key = ""

[ashm650]

I got same issue. My sample config

[model.chat.http]
kind = "openai/chat"
model_name = "qwen2.5-coder-7b-instruct" # Example model
api_endpoint = "http://127.0.0.1:1234/v1" # LM Studio server endpoint with /v1 path
api_key = "" # No API key required for local deployment

[model.completion.http]
kind = "openai/completion"
model_name = "qwen2.5-coder-7b-instruct" # Example code completion model
api_endpoint = "http://127.0.0.1:1234/v1"
api_key = ""
prompt_template = "

 {prefix} {suffix} "  # Example prompt template for CodeLlama models

were you able to resolve the issue?

[zwpaper]

Hi @ashm650 @tranghaviet , I have tested the chat LM Studio locally, and it works as expected.

I noticed you were using 'qwen2.5-coder-7b-instruct' as a completions model, but that is incorrect. You should use the 'qwen' models without the '-instruct' suffix and with the proper prompt_template. Please use this as a reference:
https://github.com/TabbyML/registry-tabby/blob/d57d6c6bb0d7a0718d401eafeb8f44ff17a81e29/models.json#L320-L335