Add support for validating tool invocation

[arturl]

I'm defining MCP tools in my server and I want to ensure that the right tools are invoked given specific user queries.

For example, I have a tool that can answer questions about the atmosphere. I want to make sure its schema is correctly defined so that if the user asks the question "why is sky blue", my tool is invoked.

This becomes even more important as I add new tools, and I want to make sure these new tools don't introduce conflicts or ambiguity. For example, if I add a tool that talks about plants, it will be invoked when the user asks "why is grass green" but the atmosphere tool still responds to the question about the sky.

I imagine the format would be something like

npx @modelcontextprotocol/inspector --cli node dist/index.js stdio --query "Why is sky blue"

And the output would be JSON containing the name of the tool.

[matteo8p]

Hi @arturl, I think to build the feature you want, the inspector needs to have built in LLM support, that way you can test whether or not the LLM will choose the right tool.

I started working on that with my fork of this inspector, the MCPJam inspector. It has LLM support where you can test your tools against a real LLM. It's not perfect yet and doesn't have cli support, but its a step closer.

I hope this inspector will eventually have LLM support too.

[arturl]

Thanks @matteo8p I will give it a try!