Ability to limit LLM call count when tools are used

[se-roberthanson]

I created a tool that allows the LLM to trigger a "place search". And if I give it specific instructions I can cause Sprint AI to make many LLM calls.

Example prompt:
"think step by step.
you can perform sequential operations.

search for places names after each color of the rainbow, performing only one search at a time.
if a search returns zero results, then skip the next color.
if a search returns at least 1 result, search places for the next color."

In my test case Spring AI made 7 calls to OpenAI. The first 6 calls resulted in a tool execution (search for "red", "orange", etc.) and the 7th call provided the combined response.

My concern is that a knowledgeable user can force a loop that will results in a very high OpenAI bill for me.

Ideally there would be some interceptor so that I can make the decision for each iteration of the tool loop (as well as ready the token counts), but even something as simple as a property that allows me to limit the number of calls made to the LLM would work.

Let me know if this already exists and I missed it.

[donhuvy]

It is very practical and meaningful. If we can set limitation of API tokens in application.yml , it is great.