Skip to content

Fix Custom Final Answer Tool for ToolCallingAgent #1255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

NoRaincheck
Copy link
Contributor

@NoRaincheck NoRaincheck commented Apr 27, 2025

An attempt at fixing #1254

It forces the ToolCallingAgent to always execute the final tool, so that custom final tools will run as intended.

If appropriate please feel free to make changes directly.

Tested locally

@albertvillanova
Copy link
Member

Thanks for the fix, @HeardACat.

I am reviewing it.

Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

I added a test and fixed the implementation to avoid calling the final_answer tool with non-JSON-serializable arguments (image, audio), which raise an error:

TypeError: Object of type AgentImage is not JSON serializable
https://github.com/huggingface/smolagents/actions/runs/14885689071/job/41804563090?pr=1255

FAILED tests/test_monitoring.py::MonitoringTester::test_streaming_agent_image_output - TypeError: Object of type AgentImage is not JSON serializable
/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/site-packages/smolagents/agents.py:1201: in execute_tool_call
    f"Error executing tool '{tool_name}' with arguments {json.dumps(arguments)}: {type(e).__name__}: {e}\n"
/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/json/__init__.py:231: in dumps
    return _default_encoder.encode(obj)
/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/json/encoder.py:199: in encode
    chunks = self.iterencode(o, _one_shot=True)
/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/json/encoder.py:257: in iterencode
    return _iterencode(o, 0)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <json.encoder.JSONEncoder object at 0x7f5f89fdbf70>
o = <smolagents.agent_types.AgentImage image mode= size=0x0 at 0x7F5E26E18940>

>       raise TypeError(f'Object of type {o.__class__.__name__} '
                        f'is not JSON serializable')
E       TypeError: Object of type AgentImage is not JSON serializable

/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/json/encoder.py:179: TypeError

@albertvillanova albertvillanova merged commit e38444c into huggingface:main May 7, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Custom Final Answer Tool does not work with ToolCallingAgent
2 participants