-
Notifications
You must be signed in to change notification settings - Fork 5.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update conversation metadata with cost and token metrics #7561
Conversation
openhands/server/conversation_manager/standalone_conversation_manager.py
Show resolved
Hide resolved
if hasattr(metrics, 'accumulated_token_usage'): | ||
token_usage = metrics.accumulated_token_usage | ||
conversation.prompt_tokens += token_usage.prompt_tokens | ||
conversation.completion_tokens += token_usage.completion_tokens |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like _accumulated_token_usage
is already a TokenUsage object which includes the additions:
https://github.com/All-Hands-AI/OpenHands/pull/7511/files#diff-9010f8fab42fcc1b1464f428310e3ef8f93d07c22b6b481bedbbc5a0d864386aR132
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good catch! reverting
This PR updates the conversation metadata to include cost/token metrics from events.
Changes
Added new fields to
ConversationMetadata
class:accumulated_cost
: Tracks the total cost of the conversationprompt_tokens
: Tracks the number of prompt tokens usedcompletion_tokens
: Tracks the number of completion tokens usedtotal_tokens
: Tracks the total number of tokens usedRenamed
_update_timestamp_for_conversation
to_update_conversation_for_event
to better reflect its expanded purposeModified
_create_conversation_update_callback
to pass the event to the callback functionUpdated the function to extract and store metrics from the event when available
All tests are passing with these changes.
To run this PR locally, use the following command: