Skip to content

Fix for removing usage in every stream chunk response. #1022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 25, 2025

Conversation

Hritik003
Copy link
Contributor

Describe the change

This PR fixes an issue where the usage field is always present in streamed completion responses, even when stream_options.include_usage is explicitly set to false.

By changing the Usage field in CompletionResponse to a pointer and tagging it with omitempty, we ensure that if usage is not included in the actual API response, it will be omitted from the parsed Go struct as well.

This improves correctness and reduces confusion when users want to opt out of usage metadata in streaming mode.


Provide OpenAI documentation link

API Reference: https://platform.openai.com/docs/api-reference/completions/create


Describe your solution

Modified the CompletionResponse struct to update the Usage field:

Usage *Usage `json:"usage,omitempty"`

This allows Go’s encoding/json to omit the field when it's not present in the streamed response. This makes the client-side response cleaner and honors the behavior expected when include_usage is set to false.

No new libraries or dependencies were added.


Tests

  • Manually tested by calling /v1/completions with both stream_options.include_usage: true and false

  • Verified that:

    • When include_usage: true, the field is present.
    • When include_usage: false, the field is omitted in streamed chunks.
  • Updated integration tests to assert presence/absence of Usage field conditionally based on stream option.


Additional context

This PR improves alignment with OpenAI’s API behavior and helps consumers avoid parsing unnecessary fields.

Issue: #1021

1. Make Usage field in completions Response to pointer.
Copy link

codecov bot commented Jun 17, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.83%. Comparing base (a931bf7) to head (a88015f).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1022   +/-   ##
=======================================
  Coverage   85.83%   85.83%           
=======================================
  Files          43       43           
  Lines        2316     2316           
=======================================
  Hits         1988     1988           
  Misses        305      305           
  Partials       23       23           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

1. Make Usage field in completions Response to pointer.
2. Add omitempty to json tag

Signed-off-by: Hritik003 <[email protected]>
@Hritik003
Copy link
Contributor Author

@sashabaranov can you please look into this?

@Hritik003
Copy link
Contributor Author

@sashabaranov A reminder to look into this?

Copy link
Owner

@sashabaranov sashabaranov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@sashabaranov sashabaranov merged commit c125ae2 into sashabaranov:master Jun 25, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants