Fix for removing usage in every stream chunk response. #1022
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe the change
This PR fixes an issue where the
usage
field is always present in streamed completion responses, even whenstream_options.include_usage
is explicitly set tofalse
.By changing the
Usage
field inCompletionResponse
to a pointer and tagging it withomitempty
, we ensure that ifusage
is not included in the actual API response, it will be omitted from the parsed Go struct as well.This improves correctness and reduces confusion when users want to opt out of usage metadata in streaming mode.
Provide OpenAI documentation link
API Reference: https://platform.openai.com/docs/api-reference/completions/create
Describe your solution
Modified the
CompletionResponse
struct to update theUsage
field:This allows Go’s
encoding/json
to omit the field when it's not present in the streamed response. This makes the client-side response cleaner and honors the behavior expected wheninclude_usage
is set tofalse
.No new libraries or dependencies were added.
Tests
Manually tested by calling
/v1/completions
with bothstream_options.include_usage: true
andfalse
Verified that:
include_usage: true
, the field is present.include_usage: false
, the field is omitted in streamed chunks.Updated integration tests to assert presence/absence of
Usage
field conditionally based on stream option.Additional context
This PR improves alignment with OpenAI’s API behavior and helps consumers avoid parsing unnecessary fields.
Issue: #1021