Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to override Content-Type header #630

Open
ashitaprasad opened this issue Mar 2, 2025 · 19 comments · May be fixed by #658 or #794
Open

Unable to override Content-Type header #630

ashitaprasad opened this issue Mar 2, 2025 · 19 comments · May be fixed by #658 or #794
Labels

Comments

@ashitaprasad
Copy link
Member

Describe the bug/problem

Create a new POST request as shown below with JSON body

Image

If you do not specify any content-type in header, the client automatically sets it to application/json; charset=utf-8.

In the headers, you can specify the Content-Type as application/json to override the default and send request. When you inspect headers in response pane you can observe that the Request header that was sent for Content-Type was application/json; charset=utf-8 instead of application/json

Image

`charset=utf-8' is automatically added into 'Content-Type', and we are unable to remove it as for some APIs, this header string can cause rejection.

Similarly, if you set text/csv as Content-Type in Request and send the response. Again charset=utf-8' is being automatically added by http` library.

Image

The objective is that user should have full control over the Content-Type header that is being sent to the server.

Relevant issues:

@AffanShaikhsurab
Copy link
Contributor

i was able ot reproduce the issue : we should allow the user to have a total control over the content type . I will look into it and see how we can solve this

@AffanShaikhsurab
Copy link
Contributor

@ashitaprasad should we also allow the user to choose the encoding ?

@Jaishree2310
Copy link

@AffanShaikhsurab Are you working on this issue ?

I wanted to work on this if needed can both work together?

@AffanShaikhsurab
Copy link
Contributor

Hi @Jaishree2310,

I've already created a pull request (#631) to address this issue. However, if you have any suggestions or improvements, feel free to review the PR and share your thoughts. I'd be happy to collaborate or make any necessary refinements.

@ashitaprasad
Copy link
Member Author

ashitaprasad commented Mar 2, 2025

@ashitaprasad should we also allow the user to choose the encoding ?

@AffanShaikhsurab The default encoding should be utf-8, if the user specifies a particular encoding using header field Content-Type like application/json; charset=utf-16 or application/json; charset=iso-8859-1 or application/json; charset=us-ascii. Then the body must be encoded using the specified encoding.
https://api.flutter.dev/flutter/dart-convert/Encoding/getByName.html

@AffanShaikhsurab
Copy link
Contributor

AffanShaikhsurab commented Mar 3, 2025

@ashitaprasad Thanks for clarifying , the current implementation in the pr satisfies the above functionality

Image

@ashitaprasad
Copy link
Member Author

@ashitaprasad Thanks for clarifying , the current implementation in the pr satisfies the above functionality

Image

No @AffanShaikhsurab it doesn't. The Request header for Content-Type being sent is application/json; charset=utf-16 but the actual body is being encoded in utf-8

@AffanShaikhsurab
Copy link
Contributor

@ashitaprasad Apologies for the oversight! I've updated it to support multiple encoding types. If you have any further suggestions, I'd be happy to implement them!

@Aman071106
Copy link
Contributor

Aman071106 commented Mar 9, 2025

@ashitaprasad

Issue Summary: Charset=utf-8 in HTTP Headers

Objective

Investigate how the http client (in Dart) handles the Content-Type header and whether charset=utf-8 is automatically appended.

Findings:

  • Default Behavior: The http package in Dart automatically appends charset=utf-8 to the Content-Type: application/json header when sending a request.
  • Expected Behavior: When sending application/json, it is expected to append charset=utf-8 unless manually overridden.
  • Request Testing: Removing charset=utf-8 from the Content-Type header does not prevent it from being appended.

Relevant Code Snippets:

Code to check auto-appended charset=utf-8:

import 'package:http/http.dart' as http;

void main() async {
  var uri = Uri.parse('https://api.apidash.dev/case/lower');
  String body = '''{
    "text": "I LOVE Flutter."
  }''';
  var headers = {
    'Content-Length': '0',
    'Content-Type': 'application/json',
  };

  final response = await http.post(uri, headers: headers, body: body);
  print(response.headers); // Check the headers here
}

Dart HTTP Documentation:
According to the Dart HTTP library documentation, the content-type header is typically auto-calculated, and for JSON, it defaults to UTF-8 encoding unless explicitly stated otherwise. Dart HTTP package documentation.

Overriding the Default Encoding:
You can override the default encoding like so:

var headers = {
  'Content-Type': 'application/json; charset=ISO-8859-1',
};

More Findings:

Some UI issues raised: #644
Even the Content-Length gets auto-calculated if it is incorrect.

Interest in Google Summer of Code (GSoC)

I am highly interested in participating in Google Summer of Code (GSoC) and am eager to contribute to open-source projects. My skills include:

Programming Languages: C++, Python
Mobile Development: Android (Kotlin), Flutter
Version Control: Git, GitHub
AI/ML: Generative AI(LLMs)
Competitive Programming: Experienced in problem-solving and algorithm design
Blockchain Development: Solidity, Hardhat, Aptos

I am particularly interested in projects related to mobile development,, generative AI. I believe my passion for learning and technical skills make me a strong candidate for contributing to innovative open-source projects during GSoC.

@ashitaprasad
Copy link
Member Author

@Aman071106 You can send a PR in case you are planning to fix the issue

@Aman071106
Copy link
Contributor

@ashitaprasad I think there is no need of changing anything as the client (httpClient) from dart has this default behaviour which we can't override. This is a good practice in many clients and if user want he can change encoding as per requirement. Instead what we can do is to show some kind of message stating the default encoding as utf-8.

@Aman071106 Aman071106 mentioned this issue Mar 10, 2025
5 tasks
@ashitaprasad
Copy link
Member Author

@Aman071106 looks like you have totally missed the point specified in the issue description. The feature requires implementation.

@Aman071106
Copy link
Contributor

Aman071106 commented Mar 10, 2025

@ashitaprasad
In response to issue [#630] regarding the inability to override the Content-Type header, I have implemented the necessary fixes to ensure that user-specified headers remain unchanged.

Issue Summary

The API Dash client was automatically appending charset=utf-8 to the Content-Type header, even when a different encoding or no charset was specified by the user. This caused API request failures in scenarios where the server expected a specific Content-Type format.

Steps to Reproduce

  1. Create a new POST request with a JSON body.
  2. Set Content-Type to application/json in the headers.
  3. Send the request and inspect the request headers in the response pane.

Observed Behavior

The Content-Type header in the request was modified to application/json; charset=utf-8, rather than respecting the user-defined value.

Expected Behavior

The Content-Type header should match exactly what the user specifies without appending any additional encoding unless explicitly provided.

Implemented Fix

Content-Length Handling

Previously, the content length was being overridden using utf8.encode(requestBody).length, leading to potential discrepancies when a different encoding was intended. This calculation has been removed to respect the user-defined Content-Length value.

// var contentLength = utf8.encode(requestBody).length;
// if (contentLength > 0) {
//   headers[HttpHeaders.contentLengthHeader] = contentLength.toString();
// }

Content-Type Handling

The client was adding charset=utf-8 automatically when no encoding was specified. Now, the implementation ensures that the exact Content-Type provided by the user is used, without any automatic modifications.

Testing and Validation

To ensure the fix works correctly, I verified the following cases:

  1. No Encoding Specified:
    • When Content-Type: application/json is set, the request should send it as is, without appending charset=utf-8.
  2. Specific Encoding Provided:
    • If Content-Type: application/json; charset=ISO-8859-1 is explicitly defined, it should be preserved in the request without alteration.

These fixes guarantee that API Dash properly respects user-specified headers, preventing unexpected modifications that may cause API request failures.
Could you clarify which part of the implementation is missing? Based on the issue description, I addressed specific part, but I’d be happy to refine or extend the implementation further.

@connectwithpankaj
Copy link

connectwithpankaj commented Mar 11, 2025

@ashitaprasad I Fix: Prevent automatic charset=utf-8 addition to Content-Type header

Image

@connectwithpankaj
Copy link

@ashitaprasad
Fix: Prevent automatic charset=utf-8 addition to Content-Type header

Step 1: Open the http Package
Navigate to the directory:
C:\Users\pksgr(windows username)\AppData\Local\Pub\Cache\hosted\pub.dev\http-1.3.0
This is the latest version of the http package on your system.

Step 2: Find the Request File
Inside http-1.3.0, go to:
lib/src/request.dart
This file is responsible for handling HTTP requests, including setting headers.

Change Summary:

  • Ensured that setting the Content-Type header does not automatically append charset=utf-8.
  • Allowed developers to control the charset parameter explicitly.

Why this change?

  • Some APIs require a specific Content-Type without charset=utf-8.
  • Previously, http package automatically added charset=utf-8, causing unexpected behavior in certain cases.

@WannaCry016
Copy link
Contributor

Hii @ashitaprasad I went through the problem so the problem was with http package it is not solved till now - that automatically encoding for certain content types like json and csv.
So I tried to solve it using dart.io Client which gives us full control over the request and it worked the request headers was sent that i wanted to without any interference but the problem is all the code is implemented with http package client and dart.io client doesnt have any functionality to get the actual request details just like http package had basehttpResponseModel.
So here one thing we could do is to set the request headers same as sent manually. So I wanted to know ur thought on this do i proceed to implement it further?.

Screenshots

Image

Image

Image

Image

u can see it is not showing the request headers in the tab as the current code is using http package and it cant get the request details as it was sent using dart.io client but it was sent correctly as u can see in the second ss which is of local server to see the sent request details

Looking forward to contribute to apidash.
Thanks

@animator
Copy link
Member

@WannaCry016 Why didn't you send a PR for review?

@WannaCry016
Copy link
Contributor

@animator I wanted to know if this approach is fine. I will send a PR soon for review.

@animator
Copy link
Member

sure @WannaCry016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment