Is it possible to use a self signed certificate with the S3 implementation?

[vikramaditya91]

Hello

I am trying to connect to a Minio endpoint with the storage_s3.rs.
The minio instance expects a self signed certificate and that is why the retrival fails with

client error (Connect): invalid peer certificate: UnknownIssuer

It seems that the reqwest client is built through opendal, but I cannot pass parameters to it.

I could not figure out how a self signed certificate could be passed to the client or if I could use my own client built like this

Client::builder()
    // ...
    .add_root_certificate(config.server_certificate)
    .danger_accept_invalid_certs(true) 

Does someone know if there is any workaround I could use?

Should this be a new feature request?

[liurenjie1024]

cc @Xuanwo Do opendal supports this?

[vikramaditya91]

I believe it always instantiates this the reqwest client this way and hence not possible?

pub(crate) static GLOBAL_REQWEST_CLIENT: LazyLock<reqwest::Client> =
    LazyLock::new(reqwest::Client::new);

https://github.com/apache/opendal/blob/a3549fcee82c8d2faf926feb2ccf4fb7f25e0f9a/core/src/raw/http_util/client.rs#L50

[Xuanwo]

OpenDAL has APIs like https://docs.rs/opendal/0.53.0/opendal/struct.Operator.html#method.update_http_client, so it's possible for users to pass any config in opendal.

The problem here is how iceberg-rust expose this API.

[vikramaditya91]

@Xuanwo if I wanted to make a PR to get this feature into iceberg rust, do you have a suggestion on how I should instrument this?

[Xuanwo]

In my opinion, the best approach is to install your self-signed certificate into the system trust store so that all your local tools can work well with the certificate. This will enhance the security of your connection to the MinIO server, ensuring that you are connecting to the correct instance.

Although it's possible for us to implement this, I really don't think using danger_accept_invalid_certs is a valid use case.

[vikramaditya91]

I agree. Let me take a shot with installing the self-signed certs to the system trust store and see how it would work.

PS: There is an open issue with reqwest which gave me the impression that I would need to use the danger_accept_invalid_certs if I wanted to use self-signed certs.

[vikramaditya91]

Thanks @Xuanwo .

It seems that the system certs were disabled in reqwest.
It was not clear to me why that is the case based on reading this.

Anyway, I enabled the system certs with the feature flag features = ["rustls-tls-native-roots-no-provider"] and that allowed me to the use the system certs.

Thanks for your help.