NoSuchMethodException due to OkHttpChannelProvider default constructor missing after R8 full mode optimization

[bubenheimer]

What version of gRPC-Java are you using?

1.49.2

What is your environment?

Android

What did you see?

When run after R8 full mode optimization, grpc throws an exception during execution:

Failed to construct OkHttpChannelProvider

java.lang.NoSuchMethodException: io.grpc.okhttp.OkHttpChannelProvider.<init> []
at java.lang.Class.getConstructor0(Class.java:2363)
at java.lang.Class.getConstructor(Class.java:1759)
at io.grpc.android.AndroidChannelBuilder.<clinit>(SourceFile:14)
at com.example.GrpcServiceModule$channel$1.invokeSuspend(SourceFile:51)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(SourceFile:6)
at kotlinx.coroutines.DispatchedTask.run(SourceFile:108)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(SourceFile:77)

This is because R8 full mode does not implicitly keep default constructors:
https://r8.googlesource.com/r8/+/refs/heads/master/compatibility-faq.md#r8-full-mode

This Proguard/R8 configuration rule avoids the issue and should be added to https://github.com/grpc/grpc-java/blob/master/android/proguard-rules.txt:

-keepclassmembers, allowoptimization class io.grpc.okhttp.OkHttpChannelProvider {
    <init>();
}

There may be additional configuration rules needed to take care of further default constructors accessed via reflection.

[ejona86]

So it keeps the class and methods (because the interface methods are used), but it doesn't have any way to construct the class.

We've taken the approach of "don't use any Proguard configuration" because it is hard to get used in all environments and have a configuration that is precise for all environments. We want to inform the optimizer via code that stuff is used, because that code may be eliminated.

But this behavior from R8 should also break using ManagedChannelBuilder.forTarget() and similar APIs. We'd need to keep the init for all NameResolvers, LoadBalancers, and ManagedChannelProviders. But apparently you didn't... So I guess it saw the META-INF/services/ and recognized those classes needed default constructors? More investigation is needed here.

[bubenheimer]

The proguard-rules.txt file that I mentioned is incorporated in the grpc-android artifact, providing Proguard configuration for grpc-android library consumers.

A good amount of grpc (OkHttp Android client) communication seems to work for me without further R8/Proguard configuration than what's referenced in this issue, I don't currently use LoadBalancer stuff AFAIK. I did encounter some intermittent issues in my cursory exploration runs today, but they may or may not be related. I'm not sure if I'll find more time to investigate before next month.

Android has default Proguard/R8 configuration files that are commonly used and that I rely on, and there is additional Proguard configuration coming from other libraries I use, and some of my own. Any of these could lead to an unexpected smooth operation of those things.

Slapping an explicit @Keep on needed constructors would likely be another way to retain them, but I don't know if it has to come from androidx.annotation (or android.annotation) or if R8 understands others, like the one from errorprone.

[bubenheimer]

I've given up on R8 full mode for now; grpc things seemed to work well for me, though, with that one added rule. I got hung up on other hard-to-debug problems. R8 full mode seemed to work for me in the past here and there, but it seems to have moved beyond real world usability. Or the other way round.

[ejona86]

Seems we won't be doing anything here for the moment. @Keep could work, but is wrong, because we really want the consumer of the API to dictate what needs to be kept, so dead-code elimination can clean up more things when possible. Since it sounds like R8 full mode in not for the feint of heart, we'll delay until someone is sticking with R8 full mode. Maybe it gets easier to work with between then and now.

Things have changed since we did grpc-android initially; we have a much better understanding of how Cronet fits into things. It might be better to add a direct dependency from grpc-android to grpc-okhttp and reference OkHttpChannelProvider directly instead of reflection. But that's this one case; there are probably other problems that would also need resolving (e.g., all the provider registries).

[bubenheimer]

@ejona86 Agreed. What you say about avoiding reflection with OkHttpChannelProvider sounds like it would make sense independent of this issue. Reflection is still unusually expensive on Android even today I believe.

[bubenheimer]

This will likely need revisiting soon, as R8 full mode is becoming the default: https://youtu.be/WZ1A7aoEHSw?t=554

[ejona86]

Well, I slept better for 3 weeks.

[temawi]

@bubenheimer, I tried to reproduce what you are seeing using the gRPC android-interop-testing project by creating a gradle.properties file with the line android.enableR8.fullMode=true.

I do see WARNING:: The option setting 'android.enableR8.fullMode=true' is experimental., so I believe R8 full mode is getting enabled. But I'm able to run the interop test client fine without the NoSuchMethodException.

Would you happen to have a project I could use to reproduce the problem?

[bubenheimer]

@temawi what's your R8 version? I print it with my my builds like this:

println("D8/R8 version: " + com.android.tools.r8.Version.getVersionString())

Reason I'm asking: not sure that current R8 still prints that message in full mode, it's not experimental anymore. I didn't always have this problem in R8 full mode, so you may not see it with an older R8 version. The R8 from Android Studio Giraffe alpha 2 is 8.1.8-dev. The R8 from the latest stable Android Studio (Electric Eel) (AGP 7.4.0) is 4.0.48, I was using nothing more recent than that, so it should suffice for your repro.

The client likely needs to be Android, and use OkHttp.

I don't have a public project of my own to repro this.