Add module-info.java

[hrchu]

So that projects depend on this can be published to a public artifact repository.
Note that this is not breaking backward compatibility. All codes except this file can be still compiled in Java 6.

[jbduncan]

Guava has an Automatic-Module-Name in its MANIFEST.MF now, so I believe this is not quite as important as it may seem. But if I'm mistaken, then I'd be more than happy to be proven wrong.

(BTW, I think I might have misunderstood something, because module-info.java is Java 9 specific, right? Would a Java 8 compiler (which I believe is what is used to compile guava-jre) or an Android compiler (for guava-android) happily process it or otherwise ignore it?)

[cpovirk]

Our current thinking is that we'll look into this next quarter. We have seen some problems from module-info files in third-party jars, since they're Java 9 .class files and not everyone uses Java 9 yet. So, if we can get away with Automatic-Module-Name, we'll continue to do so, possibly even beyond next quarter. But if there are cases in which Automatic-Module-Name isn't good enough, that would be good to know.

[hrchu]

It is possible to only compile modile-info.java in Java 9, so the jar file is still compatible to users who uses earlier Java version.

A maven example:
https://github.com/twonote/radosgw-admin4j/blob/java9/pom.xml#L127

[cpovirk]

Right, thanks. We've seen problems even when the main .class files are compiled for an older version but the module-info.class is present. As I understand it, various tools try to process all .class files, and they need to be updated to ignore module-info.class.

[hrchu]

@cpovirk could you tell me more about this problem?

[cpovirk]

I wasn't personally involved in fixing the problems, but the basic idea seems to be that people scan the whole classpath (using something like ClassPath.getTopLevelClasses -- which might be an example of something that needs updated to ignore module-info.class :)) and then try to examine/load the classes with a tool that understands only, say, Java 8.

[orionll]

It's worth mentioning that if we add module-info.java, all packages will not be open anymore.

[jbduncan]

@orionll Am I right to think/remember that in the JPMS, open packages are packages whose internal classes can be inspected with reflection?

[orionll]

@jbduncat Yes, exactly. And also private members of public classes.

[jbduncan]

@orionll Cool, thanks for confirming things for me. :)

I personally wonder how important it would be for Guava's packages to be open when used on the module path. I struggle to imagine that reflectively calling Guava's internals is a common thing to do, especially considering Guava's (IMO) pretty durn good API. 🤔

[HoldYourWaffle]

Are there any reasons for it not being open? Even if it's uncommon it might still be done by some people.

[jbduncan]

@HoldYourWaffle I think the main reason is it prevents people from using reflection to depend on internals which may change or disappear in future releases of Guava without warning.

[jbduncan]

...which by my understanding makes things easier for everyone in the long-run.

[jbduncan]

The only reason I can currently think of to have Guava's packages open in the module-info.java is if frameworks like Spring need to reflectively access its internals to do important stuff, but I don't know how important or common that is.

[orionll]

All Guava packages should be closed because as @jbduncan said the dependence on class internals is a bad practice. If someone really wants to access the internals, they can use --add-opens command line option which forces the specified package to be open.

[HoldYourWaffle]

Good point I forgot about --add-opens. Imho you should always be able to hack into internals (maybe you want to do something the developers didn't think of but it's too 'personal' that it's not worthy of a PR), with the risk of it breaking in new versions. --add-opens allows for this so it'd indeed be better to close the guava packages.

[SamCarlberg]

Guava has an Automatic-Module-Name in its MANIFEST.MF now, so I believe this is not quite as important as it may seem. But if I'm mistaken, then I'd be more than happy to be proven wrong.

It does mean that jlink will not work, since the tool requires a module name to be specified in the module-info.java file; automatic module names will not be accepted.

[hannes-transentials]

As much as I love Guava and appreciate Google's efforts, it is somehow embarrassing that a company like Google is not able to adopt Java modules within one year.

Either Google does not use Guava internally or they keep using JDK 8 and won't adopt Jigsaw.

[jbduncan]

@hannes-transentials I think it's most likely that Google have not migrated to JDK 11 and adopted modules yet simply because their internal codebase is so mind-bogglingly humongous. ;)

I say this because I remember reading somewhere (or I inferred) that they use Guava or an superset internally, and I also remember they announced a few years ago that they'd finally migrated to JDK 8 after a lot of effort. So I'm sure that they'll announce support for JDK 11 or a later LTS version (and, by extension, modules) when they fully migrate away from Java 8 and when they feel that most of us non-Googlers have moved away from Java 8 too. (I know that my company hasn't done so yet simply because Java 9 was such a freaking big, backwards-incompatible change!)

[orionll]

It's worth mentioning that adding module-info.java can break some existing tools. For example, I know that IDEA's Osmorc plugin does not work when both module-info.java exists and the library is an OSGi bundle (Guava is). So, while the tools are not ready yet, I would abstain from adding module-info.java to Guava.

[hannes-transentials]

So I either do without modularized applications or stay away from Guava (and many other popular applications)? I somehow hoped that there was some middle ground.

[jbduncan]

Well... you can use Guava as a module in a vanilla-Java modular application. But since Guava only includes an Automatic-Module-Name in its manifest, rather than going further and including amodule.info (out of necessity to stay Java-8-compatible), you won't be able to use it with jlink to create minimised modular Java applications.

Furthermore, frameworks built on top of Java that have their own programming models, like Spring, may have not fully migrated to be Java-11-compatible yet, so if you use such frameworks a lot, you may have to wait a bit longer.

That being said, if you do use a framework such a Spring, please check for yourself if Java 11 and modules work with it, since my knowledge of Spring and other frameworks is limited. :)

[overheadhunter]

out of necessity to stay Java-8-compatible

Well you can create multi-release jars, where the module-info.class file only gets included inside META-INF/versions/9 and is therefore invisible for old¹ class loaders.

---

¹ As long as no fancy custom class loaders eagerly load everything they find in a jar without reading the manifest entries.

[talios]

@hannes-transentials You could make use of something like https://github.com/moditect/moditect to adapt guava and add a module-info.java/.class at your applications side of things as a transitionary work around.

If module-info.java was to be added to guava, hopefully it'd be done so as a modular jar so we don't break java 8< versions.

[jjohannes]

Thanks so much for getting this done @sgammon and @cpovirk!
I discovered this small follow up regarding annotation processing when trying it out: #7732

[sgammon]

@bowbahdoe I kind of want to actually make them. I'll look for a low-min jacket provider 😆 Also, I just wanted to extend a thank you to @cpovirk, @bowbahdoe, @cowwoc, @ben-manes, @jbduncan, and many others in this thread who were extremely helpful and patient. This PR was easy to write because of all the research that was done ahead of time.

I'm so glad Guava will support modules now. Yay! We did it guys

[norrisjeremy]

Folks, the new release seems to have a serious regression in that it doubles the size of the Guava jar file, from approximately 3MB to 6MB.
This seems to be due to the fact that all classes are duplicated under META-INF/versions/9.
I'm not sure why it was decided to build a second Java 9 version of all classes, instead of simply putting module-info.class under META-INF/versions/9.

[cpovirk]

Yes, sorry, that was a misconfiguration, which obviously I should have tested before publishing the release :( It's mentioned in the current release notes, and I have to have a fixed version published today.

[edit: Now see #7743 Sorry for not filing such an issue.]

[sgammon]

@norrisjeremy This is probably my fault. I encountered this issue with the JAR and OSGi plugin during the JPMS PR, but thought I had solved it.

In any case, extra classes in the versioned JAR root won't preclude anything from working. The JVM will prefer those classes but otherwise they should be identical.

The next release should fix the issue.

@cpovirk sorry about this. I promise I tested as much as I feasibly could locally. I did try to build the module descriptor without sources, but at the time sources were required for other complex reasons (iirc).

[ben-manes]

I believe that the testlib's module-info should also require junit, e.g. requires transitive junit;. While javac doesn't complain to fail the build, Eclipse's ECJ does produce this build error. Historically ecj was more standard compliant than javac and helped to squash many bugs such as in generics.

The type junit.framework.TestCase cannot be resolved. It is indirectly referenced from required type com.google.common.testing.AbstractPackageSanityTests

[cpovirk]

The PR looked pretty clearly like the result of solving a dozen similar issues, so I think an extra-large jar with some -Xlint:all warnings is a pretty impressive outcome for the initial release :) (I, too, had thought that javac required you pass "real" sources along with the module-info.)

For JUnit 4:

• From my testing, it seems to be fine that JUnit 4 has only an Automatic-Module-Name. (That's good, as AFAIK the JUnit 4 folks are still hoping never to have to publish a release again :))
• I suspect that requires would be enough to satisfy the compiler. But requires transitive seems better unless you're the kind of person who doesn't believe in requires transitive at all :)
• Once we buy into requires transitive, guava-testlib should probably be using it for its dependency on guava, too, given that guava-testlib uses guava types in its API. For that matter, guava should probably be using requires transitive for its dependency on com.google.common.util.concurrent.internal, given that guava uses types from that module in its API. Maybe that would eliminate the apparent need for guava-testlib to declare what would then become a redundant dependency on c.g.c.util.concurrent.internal? (I don't think that we expose java.logging in our API (neither in guava nor guava-testlib), at least.)

[cpovirk]

Check my proposed changes here: #7748

History shows that a number of you folks know more about all this than we do :)

[cowwoc]

@cpovirk I am only concerned about https://github.com/google/guava/pull/7748/files#diff-6eabd8d281c8a36ef65bc60a72f3e4af026d190937553b6b5ac0b9ddb2a6d190R20

By adding transitive you are exporting an internal package to the public. That sounds wrong to me.

[overheadhunter]

By adding transitive you are exporting an internal package to the public. That sounds wrong to me.

I agree, maybe it is better to relocate InternalFutureFailureAccess to a public package? If com.google.common.util.concurrent.internal has never been intended to be used by lib consumers, I wouldn't even consider moving it a breaking change.

[vietj]

@sgammon thanks for your effort

[cpovirk]

Thanks for having a look.

"internal" was a poor choice of package name on my part. It is actually meant to be accessible to other users. (And among the reasons that it's a poor name is that our other internal subpackage, the one under common.base, is not meant to be accessible to other users.) We were trying to steer "normal" users away from it, but we do want code (mainly low-level libraries and frameworks) to be able to use it to maximize performance, like here in kotlinx.coroutines. Perhaps other people have wanted to use it but been scared off by the name :(

[cowwoc]

@cpovirk Backwards-compatibility aside, the right way to go would be to rename the package and then export it.

Worse-case scenario, you could deprecate the old classes, redirect calls to the new package under the hood, and add Javadoc steering users to the new package.

[cowwoc]

Hmm, where is this package anyway? :) The closest I got to finding it was https://github.com/google/guava/blob/master/guava/src/com/google/common/util/concurrent/Internal.java but this is a class, not a package.

[cowwoc]

Nevermind, found it: https://github.com/google/guava/tree/master/futures/failureaccess/src/com/google/common/util/concurrent/internal

And then I found this README.md at the top-level directory:

The modules under this directory will be released exactly once each. Once that happens, there will be no need to ever update any files here.

Oops... Are we in trouble?

[cpovirk]

The package has been around, publicly accessible, and endorsed by us for usage a few years at this point, so I think that's as good as "exported." (That includes being listed in the OSGi Export-Package line for the failureaccess jar ever since we set up OSGi there, and it includes being in the exports section of the Java module that we recently set up.) So I think it's too late to try to rename it, since that would cause the usual havoc of incompatible Guava changes.

As for the multiple releases: The main goal has been to neither add nor remove APIs. I think that 1.0.1 was to add the OSGi information, 1.0.2 was to add an Automatic-Module-Name, and 1.0.3 was to set up a proper module. It would have been idea to do that all in 1.0, but I think that the multiple releases have been only a marginal contributor to the occasional annoyances that people see with failureaccess. (But people should correct me if I'm wrong: While I hope that we don't have reason to release a 1.0.4 in the future, I would like to know both the pros and cons that such a release might have.)

[cpovirk]

I released 33.4.7 with the fixes of #7748, including requires transitive junit in guava-testlib.