HADOOP-19433. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-extras. #7586

slfan1989 · 2025-04-09T01:34:09Z

Description of PR

JIRA: HADOOP-19433. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-extras.

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2025-04-09T03:30:00Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 30s		trunk passed
+1 💚	compile	0m 28s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 27s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	checkstyle	0m 28s		trunk passed
+1 💚	mvnsite	0m 31s		trunk passed
+1 💚	javadoc	0m 34s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 27s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 46s		trunk passed
+1 💚	shadedclient	33m 47s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 19s		the patch passed
+1 💚	compile	0m 19s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 19s		the patch passed
+1 💚	compile	0m 18s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	0m 18s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 14s		the patch passed
+1 💚	mvnsite	0m 20s		the patch passed
+1 💚	javadoc	0m 20s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 18s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 40s		the patch passed
+1 💚	shadedclient	34m 24s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1m 8s	/patch-unit-hadoop-tools_hadoop-extras.txt	hadoop-extras in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		114m 32s

Reason	Tests
Failed junit tests	hadoop.mapred.tools.TestGetGroups

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/1/artifact/out/Dockerfile
GITHUB PR	#7586
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 2e0cb1878cdb 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `a75c15e`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/1/testReport/
Max. process+thread count	823 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-extras U: hadoop-tools/hadoop-extras
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

cnauroth

TestGetGroups ran tests in the base class GetGroupsTestBase, which failed. Does JUnit 5 @Disabled not disable tests in superclasses? Would we need to override each method and apply method-level @Disabled annotations?

hadoop-yetus · 2025-04-26T03:18:01Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	14m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 34s		trunk passed
+1 💚	compile	0m 27s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 27s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	checkstyle	0m 28s		trunk passed
+1 💚	mvnsite	0m 31s		trunk passed
+1 💚	javadoc	0m 33s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 26s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 42s		trunk passed
+1 💚	shadedclient	33m 56s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 20s		the patch passed
+1 💚	compile	0m 19s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 19s		the patch passed
+1 💚	compile	0m 18s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	0m 18s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 14s		hadoop-tools/hadoop-extras: The patch generated 0 new + 9 unchanged - 1 fixed = 9 total (was 10)
+1 💚	mvnsite	0m 20s		the patch passed
+1 💚	javadoc	0m 20s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 17s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 41s		the patch passed
+1 💚	shadedclient	34m 3s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1m 7s	/patch-unit-hadoop-tools_hadoop-extras.txt	hadoop-extras in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		129m 22s

Reason	Tests
Failed junit tests	hadoop.mapred.tools.TestGetGroups

Subsystem	Report/Notes
Docker	ClientAPI=1.49 ServerAPI=1.49 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/2/artifact/out/Dockerfile
GITHUB PR	#7586
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux c70e89f7739a 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `637d844`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/2/testReport/
Max. process+thread count	818 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-extras U: hadoop-tools/hadoop-extras
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-04-26T05:03:25Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	24m 2s		trunk passed
+1 💚	compile	0m 19s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 18s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	checkstyle	0m 20s		trunk passed
+1 💚	mvnsite	0m 22s		trunk passed
+1 💚	javadoc	0m 23s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 19s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 32s		trunk passed
+1 💚	shadedclient	20m 0s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 12s		the patch passed
+1 💚	compile	0m 12s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 12s		the patch passed
+1 💚	compile	0m 10s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	0m 10s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 9s		the patch passed
+1 💚	mvnsite	0m 13s		the patch passed
+1 💚	javadoc	0m 12s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 13s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 29s		the patch passed
+1 💚	shadedclient	19m 54s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 48s		hadoop-extras in the patch passed.
+1 💚	asflicense	0m 24s		The patch does not generate ASF License warnings.
		70m 53s

Subsystem	Report/Notes
Docker	ClientAPI=1.49 ServerAPI=1.49 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/4/artifact/out/Dockerfile
GITHUB PR	#7586
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 7585eb715070 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d5b60d9`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/4/testReport/
Max. process+thread count	837 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-extras U: hadoop-tools/hadoop-extras
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-04-26T05:29:42Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 58s		trunk passed
+1 💚	compile	0m 28s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 27s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	checkstyle	0m 28s		trunk passed
+1 💚	mvnsite	0m 30s		trunk passed
+1 💚	javadoc	0m 33s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 26s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 44s		trunk passed
+1 💚	shadedclient	33m 34s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 19s		the patch passed
+1 💚	compile	0m 20s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 20s		the patch passed
+1 💚	compile	0m 18s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	0m 18s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 15s		hadoop-tools/hadoop-extras: The patch generated 0 new + 9 unchanged - 1 fixed = 9 total (was 10)
+1 💚	mvnsite	0m 21s		the patch passed
+1 💚	javadoc	0m 19s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 18s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 41s		the patch passed
+1 💚	shadedclient	33m 51s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 3s		hadoop-extras in the patch passed.
+1 💚	asflicense	0m 36s		The patch does not generate ASF License warnings.
		114m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.49 ServerAPI=1.49 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/3/artifact/out/Dockerfile
GITHUB PR	#7586
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 747f8fab6f3b 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c982dc5`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/3/testReport/
Max. process+thread count	803 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-extras U: hadoop-tools/hadoop-extras
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7586/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2025-04-26T08:27:40Z

TestGetGroups ran tests in the base class GetGroupsTestBase, which failed. Does JUnit 5 @Disabled not disable tests in superclasses? Would we need to override each method and apply method-level @Disabled annotations?

@cnauroth Thank you for reviewing the code. Apologies for not noticing your comment earlier, as I was focused on upgrading other modules to JUnit 5. I believe your suggestion is correct, and I have made the necessary changes. However, I noticed that the unit tests are still being executed. I suspect this may be related to the fact that we are still in a mixed testing mode using both JUnit 4 and JUnit 5.

jojochuang · 2025-04-28T17:27:28Z

hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/mapred/tools/TestGetGroups.java


 /**
 * Tests for the MR implementation of {@link GetGroups}
 */
-@Ignore
+@Disabled


The test was disabled 14 years ago (https://issues.apache.org/jira/browse/MAPREDUCE-3860)
Maybe we should just remove it.... just saying.

jojochuang · 2025-04-28T17:30:25Z

hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/mapred/tools/TestGetGroups.java

@@ -52,4 +53,33 @@ protected Tool getTool(PrintStream o) {
    return new GetGroups(conf, o);
  }

+  @Test


Just curious. Are these test method overrides necessary?

Nvm just noticed your comments above!

HADOOP-19433. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-extras.

a75c15e

github-actions bot added trunk TOOLS labels Apr 9, 2025

cnauroth reviewed Apr 11, 2025

View reviewed changes

HADOOP-19433. Improve Some Code.

637d844

slfan1989 added 2 commits April 26, 2025 11:34

HADOOP-19433. Improve Some Code.

c982dc5

HADOOP-19433. Improve Some Code.

d5b60d9

slfan1989 marked this pull request as ready for review April 26, 2025 03:52

slfan1989 mentioned this pull request Apr 26, 2025

[DISCUSS] [JDK17] Upgrade JUnit / JavaDoc from 4 to 5. #7337

Draft

74 tasks

jojochuang reviewed Apr 28, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-19433. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-extras. #7586

HADOOP-19433. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-extras. #7586

slfan1989 commented Apr 9, 2025

hadoop-yetus commented Apr 9, 2025

cnauroth left a comment

hadoop-yetus commented Apr 26, 2025

hadoop-yetus commented Apr 26, 2025

hadoop-yetus commented Apr 26, 2025

slfan1989 commented Apr 26, 2025

jojochuang Apr 28, 2025

jojochuang Apr 28, 2025

jojochuang Apr 28, 2025

HADOOP-19433. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-extras. #7586

Are you sure you want to change the base?

HADOOP-19433. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-extras. #7586

Conversation

slfan1989 commented Apr 9, 2025

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Apr 9, 2025

cnauroth left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Apr 26, 2025

hadoop-yetus commented Apr 26, 2025

hadoop-yetus commented Apr 26, 2025

slfan1989 commented Apr 26, 2025

jojochuang Apr 28, 2025

Choose a reason for hiding this comment

jojochuang Apr 28, 2025

Choose a reason for hiding this comment

jojochuang Apr 28, 2025

Choose a reason for hiding this comment