[SPARK-51549][BUILD][3.5] Bump Parquet 1.15.1 #50528
Open
+369
−363
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes quite a few bugs on the Parquet side: https://github.com/apache/parquet-mr/blob/master/CHANGES.md#version-1140 No Using the existing unit tests No Closes apache#46447 from Fokko/fd-bump-parquet. Authored-by: Fokko Driesprong <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
This PR aims to upgrade Parquet to 1.14.2. To bring the latest bug fixes. - https://mvnrepository.com/artifact/org.apache.parquet/parquet-common/1.14.2 No. Pass the CIs. No. Closes apache#47807 from Fokko/fd-parquet. Lead-authored-by: Fokko <[email protected]> Co-authored-by: Dongjoon Hyun <[email protected]> Co-authored-by: Fokko Driesprong <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
The pr aims to upgrade `Parquet` from `1.14.2` to `1.14.3`. The full release notes: https://github.com/apache/parquet-java/releases/tag/apache-parquet-1.14.3 apache/parquet-java#3007: Ensure version specific Jackson classes are shaded apache/parquet-java#3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding No. Pass GA. No. Closes apache#48378 from panbingkun/SPARK-49903. Authored-by: panbingkun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
Bumping Apache Parquet to 1.14.4 because of a critical bug when writing a dictionary larger than 8kb. For a full overview of bugfixes, see: https://github.com/apache/parquet-java/releases/tag/apache-parquet-1.14.4 A serious issue was discovered in the 1.14.x line: https://github.com/apache/parquet-java/releases/tag/apache-parquet-1.14.4-rc2 No Existing unit tests. No Closes apache#48790 from Fokko/fd-bump-parquet-java. Lead-authored-by: Fokko Driesprong <[email protected]> Co-authored-by: Fokko <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
Bumps to the latest version of Parquet. For the full list of changes, please check the pre-release: https://github.com/apache/parquet-java/releases/tag/apache-parquet-1.15.0 Including some interesting patches for Spark, such as apache/parquet-java#3030 To bring the latest features and bug fixes for Apache Spark 4.0.0. No. Pass the CIs. No. Closes apache#48970 from Fokko/fd-parquet-1-15-0. Authored-by: Fokko <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
Bump Parquet 1.15.1. Release Notes https://github.com/apache/parquet-java/releases/tag/apache-parquet-1.15.1 No. Pass GHA. No Closes apache#50319 from pan3793/SPARK-51549. Authored-by: Cheng Pan <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
wangyum
force-pushed
the
parquet-branch-3.5
branch
from
4add2a8 to
0936ead
Compare
HyukjinKwon
approved these changes
Apr 8, 2025
wangyum
commented
Apr 8, 2025
Comment on lines
+2666
to
+2671
| <exclusions> | ||
| <exclusion> | ||
| <groupId>com.h2database</groupId> | ||
| <artifactId>h2</artifactId> | ||
| </exclusion> | ||
| </exclusions> |
There was a problem hiding this comment.
To fix:
build/sbt "sql/testOnly *.JDBCSuite"
sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.UnsupportedClassVersionError: org/h2/Driver has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
pan3793
reviewed
Apr 8, 2025
sql/core/benchmarks/BuiltInDataSourceWriteBenchmark-results.txt
Outdated
Show resolved
Hide resolved
LuciferYang
approved these changes
Apr 8, 2025
dongjoon-hyun
approved these changes
Apr 9, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Bump Parquet to 1.15.1.
Why are the changes needed?
To fix critical CVE: https://www.cve.org/CVERecord?id=CVE-2025-30065
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass GHA.
Was this patch authored or co-authored using generative AI tooling?
No.