Skip to content

Commit 10a4eaf

Browse files
JkSelfcloud-fan
authored andcommitted
[SPARK-30812][SQL] update the skew join configs by adding the prefix "skewedJoinOptimization"
### What changes were proposed in this pull request? This is a follow up in [PR#27563](apache#27563). This PR adds the prefix of "skewedJoinOptimization" in the skew join related configs. ### Why are the changes needed? address remaining address ### Does this PR introduce any user-facing change? No ### How was this patch tested? only update config and no need new ut. Closes apache#27630 from JkSelf/renameskewjoinconfig. Authored-by: jiake <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent c071522 commit 10a4eaf

File tree

1 file changed

+3
-3
lines changed
  • sql/catalyst/src/main/scala/org/apache/spark/sql/internal

1 file changed

+3
-3
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala

+3-3
Original file line numberDiff line numberDiff line change
@@ -432,22 +432,22 @@ object SQLConf {
432432
.createWithDefault(true)
433433

434434
val ADAPTIVE_EXECUTION_SKEWED_PARTITION_SIZE_THRESHOLD =
435-
buildConf("spark.sql.adaptive.optimizeSkewedJoin.skewedPartitionSizeThreshold")
435+
buildConf("spark.sql.adaptive.skewedJoinOptimization.skewedPartitionSizeThreshold")
436436
.doc("Configures the minimum size in bytes for a partition that is considered as a skewed " +
437437
"partition in adaptive skewed join.")
438438
.bytesConf(ByteUnit.BYTE)
439439
.createWithDefaultString("64MB")
440440

441441
val ADAPTIVE_EXECUTION_SKEWED_PARTITION_FACTOR =
442-
buildConf("spark.sql.adaptive.optimizeSkewedJoin.skewedPartitionFactor")
442+
buildConf("spark.sql.adaptive.skewedJoinOptimization.skewedPartitionFactor")
443443
.doc("A partition is considered as a skewed partition if its size is larger than" +
444444
" this factor multiple the median partition size and also larger than " +
445445
s" ${ADAPTIVE_EXECUTION_SKEWED_PARTITION_SIZE_THRESHOLD.key}")
446446
.intConf
447447
.createWithDefault(10)
448448

449449
val ADAPTIVE_EXECUTION_SKEWED_PARTITION_MAX_SPLITS =
450-
buildConf("spark.sql.adaptive.optimizeSkewedJoin.skewedPartitionMaxSplits")
450+
buildConf("spark.sql.adaptive.skewedJoinOptimization.skewedPartitionMaxSplits")
451451
.doc("Configures the maximum number of task to handle a skewed partition in adaptive skewed" +
452452
"join.")
453453
.intConf

0 commit comments

Comments
 (0)