-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Preserve input order of window functions when planning window operations #25589
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Preserve input order of window functions when planning window operations #25589
Conversation
Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Rishi Sajay.
|
core/trino-main/src/main/java/io/trino/sql/planner/QueryPlanner.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/sql/planner/QueryPlanner.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (1)
core/trino-main/src/main/java/io/trino/sql/planner/QueryPlanner.java:1436
- Changing the groupingBy collector to use a LinkedHashMap is the desired fix to preserve the input order; ensure that subsequent usage of this map leverages the order preservation appropriately.
.collect(Collectors.groupingBy(analysis::getWindow, LinkedHashMap::new, Collectors.toList()));
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved % comments.
core/trino-main/src/main/java/io/trino/sql/planner/QueryPlanner.java
Outdated
Show resolved
Hide resolved
602ba82
to
a6cd182
Compare
Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Rishi Sajay.
|
Map<ResolvedWindow, List<io.trino.sql.tree.FunctionCall>> functions = scopeAwareDistinct(subPlan, windowFunctions) | ||
.stream() | ||
.collect(Collectors.groupingBy(analysis::getWindow)); | ||
.collect(Collectors.groupingBy(analysis::getWindow, LinkedHashMap::new, Collectors.toUnmodifiableList())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: static import toUnmodifiableList
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
PARTITION BY category, subcategory, value, subvalue | ||
) AS rank_by_category_subcategory_value_subvalue | ||
FROM data | ||
"""; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please fix the indentation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
a6cd182
to
924b219
Compare
Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Rishi Sajay.
|
924b219
to
107f4fc
Compare
Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Rishi Sajay.
|
@RishiSajay please submit the CLA as described in https://github.com/trinodb/cla |
@cla-bot check |
Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Rishi Sajay.
|
The cla-bot has been summoned, and re-checked this pull request! |
@cla-bot check |
Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Rishi Sajay.
|
The cla-bot has been summoned, and re-checked this pull request! |
107f4fc
to
97cfcbd
Compare
Description
Fixes a bug where initial plans for the exact same query string are non-deterministic when the query includes window functions with different resolved windows over the same window. (Resolved window takes into account
partition by
,order by
, etc)Additional context and related issues
Window functions with the same window are planned in the same operation (see commit).
QueryPlanner::planWindowFunctions
takes in a list of windowFunctions and converts it into a map to be processed. However, HashMaps do not guarantee input order, leading to different initial query plans for the same query string.This PR replaces the default
HashMap
with aLinkedHashMap
to preserve the order of resolved windows (and by extension their associated window functions) when planning window functions, therefore guaranteeing the same initial plan every time the same query is run.Minimal repro and associated diff of initial query plans from 2 different runs before this PR:
Release notes
(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: