Skip to content

Commit 549cf84

Browse files
mustafasrepojayzhan211berkaysynnadaozankabak
authored
Convert first, last aggregate function to UDAF (#10648)
* move out the ordering ruel Signed-off-by: jayzhan211 <[email protected]> * introduce rule Signed-off-by: jayzhan211 <[email protected]> * revert test result Signed-off-by: jayzhan211 <[email protected]> * pass mulit order test Signed-off-by: jayzhan211 <[email protected]> * cleanup Signed-off-by: jayzhan211 <[email protected]> * with new childes Signed-off-by: jayzhan211 <[email protected]> * revert slt Signed-off-by: jayzhan211 <[email protected]> * revert back Signed-off-by: jayzhan211 <[email protected]> * rm rewrite in new child Signed-off-by: jayzhan211 <[email protected]> * backup Signed-off-by: jayzhan211 <[email protected]> * only move conversion to optimizer Signed-off-by: jayzhan211 <[email protected]> * find test that do reverse Signed-off-by: jayzhan211 <[email protected]> * add test for first and last Signed-off-by: jayzhan211 <[email protected]> * pass all test Signed-off-by: jayzhan211 <[email protected]> * upd test Signed-off-by: jayzhan211 <[email protected]> * upd test Signed-off-by: jayzhan211 <[email protected]> * cleanup Signed-off-by: jayzhan211 <[email protected]> * cleanup Signed-off-by: jayzhan211 <[email protected]> * cleanup Signed-off-by: jayzhan211 <[email protected]> * add aggregate test Signed-off-by: jayzhan211 <[email protected]> * cleanup Signed-off-by: jayzhan211 <[email protected]> * final draft Signed-off-by: jayzhan211 <[email protected]> * cleanup again Signed-off-by: jayzhan211 <[email protected]> * pull out finer ordering code and reuse Signed-off-by: jayzhan211 <[email protected]> * clippy Signed-off-by: jayzhan211 <[email protected]> * remove finer in optimize rule Signed-off-by: jayzhan211 <[email protected]> * add comments and clenaup Signed-off-by: jayzhan211 <[email protected]> * rename fun Signed-off-by: jayzhan211 <[email protected]> * rename fun Signed-off-by: jayzhan211 <[email protected]> * fmt Signed-off-by: jayzhan211 <[email protected]> * avoid unnecessary recursion and rename Signed-off-by: jayzhan211 <[email protected]> * Minor changes * Add new API for aggregate optimization * Minor changes * Minor changes * Remove old code * Minor changes * Minor changes * Minor changes * Minor changes * Minor changes * Minor changes * Update comments * Minor changes * Minor changes * Review Part 1 * TMP * Update display of aggregate fun exprs * TMP * TMP * Update tests * TMP buggy * modify name in place * Minor changes * Tmp * Tmp * Tmp * TMP * Simplifications * Tmp * Tmp * Compiles * Resolve linter errors * Resolve linter errors * Minor changes * Simplifications * Minor chagnes * Move cast to common * Minor changes * Fix test * Minor changes * Simplifications * Review * Address reviews * Address reviews * Update documentation, rename method * Minor changes --------- Signed-off-by: jayzhan211 <[email protected]> Co-authored-by: jayzhan211 <[email protected]> Co-authored-by: berkaysynnada <[email protected]> Co-authored-by: Mehmet Ozan Kabak <[email protected]>
1 parent ae10754 commit 549cf84

File tree

36 files changed

+755
-856
lines changed

36 files changed

+755
-856
lines changed

‎datafusion-cli/Cargo.lock

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎datafusion/core/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,7 @@ datafusion-functions-aggregate = { workspace = true }
105105
datafusion-functions-array = { workspace = true, optional = true }
106106
datafusion-optimizer = { workspace = true }
107107
datafusion-physical-expr = { workspace = true }
108+
datafusion-physical-expr-common = { workspace = true }
108109
datafusion-physical-plan = { workspace = true }
109110
datafusion-sql = { workspace = true }
110111
flate2 = { version = "1.0.24", optional = true }

‎datafusion/core/src/physical_optimizer/convert_first_last.rs

Lines changed: 0 additions & 260 deletions
This file was deleted.

‎datafusion/core/src/physical_optimizer/mod.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,6 @@
2424
pub mod aggregate_statistics;
2525
pub mod coalesce_batches;
2626
pub mod combine_partial_final_agg;
27-
mod convert_first_last;
2827
pub mod enforce_distribution;
2928
pub mod enforce_sorting;
3029
pub mod join_selection;
@@ -37,6 +36,7 @@ pub mod pruning;
3736
pub mod replace_with_order_preserving_variants;
3837
mod sort_pushdown;
3938
pub mod topk_aggregation;
39+
mod update_aggr_exprs;
4040
mod utils;
4141

4242
#[cfg(test)]

‎datafusion/core/src/physical_optimizer/optimizer.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@
1919
2020
use std::sync::Arc;
2121

22-
use super::convert_first_last::OptimizeAggregateOrder;
2322
use super::projection_pushdown::ProjectionPushdown;
23+
use super::update_aggr_exprs::OptimizeAggregateOrder;
2424
use crate::config::ConfigOptions;
2525
use crate::physical_optimizer::aggregate_statistics::AggregateStatistics;
2626
use crate::physical_optimizer::coalesce_batches::CoalesceBatches;

0 commit comments

Comments
 (0)