Skip to content

Planning error for compound expressions involving window functions #14910

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2010YOUY01 opened this issue Feb 27, 2025 · 5 comments · Fixed by #15033
Closed

Planning error for compound expressions involving window functions #14910

2010YOUY01 opened this issue Feb 27, 2025 · 5 comments · Fixed by #15033
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@2010YOUY01
Copy link
Contributor

Describe the bug

The following query throws planning error for DataFusion, but it runs in DuckDB and PostgreSQL

Run in datafusion-cli (commit hash a28f283)

DataFusion CLI v45.0.0
> SELECT
  t1.v1,
  SUM(t1.v1) OVER w + 1
FROM
  generate_series(1, 10000) AS t1(v1)
WINDOW
  w AS (ORDER BY t1.v1 ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW);

Error during planning: Projection references non-aggregate values: Expression t1.v1 could not be resolved from available columns: sum(t1.v1)

To Reproduce

No response

Expected behavior

No response

Additional context

No response

@2010YOUY01 2010YOUY01 added the bug Something isn't working label Feb 27, 2025
@alamb alamb added the help wanted Extra attention is needed label Feb 27, 2025
@qazxcdswe123
Copy link
Contributor

qazxcdswe123 commented Feb 28, 2025

After some digging I found that this only occur when using WINDOW clause.

So this one works

Failed with: Error during planning: Projection references non-aggregate values: Expression t1.v1 could not be resolved from available columns: sum(t1.v1)
SELECT
  t1.v1,
  SUM(t1.v1) OVER w + 1
FROM
  generate_series(1, 10) AS t1(v1)
WINDOW
  w AS (ORDER BY t1.v1);

Works:
SELECT
  t1.v1,
  SUM(t1.v1) OVER w
FROM
  generate_series(1, 10) AS t1(v1)
WINDOW
  w AS (ORDER BY t1.v1);
  
SELECT
  t1.v1,
  SUM(t1.v1) OVER (ORDER BY t1.v1) + 1
FROM
  generate_series(1, 10) AS t1(v1);

SELECT
  -- t1.v1,
  SUM(t1.v1) OVER w + 1
FROM
  generate_series(1, 10) AS t1(v1)
WINDOW
  w AS (ORDER BY t1.v1);

I think I can give it a try

@qazxcdswe123
Copy link
Contributor

take

@qazxcdswe123
Copy link
Contributor

qazxcdswe123 commented Mar 1, 2025

I dont think I can fix this lol, I tried tweaking rebase_expr and check_columns_satisfy_exprs in datafusion/sql/src/utils.rs but all failed. The planner is little too complicated for me :D

untake

How can I untake this btw

@2010YOUY01
Copy link
Contributor Author

I dont think I can fix this lol, I tried tweaking rebase_expr and check_columns_satisfy_exprs in datafusion/sql/src/utils.rs but all failed. The planner is little too complicated for me :D

untake

How can I untake this btw

Thanks for the attempt. The planning code is indeed quite complex.

@alamb
Copy link
Contributor

alamb commented Mar 5, 2025

See @chenkovsky's PR here:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants