Skip to content

Type coercion does not handle Float16 correctly #15815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
etseidl opened this issue Apr 22, 2025 · 1 comment · Fixed by #15816
Closed

Type coercion does not handle Float16 correctly #15815

etseidl opened this issue Apr 22, 2025 · 1 comment · Fixed by #15816
Assignees
@etseidl
Labels
bug Something isn't working

Comments

@etseidl
Copy link
Contributor

etseidl commented Apr 22, 2025

Describe the bug

Performing a query where a Float16 column is compared to an integer literal results in the Float16 being cast to Int64.

> explain format indent  select * from 'parquet-testing/data/float16_nonzeros_and_nans.parquet' where x > 2;
+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                                                                                                                                    |
+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan  | Filter: CAST(parquet-testing/data/float16_nonzeros_and_nans.parquet.x AS Int64) > Int64(2)                                                                                                              |
|               |   TableScan: parquet-testing/data/float16_nonzeros_and_nans.parquet projection=[x], partial_filters=[CAST(parquet-testing/data/float16_nonzeros_and_nans.parquet.x AS Int64) > Int64(2)]                |
| physical_plan | CoalesceBatchesExec: target_batch_size=8192                                                                                                                                                             |
|               |   FilterExec: CAST(x@0 AS Int64) > 2                                                                                                                                                                    |
|               |     RepartitionExec: partitioning=RoundRobinBatch(8), input_partitions=1                                                                                                                                |
|               |       DataSourceExec: file_groups={1 group: [[Users/seidl/src/datafusion/parquet-testing/data/float16_nonzeros_and_nans.parquet]]}, projection=[x], file_type=parquet, predicate=CAST(x@0 AS Int64) > 2 |
|               |                                                                                                                                                                                                         |
+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

To Reproduce

./target/debug/datafusion-cli -c "explain format indent  select * from 'parquet-testing/data/float16_nonzeros_and_nans.parquet' where x > 2;"

Expected behavior

This should instead cast 2 to Float16.

+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                                                                                                                                                                                                              |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan  | Filter: parquet-testing/data/float16_nonzeros_and_nans.parquet.x > Float16(2)                                                                                                                                                                                                     |
|               |   TableScan: parquet-testing/data/float16_nonzeros_and_nans.parquet projection=[x], partial_filters=[parquet-testing/data/float16_nonzeros_and_nans.parquet.x > Float16(2)]                                                                                                       |
| physical_plan | CoalesceBatchesExec: target_batch_size=8192                                                                                                                                                                                                                                       |
|               |   FilterExec: x@0 > 2                                                                                                                                                                                                                                                             |
|               |     RepartitionExec: partitioning=RoundRobinBatch(8), input_partitions=1                                                                                                                                                                                                          |
|               |       DataSourceExec: file_groups={1 group: [[Users/seidl/src/datafusion/parquet-testing/data/float16_nonzeros_and_nans.parquet]]}, projection=[x], file_type=parquet, predicate=x@0 > 2, pruning_predicate=x_null_count@1 != row_count@2 AND x_max@0 > 2, required_guarantees=[] |
|               |                                                                                                                                                                                                                                                                                   |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Additional context

No response
@etseidl etseidl added the bug Something isn't working label Apr 22, 2025
@etseidl
Copy link
Contributor Author

etseidl commented Apr 22, 2025

take

@etseidl etseidl mentioned this issue Apr 22, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@etseidl etseidl

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Add coercion rules for Float16 types etseidl/datafusion
1 participant
@etseidl