Skip to content

fix divide by zero not throwing proper error for decimal #3517

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 17, 2022

Conversation

kmitchener
Copy link
Contributor

Which issue does this PR close?

Closes #3498 .

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Proper error

@github-actions github-actions bot added the physical-expr Changes to the physical-expr crates label Sep 16, 2022
@kmitchener
Copy link
Contributor Author

f64 also doesn't have the precision to do Decimal128 math. I'll add another issue around fixing the math for Decimal128 so it uses i128 instead of floats. I first attempted to fix that here but I think fixing that might be more complicated than it looks. I think we can handle that separately and just focus this fix on the divide by zero error. Note also that the full precision of decimal literals is not available from SQL because the SQL planner converts all literals to floats. (not a problem in this case obviously, but would be for literals using the full 38 digits.)

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kmitchener -- the code looks good -- I think the test needs a tweak but otherwise this is looking quite good 👍

let _result = modulus_decimal(&left_decimal_array, &right_decimal_array)
.expect_err("expecting DivideByZero error");
let _result = modulus_decimal_scalar(&left_decimal_array, 0)
.expect_err("expecting DivideByZero error");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these tests would pass even on master without your code change

Perhaps we could add a check for the actual error ? With code similar to https://github.com/apache/arrow-datafusion/blob/331922036a23265084ab091fa2ab5c57da42c841/datafusion/core/tests/sql/information_schema.rs#L35-L42 ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hah, excellent! Thanks for the review, as always. Have updated with these changes.

@codecov-commenter
Copy link

codecov-commenter commented Sep 16, 2022

Codecov Report

Merging #3517 (3ddffe0) into master (25c029c) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #3517   +/-   ##
=======================================
  Coverage   85.75%   85.76%           
=======================================
  Files         299      299           
  Lines       55311    55334   +23     
=======================================
+ Hits        47432    47455   +23     
  Misses       7879     7879           
Impacted Files Coverage Δ
...sical-expr/src/expressions/binary/kernels_arrow.rs 95.03% <100.00%> (+1.00%) ⬆️
datafusion/expr/src/logical_plan/plan.rs 77.35% <0.00%> (-0.50%) ⬇️
...tafusion/core/src/physical_plan/file_format/mod.rs 96.95% <0.00%> (ø)
.../core/src/physical_plan/file_format/file_stream.rs 88.88% <0.00%> (ø)
datafusion/core/tests/sql/expr.rs 99.87% <0.00%> (+<0.01%) ⬆️
...tafusion/physical-expr/src/datetime_expressions.rs 83.70% <0.00%> (+0.18%) ⬆️
datafusion/expr/src/expr_schema.rs 63.47% <0.00%> (+0.59%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it looks great. Thanks @kmitchener

@alamb alamb merged commit b106b02 into apache:master Sep 17, 2022
@ursabot
Copy link

ursabot commented Sep 17, 2022

Benchmark runs are scheduled for baseline = c6ebf7b and contender = b106b02. b106b02 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
physical-expr Changes to the physical-expr crates
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dividing decimal type gives wrong error: "170141183460469231731687303715884105727 is too large to store in a Decimal128
4 participants