diff --git a/text/0000-floating-point-additional-precision.md b/text/0000-floating-point-additional-precision.md new file mode 100644 index 00000000000..0c0141ab43d --- /dev/null +++ b/text/0000-floating-point-additional-precision.md @@ -0,0 +1,192 @@ +- Feature Name: `allow_extra_fp_precision` +- Start Date: 2019-04-08 +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +Update the Rust specification to allow floating-point operations to provide +*more* precision than specified, but not less precision; this allows many safe +optimizations. Specify robust mechanisms to disable this behavior. + +# Motivation +[motivation]: #motivation + +Some platforms provide instructions to run a series of floating-point +operations quickly, such as fused multiply-add instructions; using these +instructions can provide performance wins up to 2x or more. These instructions +may provide *more* precision than required by IEEE floating-point operations, +such as by doing multiple operations before rounding or losing precision. +Similarly, high-performance floating-point code could perform multiple +operations with higher-precision floating-point registers before converting +back to a lower-precision format. + +In general, providing more precision than required will only bring a +calculation closer to the mathematically precise answer, never further away. + +This RFC proposes allowing floating-point types to perform intermediate +calculations using more precision than the type itself, as long as they provide +*at least* as much precision as the IEEE 754 standard requires. + +See the [prior art section](#prior-art) for precedent in several other +languages and compilers. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +Floating-point operations in Rust have a guaranteed minimum accuracy, which +specifies how far the result may differ from an infinitely accurate, +mathematically exact answer. The implementation of Rust for any target platform +must provide at least that much accuracy. In some cases, Rust can perform +operations with higher accuracy than required, and doing so provides greater +performance (such as by removing intermediate rounding steps). + +A note for users of other languages: this is *not* the equivalent of the "fast +math" option provided by some compilers. Unlike such options, this behavior +will never make any floating-point operation *less* accurate, but it can make +floating-point operations *more* accurate, making the result closer to the +mathematically exact answer. + +Due to differences in hardware, in platform libm implementations, and various +other factors, Rust cannot fully guarantee identical results on all target +platforms. (Doing so on *all* platforms would incur a massive performance +loss.) However, with some additional care, applications desiring cross-platform +identical results can potentially achieve that on multiple target platforms. In +particular, applications prioritizing identical, portable results across two or +more target platforms can disable extra floating-point precision entirely. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +Currently, Rust's [specification for floating-point +types](https://doc.rust-lang.org/reference/types/numeric.html#floating-point-types) +states only that: +> The IEEE 754-2008 "binary32" and "binary64" floating-point types are f32 and f64, respectively. + +This RFC proposes updating that definition as follows: + +The `f32` and `f64` types represent the IEEE 754-2008 "binary32" and "binary64" +floating-point types. Operations on those types must provide at least as much +precision as the IEEE standard requires; such operations may provide *more* +precision than the standard requires, such as by doing a series of operations +with higher precision before storing a value of the desired precision. + +rustc should provide a codegen (`-C`) option to disable this behavior, such as +`-C extra-fp-precision=off`; compiling with this option will disable extra +precision in all crates compiled into an application. (Cargo should provide a +means of specifying this option.) Rust should also provide attributes to +disable this behavior from within code, such as `#[extra_fp_precision(off)]`; +this attribute will disable extra precision within the module or function it is +applied to. On platforms that do not currently implement disabling extra +precision, the codegen option and attribute should produce an error (not a +warning), to avoid surprises. + +In addition, because this change makes extra floating-point precision visible +on more platforms, the Rust release notes, documentation, and similar channels +should explicitly discuss the issue of extra floating-point precision and how +to disable it. Furthermore, this change should not become part of a stable Rust +release until at least eight stable releases *after* it first becomes +implemented in the nightly compiler. + +# Drawbacks +[drawbacks]: #drawbacks + +If Rust already provided bit-for-bit identical floating-point computations +across platforms, then this change could potentially allow floating-point +computations to differ (in the amount of additional accuracy beyond the +standards requirements) by platform, enabled target features (e.g. instruction +sets), or optimization level. + +However, standards-compliant implementations of operations on floating-point +values can and do *already* vary slightly by platform, sufficiently so to +produce different binary results; in particular, floating-point operations in +Rust can already produce more precise results depending on target platform, +optimization level, the target's libm library, and the version of the target +libm. As with that existing behavior, this proposal can never make results +*less* accurate, it can only make results *more* accurate. Nonetheless, this +change potentially introduces such variations on target platforms that did not +previously have them. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +For the attribute and codegen option, we could allow code to opt in via +attribute even if disabled via codegen, and then provide a `force-off` codegen +option to override that. This would have two serious downsides, however: it +would propagate the perception of extra floating-point precision as an unsafe +optimization that requires opting into, and it would make life more difficult +for people who wish to opt out of this behavior and attempt to achieve +identical results on multiple target platforms. This RFC recommends the simpler +approach of not providing an enablement option via attribute, such that the +codegen option always force-disables extra precision everywhere. + +We could provide an option to enable extra accuracy for the default +floating-point types, but disable it by default. This would leave the majority +of floating-point code unable to use these optimizations, however; defaults +matter, and the majority of code seems likely to use the defaults. In addition, +permitting extra floating-point precision by default would match the existing +behavior of Rust, and would allow the Rust compiler to assume that code +explicitly disabling extra precision has a specific requirement to do so and +depends on that behavior. Nonetheless, this alternative would still provide the +option to produce more optimized code, making it preferable over doing nothing. +This alternative would necessitate respecifying the codegen option and +attribute to support enabling it, as well as having a force-off codegen option +to override enablement via the attribute. + +We could provide a separate set of types and allow extra accuracy in their +operations; however, this would create API incompatibilities between +floating-point functions, and the longer, less-well-known types seem unlikely +to see widespread use. Furthermore, allowing or disallowing extra accuracy +seems more closely a property of the calculation than a property of the type. + +We could provide additional methods for floating-point operations that allow +passing additional flags, including floating-point contraction. The compiler +could then fuse and otherwise optimize such operations. However, this would +make optimized floating-point code *substantially* less ergonomic, due to the +inability to use operators. To enable operators, we could additionally +implement wrapper types, as above, with the same upsides and downsides. + +We could do nothing, and require code to use `a.mul_add(b, c)` for +optimization; however, this would not allow for similar future optimizations, +and would not allow code to easily enable this optimization without substantial +code changes. + +We could narrow the scope of optimization opportunities to *only* include +floating-point contraction but not any other precision-increasing operations. +See the [future possibilities](#future-possibilities) section for further +discussion on this point. + +# Prior art +[prior-art]: #prior-art + +This has precedent in several other languages and compilers: + +- [C11](http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf) allows + extra floating-point precision with the `STDC FP_CONTRACT` pragma enabled, + and the default state of that pragma is implementation-defined. GCC, ICC, + MSVC, and some other C compilers enable this behavior by default; Clang + disables it by default, though some downstream users of Clang re-enable it + system-wide. + +- [The C++ standard](http://eel.is/c++draft/expr.pre#6) states that "The + values of the floating operands and the results of floating + expressions may be represented in greater precision and range than + that required by the type; the types are not changed thereby." + +- The [Fortran standard](https://www.fortran.com/F77_std/rjcnf0001-sh-6.html#sh-6.6.4) + states that "the processor may evaluate any mathematically equivalent + expression", where "Two arithmetic expressions are mathematically + equivalent if, for all possible values of their primaries, their + mathematical values are equal. However, mathematically equivalent + arithmetic expressions may produce different computational results." + +# Future possibilities +[future-possibilities]: #future-possibilities + +The initial implementation of this RFC can simply enable floating-point +contraction within LLVM (and equivalent options in future codegen backends). +However, this RFC also allows other precision-increasing optimizations; in +particular, this RFC would allow the implementation of f32 or future f16 +formats using higher-precision registers, without having to apply rounding +after each operation.