Skip to content

Commit 8d787d9

Browse files
authored
Update safety disclaimer (#1837)
1 parent bd33489 commit 8d787d9

File tree

1 file changed

+22
-4
lines changed

1 file changed

+22
-4
lines changed

arrow/README.md

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -49,13 +49,31 @@ The arrow crate provides the following features which may be enabled:
4949

5050
## Safety
5151

52-
TLDR: You should avoid using the `alloc` and `buffer` and `bitmap` modules if at all possible. These modules contain `unsafe` code, are easy to misuse, and are not needed for most users.
52+
Arrow seeks to uphold the Rust Soundness Pledge as articulated eloquently [here](https://raphlinus.github.io/rust/2020/01/18/soundness-pledge.html). Specifically:
5353

54-
As with all open source code, you should carefully evaluate the suitability of `arrow` for your project, taking into consideration your needs and risk tolerance prior to doing so.
54+
> The intent of this crate is to be free of soundness bugs. The developers will do their best to avoid them, and welcome help in analyzing and fixing them
5555
56-
_Background_: There are various parts of the `arrow` crate which use `unsafe` and `transmute` code internally. We are actively working as a community to minimize undefined behavior and remove `unsafe` usage to align more with Rust's core principles of safety.
56+
Where soundness in turn is defined as:
5757

58-
As `arrow` exists today, it is fairly easy to misuse the code in modules named above, leading to undefined behavior.
58+
> Code is unable to trigger undefined behaviour using safe APIs
59+
60+
One way to ensure this would be to not use `unsafe`, however, as described in the opening chapter of the [Rustonomicon](https://doc.rust-lang.org/nomicon/meet-safe-and-unsafe.html) this is not a requirement, and flexibility in this regard is actually one of Rust's great strengths.
61+
62+
In particular there are a number of scenarios where `unsafe` is largely unavoidable:
63+
64+
* Invariants that cannot be statically verified by the compiler and unlock non-trivial performance wins, e.g. values in a StringArray are UTF-8, [TrustedLen](https://doc.rust-lang.org/std/iter/trait.TrustedLen.html) iterators, etc...
65+
* FFI
66+
* SIMD
67+
68+
Additionally, this crate exposes a number of `unsafe` APIs, allowing downstream crates to explicitly opt-out of potentially expensive invariant checking where appropriate.
69+
70+
We have a number of strategies to help reduce this risk:
71+
72+
* Provide strongly-typed `Array` and `ArrayBuilder` APIs to safely and efficiently interact with arrays
73+
* Extensive validation logic to safely construct `ArrayData` from untrusted sources
74+
* All commits are verified using [MIRI](https://github.com/rust-lang/miri) to detect undefined behaviour
75+
* We provide a `force_validate` feature that enables additional validation checks for use in test/debug builds
76+
* There is ongoing work to reduce and better document the use of unsafe, and we welcome contributions in this space
5977

6078
## Building for WASM
6179

0 commit comments

Comments
 (0)