You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: arrow/README.md
+22-4Lines changed: 22 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -49,13 +49,31 @@ The arrow crate provides the following features which may be enabled:
49
49
50
50
## Safety
51
51
52
-
TLDR: You should avoid using the `alloc` and `buffer` and `bitmap` modules if at all possible. These modules contain `unsafe` code, are easy to misuse, and are not needed for most users.
52
+
Arrow seeks to uphold the Rust Soundness Pledge as articulated eloquently [here](https://raphlinus.github.io/rust/2020/01/18/soundness-pledge.html). Specifically:
53
53
54
-
As with all open source code, you should carefully evaluate the suitability of `arrow` for your project, taking into consideration your needs and risk tolerance prior to doing so.
54
+
> The intent of this crate is to be free of soundness bugs. The developers will do their best to avoid them, and welcome help in analyzing and fixing them
55
55
56
-
_Background_: There are various parts of the `arrow` crate which use `unsafe` and `transmute` code internally. We are actively working as a community to minimize undefined behavior and remove `unsafe` usage to align more with Rust's core principles of safety.
56
+
Where soundness in turn is defined as:
57
57
58
-
As `arrow` exists today, it is fairly easy to misuse the code in modules named above, leading to undefined behavior.
58
+
> Code is unable to trigger undefined behaviour using safe APIs
59
+
60
+
One way to ensure this would be to not use `unsafe`, however, as described in the opening chapter of the [Rustonomicon](https://doc.rust-lang.org/nomicon/meet-safe-and-unsafe.html) this is not a requirement, and flexibility in this regard is actually one of Rust's great strengths.
61
+
62
+
In particular there are a number of scenarios where `unsafe` is largely unavoidable:
63
+
64
+
* Invariants that cannot be statically verified by the compiler and unlock non-trivial performance wins, e.g. values in a StringArray are UTF-8, [TrustedLen](https://doc.rust-lang.org/std/iter/trait.TrustedLen.html) iterators, etc...
65
+
* FFI
66
+
* SIMD
67
+
68
+
Additionally, this crate exposes a number of `unsafe` APIs, allowing downstream crates to explicitly opt-out of potentially expensive invariant checking where appropriate.
69
+
70
+
We have a number of strategies to help reduce this risk:
71
+
72
+
* Provide strongly-typed `Array` and `ArrayBuilder` APIs to safely and efficiently interact with arrays
73
+
* Extensive validation logic to safely construct `ArrayData` from untrusted sources
74
+
* All commits are verified using [MIRI](https://github.com/rust-lang/miri) to detect undefined behaviour
75
+
* We provide a `force_validate` feature that enables additional validation checks for use in test/debug builds
76
+
* There is ongoing work to reduce and better document the use of unsafe, and we welcome contributions in this space
0 commit comments