diff --git a/text/0000-target-extension.md b/text/0000-target-extension.md new file mode 100644 index 00000000000..36c74e61b15 --- /dev/null +++ b/text/0000-target-extension.md @@ -0,0 +1,513 @@ +- Feature Name: `target extension` +- Start Date: 2017-06-27 +- RFC PR: (leave this empty) +- Rust Issue: (leave this empty) + +# Summary +[summary]: #summary + +Extend Rust target specification to follow more closely LLVM triple specification. + +The underlined purpose is to allow Rust compiler to target more easily BSD OS version. + +# Motivation +[motivation]: #motivation + +[LLVM triple](http://llvm.org/docs/LangRef.html#target-triple) specification is +more precise than the current [Rust +target](https://github.com/rust-lang/rust/blob/7ac844ffb850a73b98cd47cbdec909d1f03c7987/src/librustc_back/target/mod.rs#L228) +specification we have. + +In particular, the following elements are missing from the Rust target +definition: +- optional [OS version](https://github.com/llvm-mirror/llvm/blob/343e535d9c38cf57173ace6597380752a18a6a67/include/llvm/ADT/Triple.h#L315) +- optional [environment version](https://github.com/llvm-mirror/llvm/blob/343e535d9c38cf57173ace6597380752a18a6a67/include/llvm/ADT/Triple.h#L303) + + +Rust language is aimed to be used on different operating systems following themselves +their own rules. In particular, each operating systems have proper way to +deal with breaking changes: if Linux tends to forbid breaking changes by +policy, all others systems doesn't have such rule. As Rust language tends to be +a stable language, having a stable way to describe breaking changes on the OS +would be very valuable and could become necessary as time passes. + +LLVM deals with such changes on OS by having a different triple per OS version, +like for the following triples: + +- `x86_64-apple-darwin16.0.0` +- `x86_64-unknown-freebsd12.0` +- `x86_64-unknown-freebsd11.0` +- `i386-unknown-openbsd5.8` +- `x86_64-unknown-netbsd7.99` + + +As examples, consider the following changes in several operating systems (some +are ABI changes, others API changes) and how a crate like `libc` would have to +deal with them. Please note that some are quite old but could be considered as +representative of something that already occurred in the past. + +- **OpenBSD 5.5** does a big breaking changes in order to be compatible with + [year 2038](https://www.openbsd.org/faq/upgrade55.html#time_t): it switches + from a signed 32 bit counter to a signed 64 bit time type. + See [commit message](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/sys/_types.h?rev=1.6&content-type=text/x-cvsweb-markup) + and [diff on types](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/sys/_types.h.diff?r1=1.5&r2=1.6). + +- **OpenBSD 6.2** changes `si_addr` type (`char *` to `void *`) in + `siginfo_t` structure. + See [commit message](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/sys/siginfo.h?rev=1.12&content-type=text/x-cvsweb-markup) + and [diff on sys/siginfo.h](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/sys/siginfo.h.diff?r1=1.11&r2=1.12). + +- **FreeBSD 10** changes the `cap_rights_t` type from `uint64_t` to a structure + that they can extend in the future in a backward compatible way. + See [commit R255129](https://svnweb.freebsd.org/base?view=revision&revision=255219). + +- **FreeBSD 11** changes signature of `psignal()` to align to POSIX 2008 + (`unsigned int` to `int` argument). + See [commit R300997](https://svnweb.freebsd.org/base?view=revision&revision=300997) + and [diff on signal.h](https://svnweb.freebsd.org/base/head/include/signal.h?r1=300997&r2=300996&pathrev=300997). + +- **FreeBSD 12** removes `setkey()`, `encrypt()`, `des_setkey()` and + `des_cipher()` functions. + See [commit R306651](https://svnweb.freebsd.org/base?view=revision&revision=306651) + and [diff of unistd.h](https://svnweb.freebsd.org/base/head/include/unistd.h?r1=306651&r2=306650&pathrev=306651). + +- **FreeBSD 12** adds a new member `fb_memattr` in the middle of the + structure `fb_info` (public under `sys/fbio.h`). + See [commit R306555](https://svnweb.freebsd.org/base?view=revision&revision=306555) + and [diff of sys/fbio.h](https://svnweb.freebsd.org/base/head/sys/sys/fbio.h?r1=306555&r2=306554&pathrev=306555). + +- **FreeBSD 12** switchs `ino_t` from 32 bits to 64 bits. + See [commit R318736](https://svnweb.freebsd.org/base?view=revision&revision=318736), + [diff on types](https://svnweb.freebsd.org/base/head/sys/sys/_types.h?r1=307756&r2=318736), + the [Status Update and Call for Testing](https://lists.freebsd.org/pipermail/freebsd-fs/2017-April/024684.html), + and [diff on lang/rust (ports tree)](https://github.com/FreeBSDFoundation/freebsd/blob/bc50a841851470d98cf1c219b261133536aa7ee8/ports.patch#L402). + +- **NetBSD 7.99** (upcoming 8) adds a new member `mnt_lower` in the middle of + the structure `mount` (public under `sys/mount.h`). + See [commit message](http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/sys/mount.h?rev=1.221&content-type=text/x-cvsweb-markup) + and [diff of sys/mount.h](http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/sys/mount.h.diff?r1=1.220&r2=1.221). + +- **NetBSD 7.99** (upcoming 8) changes signature of `scandir()` function to conform to `IEEE + Std 1003.1-2008` (`const void *` to `const struct dirent **`). + See [commit message](http://cvsweb.netbsd.org/bsdweb.cgi/src/include/dirent.h?rev=1.36&content-type=text/x-cvsweb-markup&sortby=date) + and [diff to dirent.h](http://cvsweb.netbsd.org/bsdweb.cgi/src/include/dirent.h.diff?r1=1.35&r2=1.36&sortby=date). + +- **DragonFly 1.4** switches `ino_t` from 32 bits to 64 bits. + See [commit message](http://gitweb.dragonflybsd.org/dragonfly.git/commit/f91a71dd15504ebdb04387d0822771ef145b25f9?f=sys/sys/types.h) + and [diff to sys/types.h](http://gitweb.dragonflybsd.org/dragonfly.git/blobdiff/6f1e2b382f6c2ba9b43a1fc106ba998b45499eea..f91a71dd15504ebdb04387d0822771ef145b25f9:/sys/sys/types.h) + +- **MSVC 2015** has several breaking changes. + See [Visual C++ change history 2003 - 2015](https://msdn.microsoft.com/en-us/library/bb531344.aspx), + in particular section about *C Runtime* and *STL* breaking changes. + In particular: some functions like [gets](https://msdn.microsoft.com/en-us/library/2029ea5f.aspx) + or [`_cgets`](https://msdn.microsoft.com/en-us/library/3197776x.aspx) have been + removed. In order to conform to C++11, old names for type traits from an earlier + version of the C++ draft standard have been remamed. + This one is an example of environment version. + + +In the current situation, `libc` crate has no way to deal in a stable way with +these changes. It could only support two incompatible OS version together by +only defining the common subset. Depending on the breaking part, it could result +in removed feature in rustc (removing `si_addr` for OpenBSD would break stack +overflow detection), or even breaking rustc itself (removing `ino_t` for +FreeBSD). + +Additionally, in order to switch `libc` from one OS version to another, it +would be required to do a breaking change at `libc` level (incrementing major +version of `libc` itself) which is undesirable for this purpose. + + +The purpose of extending Rust `Target` type to follow LLVM Triple definition is +to be able to deal with such changes at Rust language level. As the target will +be able to distinguish between particular OS or environment versions, it +would be possible to export the information in the same way we export +`target_os`, `target_arch`, `target_endian` or `target_pointer_width`. + +This way, a crate like `libc` could export raw bindings of platform +specifically for the targeted version. + + +It should be noted that if this motivation section presents globally the +underlined problem of breaking changes in OS, which could be present in a large +broad of OS, the RFC will focus on a fewer set of OS. The RFC will target only +BSD systems where breaking changes are part of the process OS developpment. + + +# Detailed design +[design]: #detailed-design + + +## Language level: what the user will see ? + +At language level, new attributes for conditional compilation would be added: + +- `target_os_version` +- `target_env_version` + +There could be empty (""). + +```rust +extern { + // encrypt() function doesn't exist in freebsd12 + + #[cfg(all(target_os="freebsd", not(target_os_version="12")))] + pub fn encrypt(block *mut ::c_char, flag ::c_int) -> ::c_int; +} +``` + + +Another complete (and simple) example: in OpenBSD 6.2, the structure +`siginfo_t` changed: + +```rust +pub struct siginfo_t { + pub si_signo: ::c_int, + pub si_code: ::c_int, + pub si_errno: ::c_int, + + // A type correction occured in 6.2. + // Before it was a `char *` and now it is a `void *`. + #[cfg(not(any(target_os_version = "6.2", target_os_version = "6.3"))] + pub si_addr: *mut ::c_char, + #[cfg(any(target_os_version = "6.2", target_os_version = "6.3"))] + pub si_addr: *mut ::c_void, + + #[cfg(target_pointer_width = "32")] + __pad: [u8; 112], + #[cfg(target_pointer_width = "64")] + __pad: [u8; 108], +} +``` + +It would be possible to target `x86_amd64-unknown-openbsd6.1` **and** +`x86_amd64-unknown-openbsd6.2` whereas with [current libc +code](https://github.com/rust-lang/libc/blob/6ddc76a27e0678c04ec7337591f8a0e36c065664/src/unix/bsd/netbsdlike/openbsdlike/mod.rs#L106) +only one version is possible, and switching from one to the other version would +be a breaking change in `libc` (and we would lose OpenBSD 6.1 supported +version). + + +## Backend level + +### Target structure + +At the backend level, the `Target` structure gains two new members: + +- `target_os_version: String` +- `target_env_version: String` + +to represent the (possibly empty) versions of the OS and environment. + +See `librustc_back/target/`. + + +### Specifics compilations options + +It could be noted that some platforms could require additionnal compilation +options, like macOS and `-mmacosx-version-min` for specifying the minimal +version (it affects e.g. dynamic library loader). + +No additional changes is required for this support: `TargetOptions` structure +already contains array for such options: `pre_link_args`, `late_link_args` and +`post_link_args`. + +Having a per version target permits to have different options per target. + + +### Implication on targets number + +It should be noted it will implied a new target per OS version (for each +architecture), as soon as a breaking change occurs (new target required), or on +each major release (as it could be more simple for the end user to know which +target to use). + +As example, FreeBSD has currently 3 targets (one per supported architecture: +`x86_64`, `i686` and `aarch64`). If we want to be able to express targets for 3 +releases (two currently supported and one upcoming), the number of targets will +grow to 9 targets. + + +### Version tracking per OS + +The exact way to tracking the OS version (creating a new target) should be done +per OS, because OS has different expectations regarding when a breaking change +could occur accross versions. + +As example, FreeBSD keep ABI/API accross minor versions, and a breaking change +should only occur at major version (but not necessary). + +So, the targets should be (for `x86_64` architecture): +- `x86_64-unknown-freebsd10` (currently supported) +- `x86_64-unknown-freebsd11` (currently supported) +- `x86_64-unknown-freebsd12` (in development) + +At the opposite, OpenBSD only release major versions (even if expressed with +two digits version), and a breaking change could occur at each version: +- `x86_64-unknown-openbsd6.0` (currently supported) +- `x86_64-unknown-openbsd6.1` (currently supported) +- `x86_64-unknown-openbsd6.2` (in development) + + +### Default OS version for a target + +It could be noted that the current unversioned target (like +`x86_64-unknown-openbsd`) could be still used as an alias of some versioned +target. + +If so, the semantic have to be defined (tracking the oldest or most recent +supported version). + +It could be convenient thing for compiler users, but any serious work should +rely on versioned OS target (as compiling for one target version could mean +unusable binary on other OS version). + +Keeping the unversioned target would avoid a breaking change in command-line. +But the change could be useful too as it permits to downstream to be aware that +targeting particular OS doesn't mean the binary will work on other version. + + + +## Session level + +At the session level, rustc should populate and export the new attributes +(values taken from targeted backend) in the default build configuration. + + +See `librustc/session/config.rs`. + + + +## Migration path + +FreeBSD will be taken as example for the migration path. But it would apply for +others BSD OS. + +Currently, `rustc` has 3 known targets regarding FreeBSD: + +- `aarch64-unknown-freebsd` +- `i686-unknown-freebsd` +- `x86_64-unknown-freebsd` + +Assuming we want to support FreeBSD 10 and FreeBSD 11 (current supported +production releases), and FreeBSD 12 (upcoming release), the following steps +will need consideration. + + +### Extending Target definition + +- add `target_os_version` and `target_env_version` in `Target` definition. The + default value for all existing targets (including FreeBSD) will be `""` + (empty string). + +- make `rustc` to export new `target_os_version` and `target_env_version` attributes. + +- make `cargo` to export new `TARGET_OS_VERSION` and `TARGET_ENV_VERSION` + environment variables. + +At this point: + +- it should be no impact. the underline mecanism is implemented but nothing use + it. + +### Add new targets specifically for FreeBSD 10, 11 and 12 + +- add new targets, by duplicating the target code (using a function): + - FreeBSD 10 + - `aarch64-unknown-freebsd10` + - `i686-unknown-freebsd10` + - `x86_64-unknown-freebsd10` + + - FreeBSD 11 + - `aarch64-unknown-freebsd11` + - `i686-unknown-freebsd11` + - `x86_64-unknown-freebsd11` + + - FreeBSD 12 + - `aarch64-unknown-freebsd12` + - `i686-unknown-freebsd12` + - `x86_64-unknown-freebsd12` + +- in these new targets, only `llvm_target` and `target_os_version` will be changed + +At this point: + +- the `freebsd` target is still built and distributed. `freebsd10`, `freebsd11` + and `freebsd12` are available for use using usual `--target` argument of + `rustc`. there is still no changes in the Rust ecosystem. +- the number of targets will grow: for FreeBSD: from 3 to 12 targets. +- no changes in `libc`. the four targets are as functional as before and all + uses the same libc code (at this point, there is no specific code using + `target_os_version`). +- the generated code for versioned `freebsdXX` could be different as the LLVM + backend is now aware of the targeted version. +- downstream projects could start using versioned targets (the code is still + the same), but it would not be recommanded for production, but only for + testing. +- tests should be done to ensure the compatibility of the ecosystem with + versioned targets as it could break some assumption: for example, library + path will be `i686-unknown-freebsd11` instead of `i686-unknown-freebsd` in + `freebsd11` target. + +### Specific changes for FreeBSD 12 could start to occurs + +- changes libc to produce different code if `target_os_version="12"`, + specifically regarding struct changes that occured in FreeBSD 12. + +At this point: + +- when explicitly targeting `freebsd12`, the produced code will have different + structure and functions. The produced binary targets FreeBSD 12, and will not + work on FreeBSD 11 (due to the use of breaking changes that occured in FreeBSD + 12). +- the rest of the Rust ecosystem (still using `freebsd` as default build + target) is unaffected by these changes. + +### Smoothly remove unversioned Target for FreeBSD + +- replace the Target `freebsd` (unversioned) by an alias on `freebsd11`. It + ensures that all FreeBSD targets will have `target_os_version` sets to a + specific version. +- alternatively, more complex code could be used for the alias: + - on a FreeBSD host: `freebsd` will point to the host version. + - on any other host: `freebsd` will point to predefined default version. + +At this point: + +- *some code could break*. it will only concern untested code from the previous + steps. +- it is a transition period: downstream projects should start using versioned + Target for production code. For example, Rust infrastructure should stop + building `freebsd` in favor of `freebsd11`, and rustup should distribute this + specific version too (`freebsd11` has been taken only as example). +- as transition period, it should be long enough. + +### Completely remove unversioned Target for FreeBSD + +- remove the alias `freebsd` +- only keeping `freebsd10`, `freebsd11` and `freebsd12` targets + +At this point: + +- *some code could break*. Any downstream projects still using unversioned + target will break (as the target has been removed). +- this step isn't strictly necessary. an unversioned target `freebsd` could + have sens too. But care will be need when the alias will move (from + `freebsd11` to `freebsd12` for example). Removing it let downstream projects to + deal themself with such transition. + + +# How We Teach This +[how-we-teach-this]: #how-we-teach-this + +If modifying the `Target` struct is a low-level change by itself, the current +RFC proposes it in order to change an implicit paradigm in targets (the +targeting OS will be stable accross version, which is false). + +With the RFC, Rust become able to express this lack of stability on the OS, in +a stable way. In a sens, it extents the ability of Rust to targets several OS +by refining to targets several OS version. + +From downstream perspective, it permits to use Rust to target several OS +versions whereas the versions are incompatibles. + +From Rust developer perspective, it adds a new attributes for conditional +compilation. + +Regarding documentation, additions have to be done on _Rust Reference_ in order +to mention new attributes in conditional compilation attribute section. + +Visible changes should also occur on main rust-lang.org site on pages about +rustc distribution: rustup will be able to distribute binaries for more +platforms (per OS version), or about platform support. If only one version is +distributed (what it is expected), the binary will be runnable only on one +particular OS version (but will be able to produce binary for another version). + + +# Drawbacks +[drawbacks]: #drawbacks + +Do not providing a simple way to target a range of versions (for example, "from +version 12 to current version") could imply a long enumeration of version, that +could only grow with time. But two things has to be noted: first it could help +to limit the number of supported and tested versions ; secondly, the fact this +particular RFC doesn't require such construct do not forbid to propose a new +RFC later, once the necessity of such mecanism would be more evident (or not). + +At backend level, the number of targets will grow a lot. It means that not all +targets will be testable (too much required ressources and it would require a +particular OS version for testing too). + +It will require to regulary deprecate old targets (for unsupported OS version) +in order to not keep too much old stuff. The end-user has still the possibility +to use flexible target using external JSON file for these targets, if the +corresponding code for this particular version is still in `libc` crate. + +Projects using Rust with binary distribution will have to update in order to +cover a more important number of platforms. In particular rustc itself (with +and without rustup). It will mean more ressources to build more targets. But +the resulting binaries will work on all targets. + + +# Alternatives +[alternatives]: #alternatives + +## Simply appending the version in the target name + +The more simple approch is to use `target_os` with the OS version inside +(`freebsd12`). But it would require to duplicate all `libc` code (for only +small differences) at each release. Having a separated attribute is more simple. + +But without some way to express breaking changes existence at OS level, Rust is +unable to targeting simultaneous several OS version. Regarding +[issue #42681](https://github.com/rust-lang/rust/issues/42681) for FreeBSD 12, +it means Rust should either deprecating older FreeBSD versions support (whereas +FreeBSD itself still support them) or only partially supporting FreeBSD 12. + +## Runtime detection + +Runtime detection is already in use for [Android](https://github.com/rust-lang/rust/blob/13157c4ebcca735a0842bd03c3dad1de7c429f9f/src/libstd/sys/unix/android.rs#L70-L93). +It has the advantage to permit rustc to target *several* OS with the same binary. + +It is based on OS version detection (with symbol existence for example) and on +providing fallback or alternative for function calls. + +But it couldn't cover all aspects of ABI breaking, specially changes in +structures (member size or offset change). It would require to replace +structure's member access by function calls doing runtime detection. The +possible overhead could be removed by using lazy detection and caching. + +## Dynamic bindings generation + +A possible alternative is to replace `libc` with FFI bindings generation at +compile-time (using [rust-bindgen](https://github.com/servo/rust-bindgen) for +example). But it isn't suitable for cross-building. + +## Adding cfg attribute using build.rs + +It is possible to do some compile time detection (or using cargo feature) to +select a particular OS version in `libc`. It has been [proposed for resolving +the FreeBSD 12 ABI issue](https://github.com/rust-lang/libc/pull/721). + +With such code, the `libc` code is right regarding the selected version. + +The drawback is such detection is fragile, and crosscompilation more complex +(it requires cargo feature usage). + +Additionally, a larger problem is mixing code with different OS version would +be possible (no error at compile time): for example using libstd from rustup +targeting one version, and using with crate locally compiled for another +version. It would produce bad code and crash could occurs at runtime. + +# Unresolved questions +[unresolved]: #unresolved-questions + +As unresolved-question, the question about the unversioned target on +command-line is open. Does it makes sens to have it or not ? + + +# Related previous discussions +[discussions]: #previous-discussions + +- [libc#7570: How to deal with breaking changes on platform ?](https://github.com/rust-lang/libc/issues/570) +- [Rust Internals: Pre-RFC: target extension (dealing with breaking changes at OS level)](https://internals.rust-lang.org/t/pre-rfc-target-extension-dealing-with-breaking-changes-at-os-level/5289)