Tracking issue for cross-compiling doc tests (doctest-xcompile) #7040
Labels
A-doctests
Area: rustdoc --test
C-tracking-issue
Category: A tracking issue for something unstable.
Command-test
S-waiting-on-feedback
Status: An implemented feature is waiting on community feedback for bugs or design concerns.
Z-doctest-xcompile
Nightly: doctest-xcompile
Comments
Goirad
added
the
C-feature-request
ehuss
added
C-tracking-issue
bors
added a commit
that referenced
this issue
Sep 18, 2019
Added ability to crosscompile doctests This commit adds the ability to cross-compile and run doctests. Like before cargo checks if target == host, the difference is that if there is a runtool defined in config.toml, it passes the information forward to rustdoc so that it can run the doctests with that tool. If no tool is defined and the target != host, cargo instead displays a message that doctests will not be compiled because of the missing runtool. See [here](rust-lang/rust#60387) for the companion PR in the rust project that modifies rustdoc to accept the relevant options as well as allow ignoring doctests on a per target level. Partially resolves [#6460](#6460) See [here](#7040) for the tracking issue.
ehuss
changed the title
This was referenced Jun 14, 2020
Alexhuszagh
added a commit
to Alexhuszagh/cross
that referenced
this issue
May 26, 2022
This partially addresses cross-rs#225. This only works on nightly, due to the use of the unstable feature [doctest-xcompile](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#doctest-xcompile) The relevant tracking issues are: - rust-lang/cargo#7040 - rust-lang/rust#64245 If the subcommand is `test` and the compiler is nightly, we provide the `-Zdoctest-xcompile` flag if `CROSS_UNSTABLE_ENABLE_DOCTESTS=true`.
Alexhuszagh
added a commit
to Alexhuszagh/cross
that referenced
this issue
May 26, 2022
This partially addresses cross-rs#225. This only works on nightly, due to the use of the unstable feature [doctest-xcompile](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#doctest-xcompile) The relevant tracking issues are: - rust-lang/cargo#7040 - rust-lang/rust#64245 If the subcommand is `test` and the compiler is nightly, we provide the `-Zdoctest-xcompile` flag if `CROSS_UNSTABLE_ENABLE_DOCTESTS=true`.
Alexhuszagh
added a commit
to Alexhuszagh/cross
that referenced
this issue
May 26, 2022
This partially addresses cross-rs#225. This only works on nightly, due to the use of the unstable feature [doctest-xcompile](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#doctest-xcompile) The relevant tracking issues are: - rust-lang/cargo#7040 - rust-lang/rust#64245 If the subcommand is `test` and the compiler is nightly, we provide the `-Zdoctest-xcompile` flag if `CROSS_UNSTABLE_ENABLE_DOCTESTS=true`.
Alexhuszagh
added a commit
to Alexhuszagh/cross
that referenced
this issue
May 26, 2022
This partially addresses cross-rs#225. This only works on nightly, due to the use of the unstable feature [doctest-xcompile](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#doctest-xcompile) The relevant tracking issues are: - rust-lang/cargo#7040 - rust-lang/rust#64245 If the subcommand is `test` and the compiler is nightly, we provide the `-Zdoctest-xcompile` flag if `CROSS_UNSTABLE_ENABLE_DOCTESTS=true`.
bors bot
added a commit
to cross-rs/cross
that referenced
this issue
May 26, 2022
721: Add preliminary support for running doctests. r=Emilgardis a=Alexhuszagh This partially addresses #225. This only works on nightly, due to the use of the unstable feature [doctest-xcompile](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#doctest-xcompile) The relevant tracking issues are: - rust-lang/cargo#7040 - rust-lang/rust#64245 If the subcommand is `test` and the compiler is nightly, we provide the `-Zdoctest-xcompile` flag if `CROSS_UNSTABLE_ENABLE_DOCTESTS=true`. Co-authored-by: Alex Huszagh <[email protected]>
This was referenced Mar 27, 2023
ehuss
added
the
S-waiting-on-feedback
|
I have opened #15462 to propose to stabilize this. |
github-merge-queue bot
pushed a commit
that referenced
this issue
May 14, 2025
This stabilizes the doctest-xcompile feature by unconditionally enabling it. Closes #7040 Closes #12118 ## What is being stabilized? This changes it so that cargo will run doctests when using the `--target` flag for a target that is not the host. Previously, cargo would ignore doctests (and show a note if passing `--verbose`). A wrapper for running the doctest can be specified with the [`target.*.runner`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplerunner) configuration option (which is powered by the `--test-runtool` rustdoc flag). This would typically be something like qemu to run under emulation. It is my understanding that this should work just like running other kinds of tests. Additionally, the [`target.*.linker`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplelinker) config option is honored for using a custom linker. Already stabilized in rustdoc is the ability to [ignore tests per-target](https://doc.rust-lang.org/nightly/rustdoc/write-documentation/documentation-tests.html#ignoring-targets). ## Motivation The lack of doctest cross-compile support has always been simply due to the lack of functionality in rustdoc to support this. Rustdoc gained the ability to cross-compile doctests some time ago, but there were additional flags like the test runner that were not stabilized until just recently. This is intended to ensure that projects have full test coverage even when doing cross-compilation. It can be [surprising](#12118) to some that this was not happening, particularly since cargo is silent about it. ## Risks The cargo team had several conversations about how to roll out this feature. Ultimately we decided to enable it unconditionally with the understanding that most projects will probably want to have their doctests covered, and that any breakage will be a local concern that can be resolved by either fixing the test or ignoring the target. Tests in rust-lang/rust run into this issue, [particularly on android](rust-lang/rust#119147 (comment)), and those will need to be fixed before this reaches beta. This is something I am looking into. Some cross-compiling scenarios may need codegen flags that are not supported. It's not clear how common this will be, or if ignoring will be a solution, or how difficult it would be to update rustdoc and cargo to support these. Additionally, the split between RUSTFLAGS and RUSTDOCFLAGS can be cumbersome. ## Implementation history - rust-lang/rust#60387 -- Support added to rustdoc to support the `--target` flag and runtool and per-target-ignores. - #6892 -- Initial support in cargo. - #7391 -- Added unstable documentation. - #8094 -- Fix target for doc test cross compilation - #8358 -- Fixed regression with `--target=HOST` not working on stable. - #10132 -- Added note about doctests not running (under `--verbose`). - rust-lang/rust#112751 -- Fixed `--test-run-directory` interaction with `--test-runtool`. - rust-lang/rust#137096 -- Stabilization (and rename) of the rustdoc `--test-runtool` and `--test-runtool-arg` CLI args, and drops `--enable-per-target-ignores` unconditionally enabling it. ## Test coverage Cargo tests: - [artifact_dep::cross_doctests_works_with_artifacts](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/artifact_dep.rs#L1248-L1326) -- Checks that doctest has access to the artifact dependencies. - [build_script::duplicate_script_with_extra_env](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/build_script.rs#L5514-L5614) -- Checks that build-script env and cfg values are correctly handled on host versus target when cross running doctests. - [cross_compile::cross_tests](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L416-L502) -- Basic test that cross-compiled tests work. - [cross_compile::doctest_xcompile_linker](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L1139-L1182) -- Checks that the linker config argument works. - [custom_target::custom_target_minimal](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/custom_target.rs#L39-L71) -- Checks that `.json` targets work with rustdoc cross tests. - [test::cargo_test_doctest_xcompile_ignores](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/test.rs#L4743-L4777) -- Checks the `ignore-*` syntax works. - [test::cargo_test_doctest_xcompile_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4783-L4863) -- Checks runner with cross doctests. - [test::cargo_test_doctest_xcompile_no_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4869-L4907) -- Checks cross doctests without a runner. Rustdoc tests: - [run-make/doctest-runtool](https://github.com/rust-lang/rust/tree/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/run-make/doctests-runtool) -- Tests behavior of `--test-run-directory` with relative paths of the runner. - [rustdoc/doctest/doctest-runtool](https://github.com/rust-lang/rust/blob/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/rustdoc/doctest/doctest-runtool.rs) -- Tests for `--test-runtool` and `--test-runtool-arg`. ## Future concerns There have been some discussions (rust-lang/testing-devex-team#5) about changing how doctests are driven. My understanding is that stabilizing this should not affect those plans, since if cargo becomes the driver, it will simply need to build things with `--target` and use the appropriate runner. ## Change notes This PR changed tests a little: - artifact_dep::no_cross_doctests_works_with_artifacts was changed now that doctests actually work. - cross_compile::cross_tests was changed to properly check doctests. - cross_compile::no_cross_doctests dropped since it is no longer relevant. - standard_lib::doctest didn't need `-Zdoctest-xcompile` since `-Zbuild-std` no longer uses a target. - test::cargo_test_doctest_xcompile was removed since it is a duplicate of cross_compile::cross_tests I think this should probably wait until the next release cutoff, moving this to 1.89 (will update the PR accordingly if that happens).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Implementation: #6892Nightly: doctest-xcompile
Rustc tracking issue: rust-lang/rust#64245
Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#doctest-xcompile
Issues: Z-doctest-xcompile
This PR adds the unstable
-Zdoctest-xcompileflag to cargo. When present, instead of skipping doctests whenhost != targetas it does now, cargo parses therunnerfield fortargetfrom.cargo/configand passes that to rustdoc. See here for the companion PR in rust that adds the necessary functionality to rustdoc. The flag also enables a new unstable--enable-per-target-ignoresflag in rustdoc, allowing the use of compiletest-style--ignore-fooannotations for doctests so that they are ignored on a per-target basis. The motivation for these PRs, along with another modifying compiltest and bootstrap (not yet submitted), is to allow bootstrap to utilize the many doctests found in the various crates in the rust compiler, notably libstd and libcore.This should add uniformity and expand testsuites for projects that build for other platforms, since the
runnerfield is already used bycargo test.Unresolved issues
The text was updated successfully, but these errors were encountered: