From 938972409ec96336b1e41d0d23c6a23c3b248321 Mon Sep 17 00:00:00 2001 From: Eric Kidd Date: Sat, 1 Jan 2022 16:17:30 -0500 Subject: [PATCH 1/2] Upgrade baseimage to Ubuntu 20.04 Many thanks to @asaaki for having figured out the static link check was returning flase negatives with Alpine's `lld`. See: https://github.com/emk/rust-musl-builder/pull/120#issuecomment-803407719 The plan is to merge this when Rust 1.58.0 is released, so people who need 18.04 can stick with1.57.0. Closes #98. Closes #120. --- CHANGELOG.md | 6 +++++ Dockerfile | 4 ++-- README.md | 31 +++++++++++++++++++----- examples/.gitignore | 1 + test-image | 58 +++++++++++++++++++++++++-------------------- 5 files changed, 66 insertions(+), 34 deletions(-) create mode 100644 examples/.gitignore diff --git a/CHANGELOG.md b/CHANGELOG.md index 3729374..d87fa76 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). For maximum stablity, use images with tags like `ekidd/rust-musl-builder:1.46.0` or `ekidd/rust-musl-builder:nightly-2020-08-26`. These may occasionally be rebuilt, but only while they're "current", or possibly if they're recent and serious security issues are discovered in a library. +## [Unreleased] + +### Changed + +- Updated base image to Ubuntu 20.04. Major thanks to @asaaki for figuring out why the check for successful static linking was returning false negatives. + ## 2021-12-23 ### Added diff --git a/Dockerfile b/Dockerfile index 868d0b9..ee947c4 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,5 +1,5 @@ -# Use Ubuntu 18.04 LTS as our base image. -FROM ubuntu:18.04 +# Use Ubuntu 20.04 LTS as our base image. +FROM ubuntu:20.04 # The Rust toolchain to use when building our image. Set by `hooks/build`. ARG TOOLCHAIN=stable diff --git a/README.md b/README.md index 7bb9fb1..1023ba1 100644 --- a/README.md +++ b/README.md @@ -5,14 +5,12 @@ - [Source on GitHub](https://github.com/emk/rust-musl-builder) - [Changelog](https://github.com/emk/rust-musl-builder/blob/master/CHANGELOG.md) -**UPDATED:** We are now running builds on GitHub, including scheduled builds of `stable` and `beta` every Thursday! +**UPDATED:** -However, **[`rustls`](rustls) now works well** with most of the Rust ecosystem, including `reqwest`, `tokio`, `tokio-postgres`, `sqlx` and many others. The only major project which still requires `libpq` and OpenSSL is [Diesel](https://diesel.rs/). If you don't need `diesel` or `libpq`: +- We are now using Ubuntu 20.04 as our base image! +- We are now running builds on GitHub, including scheduled builds of `stable` and `beta` every Thursday! -- See if you can switch away from OpenSSL, typically by using `features` in `Cargo.toml` to ask your dependencies to use [`rustls`](rustls) instead. -- If you don't need OpenSSL, try [`cross build --target=x86_64-unknown-linux-musl --release`](https://github.com/rust-embedded/cross) to cross-compile your binaries for `libmusl`. This supports many more platforms, with less hassle! - -[rustls]: https://github.com/rustls +See our [CHANGELOG](https://github.com/emk/rust-musl-builder/blob/main/CHANGELOG.md) for more details. ## What is this? @@ -29,6 +27,27 @@ This command assumes that `$(pwd)` is readable and writable by uid 1000, gid 100 For a more realistic example, see the `Dockerfile`s for [examples/using-diesel](./examples/using-diesel) and [examples/using-sqlx](./examples/using-sqlx). +## Should you use this to distribute your Rust program? + +In general, linking against OpenSSL and/or `libpq` will limit your portability and bring you extra headaches. In an ideal world, you would avoid C libraries, replace OpenSSL with [`rustls`][rustls], and build your binary using [`cross`]](https://github.com/rust-embedded/cross). It's a super nice workflow: + +```sh +# What you would ideally do instead of using rust-musl-builder. +cross build --target=x86_64-unknown-linux-musl --release +``` + +`cross` is a drop-in replacement for `cargo`, and it will allow you cross-compile for many different platforms. Even better, it's supported by the Rust Tools team. The downside: It doesn't support OpenSSL, or anything which requires OpenSSL. So if you want to do this, you'll need to configure libraries like `hyper`, `reqwest` and `tokio-postgres` to use [`rustls`][rustls] instead. See the documentation for each library for instructions. This is often the least painful choice. + +### Use case 1 for `rust-musl-builder`: You need to link `libpq` and OpenSSL for `diesel` + +The standard PostgreSQL client library for C is `libpq`. The popular [Diesel](https://diesel.rs/) crate uses `libpq`. And `libpq` links against OpenSSL. So in order to use `diesel`, you need to link against OpenSSL. If you want to link OpenSSL and `libpq` statically, then you can't use `cross`. But you can use `rust-musl-builder`. + +### Use case 2 for `rust-musl-builder`: You need to support weird TLS certificates + +[`rustls`][rustls] is a fantastic crate and it's very easy to work with. However, it relies on `webpki` to parse TLS certificates, and `webpki` may break if it encounters weird certificates. (For example, [it requires a valid `subjectAltName`](https://github.com/briansmith/webpki/issues/11).) Unfortunately, hosted PostgreSQL servers tend to omit `subjectAltName`. This is known to be a problem with the soon-to-be-defunct Citus Data, and with Google's Cloud PostgreSQL solution. In these cases, you'll probably need OpenSSL for now. + +[rustls]: https://github.com/rustls + ## Deploying your Rust application With a bit of luck, you should be able to just copy your application binary from `target/x86_64-unknown-linux-musl/release`, and install it directly on any reasonably modern x86_64 Linux machine. In particular, you should be able make static release binaries using TravisCI and GitHub, or you can copy your Rust application into an [Alpine Linux container][]. See below for details! diff --git a/examples/.gitignore b/examples/.gitignore new file mode 100644 index 0000000..65776c3 --- /dev/null +++ b/examples/.gitignore @@ -0,0 +1 @@ +/bin/ \ No newline at end of file diff --git a/test-image b/test-image index f29ef40..6b5900c 100755 --- a/test-image +++ b/test-image @@ -9,40 +9,46 @@ docker build -t ekidd/rust-musl-builder . # Make sure we can build our example derived container. docker build -t rust-musl-zlib examples/adding-a-library -# Make sure we can build a multi-stage container. -docker build -t rust-musl-builder-using-diesel examples/using-diesel -docker run --rm rust-musl-builder-using-diesel echo "==== Verifying static linking" +# Helper function to verify static linking. +function check_static_linking() { + local image="$1" + local container_bin_path="$2" + + local bin_basename="$(basename $container_bin_path)" + local bin_path="examples/bin/$bin_basename" + + echo -e "--- Test case for $bin_basename:" + local container="$(docker create "$1")" + docker cp "$container:$container_bin_path" "$bin_path" + docker rm -f "$container" + echo 'ldd says:' + ldd "$bin_path" > .tmp-ldd-output + if ! grep "statically linked" .tmp-ldd-output ; then + rm .tmp-ldd-output + echo "[FAIL] $bin_path is not static!" 1>&2 + exit 1 + fi + rm .tmp-ldd-output + echo -e "[PASS] $bin_path binary is statically linked.\n" +} + +mkdir -p examples/bin + +# Make sure we can build a static executable using `diesel`. +docker build -t rust-musl-builder-using-diesel examples/using-diesel +docker run --rm rust-musl-builder-using-diesel +check_static_linking rust-musl-builder-using-diesel /usr/local/bin/using-diesel + # Make sure we can build a static executable using `sqlx`. docker build -t rust-musl-builder-using-sqlx examples/using-sqlx -docker run --rm rust-musl-builder-using-sqlx sh -c " -set -euo pipefail - -echo -e '--- Test case for sqlx:' -echo 'ldd says:' -if ldd /usr/local/bin/using-sqlx; then - echo '[FAIL] Executable is not static!' 1>&2 - exit 1 -fi -echo -e '[PASS] using-sqlx binary is statically linked.\n' -" +check_static_linking rust-musl-builder-using-sqlx /usr/local/bin/using-sqlx # Make sure we can build a static executable using `git2`. docker build -t rust-musl-builder-linking-with-git2 examples/linking-with-git2 -docker run --rm rust-musl-builder-linking-with-git2 bash -c " -set -euo pipefail -cd /home/rust/src - -echo -e '--- Test case for libgit2:' -echo 'ldd says:' -if ldd target/x86_64-unknown-linux-musl/debug/linking-with-git2; then - echo '[FAIL] Executable is not static!' 1>&2 - exit 1 -fi -echo -e '[PASS] libgit2 binary is statically linked.\n' -" +check_static_linking rust-musl-builder-linking-with-git2 /home/rust/src/target/x86_64-unknown-linux-musl/debug/linking-with-git2 # We're good. echo 'OK. ALL TESTS PASSED.' 1>&2 From 941e3bd9bbf2854c426cd2ae0930c9251f9ba5f1 Mon Sep 17 00:00:00 2001 From: Eric Kidd Date: Sat, 1 Jan 2022 16:30:27 -0500 Subject: [PATCH 2/2] README: Proofread --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 1023ba1..00ac7a2 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ For a more realistic example, see the `Dockerfile`s for [examples/using-diesel]( ## Should you use this to distribute your Rust program? -In general, linking against OpenSSL and/or `libpq` will limit your portability and bring you extra headaches. In an ideal world, you would avoid C libraries, replace OpenSSL with [`rustls`][rustls], and build your binary using [`cross`]](https://github.com/rust-embedded/cross). It's a super nice workflow: +In general, linking against OpenSSL and/or `libpq` will limit your portability and bring you extra headaches. In an ideal world, you would avoid C libraries, replace OpenSSL with [`rustls`][rustls], and build your binary using [`cross`](https://github.com/rust-embedded/cross). It's a super nice workflow: ```sh # What you would ideally do instead of using rust-musl-builder. @@ -44,7 +44,7 @@ The standard PostgreSQL client library for C is `libpq`. The popular [Diesel](ht ### Use case 2 for `rust-musl-builder`: You need to support weird TLS certificates -[`rustls`][rustls] is a fantastic crate and it's very easy to work with. However, it relies on `webpki` to parse TLS certificates, and `webpki` may break if it encounters weird certificates. (For example, [it requires a valid `subjectAltName`](https://github.com/briansmith/webpki/issues/11).) Unfortunately, hosted PostgreSQL servers tend to omit `subjectAltName`. This is known to be a problem with the soon-to-be-defunct Citus Data, and with Google's Cloud PostgreSQL solution. In these cases, you'll probably need OpenSSL for now. +[`rustls`][rustls] is a fantastic crate and it's very easy to work with. However, it relies on `webpki` to parse TLS certificates, and `webpki` may break if it encounters weird certificates. For example, [it requires a valid `subjectAltName`](https://github.com/briansmith/webpki/issues/11). Unfortunately, hosted PostgreSQL servers tend to omit `subjectAltName`. This is known to be a problem with the soon-to-be-defunct Citus Data, and with Google's Cloud PostgreSQL solution. In these cases, you'll probably need OpenSSL for now. [rustls]: https://github.com/rustls