Skip to content

Add limited android test support #132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 20, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ matrix:
- env: TARGET=x86_64-unknown-linux-musl STD=1 OPENSSL=0.5.5 RUN=1 DEPLOY=1

# Android
- env: TARGET=aarch64-linux-android CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=arm-linux-androideabi CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=armv7-linux-androideabi CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=i686-linux-android CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=x86_64-linux-android CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=aarch64-linux-android CPP=1 STD=1 OPENSSL=0.5.5 RUN=1
- env: TARGET=arm-linux-androideabi CPP=1 STD=1 OPENSSL=0.5.5 RUN=1
- env: TARGET=armv7-linux-androideabi CPP=1 STD=1 OPENSSL=0.5.5 RUN=1
- env: TARGET=i686-linux-android CPP=1 STD=1 OPENSSL=0.5.5 RUN=1
- env: TARGET=x86_64-linux-android CPP=1 STD=1 OPENSSL=0.5.5 RUN=1

# OSX
- env: TARGET=i686-apple-darwin DYLIB=1 STD=1 RUN=1
Expand All @@ -41,7 +41,7 @@ matrix:

# BSD
- env: TARGET=i686-unknown-freebsd DYLIB=1 STD=1 OPENSSL=0.7.17
- env: TARGET=x86_64-unknown-dragonfly DYLIB=1 OPENSSL=0.5.5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: alignment regression :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a fix, isn't it?

- env: TARGET=x86_64-unknown-dragonfly DYLIB=1 OPENSSL=0.5.5
- env: TARGET=x86_64-unknown-freebsd DYLIB=1 STD=1 OPENSSL=0.5.5
- env: TARGET=x86_64-unknown-netbsd DYLIB=1 STD=1 OPENSSL=0.7.17

Expand Down
16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,16 +189,16 @@ worst, "hang" (never terminate).

| Target | libc | GCC | OpenSSL | C++ | QEMU | `test` |
|--------------------------------------|--------|---------|---------|:---:|-------|:------:|
| `aarch64-linux-android` | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `aarch64-linux-android` [5] | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `aarch64-unknown-linux-gnu` | 2.19 | 4.8.2 | 1.0.2k | ✓ | 2.8.0 | ✓ |
| `arm-linux-androideabi` | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `arm-linux-androideabi` [5] | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `arm-unknown-linux-gnueabi` | 2.19 | 4.8.2 | 1.0.2k | ✓ | 2.8.0 | ✓ |
| `arm-unknown-linux-musleabi` | 1.1.15 | 5.3.1 | N/A | | 2.8.0 | ✓ |
| `armv7-linux-androideabi` | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `armv7-linux-androideabi` [5] | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `armv7-unknown-linux-gnueabihf` | 2.15 | 4.6.2 | 1.0.2k | ✓ | 2.8.0 | ✓ |
| `armv7-unknown-linux-musleabihf` | 1.1.15 | 5.3.1 | N/A | | 2.8.0 | ✓ |
| `asmjs-unknown-emscripten` [4] | 1.1.15 | 1.37.13 | N/A | ✓ | N/A | ✓ |
| `i686-linux-android` | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `i686-linux-android` [5] | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `i686-pc-windows-gnu` | N/A | 6.2.0 | N/A | ✓ | N/A | ✓ |
| `i686-unknown-freebsd` [1] | 10.2 | 5.3.0 | 1.0.2k | | N/A | |
| `i686-unknown-linux-gnu` | 2.15 | 4.6.2 | 1.0.2k | ✓ | N/A | ✓ |
Expand All @@ -217,7 +217,7 @@ worst, "hang" (never terminate).
| `thumbv7em-none-eabihf` [3] | 2.2.0 | 5.3.1 | N/A | | N/A | |
| `thumbv7m-none-eabi` [3] | 2.2.0 | 5.3.1 | N/A | | N/A | |
| `wasm32-unknown-emscripten` [4] | 1.1.15 | 1.37.13 | N/A | ✓ | N/A | ✓ |
| `x86_64-linux-android` | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `x86_64-linux-android` [5] | N/A | 4.9 | 1.0.2k | ✓ | N/A | |
| `x86_64-pc-windows-gnu` | N/A | 6.2.0 | N/A | ✓ | N/A | ✓ |
| `x86_64-unknown-dragonfly` [1] [2] | 4.6.0 | 5.3.0 | 1.0.2k | | N/A | ✓ |
| `x86_64-unknown-freebsd` [1] | 10.2 | 5.3.0 | 1.0.2k | | N/A | |
Expand All @@ -235,6 +235,12 @@ where libc was extracted.
[4] libc = musl, gcc = emcc; Some projects that use libc may fail due to wrong
definitions (will be fixed by https://github.com/rust-lang/libc/pull/610)

[5] Only works with native tests, that is, tests that do not depends on the
Android Runtime. For i686 some tests may fails with the error `assertion
failed: signal(libc::SIGPIPE, libc::SIG_IGN) != libc::SIG_ERR`, see
[issue #140](https://github.com/japaric/cross/issues/140) for more
information.

## Debugging

### QEMU_STRACE (v0.1.9+)
Expand Down
12 changes: 11 additions & 1 deletion docker/aarch64-linux-android/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,23 @@ RUN apt-get update && \
COPY xargo.sh /
RUN bash /xargo.sh

COPY qemu.sh /
RUN bash /qemu.sh aarch64 android

COPY android-ndk.sh /
RUN bash /android-ndk.sh arm64 21
ENV PATH=$PATH:/android-ndk/bin

COPY openssl.sh /
RUN bash /openssl.sh linux-generic64 aarch64-linux-android- -mandroid -fomit-frame-pointer

COPY android-system.sh /
RUN bash /android-system.sh arm64

# Libz is distributed in the android ndk, but for some unknown reason it is not
# found in the build process of some crates, so we explicit set the DEP_Z_ROOT
ENV CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER=aarch64-linux-android-gcc \
CARGO_TARGET_AARCH64_LINUX_ANDROID_RUNNER=qemu-aarch64 \
CC_aarch64_linux_android=aarch64-linux-android-gcc \
CXX_aarch64_linux_android=aarch64-linux-android-g++ \
DEP_Z_ROOT=/android-ndk/sysroot/usr/ \
Expand All @@ -30,4 +37,7 @@ ENV CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER=aarch64-linux-android-gcc \
OPENSSL_INCLUDE_DIR=/openssl/include \
OPENSSL_LIB_DIR=/openssl/lib \
RUST_TEST_THREADS=1 \
HOME=/tmp/
HOME=/tmp/ \
TMPDIR=/tmp/ \
ANDROID_ROOT=/system \
ANDROID_DATA=/
102 changes: 102 additions & 0 deletions docker/android-system.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
set -ex

main() {
local arch=$1
local td=$(mktemp -d)
pushd $td

local dependencies=(
ca-certificates
curl
gcc-multilib
git
g++-multilib
make
python
)

# fake java and javac, it is not necessary for what we build, but the build
# script ask for it
cat << EOF > /usr/bin/java
#!/bin/bash
echo "java version \"1.7.0\""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an interesting hack. I suppose that all android programs read the java version (e.g. by calling java --version) at runtime or something like that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you are right.

echo "OpenJDK Runtime Environment (IcedTea 2.6.9)"
echo "OpenJDK 64-Bit Server VM (build 24.131-b00, mixed mode)"
EOF

cat << EOF > /usr/bin/javac
#!/bin/bash
echo "javac 1.7.0"
EOF

chmod +x /usr/bin/java
chmod +x /usr/bin/javac

# more faking
export ANDROID_JAVA_HOME=/tmp
mkdir /tmp/lib/
touch /tmp/lib/tools.jar

apt-get update
local purge_list=(default-jre)
for dep in ${dependencies[@]}; do
if ! dpkg -L $dep; then
apt-get install --no-install-recommends -y $dep
purge_list+=( $dep )
fi
done

curl -O https://storage.googleapis.com/git-repo-downloads/repo
chmod +x repo

# this is the minimum set of modules that are need to build bionic
# this was created by trial and error
./repo init -u https://android.googlesource.com/platform/manifest -b android-5.0.0_r1
./repo sync bionic
./repo sync build
./repo sync external/compiler-rt
./repo sync external/jemalloc
./repo sync external/libcxx
./repo sync external/libcxxabi
./repo sync external/stlport
./repo sync prebuilts/clang/linux-x86/host/3.5
./repo sync system/core
case $arch in
arm)
./repo sync prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8
;;
arm64)
./repo sync prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8
./repo sync prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9
;;
x86)
./repo sync prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.8
;;
x86_64)
./repo sync prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.8
;;
esac

# avoid build tests
rm bionic/linker/tests/Android.mk bionic/tests/Android.mk

source build/envsetup.sh
lunch aosp_$arch-user
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lunch? not launch? (probably intentional)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lunch is a command of the android build system.

mmma bionic/

if [ $arch = "arm" ]; then
mv out/target/product/generic/system/ /
else
mv out/target/product/generic_$arch/system/ /
fi

# clean up
apt-get purge --auto-remove -y ${purge_list[@]}

popd

rm -rf $td
rm $0
}

main "${@}"
12 changes: 11 additions & 1 deletion docker/arm-linux-androideabi/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,23 @@ RUN apt-get update && \
COPY xargo.sh /
RUN bash /xargo.sh

COPY qemu.sh /
RUN bash /qemu.sh arm android

COPY android-ndk.sh /
RUN bash /android-ndk.sh arm 21
ENV PATH=$PATH:/android-ndk/bin

COPY openssl.sh /
RUN bash /openssl.sh android arm-linux-androideabi-

COPY android-system.sh /
RUN bash /android-system.sh arm

# Libz is distributed in the android ndk, but for some unknown reason it is not
# found in the build process of some crates, so we explicit set the DEP_Z_ROOT
ENV CARGO_TARGET_ARM_LINUX_ANDROIDEABI_LINKER=arm-linux-androideabi-gcc \
CARGO_TARGET_ARM_LINUX_ANDROIDEABI_RUNNER=qemu-arm \
CC_arm_linux_androideabi=arm-linux-androideabi-gcc \
CXX_arm_linux_androideabi=arm-linux-androideabi-g++ \
DEP_Z_ROOT=/android-ndk/sysroot/usr/ \
Expand All @@ -30,4 +37,7 @@ ENV CARGO_TARGET_ARM_LINUX_ANDROIDEABI_LINKER=arm-linux-androideabi-gcc \
OPENSSL_INCLUDE_DIR=/openssl/include \
OPENSSL_LIB_DIR=/openssl/lib \
RUST_TEST_THREADS=1 \
HOME=/tmp/
HOME=/tmp/ \
TMPDIR=/tmp/ \
ANDROID_ROOT=/system \
ANDROID_DATA=/
12 changes: 11 additions & 1 deletion docker/armv7-linux-androideabi/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,23 @@ RUN apt-get update && \
COPY xargo.sh /
RUN bash /xargo.sh

COPY qemu.sh /
RUN bash /qemu.sh arm android

COPY android-ndk.sh /
RUN bash /android-ndk.sh arm 21
ENV PATH=$PATH:/android-ndk/bin

COPY openssl.sh /
RUN bash /openssl.sh android-armv7 arm-linux-androideabi-

COPY android-system.sh /
RUN bash /android-system.sh arm

# Libz is distributed in the android ndk, but for some unknown reason it is not
# found in the build process of some crates, so we explicit set the DEP_Z_ROOT
ENV CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=arm-linux-androideabi-gcc \
CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_RUNNER=qemu-arm \
CC_armv7_linux_androideabi=arm-linux-androideabi-gcc \
CXX_armv7_linux_androideabi=arm-linux-androideabi-g++ \
DEP_Z_ROOT=/android-ndk/sysroot/usr/ \
Expand All @@ -30,4 +37,7 @@ ENV CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=arm-linux-androideabi-gcc \
OPENSSL_INCLUDE_DIR=/openssl/include \
OPENSSL_LIB_DIR=/openssl/lib \
RUST_TEST_THREADS=1 \
HOME=/tmp/
HOME=/tmp/ \
TMPDIR=/tmp/ \
ANDROID_ROOT=/system \
ANDROID_DATA=/
22 changes: 20 additions & 2 deletions docker/i686-linux-android/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,31 @@ RUN apt-get update && \
COPY xargo.sh /
RUN bash /xargo.sh

# We could supposedly directly run i686 binaries like we do for x86_64, but
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is ... interesting. Any idea why this happen? IIRC, this also happens with the i686-musl target. A panicking program cross compiled for this target will fail in a similar way when executed on x86_64. But I never tried running running that binary under qemu-i386. Cross compiling that binary for i686-gnu and running it on x86_64 works fine though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it seams I got this wrong. Using qemu is enough to pass the ci/script.sh testing, but it fails in in the ripgrep tests. I was unable to reproduce the i686-musl failure... Running cross test --target i686-unknown-linux-musl in the ripgrep repository worked as expected. I tested a simple program that panics, and it also worked.

Maybe this is a std issue? How should we proceed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it seems that unwinding on i686-unknown-linux-musl was fixed a while ago but didn't notice.

i686-unknown-linux-musl is not tested in rust-lang/rust so what you are reporting may indeed be a std bug.

# doing so generates an assertion failure:
# ... assertion failed: signal(libc::SIGPIPE, libc::SIG_IGN) != libc::SIG_ERR
# ... src/libstd/sys/unix/mod.rs
# fatal runtime error: failed to initiate panic, error 5
#
# Running with qemu works as expected
COPY qemu.sh /
RUN bash /qemu.sh i386 android

COPY android-ndk.sh /
RUN bash /android-ndk.sh x86 21
ENV PATH=$PATH:/android-ndk/bin

# Build with no-asm to make openssl linked binaries position-independent (PIE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting finding. I wonder if no-asm would fix OpenSSL builds on i686-musl. IIRC, when cross compiling OpenSSL for i686-musl caused some PIC/PIE errors later on in the linking process.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried and it didn't work...

Copy link
Contributor

@japaric japaric Oct 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Doesn't matter; it seems that there may be a solution for that issue in #27 (comment) .

COPY openssl.sh /
RUN bash /openssl.sh android-x86 i686-linux-android-
RUN bash /openssl.sh android-x86 i686-linux-android- no-asm

COPY android-system.sh /
RUN bash /android-system.sh x86

# Libz is distributed in the android ndk, but for some unknown reason it is not
# found in the build process of some crates, so we explicit set the DEP_Z_ROOT
ENV CARGO_TARGET_I686_LINUX_ANDROID_LINKER=i686-linux-android-gcc \
CARGO_TARGET_I686_LINUX_ANDROID_RUNNER=qemu-i386 \
CC_i686_linux_android=i686-linux-android-gcc \
CXX_i686_linux_android=i686-linux-android-g++ \
DEP_Z_ROOT=/android-ndk/sysroot/usr/ \
Expand All @@ -30,4 +45,7 @@ ENV CARGO_TARGET_I686_LINUX_ANDROID_LINKER=i686-linux-android-gcc \
OPENSSL_INCLUDE_DIR=/openssl/include \
OPENSSL_LIB_DIR=/openssl/lib \
RUST_TEST_THREADS=1 \
HOME=/tmp/
HOME=/tmp/ \
TMPDIR=/tmp/ \
ANDROID_ROOT=/system \
ANDROID_DATA=/
Loading