CPython 3.5 manylinux s390x failed

[caishanli]

Hi, I use cibuildwheel to build ncnn, build log is here, and workflow file is release-python.yml

[Czaki]

It looks like lack of SSL library in docker image. Could You fix it using yum install for the proper package.

[caishanli]

It looks like lack of SSL library in docker image. Could You fix it using yum install for the proper package.

error msg is ERROR: Could not build wheels for cmake which use PEP 517 and cannot be installed directly
You mean install cmake via yum?

[henryiii]

These are being added soon, see scikit-build/cmake-python-distributions#130 - but for now, you can turn them off when building on s390x in your pyproject.yaml via a selector and use the built-in CMake in those images instead.

[henryiii]

(You should also be able to add the requirements for CMake to build from SDist, which also would fix it - I expect it's SSL that's missing?)

[henryiii]

Why are you shipping 3.5 + s390x? Shouldn't you start with 3.6? We are currently dropping 3.5 and 3.5 has already been removed from the manylinux images, including s390x images.

[caishanli]

Why are you shipping 3.5 + s390x? Shouldn't you start with 3.6? We are currently dropping 3.5 and 3.5 has already been removed from the manylinux images already, including s390x images.

3.5+s390x is auto build
yml is here, anything wrong?

[henryiii]

Okay, the next release will build only 3.6+, so just making sure you were not needing that specific version for something.

[henryiii]

Something roughly like this should work for your pyproject.toml until new wheels are released. I'm only guessing Ninja has similar issues.

[build-system]
requires = [
    "setuptools>=42",
    "wheel",
    "ninja; sys_platform != 'win32' and platform_machine != 's390x'",
    "cmake>=3.12; platform_machine != 's390x'",
]
build-backend = "setuptools.build_meta"

[caishanli]

Something roughly like this should work for your pyproject.toml until new wheels are released. I'm only guessing Ninja has similar issues.

[build-system]
requires = [
    "setuptools>=42",
    "wheel",
    "ninja; sys_platform != 'win32' and platform_machine != 'aarch64'",
    "cmake>=3.12; platform_machine != 'aarch64'",
]
build-backend = "setuptools.build_meta"

ninja is ok on aarch64, see build log here

[henryiii]

Oops, please replace aarch64 with s390x above. 😳

[caishanli]

Oops, please replace aarch64 with s390x above.

still failed with s390x, see log

[mayeut]

Only the build for CMake wheel was failing in the original log so I guess you can still rely on Ninja being built from source correctly and use the manylinux provided cmake:

[build-system]
requires = [
    "setuptools>=42",
    "wheel",
    "ninja; sys_platform != 'win32'",
    "cmake>=3.12; platform_machine != 's390x'",
]
build-backend = "setuptools.build_meta"

[henryiii]

I just released CMake 3.20.2 on PyPI which contains Apple Silicon, S390x, and PowerPC builds now too (thanks to @mayeut). So this should "just work" now (though Ninja may still need some work).

[henryiii]

Closable, @caishanli?

[caishanli]

in testing now...
aarch64 takes long time to build
and i will report the result

[caishanli]

Closable, @caishanli?

still failed. see build log here

[mayeut]

see build log here

@caishanli,
The error is now completely different. All tools seem to install correctly.
The CMakeLists.txt has a fallback to x86 arch with sse2 enabled which seems wrong for s390x.
This is what is now failing:

/opt/rh/devtoolset-9/root/usr/bin/c++  -I../../src/layer/x86 -I../../src -Isrc -I../../src/layer -O3 -DNDEBUG -fPIC -DNCNN_STATIC_DEFINE -Wall -Wextra -Wno-unused-function -Ofast -ffast-math -fvisibility=hidden -fvisibility-inlines-hidden -msse2 -msse -fopenmp -pthread -MD -MT src/CMakeFiles/ncnn.dir/allocator.cpp.o -MF src/CMakeFiles/ncnn.dir/allocator.cpp.o.d -o src/CMakeFiles/ncnn.dir/allocator.cpp.o -c ../../src/allocator.cpp
929
    c++: error: unrecognized command line option ‘-msse2’
930
    c++: error: unrecognized command line option ‘-msse’

[mayeut]

fixed by Tencent/ncnn#3396 which indeed includes a fix in CMakeLists.txt.