Add support for aarch64 wheels

[janaknat]

Travis-CI publicly announced Graviton 2 support (https://blog.travis-ci.com/AWS-Graviton-2-support-comes-to-Travis-CI)

The tag needed to run builds on a travis-ci graviton2 system is:

- os: linux
  arch: arm64-graviton2
  dist: local
  virt: lxd
  group: edge

Graviton2 support is in very early stages, there are limitations and everything runs in an lxd container. This is in early adoption as of now.

I've been able to use that to build aarch64 wheels. However, I do run into the travis-ci 50 minute timeout issue when running the pytest, even with 'test_mode=fast'. On the occasion where all the tests are done, I run into two failures.

1. test_infeasible_bounds_2: Checking to see if there is an issue with the way infinity is represented in aarch64 vs x86
2. test_maxiter_worsening: Any suggestions?

travis.yml: https://github.com/janaknat/scipy-wheel/blob/master/.travis.yml
travis-ci build log: https://travis-ci.com/github/janaknat/scipy-wheel/jobs/365931011
sha256 entry in scipy: https://github.com/janaknat/scipy/blob/master/tools/openblas_support.py#L37

Is there any interest in pursuing this to generate aarch64 wheels? If so, any pointers on investigating these failing tests?

[tylerjereddy]

This is really cool stuff--I'm pretty sure there is long-term interest in supporting ARM wheels yes. For example, they are included in the PEP 599 manylinux 2014 description, and I believe NumPy may have experimental support now?

I'm sure there is a simple reason, but what is the advantage of choosing arm64-graviton2 as you mention vs. plain arm64 architecture selected for Travis in the SciPy main repo?

[mattip]

You may want to take a look at odidev's fork where @odidev has been trying to overcome the timeouts on arm64. It seems splitting the job into build and test helped, there are just a few test failures left.

[janaknat]

@tylerjereddy The build times when using the arm64-graviton2 is significantly lesser when compared to arm64. This should help out in avoiding the timeouts with arm64. When building scikit-learn, for example, my build times were almost cut in half with arm64-graviton2. Just need to figure out why the pytests are taking longer on ARM.
@mattip I'll take a look at odidev's fork to fix the timeout caused by the pytests.

[janaknat]

Looking at the discussion in scipy/scipy#11170, it looks like odidev has a PR prepped for this.

@mattip I noticed that when scipy is building its wheel, numpy also goes through the same process. Numpy has wheels for the latest version (v1.19.1). Are there any plans to release wheels for older versions which other projects depend on (scipy-wheels depends on v1.14.5 & 1.17.3). That could reduce the build time.

[mattip]

No, there are no plans for NumPy to build older wheels. Isn't there a way to cache them here?

[janaknat]

Ok. I'm not aware of the way to cache wheels. I know the wheel created lives in the wheelhouse. Numpy gets built once during the build phase and once again during the test phase.

@tylerjereddy are y'all open to moving to the latest numpy as a dependency?

[rgommers]

are y'all open to moving to the latest numpy as a dependency?

No, but that's not needed either. Adding aarch64 SciPy wheels on PyPI that have `install_requires=['numpy>=1.19.1'], but leaving sdists and other wheels alone should be fine.

[rgommers]

Or actually, that's not even needed. By default pip will pick up the latest version of NumPy, which does have wheels. So nothing to do here.

[mattip]

If the SciPy maintainers agree to use a later version for ARM64, you can tell multibuild to use it via NP_BUILD_DEP and NP_TEST_DEP.

[geoffreyblake]

@janaknat have you tested if pip will pick up the latest numpy wheels for scipy to confirm that nothing is needed here?

[janaknat]

@geoffreyblake When I ran the scipy-wheels travis job, the numpy wheel is always built. NP_BUILD_DEP is set to numpy==1.14.5 for python 3.6, 3.7 and numpy==1.17.3 for python 3.8. For aarch64, numpy wheels exists only for 1.19.1. I don't see the latest numpy being picked up by pip.

[tylerjereddy]

See also #93.

Perhaps the assumption on always using the latest NumPy with pip is based on build isolation, which maybe is not being used in some of the cases observed? I suspect Ralf is referring to pyproject.toml + PEP 518 stuff, instead of the --no-build-isolation bypass to use what is in the env, though I could be wrong.

Presumably for ARM, the support range is so narrow we don't have to/can't pin to older dep versions for support anyway.

[tylerjereddy]

I'll close this since SciPy 1.5.3 is out with support for Linux ARM64 wheels (Python 3.6, 3.7, 3.8).

If issues show up with those wheels, or for other things like upcoming MacOS ARM binaries, let's open separate issues for that.