Skip to content

Commit dafc15b

Browse files
wshankswshanks
authored
Use pypa/build and tox-uv in CI testing and publishing (#1530)
pypa/build builds an sdist and then a wheel from the sdist which helps avoid leaving files out of the sdist. Additionally, pypa/build formats the package file names in the way that PyPI will soon require while `python setup.py` does not. Setting up the .pkg environment led to tox errors when `package` was set to different values in different environments so they were all set to editable by default but they can be overridden with `--override testenv.package=wheel`. Additionally, the shared `.tox/lint` `envdir` in `tox.ini` was dropped because with `tox` version 4 it resulted in the environment being rebuilt every time a different command sharing that environment directory was run (with `tox` 3 the environment was reused). Instead, a recommendation to use `tox-uv` was added to the documentation. `uv` is quick at creating similar virtual environments so it allows using many `tox` environments without worrying about speed or disk space (because it uses hard links). Explicit installation of `pyarrow` was removed from `tox.ini` because Pandas no longer issues a deprecation warning when `pyarrow` is not installed. Along with suggesting using `tox-uv` in the documentation, the GitHub workflows were updated to use `uv` and `tox-uv`. Also, the version of Python used for running linting and building the release artifacts was updated from 3.9 to 3.12 Closes #1527
1 parent 058e93e commit dafc15b

File tree

8 files changed

+90
-95
lines changed

8 files changed

+90
-95
lines changed

‎.github/workflows/cron-staging.yml

Lines changed: 17 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -26,25 +26,16 @@ jobs:
2626
uses: actions/setup-python@v5
2727
with:
2828
python-version: ${{ matrix.python-version }}
29-
- name: Pip cache
30-
uses: actions/cache@v4
29+
- name: Set up uv
30+
uses: astral-sh/setup-uv@v5
3131
with:
32-
path: ~/.cache/pip
33-
key: ${{ runner.os }}-${{ matrix.python-version }}-pip-tests-${{ hashFiles('pyproject.toml') }}
34-
restore-keys: |
35-
${{ runner.os }}-${{ matrix.python-version }}-pip-tests-
36-
${{ runner.os }}-${{ matrix.python-version }}-pip-
37-
${{ runner.os }}-${{ matrix.python-version }}
38-
- name: Install Deps
39-
run: python -m pip install -U tox setuptools virtualenv wheel
40-
- name: Install and Run Tests (Windows and Linux)
41-
run: tox -e qiskit-main
42-
if: runner.os != 'macOS'
43-
- name: Install and Run Tests (Macs only)
44-
run: tox -e qiskit-main
45-
if: runner.os == 'macOS'
46-
env:
47-
TEST_TIMEOUT: 120
32+
enable-cache: true
33+
cache-dependency-glob: |
34+
**/tox.ini
35+
**/pyproject.toml
36+
cache-suffix: testsqiskitmain
37+
- name: Install and Run Tests
38+
run: uvx --with tox-uv tox -e qiskit-main
4839
docs:
4940
if: github.repository_owner == 'Qiskit-Community'
5041
name: docs
@@ -57,17 +48,19 @@ jobs:
5748
uses: actions/setup-python@v5
5849
with:
5950
python-version: 3.12
60-
- name: Pip cache
61-
uses: actions/cache@v4
51+
- name: Set up uv
52+
uses: astral-sh/setup-uv@v5
6253
with:
63-
path: ~/.cache/pip
64-
key: ${{ runner.os }}-pip-docs-${{ hashFiles('pyproject.toml') }}
54+
enable-cache: true
55+
cache-dependency-glob: |
56+
**/tox.ini
57+
**/pyproject.toml
58+
cache-suffix: docsqiskitmain
6559
- name: Install Deps
6660
run: |
67-
python -m pip install -U tox
6861
sudo apt-get install -y pandoc graphviz
6962
- name: Build Docs
70-
run: tox -edocs-qiskit-main
63+
run: uvx --with tox-uv tox -edocs-qiskit-main
7164
- name: Compress Artifacts
7265
run: |
7366
mkdir artifacts

‎.github/workflows/docs_dev.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@ jobs:
1818
python-version: '3.12'
1919
- name: Install dependencies
2020
run: |
21-
python -m pip install --upgrade pip
22-
pip install -U virtualenv setuptools wheel tox
2321
sudo apt-get install graphviz pandoc
22+
- name: Set up uv
23+
uses: astral-sh/setup-uv@v5
2424
- name: Build docs dev
25-
run: EXPERIMENTS_DEV_DOCS=1 PROD_BUILD=1 RELEASE_STRING=`git describe` tox -edocs
25+
run: EXPERIMENTS_DEV_DOCS=1 PROD_BUILD=1 RELEASE_STRING=`git describe` uvx --with tox-uv tox run -edocs
2626
- name: Bypass Jekyll Processing # Necessary for setting the correct css path
2727
run: touch docs/_build/html/.nojekyll
2828
- name: Deploy

‎.github/workflows/docs_release.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,15 +17,15 @@ jobs:
1717
uses: actions/setup-python@v5
1818
with:
1919
python-version: '3.12'
20+
- name: Set up uv
21+
uses: astral-sh/setup-uv@v5
2022
- name: Install dependencies
2123
run: |
22-
python -m pip install --upgrade pip
23-
pip install -U virtualenv setuptools wheel tox
2424
sudo apt-get install graphviz pandoc
2525
- name: Build docs
2626
env:
2727
QISKIT_DOCS_BUILD_TUTORIALS: 'always'
28-
run: PROD_BUILD=1 tox -edocs
28+
run: PROD_BUILD=1 uvx --with tox-uv tox -edocs
2929
- name: Bypass Jekyll Processing # Necessary for setting the correct css path
3030
run: touch docs/_build/html/.nojekyll
3131
- name: Deploy

‎.github/workflows/docs_stable.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,15 +17,15 @@ jobs:
1717
uses: actions/setup-python@v5
1818
with:
1919
python-version: '3.12'
20+
- name: Set up uv
21+
uses: astral-sh/setup-uv@v5
2022
- name: Install dependencies
2123
run: |
22-
python -m pip install --upgrade pip
23-
pip install -U virtualenv setuptools wheel tox
2424
sudo apt-get install graphviz pandoc
2525
- name: Build docs stable
2626
env:
2727
QISKIT_DOCS_BUILD_TUTORIALS: 'always'
28-
run: PROD_BUILD=1 tox -e docs
28+
run: PROD_BUILD=1 uvx --with tox-uv tox -e docs
2929
- name: Bypass Jekyll Processing # Necessary for setting the correct css path
3030
run: touch docs/_build/html/.nojekyll
3131
- name: Set current version
@@ -35,4 +35,4 @@ jobs:
3535
uses: JamesIves/github-pages-deploy-action@v4
3636
with:
3737
folder: docs/_build/html
38-
target-folder: stable/${{ env.version }}
38+
target-folder: stable/${{ env.version }}

‎.github/workflows/main.yml

Lines changed: 28 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -31,15 +31,14 @@ jobs:
3131
uses: actions/setup-python@v5
3232
with:
3333
python-version: ${{ matrix.python-version }}
34-
- name: Pip cache
35-
uses: actions/cache@v4
34+
- name: Set up uv
35+
uses: astral-sh/setup-uv@v5
3636
with:
37-
path: ~/.cache/pip
38-
key: ${{ runner.os }}-${{ matrix.python-version }}-pip-tests-${{ hashFiles('pyproject.toml') }}
39-
restore-keys: |
40-
${{ runner.os }}-${{ matrix.python-version }}-pip-tests-
41-
${{ runner.os }}-${{ matrix.python-version }}-pip-
42-
${{ runner.os }}-${{ matrix.python-version }}
37+
enable-cache: true
38+
cache-dependency-glob: |
39+
**/tox.ini
40+
**/pyproject.toml
41+
cache-suffix: tests
4342
- name: Stestr cache
4443
uses: actions/cache@v4
4544
with:
@@ -48,38 +47,31 @@ jobs:
4847
restore-keys: |
4948
stestr-${{ runner.os }}-
5049
stestr-
51-
- name: Install Deps
52-
run: python -m pip install -U tox setuptools virtualenv wheel
53-
- name: Install and Run Tests (Windows and Linux)
54-
run: tox -e py
55-
if: runner.os != 'macOS'
56-
- name: Install and Run Tests (Macs only)
57-
run: tox -e py
58-
if: runner.os == 'macOS'
59-
env:
60-
TEST_TIMEOUT: 120
50+
- name: Install and Run Tests
51+
run: uvx --with tox-uv tox --override testenv.package=external run -e py
6152
- name: Clean up stestr cache
62-
run: tox exec -epy -- stestr history remove all
53+
run: uvx --with tox-uv tox exec -epy -- stestr history remove all
6354

6455
lint:
6556
if: github.repository_owner == 'Qiskit-Community'
6657
name: lint
6758
runs-on: ubuntu-latest
6859
steps:
6960
- uses: actions/checkout@v4
70-
- name: Set up Python 3.9
61+
- name: Set up Python 3.12
7162
uses: actions/setup-python@v5
7263
with:
73-
python-version: 3.9
74-
- name: Pip cache
75-
uses: actions/cache@v4
64+
python-version: 3.12
65+
- name: Set up uv
66+
uses: astral-sh/setup-uv@v5
7667
with:
77-
path: ~/.cache/pip
78-
key: ${{ runner.os }}-pip-lint-${{ hashFiles('pyproject.toml') }}
79-
- name: Install Deps
80-
run: python -m pip install -U tox
68+
enable-cache: true
69+
cache-dependency-glob: |
70+
**/tox.ini
71+
**/pyproject.toml
72+
cache-suffix: lint
8173
- name: Run lint
82-
run: tox -elint
74+
run: uvx --with tox-uv tox run -elint
8375
docs:
8476
if: github.repository_owner == 'Qiskit-Community'
8577
name: docs
@@ -92,17 +84,19 @@ jobs:
9284
uses: actions/setup-python@v5
9385
with:
9486
python-version: 3.12
95-
- name: Pip cache
96-
uses: actions/cache@v4
87+
- name: Set up uv
88+
uses: astral-sh/setup-uv@v5
9789
with:
98-
path: ~/.cache/pip
99-
key: ${{ runner.os }}-pip-docs-${{ hashFiles('pyproject.toml') }}
90+
enable-cache: true
91+
cache-dependency-glob: |
92+
**/tox.ini
93+
**/pyproject.toml
94+
cache-suffix: docstest
10095
- name: Install Deps
10196
run: |
102-
python -m pip install -U tox
10397
sudo apt-get install -y pandoc graphviz
10498
- name: Build Docs
105-
run: tox -edocs-parallel
99+
run: uvx --with tox-uv tox run -edocs-parallel
106100
- name: Compress Artifacts
107101
run: |
108102
mkdir artifacts

‎.github/workflows/release.yml

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,12 @@ jobs:
1515
- uses: actions/setup-python@v5
1616
name: Install Python
1717
with:
18-
python-version: '3.9'
18+
python-version: '3.12'
1919
- name: Install Deps
20-
run: pip install -U wheel
20+
run: pip install -U build
2121
- name: Build Artifacts
2222
run: |
23-
python setup.py sdist
24-
python setup.py bdist_wheel
23+
python -m build
2524
shell: bash
2625
- uses: actions/upload-artifact@v4
2726
with:

‎CONTRIBUTING.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,11 +95,24 @@ python when running. Additionally, the environment that tox sets up matches the
9595
environment more closely and it runs the tests in parallel (resulting in much faster
9696
execution). To run tests on all installed supported python versions and lint/style
9797
checks you can simply run `tox`. Or if you just want to run the tests once for a
98-
specific python version such as 3.10: `tox -epy310`.
98+
specific python version such as 3.10: `tox run -epy310`. Using `tox run -epy`
99+
will run the tests with the same Python version as used to install `tox`.
100+
101+
> [!TIP]
102+
> Install `tox` with [tox-uv](https://github.com/tox-dev/tox-uv) using `pip
103+
> install tox tox-uv` (or `uv pip install tox tox-uv`) for a smoother
104+
> experience working with the Qiskit Experiments `tox` environments. The
105+
> `tox.ini` file defines several similar environments because `tox` ties an
106+
> environment to a single set of commands. It can be slow to recreate each of
107+
> these environments when switching between commands and updating dependencies.
108+
> When `tox-uv` is installed, `tox` uses [uv](https://docs.astral.sh/uv/)
109+
> instead of `pip` for package installation. `uv` makes better use of caching
110+
> and hardlinking to set up similar environments much more quickly than `pip`
111+
> does.
99112
100113
If you just want to run a subset of tests you can pass a selection regex to the test
101114
runner. For example, if you want to run all tests that have "dag" in the test id you can
102-
run: `tox -- dag`. You can pass arguments directly to the test runner after the bare
115+
run: `tox run -epy -- dag`. You can pass arguments directly to the test runner after the bare
103116
`--`. To see all the options on test selection you can refer to the stestr manual:
104117
https://stestr.readthedocs.io/en/stable/MANUAL.html#test-selection
105118

‎tox.ini

Lines changed: 17 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ envlist = py312,py311,py310,py39,py38,lint
44

55
[testenv]
66
package = editable
7+
package_env = .pkg
8+
wheel_build_env = .pkg
79
setenv =
810
VIRTUAL_ENV={envdir}
911
QISKIT_SUPPRESS_PACKAGING_WARNINGS=Y
@@ -23,6 +25,20 @@ passenv =
2325
QE_USE_TESTTOOLS
2426
commands = stestr run {posargs}
2527

28+
[testenv:.pkg]
29+
# When testenv.package is set to external these settings will be used to build
30+
# the wheel with pypa/build instead of tox's internal PEP517 support. pypa/build
31+
# builds the sdist first and then builds the wheel from the sdist while tox
32+
# builds the wheel directly and can miss packaging problems like a missing entry
33+
# in MANIFEST.in.
34+
deps = build
35+
# tox does not do variable substitution in this environment so we have to write
36+
# out .tox/dist and clean it explicitly rather than using {envtmpdir}
37+
package_glob = .tox/dist/*.whl
38+
commands =
39+
python -c "import shutil; shutil.rmtree('.tox/dist', ignore_errors=True)"
40+
python -m build -o .tox/dist
41+
2642
[testenv:cover]
2743
basepython = python3
2844
setenv =
@@ -34,23 +50,19 @@ commands =
3450
coverage3 lcov
3551

3652
[testenv:qiskit-main]
37-
package = editable
3853
deps =
3954
{[testenv]deps}
4055
git+https://github.com/Qiskit/qiskit
4156
commands = stestr run {posargs}
4257

4358

4459
[testenv:lint]
45-
envdir = .tox/lint
4660
commands =
4761
black --check qiskit_experiments test tools
4862
pylint -rn {posargs} --rcfile={toxinidir}/.pylintrc qiskit_experiments/ test/ tools/
4963
python {toxinidir}/tools/verify_headers.py
5064

5165
[testenv:lint-incr]
52-
envdir = .tox/lint
53-
basepython = python3
5466
allowlist_externals = git
5567
commands =
5668
black --check {posargs} qiskit_experiments test tools
@@ -59,57 +71,43 @@ commands =
5971
python {toxinidir}/tools/verify_headers.py qiskit_experiments test tools
6072

6173
[testenv:black]
62-
envdir = .tox/lint
6374
commands = black {posargs} qiskit_experiments test tools
6475

6576
[testenv:docs]
66-
package = wheel
6777
passenv =
6878
EXPERIMENTS_DEV_DOCS
6979
PROD_BUILD
7080
RELEASE_STRING
7181
VERSION_STRING
72-
deps =
73-
{[testenv]deps}
74-
pyarrow
7582
setenv =
7683
PYDEVD_DISABLE_FILE_VALIDATION = 1
7784
commands =
7885
sphinx-build -T -W --keep-going -b html {posargs} docs/ docs/_build/html
7986

8087
[testenv:docs-parallel]
81-
package = wheel
8288
passenv =
8389
EXPERIMENTS_DEV_DOCS
8490
PROD_BUILD
8591
RELEASE_STRING
8692
VERSION_STRING
87-
deps =
88-
{[testenv]deps}
89-
pyarrow
9093
setenv =
9194
PYDEVD_DISABLE_FILE_VALIDATION = 1
9295
commands =
9396
sphinx-build -j auto -T -W --keep-going -b html {posargs} docs/ docs/_build/html
9497

9598
[testenv:docs-minimal]
96-
package= wheel
9799
passenv =
98100
EXPERIMENTS_DEV_DOCS
99101
PROD_BUILD
100102
RELEASE_STRING
101103
VERSION_STRING
102-
deps =
103-
{[testenv]deps}
104-
pyarrow
105104
setenv =
106105
QISKIT_DOCS_SKIP_EXECUTE = 1
107106
PYDEVD_DISABLE_FILE_VALIDATION = 1
108107
commands =
109108
sphinx-build -T -W --keep-going -b html {posargs} docs/ docs/_build/html
110109

111110
[testenv:docs-qiskit-main]
112-
package = editable
113111
passenv =
114112
EXPERIMENTS_DEV_DOCS
115113
PROD_BUILD
@@ -118,7 +116,6 @@ passenv =
118116
deps =
119117
{[testenv]deps}
120118
git+https://github.com/Qiskit/qiskit
121-
pyarrow
122119
setenv =
123120
PYDEVD_DISABLE_FILE_VALIDATION = 1
124121
commands =
@@ -127,5 +124,4 @@ commands =
127124
[testenv:docs-clean]
128125
skip_install = true
129126
deps =
130-
allowlist_externals = rm
131-
commands = rm -rf {toxinidir}/docs/stubs/ {toxinidir}/docs/_build
127+
commands = python -c "import shutil; shutil.rmtree('{toxinidir}/docs/stubs/', ignore_errors=True); shutil.rmtree('{toxinidir}/docs/_build', ignore_errors=True)"

0 commit comments

Comments
 (0)