enable python 3.13t (freethreading) in binary validation workflow #3049
+8
−10
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary:
# context
* torchrec github actions re-uses lots of workflows from pytorch-test-infra
* in the build-wheels-linux it disables rocm
```
generate-matrix:
uses: pytorch/test-infra/.github/workflows/generate_binary_build_matrix.yml@main
with:
package-type: wheel
os: linux
test-infra-repository: pytorch/test-infra
test-infra-ref: main
with-rocm: false
```
* but in the validate-binaries it uses default, which generates rocm arch for testing
```
validate-binaries:
uses: pytorch/test-infra/.github/workflows/validate-domain-library.yml@main
with:
package_type: "wheel"
os: "linux"
channel: ${{ inputs.channel }}
repository: "pytorch/torchrec"
smoke_test: "source ./.github/scripts/validate_binaries.sh"
with_cuda: enable
with_rocm: false <---- this line is added by this diff
```
* since there's no ROCM support so the validate_binaries.sh script has to do an early return
```
if [[ ${MATRIX_GPU_ARCH_TYPE} = 'rocm' ]]; then
echo "We don't support rocm"
exit 0
fi
```
WARNING: although the validate-binaries.sh script does early return but it still consume github runner to set up the docker and so. Pure waste of resources.
* this entire thing is unnecessary if we just exclude ROCM arch in the config
Reviewed By: aporialiao
Differential Revision: D76107773
Summary: # context * python 3.13t was never supported in the github "binary validation" workflow due to lack of conda native support ``` $ conda create -n py313t python=3.13t Collecting package metadata (current_repodata.json): done Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: failed PackagesNotFoundError: The following packages are not available from current channels: - python=3.13t Current channels: - https://repo.anaconda.com/pkgs/main/linux-64 - https://repo.anaconda.com/pkgs/main/noarch - https://repo.anaconda.com/pkgs/r/linux-64 - https://repo.anaconda.com/pkgs/r/noarch To search for alternate channels that may provide the conda package you're looking for, navigate to https://anaconda.org and use the search bar at the top of the page. ``` * therefore the validation workflow for python3.13t is alway skipped (early return) ``` if [[ ${MATRIX_PYTHON_VERSION} = '3.13t' ]]; then echo "Conda doesn't support 3.13t yet, you can just try \`conda create -n test python=3.13t\`" exit 0 fi ``` * the workaround is to use conda-forge channel to get the freethreading python https://conda-forge.org/blog/2024/09/26/python-313/ ``` conda create -n py313 python=3.13 python-freethreading -c conda-forge ``` Differential Revision: D76109652
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D76109652 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
context
https://conda-forge.org/blog/2024/09/26/python-313/
Differential Revision: D76109652