Enable Multi-python support extension for Hue #4106
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Introduced infrastructure for building and packaging Hue against multiple Python versions (3.8, 3.9, 3.10, 3.11). - Refactored `Makefile`, `Makefile.vars`, and various component-level Makefiles to support per-Python virtual environments under `build/venvs/`. - Removed obsolete `requirements.txt` and `base_requirements.txt`; replaced with dynamic `generate_requirements.py` that resolves architecture- and Python-version-specific dependencies at build time. - Updated environment bootstrapping (`virtual-env`) logic: * More robust handling of Python/PIP binaries across platforms. * Support for `virtualenv-make-relocatable` and `.pth` path rewriting via a new `relocatable.py` tool. - Enhanced build toolchain for Cloudera: * `build_hue_cloudera.sh` and `build_hue_common.sh` now iterate builds over supported Python versions. * SQLite3 installation now checks for presence and adds proper bin path. * Redundant shell logic simplified; system-level dependency installation made cleaner. - Modernized PEP 517-compliant editable install (`pip install -e .`) for all Python apps/libs (removed `setup.py develop` usage). - Build output (`make prod`, `make release`) now clearly logs artifact paths and validates packaging exclusions. - Added `huecheck` target for post-build diagnostics.
ranade1
requested review from
wing2fly,
amitsrivastava,
athithyaaselvam,
agl29 and
Harshg999
|
|
amitsrivastava
approved these changes
Apr 11, 2025
Backend Code Coverage Report •
Pytest Report
|
athithyaaselvam
approved these changes
Apr 11, 2025
wing2fly
reviewed
Apr 11, 2025
| @@ -138,30 +144,25 @@ PYBABEL := $(ROOT)/build/env/bin/pybabel | |||
| ############################## | |||
| # Path to files for pip requirements | |||
| ############################## | |||
| REQUIREMENT_FILE := $(ROOT)/desktop/core/requirements.txt | |||
| REQUIREMENT_DOT_FILE := $(ROOT)/desktop/core/.requirements | |||
| REQUIREMENT_TEST_FILE := $(ROOT)/desktop/core/test_requirements.txt | |||
There was a problem hiding this comment.
@ranade1 did you add this file to the new generator? See here https://github.com/cloudera/hue/blob/master/Makefile#L114-L116. We need to install pytest packages.
agl29
reviewed
Apr 11, 2025
| export SYS_PYTHON="$PYTHON39_PATH/bin/python3.9" | ||
| export SYS_PIP="$PYTHON39_PATH/bin/pip3.9" | ||
| export VIRTUAL_ENV_VERSION="20.19.0" | ||
| elif [[ $PYTHON_VER == "python3.11" && ( $DOCKEROS == "redhat9" || $DOCKEROS == "redhat8" || $DOCKEROS == "sles15" ) ]]; then |
There was a problem hiding this comment.
Need to add $DOCKEROS == "redhat8-arm64" in $PYTHON_VER == "python3.11"
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Makefile,Makefile.vars, and various component-level Makefiles to support per-Python virtual environments underbuild/venvs/.requirements.txtandbase_requirements.txt; replaced with dynamicgenerate_requirements.pythat resolves architecture- and Python-version-specific dependencies at build time.virtual-env) logic:virtualenv-make-relocatableand.pthpath rewriting via a newrelocatable.pytool.build_hue_cloudera.shandbuild_hue_common.shnow iterate builds over supported Python versions.pip install -e .) for all Python apps/libs (removedsetup.py developusage).make prod,make release) now clearly logs artifact paths and validates packaging exclusions.huechecktarget for post-build diagnostics.How was this patch tested?
Please review Hue Contributing Guide before opening a pull request.