Skip to content

build: use poetry for building #66

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 12, 2021
Merged

build: use poetry for building #66

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 41 additions & 57 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,61 +9,45 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ 3.6, 3.7, 3.8, 3.9 ]
python-version: [3.7, 3.8, 3.9]
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Patch for Python 3.6
if: matrix.python-version == 3.6
run: |
pip install strip-hints==0.1.9
./patch-python3.6.sh

git config --global user.email "[email protected]"
git config --global user.name "Github Actions"

git add --all
git commit --message "Python 3.6 patch"
latest_tag=$(git describe --abbrev=0 --tags)
git tag --force ${latest_tag}

- name: Install dependencies
run: |
pip install -r requirements.txt

- name: Install dev dependencies
run: |
pip install -r dev_requirements.txt

- name: Test
run: |
pytest

- name: Test install package
env:
PYTHON_VERSION: ${{ matrix.python-version }}
run: |
python setup.py bdist_wheel --python-tag py${PYTHON_VERSION//.}

- name: Test install package
run: |
pip install virtualenv
virtualenv installable
source installable/bin/activate

pip install dist/$(ls dist)

mkdir test_install
cd test_install
python -c "import datastream"

- name: PyPi Publish
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
run:
twine upload dist/*
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
source $HOME/.poetry/env
poetry install

- name: Run tests
run: |
source $HOME/.poetry/env
poetry run pytest

- name: Build wheels
run: |
source $HOME/.poetry/env
poetry version $(git tag --points-at HEAD)
poetry build

- name: Test install package
run: |
source $HOME/.poetry/env
mkdir test_install
cd test_install
poetry init
poetry add ../dist/$(ls dist/*.whl)

poetry run python -c "import datastream"

- name: Upload
env:
USERNAME: __token__
PASSWORD: ${{ secrets.PYPI_TOKEN }}
run: |
source $HOME/.poetry/env
poetry publish --username=$USERNAME --password=$PASSWORD
114 changes: 63 additions & 51 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,58 +3,70 @@ name: Test
on: [push]

jobs:
build:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ 3.6, 3.7, 3.8, 3.9 ]
python-version: [3.7, 3.8, 3.9]
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Cache pip
uses: actions/cache@v2
with:
# This path is specific to Ubuntu
path: ~/.cache/pip
# Look to see if there is a cache hit for the corresponding requirements file
key: ${{ runner.os }}-python${{ matrix.python-version}}-pip-${{ hashFiles('requirements.txt') }}-${{ hashFiles('dev_requirements.txt') }}-${ GITHUB_REF }

- name: Patch for Python 3.6
if: matrix.python-version == 3.6
run: |
pip install strip-hints==0.1.9
./patch-python3.6.sh

- name: Install dependencies
run: |
pip install -r requirements.txt

- name: Install dev dependencies
run: |
pip install -r dev_requirements.txt

- name: Test
run: |
pytest

- name: Build package
env:
PYTHON_VERSION: ${{ matrix.python-version }}
run: |
python setup.py bdist_wheel --python-tag py${PYTHON_VERSION//.}

- name: Test install package
run: |
pip install virtualenv
virtualenv installable
source installable/bin/activate

pip install dist/$(ls dist)

mkdir test_install
cd test_install
python -c "import datastream"
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Cache pip
uses: actions/cache@v2
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('pyproject.toml') }}-${{ hashFiles('poetry.lock') }}-${ GITHUB_REF }
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-

- name: Install dependencies
run: |
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
source $HOME/.poetry/env
poetry install

- name: Run tests
run: |
source $HOME/.poetry/env
poetry run pytest

- name: Build wheels
run: |
source $HOME/.poetry/env
poetry build

build-docs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.8]

steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Cache pip
uses: actions/cache@v2
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('docs/source/requirements.txt') }}-${ GITHUB_REF }
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r docs/source/requirements.txt

- name: Build html
run: |
(cd docs && make html)
6 changes: 6 additions & 0 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ sampling, and finally converting to a ``torch.utils.data.DataLoader``.
Install
=======

.. code-block::

poetry add pytorch-datastream

Or, for the old-timers:

.. code-block::

pip install pytorch-datastream
Expand Down
3 changes: 0 additions & 3 deletions dev_requirements.txt
View file
Open in desktop

This file was deleted.

6 changes: 3 additions & 3 deletions docs/source/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ mccabe==0.6.1
more-itertools==8.3.0
numpy==1.18.5
packaging==20.4
pandas==1.0.4
pandas==1.1.5
pkginfo==1.5.0.1
pluggy==0.13.1
py==1.10.0
Expand All @@ -34,14 +34,14 @@ pydantic==1.8.2
Pygments==2.7.4
pylint==2.5.3
pyparsing==2.4.7
pyspark==2.4.1
pyspark==3.0.3
pytest==5.4.3
python-dateutil==2.8.1
pytz==2020.1
PyYAML==5.4
readme-renderer==26.0
recommonmark==0.6.0
requests==2.23.0
requests==2.26.0
requests-toolbelt==0.9.1
SecretStorage==3.1.2
six==1.15.0
Expand Down
17 changes: 0 additions & 17 deletions patch-python3.6.sh
View file
Open in desktop

This file was deleted.

Loading