maps generation is intermitently failing

[benoit74]

No description provided.

[benoit74]

Problem is timeouts at install of packaging package, probably linked to slowness of storage machine

[rgaudin]

Indeed. Although error looks network related,

Collecting packaging
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe7f68144c0>: Failed to establish a new connection: [Errno -3] Try again')': /packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe7f6815210>: Failed to establish a new connection: [Errno -3] Try again')': /packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe7f6816200>: Failed to establish a new connection: [Errno -3] Try again')': /packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe7f68147c0>: Failed to establish a new connection: [Errno -3] Try again')': /packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe7f68174c0>: Failed to establish a new connection: [Errno -3] Try again')': /packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl
ERROR: Could not install packages due to an OSError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl (Caused by NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe7f68162f0>: Failed to establish a new connection: [Errno -3] Try again'))


[notice] A new release of pip is available: 23.0.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
stream closed EOF for zim/maps-generator-29039820-qd8bj (alpine)

I could successfully install it on very same image using very same command though…

❯ docker run -it docker.io/library/python:3.10-alpine /bin/sh
Unable to find image 'python:3.10-alpine' locally
3.10-alpine: Pulling from library/python
f18232174bc9: Pull complete
3588bd7cf5ec: Pull complete
3d2e6691f0b2: Pull complete
e5c20ecf2705: Pull complete
Digest: sha256:4c4097e46608e9b9025a486e3b72d628ac8947a3caa20f142c2e8c0029a12b21
Status: Downloaded newer image for python:3.10-alpine
/ # python3 -m pip install packaging
Collecting packaging
  Downloading packaging-24.2-py3-none-any.whl (65 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 515.0 kB/s eta 0:00:00
Installing collected packages: packaging
Successfully installed packaging-24.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip

We can see that even the last working run had to retry the installation

Collecting packaging
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa5493084c0>: Failed to establish a new connection: [Errno -3] Try again')': /packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl
  Downloading packaging-24.2-py3-none-any.whl (65 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 46.6 kB/s eta 0:00:00
Installing collected packages: packaging
Successfully installed packaging-24.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
Generating releases map for Kiwix…
Rewriting /data/maps/kiwix-releases.map...
> OK. Wrote 52 redirects
Generating nightly map for Kiwix…
> OK. Wrote 47 redirects
Generating releases map for openZIM…
Rewriting /data/maps/openzim-releases.map...
> OK. Wrote 34 redirects
Generating nightyly map for openZIM…
> OK. Wrote 27 redirects
Starting create_wp1_redirects...
> done
stream closed EOF for zim/maps-generator-29046840-bqhxk (alpine)