3.23.1

This release bump was used to trigger the CI builds again and properly tag the image.

Full Changelog: 3.23.0...3.23.1

3.23.0

• remove daily ingest-file build
• add GHA that builds a docker image tagged ingest-file-base when new code is pushed to Dockerfile.base or to .github/workflows/docker-base.yml, and every day at midnight
• add .pre-commit-config.yaml (pip install pre-commit); create requirements.txt and requirements-dev.txt from pyproject.toml using the pre-commit hook && other convenience checks
• split the contents of the Dockerfile into Dockerfile.base which contails all dependencies aside from the ingestors library and Dockerfile which installs ingestors. Additionally, Dockerfile.test install the dev dependencies
• add build-macos to Makefile to address docker/buildx#426 (comment)
• add LD_PRELOAD paths for both MacOS and Linux systems in docker-compose.yml

What's Changed

• Chore/upgrade build by @simonwoerpel in #10

New Contributors

• @simonwoerpel made their first contribution in #10

Full Changelog: 3.22.2...3.23.0

v3.22.2 (security fix)

Release notes

• OpenAleph ingest-file 3.22.2 (this fork, pre-RabbitMQ)
• Aleph upstream ingest-file 4.1.2 <- use this if you are running Aleph v4

This version contains a patch for a security vulnerability in ingest-file, the component that processes files uploaded to Aleph and OpenAleph. We recommend that you update your instances you operate to use the latest patched release of ingest-file.

Please find detailed information about the patched vulnerability below.

How to update

If you operate Aleph using Docker Compose, update the ingest-file service in your Docker Compose configuration to use the image ghcr.io/openaleph/ingest-file:3.22.2.

Summary

Previous versions of ingest-file handled 7zip archives containing symbolic links insecurely. When processing 7zip archives, ingest-file followed symbolic links even if they were targeting files outside of the archive. A maliciously crafted archive would allow an attacker to access arbitrary files in the ingest-file container.

Depending on the exact configuration and deployment method, this might include:

• Access to files uploaded to Aleph if using the file archive (rather than object storage such as S3 or Google Cloud Storage) as the file archive is mounted into the container.
• Access to environment variables.
• Access to secrets mounted into the container.

Affected versions

All versions of ingest-file prior to 3.22.2 (this release) are affected.

Solution

ingest-file 3.22.2 and 4.1.2 contains a patch for the security vulnerability. 7zip archives containing symbolic links are now validated and archives containing symbolic links pointing to files outside of the archive are rejected.

Credits

• Responsibly disclosed by InterSecLab
• Patch by @catileptic
• Research, Testing, Validation: Alex Ștefănescu, Till Prochaska, Jan Strozyk, Friedrich Lindenberg, Simon Wörpel