Skip to content

Fix ImageMobject 3D rotation/flipping and remove resampling algorithms lanczos (antialias), box and hamming #4266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

chopan050
Copy link
Contributor

@chopan050 chopan050 commented Jun 4, 2025

Fixes #2412
Therefore, also fixes its duplicates #2979, #3131, #3559 and part of #3646.

As I commented in the "Image issues Tracking" issue #3482:

The main problem is that the implementation of the Camera.display_image_mobject() method is incomplete:

  • It only resizes and rotates the image.
  • It does not shear the image or apply perspective to it. Therefore, it always looks rectangular and faces the camera, even when you 3D rotate it. Take a look at this TODO.
  • In facing the camera, the image always preserves its 2D orientation (clockwise - counterclockwise). Thus, flipping the image (i.e. rotating it 180° about the Y axis) does not work as expected. A proper flipping would also change the image's orientation, which is currently not happening.

The solution I described, which was also discussed in some of the issues I mentioned before, consists of leveraging PIL.Image.Image.transform() to rewrite Camera.display_image_mobject(). Figuring out how to use PIL.Image.Image.transform() was the actual challenge, but this StackOverflow discussion shed light on this problem (thank you very much, mmgp ❤️).

I'll use the example provided by @hennels on issue #2412 (thanks!). I'm including the generated video after applying these changes.

There are still some issues, though:

  • The image is showing in front of the cube when it shouldn't. This is a known Manim issue which falls outside of this PR's scope.
  • There is a brief glitch at the beginning of this video when the image is perpendicular to the camera. Maybe floating point error? I couldn't figure out how to fix it, but this video is in medium quality and the glitch doesn't show up when I render the scene in high quality. Key point: high quality = more pixels = bigger pixel coordinates
ImageIn3D_new.mp4

However, this PR must introduce a...

BREAKING CHANGE 💥

PIL.Image.Image.transform() only supports the NEAREST, BILINEAR and BICUBIC resampling algorithms. It does not support BOX, HAMMING or LANCZOS. That is a big issue, since Camera.display_image_mobject() runs for every frame and you can't opt out of using it. Even in 2D scenes where there's no actual need for a perspective transform, one still has to account for possible shears of an image, which seems to require .transform() anyways. There doesn't seem to exist a proper fix to the 3D rotation / flipping / shear issue without using PIL.Image.Image.transform().

If such fix does not exist, then it is not possible to support BOX, HAMMING or LANCZOS. Therefore, I removed support for them in this PR by removing the following constants:

RESAMPLING_ALGORITHM["box"]
RESAMPLING_ALGORITHM["hamming"]
RESAMPLING_ALGORITHM["lanczos"]
RESAMPLING_ALGORITHM["antialias"]  # alias for "lanczos"

I also rewrote tests/test_graphical_units/test_img_and_svg.py::test_ImageInterpolation to remove those now unsupported resamping algorithms and rewrote AbstractImageMobject.set_resampling_algorithm() to better capture invalid values.

Reviewer Checklist

  • The PR title is descriptive enough for the changelog, and the PR is labeled correctly
  • If applicable: newly added non-private functions and classes have a docstring including a short summary and a PARAMETERS section
  • If applicable: newly added functions and classes are tested

@chopan050 chopan050 force-pushed the fix-image-rotation branch from 7b059ef to 6aea7b6 Compare June 5, 2025 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🆕 New
Development

Successfully merging this pull request may close these issues.

3D rotating and flipping does not work as expected for ImageMobject
1 participant