Rotated bboxes transforms #9104

AntoineSimoulin · 2025-06-10T22:11:23Z

Add Transforms support for Rotated Boxes

This PR implements the last transforms for rotated boxes and follows what has been implemented in #9095 and #9084. This PR implements in particular the following modifications :

Add support for perspective for rotated boxes;
Fix missing tests for affine transformation and rotated boxes;
Fix the _affine_bounding_boxes_with_expand function for rotated boxes when expand=True;
Fix clamp_bounding_boxes function with behavior detailed below;
Add support for elastic for rotated boxes;
Add support for crop for rotated boxes.
Add missing tests for TestConvertBoundingBoxFormat;
Remove the SUPPORTED_BOX_FORMATS and NEW_BOX_FORMATS variable in the tests as tests for transform now full cover rotated boxes
Add support for sanitize for rotated boxes

Details on the clamping function

For the clamping, we re-order the point of the box such that the point with the lowest value on the x-axis is the point 1 (c.f. _order_bounding_boxes_points). Given the position of the 4 vertices with respect to the y-axis (c.f. cases above), we are going to adjust the points (x1, y1), (x2, y2), and (x4, y4) to make sure the point (x1, y1) is on the right side of the y-axis. We loop through the four vertices of the rotated box and apply the same operation. In the end we are guaranteed that the bounding box will be within the canvas size and will be completely included within the area of the original box.

We propose some illustration examples below (original boxes in grey and corresponding clamped boxes in blue.

Please note that depending on the order in which we loop through the vertices, we are not guaranteed the output boxes is the box with the largest area that meet the condition above (we might be too aggressive with the clamping. This can occur if the box is largely out of bounds along multiple axis).

Test plan

Please run the following tests:

pytest test/test_transforms_v2.py -vvv -k "TestPerspective and test_kernel_bounding_boxes"
pytest test/test_transforms_v2.py -vvv -k "TestPerspective and test_correctness_perspective_bounding_boxes"

pytest test/test_transforms_v2.py -vvv -k "TestAffine and test_transform_bounding_boxes_correctness"

pytest test/test_transforms_v2.py -vvv -k "TestRotate and test_kernel_bounding_boxes"
pytest test/test_transforms_v2.py -vvv -k "TestRotate and test_functional_bounding_boxes_correctness"
pytest test/test_transforms_v2.py -vvv -k "TestRotate and test_transform_bounding_boxes_correctness"

pytest test/test_transforms_v2.py -vvv -k "TestClampBoundingBoxes and test_kernel"
pytest test/test_transforms_v2.py -vvv -k "TestClampBoundingBoxes and test_functional"

pytest test/test_transforms_v2.py -vvv -k "TestElastic and test_kernel_bounding_boxes"

pytest test/test_transforms_v2.py -vvv -k "TestConvertBoundingBoxFormat and test_kernel"
pytest test/test_transforms_v2.py -vvv -k "TestConvertBoundingBoxFormat and test_kernel_noop"

Test Plan: ```bash pytest test/test_transforms_v2.py -vvv -k "TestPerspective and test_kernel_bounding_boxes" pytest test/test_transforms_v2.py -vvv -k "TestPerspective and test_correctness_perspective_bounding_boxes" ```

Test Plan: Unit tests: ```bash pytest test/test_transforms_v2.py -vvv -k "TestAffine and test_transform_bounding_boxes_correctness" ```

Test Plan: Unit tests: ```bash pytest test/test_transforms_v2.py -vvv -k "TestRotate and test_kernel_bounding_boxes" pytest test/test_transforms_v2.py -vvv -k "TestRotate and test_functional_bounding_boxes_correctness" pytest test/test_transforms_v2.py -vvv -k "TestRotate and test_transform_bounding_boxes_correctness" ```

Test Plan: Unit tests: ```bash pytest test/test_transforms_v2.py -vvv -k "TestClampBoundingBoxes and test_kernel" pytest test/test_transforms_v2.py -vvv -k "TestClampBoundingBoxes and test_functional" ```

Test Plan: Unit tests: ```bash pytest test/test_transforms_v2.py -vvv -k "TestElastic and test_kernel_bounding_boxes" ```

Test Plan: Unit tests: ```bash pytest test/test_transforms_v2.py -vvv -k "TestConvertBoundingBoxFormat and test_kernel" pytest test/test_transforms_v2.py -vvv -k "TestConvertBoundingBoxFormat and test_kernel_noop" ```

pytorch-bot · 2025-06-10T22:11:26Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/9104

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 8 New Failures

As of commit 4a02ba0 with merge base fcca6ff ():

NEW FAILURES - The following jobs have failed:

Tests / unittests-linux (3.9, linux.g5.4xlarge.nvidia.gpu, cuda, 11.8) / linux-job (gh)
test/test_transforms_v2.py::TestClampBoundingBoxes::test_kernel[cuda-dtype0-BoundingBoxFormat.CXCYWHR]
Tests / unittests-macos (3.10, macos-m1-stable) / macos-job (gh)
test/test_transforms_v2.py::TestCrop::test_transform_bounding_boxes_correctness[3-cpu-dtype0-BoundingBoxFormat.CXCYWHR-output_size1]
Tests / unittests-macos (3.12, macos-m1-stable) / macos-job (gh)
test/test_transforms_v2.py::TestCrop::test_transform_bounding_boxes_correctness[3-cpu-dtype0-BoundingBoxFormat.CXCYWHR-output_size1]
Tests / unittests-macos (3.9, macos-m1-stable) / macos-job (gh)
test/test_transforms_v2.py::TestCrop::test_transform_bounding_boxes_correctness[3-cpu-dtype0-BoundingBoxFormat.CXCYWHR-output_size1]
Tests / unittests-windows (3.10, windows.4xlarge, cpu) / windows-job (gh)
test/test_transforms_v2.py::TestCrop::test_transform_bounding_boxes_correctness[3-cpu-dtype0-BoundingBoxFormat.CXCYWHR-output_size1]
Tests / unittests-windows (3.11, windows.4xlarge, cpu) / windows-job (gh)
test/test_transforms_v2.py::TestCrop::test_transform_bounding_boxes_correctness[3-cpu-dtype0-BoundingBoxFormat.CXCYWHR-output_size1]
Tests / unittests-windows (3.12, windows.4xlarge, cpu) / windows-job (gh)
test/test_transforms_v2.py::TestCrop::test_transform_bounding_boxes_correctness[3-cpu-dtype0-BoundingBoxFormat.CXCYWHR-output_size1]
Tests / unittests-windows (3.9, windows.4xlarge, cpu) / windows-job (gh)
test/test_transforms_v2.py::TestCrop::test_transform_bounding_boxes_correctness[3-cpu-dtype0-BoundingBoxFormat.CXCYWHR-output_size1]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

torchvision/transforms/v2/functional/_meta.py

NicolasHug · 2025-06-11T10:11:07Z

torchvision/transforms/v2/functional/_meta.py

+    cond_a = x1.lt(0).logical_and(x2.ge(0)).logical_and(x3.ge(0)).logical_and(x4.ge(0))
+    cond_a = cond_a.logical_and(area(case_a) > area(case_b))
+    cond_a = cond_a.logical_or(x1.lt(0).logical_and(x2.ge(0)).logical_and(x3.ge(0)).logical_and(x4.le(0)))
+    cond_b = x1.lt(0).logical_and(x2.ge(0)).logical_and(x3.ge(0)).logical_and(x4.ge(0))
+    cond_b = cond_b.logical_and(area(case_a) <= area(case_b))
+    cond_b = cond_b.logical_or(x1.lt(0).logical_and(x2.le(0)).logical_and(x3.ge(0)).logical_and(x4.ge(0)))
+    cond_c = x1.lt(0).logical_and(x2.le(0)).logical_and(x3.ge(0)).logical_and(x4.le(0))
+    cond_d = x1.lt(0).logical_and(x2.le(0)).logical_and(x3.le(0)).logical_and(x4.le(0))


For all of these, is there a particular reasons to use the methods? If not maybe we can rely on the plain operators like < etc.?

I am not sure how to do this as the operation needs to be applied along the axis. Operators such as AND and OR typically reduce the results to a single bolean value. So we will need to use logical_and. I can eventually refactor with < and > if this makes the code more readable.

test/test_transforms_v2.py

AntoineSimoulin · 2025-06-13T03:17:40Z

Hey @NicolasHug I publish a fix which should fix the test and address your comments. Here is the list of the modifications:

Modify the make_bounding_boxes function to add clamping and padding, this ensuring that rotated boxes are build within the range of the canvas size;
Re-placing the reference_perspective_bounding_boxes function within the TestPerspective class to reduce the number of lines modified in this PR and since this function is only used within the class;
Decreasing the tightness for the test in TestAffine to atol=1e-5, rtol=2e-5 as the rotation angle had slightly higher variation when computed with the test function. Also let some tolerance for TestConvertBoundingBoxFormat;
Not applying the function _parallelogram_to_bounding_boxes to int rotated box as the truncation of the point from float to int does not preserve the rectangular shape of the box;
Apply clamping after resizing rotated bounding boxes;
Improve docstring for the _clamp_rotated_bounding_boxes function.

Please run the tests with:

pytest test/test_transforms_v2.py -k box -v
...
2372 passed, 1432 skipped, 5025 deselected in 67.68s (0:01:07)

NicolasHug · 2025-06-13T11:55:04Z

torchvision/transforms/v2/functional/_geometry.py

+    if int_dtype:
+        # Does not apply the transformation to `int` boxes as the rounding error
+        # will typically not ensure the resulting box has a rectangular shape.
+        return parallelogram.clone()


Is it better to return the parallelogram as-is, i.e. really not a rectangle, or still try to do the conversion and return something that is closer to a rectangle?

Separately it makes me wonder, maybe we should completely prevent rotated bounding boxes of integer dtype? That would probably make our life a lot easier, and users probably should be running the whole transform pipeline in float anyway, so as to avoid rounding errors compounding?

AntoineSimoulin added 8 commits June 10, 2025 14:48

Update perspective_bounding_boxes for rotated boxes

580ae4b

Test Plan: ```bash pytest test/test_transforms_v2.py -vvv -k "TestPerspective and test_kernel_bounding_boxes" pytest test/test_transforms_v2.py -vvv -k "TestPerspective and test_correctness_perspective_bounding_boxes" ```

Add affine transformation tests for rotated boxes

8dc9ce4

Test Plan: Unit tests: ```bash pytest test/test_transforms_v2.py -vvv -k "TestAffine and test_transform_bounding_boxes_correctness" ```

Update clamp_bounding_boxes for rotated boxes

1105aa1

Test Plan: Unit tests: ```bash pytest test/test_transforms_v2.py -vvv -k "TestClampBoundingBoxes and test_kernel" pytest test/test_transforms_v2.py -vvv -k "TestClampBoundingBoxes and test_functional" ```

Update elastic_bounding_boxes for rotate boxes

3966ff2

Test Plan: Unit tests: ```bash pytest test/test_transforms_v2.py -vvv -k "TestElastic and test_kernel_bounding_boxes" ```

Add convert tests for rotated boxes

5a35f3f

Test Plan: Unit tests: ```bash pytest test/test_transforms_v2.py -vvv -k "TestConvertBoundingBoxFormat and test_kernel" pytest test/test_transforms_v2.py -vvv -k "TestConvertBoundingBoxFormat and test_kernel_noop" ```

Remove unused SUPPORTED_BOX_FORMATS variable

72b33b0

Update sanitize_bounding_boxes for rotated boxes

dffd5ae

facebook-github-bot added the cla signed label Jun 10, 2025

NicolasHug reviewed Jun 11, 2025

View reviewed changes

Fix failing tests and answer comments

aac8e1e

Simplify elastic test

4a02ba0

NicolasHug approved these changes Jun 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Rotated bboxes transforms #9104

Rotated bboxes transforms #9104

Uh oh!

AntoineSimoulin commented Jun 10, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Jun 10, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NicolasHug Jun 11, 2025

Uh oh!

AntoineSimoulin Jun 13, 2025

Uh oh!

Uh oh!

AntoineSimoulin commented Jun 13, 2025 •

edited

Loading

Uh oh!

NicolasHug Jun 13, 2025 •

edited

Loading

Uh oh!

Uh oh!

Rotated bboxes transforms #9104

Are you sure you want to change the base?

Rotated bboxes transforms #9104

Uh oh!

Conversation

AntoineSimoulin commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add Transforms support for Rotated Boxes

Details on the clamping function

Test plan

Uh oh!

pytorch-bot bot commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/9104

❌ 8 New Failures

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NicolasHug Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

AntoineSimoulin Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AntoineSimoulin commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NicolasHug Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AntoineSimoulin commented Jun 10, 2025 •

edited

Loading

pytorch-bot bot commented Jun 10, 2025 •

edited

Loading

AntoineSimoulin commented Jun 13, 2025 •

edited

Loading

NicolasHug Jun 13, 2025 •

edited

Loading