fix: [native_iceberg_compat / native_datafusion] Fall back to Spark for maps containing structs

[andygrove]

Which issue does this PR close?

Workaround for #1754

Rationale for this change

Fix the following Spark SQL tests in core-2.

- Spark vectorized reader - without partition data column - select a single complex field from a map entry and in clause *** FAILED *** (290 milliseconds)
- Spark vectorized reader - with partition data column - select a single complex field from a map entry and in clause *** FAILED *** (269 milliseconds)
- Non-vectorized reader - without partition data column - select a single complex field from a map entry and in clause *** FAILED *** (263 milliseconds)
- Non-vectorized reader - with partition data column - select a single complex field from a map entry and in clause *** FAILED *** (322 milliseconds)
- Spark vectorized reader - without partition data column - select nested field from a complex map key using map_keys *** FAILED *** (315 milliseconds)
- Spark vectorized reader - with partition data column - select nested field from a complex map key using map_keys *** FAILED *** (278 milliseconds)
- Non-vectorized reader - without partition data column - select nested field from a complex map key using map_keys *** FAILED *** (261 milliseconds)
- Non-vectorized reader - with partition data column - select nested field from a complex map key using map_keys *** FAILED *** (312 milliseconds)
- Spark vectorized reader - without partition data column - select nested field from a complex map value using map_values *** FAILED *** (268 milliseconds)
- Spark vectorized reader - with partition data column - select nested field from a complex map value using map_values *** FAILED *** (262 milliseconds)
- Non-vectorized reader - without partition data column - select nested field from a complex map value using map_values *** FAILED *** (264 milliseconds)
- Non-vectorized reader - with partition data column - select nested field from a complex map value using map_values *** FAILED *** (308 milliseconds)
- Spark vectorized reader - without partition data column - SPARK-40033: Schema pruning support through element_at *** FAILED *** (554 milliseconds)
- Spark vectorized reader - with partition data column - SPARK-40033: Schema pruning support through element_at *** FAILED *** (510 milliseconds)
- Non-vectorized reader - without partition data column - SPARK-40033: Schema pruning support through element_at *** FAILED *** (525 milliseconds)
- Non-vectorized reader - with partition data column - SPARK-40033: Schema pruning support through element_at *** FAILED *** (488 milliseconds)

What changes are included in this PR?

How are these changes tested?

[comphead]

thanks @andygrove

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 87.50000% with 1 line in your changes missing coverage. Please review.

Project coverage is 58.62%. Comparing base (f09f8af) to head (843f9e8).
Report is 205 commits behind head on main.

Files with missing lines	Patch %	Lines
.../main/scala/org/apache/comet/DataTypeSupport.scala	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #1764      +/-   ##
============================================
+ Coverage     56.12%   58.62%   +2.49%     
- Complexity      976     1138     +162     
============================================
  Files           119      130      +11     
  Lines         11743    12671     +928     
  Branches       2251     2367     +116     
============================================
+ Hits           6591     7428     +837     
- Misses         4012     4058      +46     
- Partials       1140     1185      +45     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kazuyukitanimura]

Now we are doing the data type support check for maps of struct. Dow we still have to ignore these tests?
If so, is there a way to ignore only for native datafusion/iceberg-compat?

[andygrove]

This test suite is specific to native datafusion/iceberg-compat scans. The fallback is just a temporary workaround until we have the real fix for #1754, then we can re-enable these tests.

[andygrove]

One test keeps failing without any actual test failures. This is happening on other PRs as well.

[kazuyukitanimura]

pending with CI

[andygrove]

Moving this to draft because we may want to merge #1771 instead