XTable 3rd Release

[vinishjail97]

XTable 3rd release updates.

[vinishjail97]

Release Notes for release-0.3.0-incubating-rc1

Apache XTable™ (Incubating) is a cross-table converter for table formats that facilitates omni-directional interoperability across data processing systems and query engines. Currently, Apache XTable™ supports widely adopted open-source table formats such as Apache Hudi, Apache Iceberg, and Delta Lake.

✨ What's Changed

🔄 Core Functionality Enhancements

• Catalog Sync:

  • Introduced CatalogSyncClient and CatalogSync interfaces.
  • Implemented Glue and HMS catalog sync for Iceberg, Delta, and Hudi.
  • Additional information can be found in docs.

• Table Format Sync Improvements:

  • Added support for continuous sync using RunSync.
  • Support restore/rollback sync during conversion across all three formats.
  • Added support for bucket partition transform (Iceberg → Delta).
  • Synced schema field comments and record counts in Delta source.
  • Improved decimal field scale handling in iceberg and fixed bug in column stats.
  • Supporting timestamp_ntz type in conversion logic.

• Breaking changes

  • We have refactored SyncStatusCode and ErrorDetails into dedicated classes, refactor code by fixing the import path if you are using these classes.

🧪 Testing & Quality

• Fixed flaky tests and normalized base paths for test stability.
• Addressed minor issues found via static analysis tooling.
• Dockerfile syntax fix and test class name corrections.

📘 Documentation & Community

• Added downloads page for 0.2.0-incubating release.
• Introduced RFC template and new change proposals.
• Updated community sync and blog pages.

🧰 Infrastructure & Maintenance

• Upgraded parquet-avro to 1.15.1 to fix critical vulnerabilities found recently.
• Upgraded Hadoop and AWS SDK dependencies.
• Refactored RunSync for better modularity and reusability.

Detailed Change Log

• Iceberg and Delta target: sync schema field comments by @emilie-wang in Iceberg and Delta target: sync schema field comments #575
• [551] Add downloads page for 0.2.0-incubating release by @vinishjail97 in [551] Add downloads page for 0.2.0-incubating release #576
• [551] Try enabling keep_files as false in GH action for downloads by @vinishjail97 in [551] Try enabling keep_files as false in GH action for downloads #577
• [DOCS] Remove info on snowflake public preview for Iceberg by @sagarlakshmipathy in [DOCS] Remove info on snowflake public preview for Iceberg #580
• Add 2 New Community Blogs to the Apache XTable Blogs section by @dipankarmazumdar in Add 2 New Community Blogs to the Apache XTable Blogs section #585
• Site content is not deployed due to missing .asf.yaml file by @zabetak in Site content is not deployed due to missing .asf.yaml file #587
• Execute correct delete operation in delta tests by @ashvina in Execute correct delete operation in delta tests #589
• Fix commit log parsing of Delta tables with delete vector by @ashvina in Fix commit log parsing of Delta tables with delete vector #596
• Normalize basePath in targetTables in ConversionController by @vinishjail97 in Normalize basePath in targetTables in ConversionController #602
• [590] Add RFC template for XTable by @vinishjail97 in [590] Add RFC template for XTable #604
• [590] Add interfaces for CatalogSyncClient and CatalogSync by @vinishjail97 in [590] Add interfaces for CatalogSyncClient and CatalogSync #603
• [590] Add RFC template for XTable by @vinishjail97 in [590] Add RFC template for XTable  #610
• [590] Hadoop and AWS sdk version upgrade by @kroushan-nit in [590] Hadoop and AWS sdk version upgrade #614
• [619] Maintenance: Update dependencies, fix test class names by @the-other-tim-brown in [619] Maintenance: Update dependencies, fix test class names #620
• [608] Ensure column stats for decimal fields have proper scale set by @the-other-tim-brown in [608] Ensure column stats for decimal fields have proper scale set #617
• [615] Add Iceberg stats when reading snapshot by @the-other-tim-brown in [615] Add Iceberg stats when reading snapshot #616
• Add community sync page for XTable by @vinishjail97 in Add community sync page for XTable #625
• Update GMeet link and add today's recording by @vinishjail97 in Update GMeet link and add today's recording #628
• [590] Add RFC for XCatalogSync - Synchronize tables across catalogs by @vinishjail97 in [590] Add RFC for XCatalogSync - Synchronize tables across catalogs #605
• [590] Add RunCatalogSync utility for synchronizing tables across catalogs by @vinishjail97 in [590] Add RunCatalogSync utility for synchronizing tables across catalogs #591
• Delta source: add record count for DataFile by @emilie-wang in Delta source: add record count for DataFile #623
• [site] Update blogs by @bhasudha in [site] Update blogs #635
• [590] Add Iceberg Glue Catalog Sync implementation by @kroushan-nit in [590] Add Iceberg Glue Catalog Sync implementation #632
• [590] Add Iceberg HMS Catalog Sync implementation by @kroushan-nit in [590] Add Iceberg HMS Catalog Sync implementation #633
• [590] Add Delta Glue Catalog Sync implementation by @kroushan-nit in [590] Add Delta Glue Catalog Sync implementation #637
• [590] Add Delta HMS Catalog Sync implementation by @kroushan-nit in [590] Add Delta HMS Catalog Sync implementation #638
• [590] Add Hudi HMS Catalog Sync implementation by @vamsikarnika in [590] Add Hudi HMS Catalog Sync implementation #639
• Update xtable-api sources by @emmanuel-ferdman in Update xtable-api sources #651
• [Enhancement] Support restore/rollback sync during conversion (1/2) by @danielhumanmod in [Enhancement] Support restore/rollback sync during conversion (1/2) #569
• [590] Add Hudi HMS Catalog Sync Implementation by @vamsikarnika in [590] Add Hudi HMS Catalog Sync Implementation #648
• [590] Add Hudi Glue Catalog Sync Implementation by @vamsikarnika in [590] Add Hudi Glue Catalog Sync Implementation #649
• Add change proposal for conversion of deletion vectors by @ashvina in Add change proposal for conversion of deletion vectors #634
• Add base representation for storage files by @ashvina in Add base representation for storage files #653
• Fix flaky test, ignore order of elements in list validation by @ashvina in Fix flaky test, ignore order of elements in list validation #662
• [594] Support continuous conversion with the RunSync tool by @the-other-tim-brown in [594] Support continuous conversion with the RunSync tool #593
• [MINOR] fixes recommended from static analysis tooling by @the-other-tim-brown in [MINOR] fixes recommended from static analysis tooling #664
• Modified incremental safe check which does not depend on snapshots list order by @brishi19791 in Modified incremental safe check which does not depend on snapshots list order #665
• [668] Support bucket partition transform for Iceberg Sources and Delta Targets by @the-other-tim-brown in [668] Support bucket partition transform for Iceberg Sources and Delta Targets #670
• Handle timestamp_ntz in delta and iceberg by @vinishjail97 in Handle timestamp_ntz in delta and iceberg #647
• [MINOR] Fix docker file syntax issue by @the-other-tim-brown in [MINOR] Fix docker file syntax issue #674
• [528] Refactor RunSync Class for Improved Modularity and Reusability by @vaibhavk1992 in [528] Refactor RunSync Class for Improved Modularity and Reusability #675
• Add how-to docs for catalog sync by @vinishjail97 in Add how-to docs for catalog sync #673
• [MINOR] Add mailing list for discussions, update issues list in asf.yaml by @the-other-tim-brown in [MINOR] Add mailing list for discussions, update issues list in asf.yaml #676
• [MINOR] Fix mailing list in asf.yaml by @the-other-tim-brown in [MINOR] Fix mailing list in asf.yaml #677

New Contributors

• @kroushan-nit made their first contribution in [590] Hadoop and AWS sdk version upgrade #614
• @bhasudha made their first contribution in [site] Update blogs #635
• @brishi19791 made their first contribution in Modified incremental safe check which does not depend on snapshots list order #665
• @vaibhavk1992 made their first contribution in [528] Refactor RunSync Class for Improved Modularity and Reusability #675

Full Changelog: 0.2.0-incubating...0.3.0-incubating-rc1

[vinishjail97]

Release Notes for release-0.3.0-incubating-rc2

Apache XTable™ (Incubating) is a cross-table converter for table formats that facilitates omni-directional interoperability across data processing systems and query engines. Currently, Apache XTable™ supports widely adopted open-source table formats such as Apache Hudi, Apache Iceberg, and Delta Lake.

✨ What's Changed

🔄 Core Functionality Enhancements

• Catalog Sync:

  • Introduced CatalogSyncClient and CatalogSync interfaces.
  • Implemented Glue and HMS catalog sync for Iceberg, Delta, and Hudi.
  • Additional information can be found in docs.

• Table Format Sync Improvements:

  • Added support for continuous sync using RunSync.
  • Support restore/rollback sync during conversion across all three formats.
  • Added support for bucket partition transform (Iceberg → Delta).
  • Synced schema field comments and record counts in Delta source.
  • Improved decimal field scale handling in iceberg and fixed bug in column stats.
  • Supporting timestamp_ntz type in conversion logic.

• Breaking changes

  • We have refactored SyncStatusCode and ErrorDetails into dedicated classes, refactor code by fixing the import path if you are using these classes.

🧪 Testing & Quality

• Fixed flaky tests and normalized base paths for test stability.
• Addressed minor issues found via static analysis tooling.
• Dockerfile syntax fix and test class name corrections.

📘 Documentation & Community

• Added downloads page for 0.2.0-incubating release.
• Introduced RFC template and new change proposals.
• Updated community sync and blog pages.

🧰 Infrastructure & Maintenance

• Upgraded parquet-avro to 1.15.1 to fix critical vulnerabilities found recently.
• Upgraded Hadoop and AWS SDK dependencies.
• Refactored RunSync for better modularity and reusability.

Detailed Change Log

• Iceberg and Delta target: sync schema field comments by @emilie-wang in Iceberg and Delta target: sync schema field comments #575
• [551] Add downloads page for 0.2.0-incubating release by @vinishjail97 in [551] Add downloads page for 0.2.0-incubating release #576
• [551] Try enabling keep_files as false in GH action for downloads by @vinishjail97 in [551] Try enabling keep_files as false in GH action for downloads #577
• [DOCS] Remove info on snowflake public preview for Iceberg by @sagarlakshmipathy in [DOCS] Remove info on snowflake public preview for Iceberg #580
• Add 2 New Community Blogs to the Apache XTable Blogs section by @dipankarmazumdar in Add 2 New Community Blogs to the Apache XTable Blogs section #585
• Site content is not deployed due to missing .asf.yaml file by @zabetak in Site content is not deployed due to missing .asf.yaml file #587
• Execute correct delete operation in delta tests by @ashvina in Execute correct delete operation in delta tests #589
• Fix commit log parsing of Delta tables with delete vector by @ashvina in Fix commit log parsing of Delta tables with delete vector #596
• Normalize basePath in targetTables in ConversionController by @vinishjail97 in Normalize basePath in targetTables in ConversionController #602
• [590] Add RFC template for XTable by @vinishjail97 in [590] Add RFC template for XTable #604
• [590] Add interfaces for CatalogSyncClient and CatalogSync by @vinishjail97 in [590] Add interfaces for CatalogSyncClient and CatalogSync #603
• [590] Add RFC template for XTable by @vinishjail97 in [590] Add RFC template for XTable  #610
• [590] Hadoop and AWS sdk version upgrade by @kroushan-nit in [590] Hadoop and AWS sdk version upgrade #614
• [619] Maintenance: Update dependencies, fix test class names by @the-other-tim-brown in [619] Maintenance: Update dependencies, fix test class names #620
• [608] Ensure column stats for decimal fields have proper scale set by @the-other-tim-brown in [608] Ensure column stats for decimal fields have proper scale set #617
• [615] Add Iceberg stats when reading snapshot by @the-other-tim-brown in [615] Add Iceberg stats when reading snapshot #616
• Add community sync page for XTable by @vinishjail97 in Add community sync page for XTable #625
• Update GMeet link and add today's recording by @vinishjail97 in Update GMeet link and add today's recording #628
• [590] Add RFC for XCatalogSync - Synchronize tables across catalogs by @vinishjail97 in [590] Add RFC for XCatalogSync - Synchronize tables across catalogs #605
• [590] Add RunCatalogSync utility for synchronizing tables across catalogs by @vinishjail97 in [590] Add RunCatalogSync utility for synchronizing tables across catalogs #591
• Delta source: add record count for DataFile by @emilie-wang in Delta source: add record count for DataFile #623
• [site] Update blogs by @bhasudha in [site] Update blogs #635
• [590] Add Iceberg Glue Catalog Sync implementation by @kroushan-nit in [590] Add Iceberg Glue Catalog Sync implementation #632
• [590] Add Iceberg HMS Catalog Sync implementation by @kroushan-nit in [590] Add Iceberg HMS Catalog Sync implementation #633
• [590] Add Delta Glue Catalog Sync implementation by @kroushan-nit in [590] Add Delta Glue Catalog Sync implementation #637
• [590] Add Delta HMS Catalog Sync implementation by @kroushan-nit in [590] Add Delta HMS Catalog Sync implementation #638
• [590] Add Hudi HMS Catalog Sync implementation by @vamsikarnika in [590] Add Hudi HMS Catalog Sync implementation #639
• Update xtable-api sources by @emmanuel-ferdman in Update xtable-api sources #651
• [Enhancement] Support restore/rollback sync during conversion (1/2) by @danielhumanmod in [Enhancement] Support restore/rollback sync during conversion (1/2) #569
• [590] Add Hudi HMS Catalog Sync Implementation by @vamsikarnika in [590] Add Hudi HMS Catalog Sync Implementation #648
• [590] Add Hudi Glue Catalog Sync Implementation by @vamsikarnika in [590] Add Hudi Glue Catalog Sync Implementation #649
• Add change proposal for conversion of deletion vectors by @ashvina in Add change proposal for conversion of deletion vectors #634
• Add base representation for storage files by @ashvina in Add base representation for storage files #653
• Fix flaky test, ignore order of elements in list validation by @ashvina in Fix flaky test, ignore order of elements in list validation #662
• [594] Support continuous conversion with the RunSync tool by @the-other-tim-brown in [594] Support continuous conversion with the RunSync tool #593
• [MINOR] fixes recommended from static analysis tooling by @the-other-tim-brown in [MINOR] fixes recommended from static analysis tooling #664
• Modified incremental safe check which does not depend on snapshots list order by @brishi19791 in Modified incremental safe check which does not depend on snapshots list order #665
• [668] Support bucket partition transform for Iceberg Sources and Delta Targets by @the-other-tim-brown in [668] Support bucket partition transform for Iceberg Sources and Delta Targets #670
• Handle timestamp_ntz in delta and iceberg by @vinishjail97 in Handle timestamp_ntz in delta and iceberg #647
• [MINOR] Fix docker file syntax issue by @the-other-tim-brown in [MINOR] Fix docker file syntax issue #674
• [528] Refactor RunSync Class for Improved Modularity and Reusability by @vaibhavk1992 in [528] Refactor RunSync Class for Improved Modularity and Reusability #675
• Add how-to docs for catalog sync by @vinishjail97 in Add how-to docs for catalog sync #673
• [MINOR] Add mailing list for discussions, update issues list in asf.yaml by @the-other-tim-brown in [MINOR] Add mailing list for discussions, update issues list in asf.yaml #676
• [MINOR] Fix mailing list in asf.yaml by @the-other-tim-brown in [MINOR] Fix mailing list in asf.yaml #677
• [678] Add contribution guidelines to reflect stale PR's guidelines by @vinishjail97 in [678] Add contribution guidelines to reflect stale PR's guidelines #684
• [678] Avoid skipping xtable-aws and xtable-hive-metastore in mvn deploy by @vinishjail97 in [678] Avoid skipping xtable-aws and xtable-hive-metastore in mvn deploy #679
• [678] Introduce maturity-level in xtable-api and Refactor SyncStatusCode into separate class by @vinishjail97 in [678] Introduce maturity-level in xtable-api and Refactor SyncStatusCode into separate class #685
• Bump org.apache.parquet:parquet-avro from 1.12.3 to 1.15.1 by @dependabot in Bump org.apache.parquet:parquet-avro from 1.12.3 to 1.15.1 #686
• Enable squash and merge in .asf.yaml for xtable by @vinishjail97 in Enable squash and merge in .asf.yaml for xtable #688
• [678] Add LICENSE for xtable-aws module by @vinishjail97 in [678] Add LICENSE for xtable-aws module #690
• [678] Add LICENSE for xtable-hive-metastore module by @vinishjail97 in [678] Add LICENSE for xtable-hive-metastore module #691
• Docs: Update Footer Copyright Year by @omkenge in Docs: Update Footer Copyright Year #654
• [692] Include 2025 in NOTICE file by @vinishjail97 in [692] Include 2025 in NOTICE file #695
• [692] Exclude jol-core in xtable bundled dependencies by @vinishjail97 in [692] Exclude jol-core in xtable bundled dependencies #696
• Remove parameters from Glue partition columns by @kroushan-nit in Remove parameters from Glue partition columns #697
• [692] Ensure LICENSE content is different for bundled jars by @vinishjail97 in [692] Ensure LICENSE content is different for bundled jars #699

New Contributors

• @kroushan-nit made their first contribution in [590] Hadoop and AWS sdk version upgrade #614
• @bhasudha made their first contribution in [site] Update blogs #635
• @brishi19791 made their first contribution in Modified incremental safe check which does not depend on snapshots list order #665
• @vaibhavk1992 made their first contribution in [528] Refactor RunSync Class for Improved Modularity and Reusability #675
• @omkenge made their first contribution in Docs: Update Footer Copyright Year #654

Full Changelog: 0.2.0-incubating...0.3.0-incubating-rc2