Skip to content

Fix concurrency issue with load snapshot from DeltaLakeMetadata #25588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

vlad-lyutenko
Copy link
Contributor

@vlad-lyutenko vlad-lyutenko commented Apr 15, 2025

Description

If getTableHandle called concurrently for same table, getSnapshot method could produce concurrency issue:

steps to reproduce:

  • create some table: CREATE table my_catalog_delta.my_schema.my_table_1 AS SELECT 1 as a;
  • run query:
SELECT
  t.table_name,
  t.table_type,
  mv.catalog_name IS NOT NULL as is_materialized_view
FROM
  my_catalog_delta.information_schema.tables t
  LEFT JOIN system.metadata.materialized_views mv 
    ON 
       t.table_catalog = mv.catalog_name AND 
       t.table_schema = mv.schema_name AND 
       t.table_name = mv.name
WHERE
  t.table_schema = 'my_schema'
  AND t.table_name = 'my_table_1'
  AND t.table_type IN ('BASE TABLE', 'VIEW')
ORDER by
  t.table_name

If you repeat it a few times, you will notice that sometime query failed with : latestTableVersions changed concurrently exception.

It happens because there is multiple concurrent calls to getSnapshot and first call updated latestTableVersions map but not yet queriedSnapshots map and second call see these 2 tables in inconsistent state ().

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Apr 15, 2025
@vlad-lyutenko vlad-lyutenko requested a review from ebyhr April 15, 2025 18:08
@github-actions github-actions bot added the delta-lake Delta Lake connector label Apr 15, 2025
@vlad-lyutenko vlad-lyutenko force-pushed the vlad-lyutenko/fix-conc-load-snapshot branch from b8b0622 to 93faeca Compare April 15, 2025 19:28
@vlad-lyutenko vlad-lyutenko force-pushed the vlad-lyutenko/fix-conc-load-snapshot branch from 93faeca to 55b9365 Compare April 15, 2025 19:58
@ebyhr
Copy link
Member

ebyhr commented Apr 15, 2025

I can't reproduce the issue. Did you use DefaultDeltaLakeQueryRunnerMain?

@chenjian2664
Copy link
Contributor

I am confusing how it can happen, DeltaLakeMetadata creates the latestTableVersions everytime, which means to me is that only the case: there exists multiple threads arrive this method in one transaction(query) ?
Multiple run should not hit this case

I see the code in DeltaLakeTransactionManager:

    public void begin(ConnectorTransactionHandle transaction)
    {
        MemoizedMetadata previousValue = transactions.putIfAbsent(transaction, new MemoizedMetadata());
        checkState(previousValue == null);
    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector
Development

Successfully merging this pull request may close these issues.

3 participants