Fetch table metadata in parallel and asynchronously and add them to a queue for faster processing #322

abhisheknath2011 · 2025-05-14T23:51:38Z

Summary

Issue] Briefly discuss the summary of the changes made in this
pull request in 2-3 lines.

The jobs scheduler for each job type fetches table metadata as a first step before start submitting jobs. The metadata discovery process takes around 1.5 - 2 hours for 30k+ tables and this number is eventually going to increase as more tables are onboarded to openhouse. As a result, job submission is delayed. In order to accelerate the job submission it is required to improve the table metadata discovery. Hence this PR fetches table metadata in parallel and asynchronously. The fetched table metadata is added to a queue for faster processing. The below are the changes done in this PR.

Leverages reactive programming using Flux and Mono to fetch table metadata in parallel and asynchronously so that the main thread is not blocked.
Once table metadata is fetched for a table, it is added to a in memory queue.
On the other hand, table metadata is consumed from the queue in parallel and the job is submitted for the table.
Flux sends table metadata discovery completion signal as part of onAfterTerminate which ensures that all the parallel threads finishes and were shutdown. Based on this signal queue consumer terminates.
Parallel fetch table metadata and number of parallel threads are configured through configs. So the parallel fetch table metadata can be enabled and rolled out for each job type through config.

Note: As this feature is configurable we should be able to enable and roll this out by job type.

Changes

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

Manually Tested on local docker setup. Please include commands ran, and their output.
Added new tests for the changes made.
Updated existing tests to reflect the changes made.
No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
Some other form of testing like staging or soak time in production. Please explain.

Tested on docker. Printed the output only for validation. Fetched metadata count and submitted job count matches which is 30 (Refer to the end of the log).

anath1@anath1-mn4233 oh-hadoop-spark % docker compose --profile with_jobs_scheduler run openhouse-jobs-scheduler - \
    --type SNAPSHOTS_EXPIRATION --cluster local --tablesURL http://openhouse-tables:8080 --jobsURL http://openhouse-jobs:8080 -\
    --tableMinAgeThresholdHours 0 --taskPollIntervalMs 5000 --parallelMetadataFetchMode true --numParallelMetadataFetch 4


WARN[0000] /Users/anath1/IdeaProjects/openhouse/infra/recipes/docker-compose/oh-hadoop-spark/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 
WARN[0000] Found orphan containers ([oh-hadoop-spark-openhouse-jobs-scheduler-run-a1931e6154a1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. 
[+] Creating 8/8
 ✔ Container oh-hadoop-spark-prometheus-1  Running                                                                                                                                                                                              0.0s 
 ✔ Container local.opa                     Running                                                                                                                                                                                              0.0s 
 ✔ Container local.datanode                Running                                                                                                                                                                                              0.0s 
 ✔ Container local.mysql                   Running                                                                                                                                                                                              0.0s 
 ✔ Container local.namenode                Running                                                                                                                                                                                              0.0s 
 ✔ Container local.openhouse-housetables   Running                                                                                                                                                                                              0.0s 
 ✔ Container local.openhouse-tables        Running                                                                                                                                                                                              0.0s 
 ✔ Container local.openhouse-jobs          Running                                                                                                                                                                                              0.0s 
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2025-05-16 18:19:58,009 main WARN JNDI lookup class is not available because this JRE does not support JNDI. JNDI string lookups will not be available, continuing configuration. Ignoring java.lang.IllegalStateException: JNDI must be enabled by setting log4j2.enableJndiLookup=true
2025-05-16 18:19:58 INFO  OtelConfig:70 - initializing open-telemetry sdk
2025-05-16 18:19:58 INFO  Reflections:219 - Reflections took 58 ms to scan 1 urls, producing 4 keys and 12 values
2025-05-16 18:19:58 INFO  JobsScheduler:122 - Starting scheduler
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  JobsScheduler:171 - Fetching task list based on the job type: SNAPSHOTS_EXPIRATION
2025-05-16 18:19:58 INFO  OperationTasksBuilder:255 - Got all tables: class GetAllTablesResponseBody {
    results: [class GetTableResponseBody {
        tableId: t1
        databaseId: d3
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t2
        databaseId: d3
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t3
        databaseId: d3
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t4
        databaseId: d3
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t5
        databaseId: d3
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }]
} for database d3
2025-05-16 18:19:58 INFO  OperationTasksBuilder:255 - Got all tables: class GetAllTablesResponseBody {
    results: [class GetTableResponseBody {
        tableId: t1
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t10
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t2
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t3
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t4
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t5
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t6
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t7
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t8
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t9
        databaseId: d1
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }]
} for database d1
2025-05-16 18:19:58 INFO  OperationTasksBuilder:255 - Got all tables: class GetAllTablesResponseBody {
    results: [class GetTableResponseBody {
        tableId: t1
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t10
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t2
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t3
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t4
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t5
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t6
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t7
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t8
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t9
        databaseId: d2
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }]
} for database d2
2025-05-16 18:19:58 INFO  OperationTasksBuilder:255 - Got all tables: class GetAllTablesResponseBody {
    results: [class GetTableResponseBody {
        tableId: t1
        databaseId: d4
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t2
        databaseId: d4
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t3
        databaseId: d4
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t4
        databaseId: d4
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }, class GetTableResponseBody {
        tableId: t5
        databaseId: d4
        clusterId: null
        tableUri: null
        tableUUID: null
        tableLocation: null
        tableVersion: null
        tableCreator: null
        schema: null
        lastModifiedTime: 0
        creationTime: 0
        tableProperties: null
        timePartitioning: JsonNullable[null]
        clustering: JsonNullable[null]
        policies: JsonNullable[null]
        tableType: null
    }]
} for database d4
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t2, creationTimeMs=1747418467199, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t10, creationTimeMs=1747418815470, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t4, creationTimeMs=1747418988766, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d4, table name: t4 
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t7, creationTimeMs=1747418563114, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t10 
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t2 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t3, creationTimeMs=1747418488286, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t1, creationTimeMs=1747418656618, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t1, creationTimeMs=1747418890838, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d3, table name: t1 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t7, creationTimeMs=1747418774043, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t5, creationTimeMs=1747418946376, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d3, table name: t5 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t1 
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t8, creationTimeMs=1747418788278, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t1, creationTimeMs=1747418890838, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t10, creationTimeMs=1747418815470, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t7 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t4, creationTimeMs=1747418988766, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t1, creationTimeMs=1747418656618, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t7 
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t8 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t5, creationTimeMs=1747418946376, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t2, creationTimeMs=1747418467199, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t7, creationTimeMs=1747418774043, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t8, creationTimeMs=1747418788278, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t3 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t7, creationTimeMs=1747418563114, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t3, creationTimeMs=1747418488286, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t5, creationTimeMs=1747418721033, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t5 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t5, creationTimeMs=1747418721033, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t9, creationTimeMs=1747418609799, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t6, creationTimeMs=1747418546511, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t9 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t4, creationTimeMs=1747418706289, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t4 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t6 
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t9, creationTimeMs=1747418609799, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:58 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:58 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:58 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:58 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:58 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t4, creationTimeMs=1747418706289, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t6, creationTimeMs=1747418546511, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t4, creationTimeMs=1747418505361, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t4 
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t2, creationTimeMs=1747418674429, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t2 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t1, creationTimeMs=1747419033490, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d4, table name: t1 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t8, creationTimeMs=1747418584959, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t1, creationTimeMs=1747418440801, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t3, creationTimeMs=1747418919201, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d3, table name: t3 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t1, creationTimeMs=1747419033490, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t1 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t4, creationTimeMs=1747418505361, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t2, creationTimeMs=1747418674429, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t5, creationTimeMs=1747418523130, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t3, creationTimeMs=1747418919201, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t4, creationTimeMs=1747418932819, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t6, creationTimeMs=1747418758481, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d3, table name: t4 
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t6 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t2, creationTimeMs=1747418905701, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t5 
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t1, creationTimeMs=1747418440801, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d3, table name: t2 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t4, creationTimeMs=1747418932819, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t8 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t6, creationTimeMs=1747418758481, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t5, creationTimeMs=1747418523130, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d3, tableName=t2, creationTimeMs=1747418905701, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t5, creationTimeMs=1747418969095, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d4, table name: t5 
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t8, creationTimeMs=1747418584959, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t3, creationTimeMs=1747418692860, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t3 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t2, creationTimeMs=1747419019296, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d4, table name: t2 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t5, creationTimeMs=1747418969095, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t10, creationTimeMs=1747418627250, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d1, table name: t10 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t2, creationTimeMs=1747419019296, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t9, creationTimeMs=1747418801755, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d2, table name: t9 
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  OperationTasksBuilder:277 - Got table metadata for : TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t3, creationTimeMs=1747419003234, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t3, creationTimeMs=1747418692860, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d1, tableName=t10, creationTimeMs=1747418627250, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  OperationTasksBuilder:224 - Got table metadata for database name: d4, table name: t3 
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 86c24a75-24eb-4ade-a9c9-8a48da7585fc
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  WebClientFactory:121 - Using connection pool strategy
2025-05-16 18:19:59 INFO  WebClientFactory:218 - Creating custom connection provider
2025-05-16 18:19:59 INFO  WebClientFactory:196 - Client session id: 5e4dbdbe-fdc9-45b6-bf5d-ba319406e4d5
2025-05-16 18:19:59 INFO  WebClientFactory:209 - Client name: null
2025-05-16 18:19:59 INFO  JobsScheduler:542 - The metadata fetched count: 30 for the job type: SNAPSHOTS_EXPIRATION
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d2, tableName=t9, creationTimeMs=1747418801755, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)
2025-05-16 18:19:59 INFO  JobsScheduler:543 - The total operation tasks submitted: 30 for the job type: SNAPSHOTS_EXPIRATION
2025-05-16 18:19:59 INFO  OperationTask:84 - Launching job for TableMetadata(super=Metadata(creator=DUMMY_ANONYMOUS_USER), dbName=d4, tableName=t3, creationTimeMs=1747419003234, isPrimary=true, isTimePartitioned=true, isClustered=true, jobExecutionProperties={}, retentionConfig=null, historyConfig=null, replicationConfig=null)

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Additional Information

Breaking Changes
Deprecations
Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.

… queue for faster processing

apps/spark/src/main/java/com/linkedin/openhouse/jobs/scheduler/JobsScheduler.java

apps/spark/src/main/java/com/linkedin/openhouse/jobs/scheduler/tasks/OperationTasksBuilder.java

… refactoring and cleanup

teamurko

Thanks @abhisheknath2011, generally looks good

apps/spark/src/main/java/com/linkedin/openhouse/jobs/scheduler/JobsScheduler.java

apps/spark/src/main/java/com/linkedin/openhouse/jobs/scheduler/tasks/OperationTasksBuilder.java

apps/spark/src/main/java/com/linkedin/openhouse/jobs/scheduler/JobsScheduler.java

…and we don't have replication task in OSS repo

apps/spark/src/main/java/com/linkedin/openhouse/jobs/scheduler/JobsScheduler.java

Fetch table metadata in parallel and asynchronously and add them to a…

b7c8266

… queue for faster processing

teamurko reviewed May 15, 2025

View reviewed changes

apps/spark/src/main/java/com/linkedin/openhouse/jobs/scheduler/JobsScheduler.java Outdated Show resolved Hide resolved

apps/spark/src/main/java/com/linkedin/openhouse/jobs/scheduler/tasks/OperationTasksBuilder.java Outdated Show resolved Hide resolved

Added configs for enabling parallel metadata fetch, terminate signal,…

364ef2a

… refactoring and cleanup

abhisheknath2011 marked this pull request as ready for review May 16, 2025 18:35

abhisheknath2011 added 2 commits May 16, 2025 11:49

Added comment and removed unintended changes

f770fb6

Changing some info messages to debug which were added for testing

37eebf3

teamurko reviewed May 21, 2025

View reviewed changes

abhisheknath2011 added 3 commits May 21, 2025 10:00

Moved output args to the end of the args

f4b6f85

Apply table metadata filter

0a5ac00

Remove metadata filter for replication as filtering is internal repo …

98ccc70

…and we don't have replication task in OSS repo

teamurko previously approved these changes May 21, 2025

View reviewed changes

apps/spark/src/main/java/com/linkedin/openhouse/jobs/scheduler/JobsScheduler.java Show resolved Hide resolved

Updated doc

5c8f22a

abhisheknath2011 dismissed teamurko’s stale review via 5c8f22a May 21, 2025 19:23

teamurko self-requested a review May 21, 2025 19:25

teamurko approved these changes May 21, 2025

View reviewed changes

abhisheknath2011 merged commit 91ce21c into linkedin:main May 21, 2025
1 check passed

abhisheknath2011 mentioned this pull request May 30, 2025

Job submission improvement: separation of job submission and status check #326

Open

17 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fetch table metadata in parallel and asynchronously and add them to a queue for faster processing #322

Fetch table metadata in parallel and asynchronously and add them to a queue for faster processing #322

Uh oh!

abhisheknath2011 commented May 14, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

teamurko left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Fetch table metadata in parallel and asynchronously and add them to a queue for faster processing #322

Fetch table metadata in parallel and asynchronously and add them to a queue for faster processing #322

Uh oh!

Conversation

abhisheknath2011 commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Testing Done

Additional Information

Uh oh!

Uh oh!

Uh oh!

teamurko left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

abhisheknath2011 commented May 14, 2025 •

edited

Loading