Skip to content

Task handling is incomplete #774

@snazy

Description

@snazy

Describe the bug

Polaris uses some asynchronously executed tasks to run operations for table and manifest file cleanup. Those tasks are potentially executed in a separate thread in the same JVM. There is however no guarantee that those tasks will eventually run for multiple reasons:

  • Tasks (e.g. via org.apache.polaris.service.catalog.BasePolarisCatalog#dropTable) are triggered after the fact.
  • Although tasks are persisted, there is no mechanism to pick up tasks that did not start or did not finish ("long lasting" failures, JVM terminates).

Overall this means that for example a "drop table with purge" returns a successful result to the user, the actual purge may never ever happen.

To Reproduce

No response

Actual Behavior

No response

Expected Behavior

No response

Additional context

No response

System information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions