Skip to content

Stop copying Exprs and LogicalPlans so much during Common Subexpression Elimination #9873

Closed
@alamb

Description

@alamb

Is your feature request related to a problem or challenge?

The common subexpression elimination pass copies many Exprs around. You can see this performance impact this has by looking at the screenshot from #9637 (comment)

While we will fix the copying plan problem in #9637 I think there is more work to be done in the common sub expression code itself, which copies a significant number of Exprs and Strings around

312892336-1c8214cc-09ec-41b2-aa5d-f52a5dfa4226

Describe the solution you'd like

Figure out how to avoid cloneing Exprs in the https://github.com/apache/arrow-datafusion/blob/main/datafusion/optimizer/src/common_subexpr_eliminate.rs

  1. The Exprs themselves
  2. Avoid creating Strings for Identifier

We should see a significant improvement in the sql_planner benchmarks:

cargo bench --bench sql_planner

Describe alternatives you've considered

No response

Additional context

I noticed this while reviewing #9871

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions