Add H2O.ai Database-like Ops benchmark to `dfbench`

[alamb]

Is your feature request related to a problem or challenge?

Follow on to #7052
There is an interesting database benchark called "H20.ai database like benchmark" that DuckDB seems to have revived (perhaps because the original went dormant with very old with very old/ slow duckdb results). More background here: https://duckdb.org/2023/04/14/h2oai.html#results

@Dandandan added a new solution for datafusion here: duckdblabs/db-benchmark#18

However, there is no easy way to run the h2o benchmark within the datafusion repo. There is an old version of some of these benchmarks in the code: https://github.com/apache/arrow-datafusion/blob/main/benchmarks/src/bin/h2o.rs

Describe the solution you'd like

I would like someone to make it easy to run the h20.ai benchmark in the datafusion repo.

Ideally this would look like

# generate data
./benchmarks/bench.sh data h20.ai
# run 
./benchmarks/bench.sh run h20.ai

I would expect to be able to run the individual queries like this

cargo run  --bin dfbench -- h2o.ai --query=3

Some steps might be

1. port the existing benchmark script to dfbench following the model in Add parquet-filter and sort benchmarks to dfbench #7120
2. update bench.sh, following the model of existing benchmarks
3. Update the documentation

Describe alternatives you've considered

We could also simply remove the h20.ai benchmark script as it is not clear how important it will be long term

Additional context

I think this is a good first issue as the task is clear, and there are existing patterns in bench.sh, dfbench and in

[palash25]

I would like to work on this

[alamb]

Thank you @palash25

[palash25]

sorry for the inactivity on this. my RSI came back so i was taking a break from typing, i will try to submit the PR in a day or two.

[alamb]

No problem -- I hope you feel better soon

[drewhayward]

Is this something that's still wanted? I took a look at doing this but it looks like the data isn't hosted on the benchmark repo, just data gen scripts in R.

[alamb]

Is this something that's still wanted? I took a look at doing this but it looks like the data isn't hosted on the benchmark repo, just data gen scripts in R.

I think it would be useful. Thank you

I think figuring out how to generate the data locally would be super valuable -- perhaps we can use a docker like approach as we do for tpch:

datafusion/benchmarks/bench.sh

Lines 286 to 292 in 89677ae

	FILE="${TPCH_DIR}/supplier.tbl"
	if test -f "${FILE}"; then
	echo " tbl files exist ($FILE exists)."
	else
	echo " creating tbl files with tpch_dbgen..."
	docker run -v "${TPCH_DIR}":/data -it --rm ghcr.io/scalytics/tpch-docker:main -vf -s ${SCALE_FACTOR}
	fi

So it would run like

./bench.sh data h2o

Which would leave data in datafusion/benchmarks/data/h2o

🤔

[alamb]

Now that @Rachelint and @2010YOUY01 and others have started working on

• [EPIC] Improved performance in H2O.ai benchmarks #13548

I think this issue is more important than ever

I think the hardest part of this task is actually generating the benchmark data
Thankfully @MrPowers has created falsa to generate the dataset (so we don't need R installed):

Here are the instructions for generating data: https://github.com/MrPowers/mrpowers-benchmarks?tab=readme-ov-file#running-the-benchmarks-on-your-machine

Here are the queries:

• https://github.com/MrPowers/mrpowers-benchmarks/blob/main/benchmarks/datafusion_h2o_groupby_queries.py
• https://github.com/MrPowers/mrpowers-benchmarks/blob/main/benchmarks/datafusion_h2o_join_queries.py

[drewhayward]

I don't currently have bandwidth to take this across the finish line but I did get the datagen working via Docker

Commit with data gen

[alamb]

Thanks @drewhayward -- maybe someone else can pick it up from there

[zhuqi-lucas]

take