Skip to content

Add "end to end parquet reading test" for WASM #15357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alamb opened this issue Mar 22, 2025 · 1 comment · Fixed by #15362
Closed

Add "end to end parquet reading test" for WASM #15357

alamb opened this issue Mar 22, 2025 · 1 comment · Fixed by #15362
Assignees
Labels
good first issue Good for newcomers

Comments

@alamb
Copy link
Contributor

alamb commented Mar 22, 2025

I think the current code tests the re-exported Parquet functionalities, not touching the DataFusion-related code. Ideally, we should test the end-to-end Parquet reading process.

The process roughly looks like this:

  1. Create a in-memory object_store, and put the Parquet data you generated into the object_store.
  2. Register the object_store along with the path to the DataFusion.
  3. Run a SQL query from the DataFusion side to see if the results can be read back.

A loosely related test can be found here: https://github.com/XiangpengHao/parquet-viewer/blob/main/src/tests.rs#L9

Originally posted by @XiangpengHao in #15325 (comment)

@jsai28
Copy link
Contributor

jsai28 commented Mar 22, 2025

take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants