Skip to content

chore: Upgrade datafusion/arrow/parquet #1286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Conversation

AdamGS
Copy link

@AdamGS AdamGS commented May 2, 2025

Which issue does this PR close?

  • Closes #.

What changes are included in this PR?

Mostly dependency changes with very minor code changes. There are two open things in my mind:

  1. Seems like the AsyncFileRead trait changed and the options struct is very private, so maybe there's some other type that can be pulled from parquet-rs to replace ArrowFileReader. IDK if its desirable but I do think it might make sense to make some changes
  2. There some dependency issue I'm not sure is resolvable without bumping the msrv, right now if you run CI on this branch that check will fail with:
error[E0599]: no function or associated item named `into_compile_error` found for struct `syn::Error` in the current scope
   --> /Users/adamgs/.cargo/registry/src/index.crates.io-6f17d22bba15001f/motore-macros-0.4.0/src/lib.rs:38:28
    |
38  |         return syn::Error::into_compile_error(err).into();
    |                            ^^^^^^^^^^^^^^^^^^ function or associated item not found in `syn::Error`
    |
note: if you're trying to build a new `syn::Error` consider using one of the following associated functions:
      syn::Error::new
      syn::Error::new_spanned
   --> /Users/adamgs/.cargo/registry/src/index.crates.io-6f17d22bba15001f/syn-1.0.44/src/error.rs:133:5
    |
133 |     pub fn new<T: Display>(span: Span, message: T) -> Self {
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
157 |     pub fn new_spanned<T: ToTokens, U: Display>(tokens: T, message: U) -> Self {
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: there is a method `to_compile_error` with a similar name, but with different arguments
   --> /Users/adamgs/.cargo/registry/src/index.crates.io-6f17d22bba15001f/syn-1.0.44/src/error.rs:193:5
    |
193 |     pub fn to_compile_error(&self) -> TokenStream {
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I'm leaving it here until I have some more time to play around with these issues and to get some feedback if anything here is actually welcome.

Are these changes tested?

@AdamGS AdamGS marked this pull request as draft May 2, 2025 10:20
@AdamGS AdamGS changed the title Chore: Upgrade datafusion/arrow/parquet chore: Upgrade datafusion/arrow/parquet May 2, 2025
@sundy-li
Copy link
Contributor

sundy-li commented May 5, 2025

I will start to continue this pr now.

Xuanwo added a commit that referenced this pull request May 13, 2025
## Which issue does this PR close?

continue #1286 

- Closes #.

## What changes are included in this PR?

bump up arrow/parquet/datafusion 

## Are these changes tested?

Y

---------

Signed-off-by: Xuanwo <[email protected]>
Co-authored-by: Xuanwo <[email protected]>
@Xuanwo
Copy link
Member

Xuanwo commented May 14, 2025

#1294 has been merged. I'm going to close this one now.

Thank you @AdamGS for working on this.

@Xuanwo Xuanwo closed this May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants