Skip to content

Proposal: improve SchemaBuilder for "update fields" usecase #6576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Oct 16, 2024

Which issue does this PR close?

Closes #6575

Rationale for this change

See #6575 -- tldr is that it is very easy to forget to copy schema metadata when creating new schemas

What changes are included in this PR?

  1. Add examples to SchemaBuilder
  2. Update documentation to make it more likely people will find SchemaBuilder
  3. Add clear_fields and with_field and build for a more builder like experience

Are there any user-facing changes?

Yes new APIs and docs

@github-actions github-actions bot added the arrow Changes to the arrow crate label Oct 16, 2024
@alamb alamb marked this pull request as ready for review October 16, 2024 20:54
Comment on lines +162 to +164
/// consume the builder and return the final [`Schema`]
///
/// (synonym for [`Self::finish`]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// consume the builder and return the final [`Schema`]
///
/// (synonym for [`Self::finish`]
/// Consume this [`SchemaBuilder`] yielding the final [`Schema`]
///
/// Synonym for [`Self::finish`]

@@ -45,6 +71,18 @@ impl SchemaBuilder {
}
}

/// Clears any fields currently in this builder.
pub fn clear_fields(mut self) -> Self {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub fn clear_fields(mut self) -> Self {
pub fn clear(mut self) -> Self {

I can see the desire to distinguish this from metadata, but given the other fields aren't push_field, etc... Maybe this is more consistent?

}

/// Appends a new field to this [`SchemaBuilder`] and returns self
pub fn with_field(mut self, field: impl Into<FieldRef>) -> Self {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a with_metadata as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, will add

Comment on lines +47 to +50
/// let projected_schema = SchemaBuilder::from(&schema)
/// .clear_fields()
/// .with_field(schema.field(1).clone())
/// .build();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// let projected_schema = SchemaBuilder::from(&schema)
/// .clear_fields()
/// .with_field(schema.field(1).clone())
/// .build();
/// let projected_schema = SchemaBuilder::from(schema.metadata())
/// .with_field(schema.field(1).clone())
/// .build();

I wonder if we could make this work, it might be a bit cleaner?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree -- will give it a try

@alamb
Copy link
Contributor Author

alamb commented Oct 16, 2024

FYI @wiedld

@alamb
Copy link
Contributor Author

alamb commented Oct 21, 2024

I plan to return to this PR later in this week

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ability to create Schema with new fields but same everytthing else
2 participants