Skip to content

Ensure Partition Specs can only contain primitive types #718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
c-thiel opened this issue Nov 27, 2024 · 2 comments · May be fixed by #780 or #1285
Open

Ensure Partition Specs can only contain primitive types #718

c-thiel opened this issue Nov 27, 2024 · 2 comments · May be fixed by #780 or #1285
Labels
good first issue Good for newcomers

Comments

@c-thiel
Copy link
Collaborator

c-thiel commented Nov 27, 2024

Following the discussion here: #349 (comment) and the spec, partitions must only produce primitive types.

This is currently not guranteed, neither by deserialization nor for add_unbound_field. At least adding a void transform can result in complex types.
@c-thiel c-thiel mentioned this issue Nov 27, 2024
Merged
@c-thiel c-thiel added the good first issue Good for newcomers label Nov 28, 2024
@Lordworms Lordworms linked a pull request Dec 12, 2024 that will close this issue
Open
@liurenjie1024
Copy link
Contributor

Instead of ensuring primitive types only, should we add check in Transform, e.g. check each transform supports targeting type?

@jagdeeps91 jagdeeps91 linked a pull request May 2, 2025 that will close this issue
Open
@jagdeeps91
Copy link

I have published a pull request, which removes an if check. Now the code validates that the source column type is Primitive for all transforms including Void:
#1285

@c-thiel since you authored the original code, let me know if this if check should be kept:
https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/spec/partition.rs#L582

This is my first pull request to iceberg-rust, looking for feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
3 participants
@liurenjie1024 @jagdeeps91 @c-thiel