Skip to content

std.tar: add support for file path in pax attributes #16990

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 8, 2023

Conversation

189900
Copy link
Contributor

@189900 189900 commented Aug 27, 2023

Handles .extended_header type to parse PAX attributes and check if they override the path of the next file. Increases maximum file path length from 255 to 1014 std.fs.MAX_PATH_BYTES.

Fixes #15342

@truemedian
Copy link
Contributor

Is there a reason you chose 1014 specifically instead of something like std.fs.MAX_PATH_BYTES?

@189900
Copy link
Contributor Author

189900 commented Aug 27, 2023

I went with 1014 to maintain the existing read buffer characteristics and avoid extra overhead.

We can definitely switch to std.fs.MAX_PATH_BYTES if that would be more preferable.

@truemedian
Copy link
Contributor

PAX path attributes technically have no length limitation, so we need to allocate as much space as we can reasonably handle so that the issue of not being able to handle long file paths doesn't appear again. Any path > MATH_PATH_BYTES can't be sent to the os anyway, so it's a good upper limit.

@squeek502
Copy link
Collaborator

squeek502 commented Aug 27, 2023

Going to link to my idea for parsing one-component-at-a-time to support arbitrarily long file paths without needing an allocator: #15382 (comment)

Just using a buffer length of MAX_PATH_BYTES might be good enough without the added complication of the one-component-at-a-time strategy, though.

@189900 189900 marked this pull request as draft August 28, 2023 05:50
@189900 189900 force-pushed the tar-pax-path-attribute branch from 67bebff to d8e7e00 Compare August 28, 2023 19:53
Handles .extended_header type to parse PAX attributes and check if they override
the path of the next file. Increases file path limit to std.fs.MAX_PATH_BYTES.

Fixes ziglang#15342
@189900 189900 force-pushed the tar-pax-path-attribute branch from d8e7e00 to 3227ca5 Compare August 29, 2023 17:45
@189900
Copy link
Contributor Author

189900 commented Aug 29, 2023

Thank you both. Updated the branch to support std.fs.MAX_PATH_BYTES. A sample tar with a long path: 4k.tar.gz

@189900 189900 marked this pull request as ready for review August 29, 2023 19:31
@Vexu Vexu merged commit 39a98dc into ziglang:master Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[std.http] zig build fails to download large archives from github
4 participants