Skip to content

Add the object length to object mappings #3457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
teor2345 opened this issue Mar 25, 2025 · 0 comments
Open

Add the object length to object mappings #3457

teor2345 opened this issue Mar 25, 2025 · 0 comments
Labels
improvement it is already working, but can be better networking Subspace networking (DSN)

Comments

@teor2345
Copy link
Member

The object retrieval code downloads one or two pieces to determine how long an object is. But the archiver can read the length of the object from the block data.

Including the object length in the block data would improve object fetching latency, because all the object's pieces could be downloaded at the same time.

As part of this change, mappings could use the offset of the first data byte, rather than the offset of the encoded length. This would save us 1 downloaded piece, in some rare edge cases.

See #3318 and #3362 for more background.

Implementation Strategy

The object mappings are initially generated by the runtime, then modified by the archiver. To apply to all blocks, the archiver should:

  • add the length to the mapping
  • update the offset and piece index to exclude the encoded object length bytes

Updating the runtime code would only add the length to mappings for new blocks.

@teor2345 teor2345 added improvement it is already working, but can be better networking Subspace networking (DSN) labels Mar 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement it is already working, but can be better networking Subspace networking (DSN)
Projects
None yet
Development

No branches or pull requests

1 participant