Skip to content

feat: make store fetch messages by hash #2351

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
weboko opened this issue Apr 11, 2025 · 3 comments · May be fixed by #2330
Open

feat: make store fetch messages by hash #2351

weboko opened this issue Apr 11, 2025 · 3 comments · May be fixed by #2330
Assignees

Comments

@weboko
Copy link
Collaborator

weboko commented Apr 11, 2025

Description

The Waku Store protocol specification supports querying messages by their hash, but our current implementation makes this impossible due to validation logic conflicts. When attempting to use messageHashes as a query parameter, users encounter a catch-22:

  1. Using queryGenerator requires passing decoders, which automatically sets pubsubTopic and contentTopics
  2. The validation logic then rejects the query with an error:
if (
  params.messageHashes &&
  params.messageHashes.length > 0 &&
  (params.pubsubTopic ||
    (params.contentTopics && params.contentTopics.length > 0) ||
    params.timeStart ||
    params.timeEnd)
) {
  throw new Error(
    "Message hash lookup queries cannot include content filter criteria"
  );
}

This makes it currently impossible to use the message hash lookup feature that's specified in the protocol, despite it being exposed in our API interfaces.

User Story

As a developer, I want to query specific messages by their hash directly, so that I can efficiently retrieve exactly the messages I need without having to filter through time-based or content-based results.

Proposed Solution / Feature Design

Fix the validation logic to properly handle message hash queries:

@chair28980 chair28980 added this to Waku Apr 11, 2025
@weboko weboko moved this to Triage in Waku Apr 11, 2025
@weboko
Copy link
Collaborator Author

weboko commented Apr 11, 2025

@adklempner to add description

also we need to add E2E interop tests for this usage of Store

@adklempner
Copy link
Member

related to #2330

@weboko weboko linked a pull request May 13, 2025 that will close this issue
5 tasks
@adklempner adklempner moved this from Triage to Code Review / QA in Waku May 15, 2025
@adklempner
Copy link
Member

Issue Summary: Store v3 Message Hash Query Fix

Problem

The Waku Store v3 message hash query test was failing in js-waku CI. Initial investigation suggested it was an nwaku issue, but further analysis revealed the problem was in the js-waku implementation.

Root Cause

  1. Missing pubsubTopic in test: The test was not specifying the pubsubTopic when querying by message hash
  2. Incorrect RPC validation: The js-waku RPC validation was incorrectly rejecting hash queries that included pubsubTopic
  3. Wrong assumption: The js-waku implementation assumed hash queries should not include pubsubTopic, but according to Waku protocol, pubsubTopic should always be specified

Solution

The following changes were made to fix the issue:

  1. Updated the test (packages/tests/tests/store/message_hash.spec.ts):

    // Added pubsubTopic to the query options
    for await (const page of waku.store.queryGenerator([TestDecoder], {
      messageHashes,
      pubsubTopic: TestDecoder.pubsubTopic
    })) {
  2. Fixed RPC validation (packages/core/src/lib/store/rpc.ts):

    // Changed validation to allow pubsubTopic in hash queries
    if (isHashQuery) {
      // Message hash lookup queries cannot include content topics or time filters
      // but pubsubTopic is allowed/required
      if (hasContentTopics || hasTimeFilter) {
        throw new Error(
          "Message hash lookup queries cannot include content filter criteria (contentTopics, timeStart, or timeEnd)"
        );
      }
    }
  3. Updated store logic (packages/sdk/src/store/store.ts):

    // Use pubsubTopic from options if provided, otherwise from first decoder
    pubsubTopic = options.pubsubTopic || decoders[0]?.pubsubTopic || "";

Test Result

After applying these fixes, the test now passes successfully:

✔ can query messages by message hash (201ms)

The nwaku logs confirm that queries now include the proper pubsubTopic and messages are successfully retrieved.

Key Takeaway

The issue was not in nwaku but in js-waku's implementation and test. The Waku protocol requires pubsubTopic to be specified for all store queries, including hash queries. The js-waku implementation incorrectly assumed hash queries should exclude pubsubTopic.

Related PR

This investigation was done while analyzing failing CI tests in js-waku PR #2250.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Code Review / QA
Development

Successfully merging a pull request may close this issue.

2 participants