Skip to content

ES|QL Reranker command #123074

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 102 commits into from
Apr 4, 2025
Merged

ES|QL Reranker command #123074

merged 102 commits into from
Apr 4, 2025

Conversation

afoucret
Copy link
Contributor

@afoucret afoucret commented Feb 20, 2025

Implementation of the RERANK command

Command syntax

| RERANK "query text" ON title,description WITH `reranker-inference-id`

where

  • query text is a constant string (can use a param instead ?queryText)
  • ON clause contains one or several field separated by comma
    • it is also possible to rename a field here (eg ON title, description=overview)
    • it is also possible to create computed fields here (eg ON title, short_description=SUBSTRING(description,0 100))
  • WITH the name of inference endpoint to be used for reranking. The inference endpoint task type has to be rerank

Done:

  • Grammar and parsing of the command
  • Logical plan
  • Pre-analysis and Analysis
    • Detect missing inference endpoint
    • Ensure all expression are resolved
  • Logical plan optimization
    • Default limits and sort order for the rerank command
  • Physical plan and planing
  • Reranker async operator

Follow-up:

@afoucret afoucret force-pushed the esql-reranker-boostrap branch from 9d2471e to fae22df Compare February 21, 2025 13:42
@afoucret afoucret mentioned this pull request Mar 7, 2025
16 tasks
@afoucret afoucret force-pushed the esql-reranker-boostrap branch from 4a5a19e to 132825d Compare April 2, 2025 05:38
@afoucret afoucret requested a review from jimczi April 2, 2025 06:16
@afoucret afoucret removed auto-backport Automatically create backport pull requests when merged v8.19.0 labels Apr 2, 2025
Copy link
Contributor

@jimczi jimczi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@afoucret afoucret enabled auto-merge (squash) April 3, 2025 13:39
@afoucret afoucret removed the test-release Trigger CI checks against release build label Apr 4, 2025
@afoucret afoucret disabled auto-merge April 4, 2025 09:14
@afoucret afoucret enabled auto-merge (squash) April 4, 2025 09:54
@afoucret afoucret merged commit a4a2714 into elastic:main Apr 4, 2025
17 checks passed
dnhatn added a commit that referenced this pull request Apr 7, 2025
If the clusters don't support inference test services, skip tests that require inference services. 
Hence, we should check for rerank tests.

Relates #123074
andreidan pushed a commit to andreidan/elasticsearch that referenced this pull request Apr 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>feature :Search Relevance/Ranking Scoring, rescoring, rank evaluation. Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants