Skip to content

Nate/vllm-no-fim #5833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 26, 2025
Merged

Nate/vllm-no-fim #5833

merged 6 commits into from
May 26, 2025

Conversation

sestinj
Copy link
Contributor

@sestinj sestinj commented May 24, 2025

Description

vllm doesn't support fim, and shouldn't through the proxy

Checklist

  • I've read the contributing guide
  • The relevant docs, if any, have been updated or created
  • The relevant tests, if any, have been updated or created

Tests

A test suite was added to check the supportsFim method for many ILLM classes

@sestinj sestinj requested a review from a team as a code owner May 24, 2025 21:56
@sestinj sestinj requested review from Patrick-Erichsen and removed request for a team May 24, 2025 21:56
Copy link

cubic-dev-ai bot commented May 24, 2025

Your cubic subscription is currently inactive. Please reactivate your subscription to receive AI reviews and use cubic.

Copy link

netlify bot commented May 24, 2025

Deploy Preview for continuedev canceled.

Name Link
🔨 Latest commit 0c18c51
🔍 Latest deploy log https://app.netlify.com/projects/continuedev/deploys/6833f2956cef940008fb7000

@dosubot dosubot bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label May 24, 2025
Copy link
Collaborator

@Patrick-Erichsen Patrick-Erichsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice tests 👌 just one small comment but the logic for that file is all similarly hard to follow so unrelated to this particular PR


export const PROVIDER_TOOL_SUPPORT: Record<string, (model: string) => boolean> =
{
"continue-proxy": (model) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this logic is quite difficult to follow

@github-project-automation github-project-automation bot moved this from Todo to In Progress in Issues and PRs May 24, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label May 24, 2025
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels May 26, 2025
@continue-code-review
Copy link

🚨 Code Review Error

GitHub API error (422): Unprocessable Entity

Please check the logs or contact the maintainers for assistance.

@sestinj sestinj merged commit 2726f3b into main May 26, 2025
35 checks passed
@sestinj sestinj deleted the nate/vllm-no-fim branch May 26, 2025 05:20
@github-project-automation github-project-automation bot moved this from In Progress to Done in Issues and PRs May 26, 2025
@github-actions github-actions bot locked and limited conversation to collaborators May 26, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants