Add Salesforce xLAM-2 series of model handlers and update vLLM version from 0.6.3 to 0.6.5 #972
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for the Salesforce xLAM-2 series of models and updates the vLLM dependency from 0.6.3 to 0.6.5 to address a long context issue with Qwen models (referenced in vllm-project/vllm#9769).
Changes
Added new Salesforce xLAM-2 model handlers:
SalesforceLlamaHandler
for Llama-based models (xLAM-2-70b-fc-r, xLAM-2-8b-fc-r)SalesforceQwenHandler
for Qwen-based models (xLAM-2-32b-fc-r, xLAM-2-3b-fc-r, xLAM-2-1b-fc-r)Updated model metadata and supported models documentation to include the new xLAM-2 models
Bumped vLLM version from 0.6.3 to 0.6.5 in pyproject.toml to fix Qwen's long context issue
Testing
The implementation has been tested with the supported xLAM-2 models to ensure proper function calling capabilities and compatibility with the Berkeley Function Call Leaderboard evaluation framework.