[Feature]: Support Pipeline Parallelism on Llama-4-Maverick-17B-128E

[Edwinhr716]

🚀 The feature, motivation and pitch

I'm attempting to deploy Llama-4-Maverick-17B-128E across 16 H100s on two nodes, running this command:

python3 -m vllm.entrypoints.openai.api_server --port 8080 --model meta-llama/Llama-4-Maverick-17B-128E-Instruct --tensor-parallel-size 8 --pipeline-parallel-size 2

I got this message saying that PP isn't supported

NotImplementedError: Pipeline parallelism is not supported for this model. Supported models implement the `SupportsPP` interface.

Llama-4-Maverick-17B-128E is a large LLM that most people will be running across multiple GPU nodes.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[houseroad]

I think trunk should already have the PP support. @zhewenl, could you help on verification?

[zhewenl]

@Edwinhr716 rebase to latest main and try again, it should be supported: https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/mllama4.py#L668-L669
(unable to reproduce your issue: https://gist.github.com/zhewenl/c2a946bbc0c24450bd469aa29f836784)

[Edwinhr716]

sweet, so should be available on 0.8.4 release?

[houseroad]

It will. Before it's ready, feel free to try our nightly: https://docs.vllm.ai/en/latest/getting_started/installation/gpu.html#pre-built-wheels

[houseroad]

Okay, i will close this issue, feel free to re-open if anything else is needed :-)