How do let vllm v1 to use a customized stat logger extended from StatLoggerBase? #14314
liuzijing2014
announced in
Q&A
Replies: 1 comment
-
Any idea if this is going to be supported? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
For V0, we are allowed to pass in customized stat loggers when we init the llm_engine:
vllm/vllm/engine/async_llm_engine.py
Lines 633 to 658 in 53ea6ad
However, looking at v1, it seems that we are no longer taking in stat_logger:
vllm/vllm/v1/engine/async_llm.py
Lines 39 to 63 in 53ea6ad
The purpose is to log stats to a customized destination for persistent storage and analysis.
Beta Was this translation helpful? Give feedback.
All reactions