Introduce VLLM_VERBOSITY environment variable
#1165
Conversation
danilopeixoto
changed the title
VLLM_VERBOSITY environment variable
| @@ -76,5 +76,5 @@ async def stream_results() -> AsyncGenerator[bytes, None]: | |||
| uvicorn.run(app, | |||
| host=args.host, | |||
| port=args.port, | |||
| log_level="debug", | |||
| log_level="warning", | |||
There was a problem hiding this comment.
Here we log only warnings and critical errors for uvicorn dependency (internal).
There was a problem hiding this comment.
Would it be better to omit this argument completely so that the uvicorn log level can be set with its own environment variable? See https://www.uvicorn.org/settings/
| @@ -26,15 +36,23 @@ def format(self, record): | |||
| _default_handler = None | |||
|
|
|||
|
|
|||
| def _get_default_logging_level(): | |||
| verbosity = os.getenv("VLLM_VERBOSITY", "info").lower() | |||
There was a problem hiding this comment.
For vLLM logs, we allow debug, info, warning, error and critical levels.
The default is info. The default can be overridden by VLLM_VERBOSITY variable.
Users may want to set it to warning in production.
| global _default_handler | ||
| if _default_handler is None: | ||
| _default_handler = logging.StreamHandler(sys.stdout) | ||
| _default_handler.flush = sys.stdout.flush # type: ignore | ||
| _default_handler.setLevel(logging.INFO) | ||
| _default_handler.setLevel(logging.DEBUG) |
There was a problem hiding this comment.
The default handler goes down to debug level, however, the root is defining the effective level from user selection (VLLM_VERBOSITY) or default.
|
Hi @zhuohan123! |
|
+1 for this change, being able to change the log level via environment variable would be very useful. |
Introduce
VLLM_VERBOSITYenvironment variable and useinfolog level by default to reduce IO operations in production.For dependencies use
warninglevel.