Commits on Oct 19, 2024

1. Make benchmarks use EngineArgs …

   Update the benchmark scripts to directly use the CLI arguments
   provided by EngineArgs instead of duplicating a subset of these
   arguments in each benchmark script.
   
   Currently the CLI arguments are duplicated, forcing changes to be
   made in multiple locations and resulting in some useful vLLM options
   not being exposed in the scripts.  For example, the
   --num-scheduler-steps option is currently available in
   benchmark_throughput.py but not benchmark_latency.py, making it
   difficult to understand the latency impacts of this option.  As
   another example, the benchmark_prioritization.py script appears to
   be broken currently because it was not updated to expose the
   --scheduling-policy option which is required for enabling priority.
   
   These maintenance challenges are eliminated by using
   EngineArgs.add_cli_args to add support for all engine arguments
   directly, and then passing these options to the engine
   initialization.
   
   One minor change in behavior is that when benchmark_throughput.py
   runs in async mode it no longer includes hard-coded settings for
   worker_use_ray=False (which is deprecated anyway) and
   disable_log_requests=True (but the user now has the option to pass
   --disable-log-requests on the command-line).
   
   Similarly, benchmark_prefix_caching no longer has hard-coded values
   for trust_remote_code=True and enforce_eager=True, but these may
   now be passed on the command-line.

   

   JArnoldAMD committed Oct 19, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for b104dc2
   • Browse repository at this point

   Copy the full SHA

   b104dc2 View commit details

   Browse the repository at this point in the history