cli.vllm_serve

cli.vllm_serve

CLI to start the vllm server for online RL

Classes

Name	Description
VllmServeArguments	Arguments for vLLM’s native OpenAI server as used by TRL trainers

VllmServeArguments

cli.vllm_serve.VllmServeArguments(
    model,
    revision=None,
    tensor_parallel_size=1,
    data_parallel_size=1,
    pipeline_parallel_size=1,
    nnodes=1,
    node_rank=0,
    master_addr=None,
    host='0.0.0.0',
    port=8000,
    gpu_memory_utilization=0.9,
    dtype='auto',
    max_model_len=None,
    enable_prefix_caching=None,
    enforce_eager=False,
    kv_cache_dtype='auto',
    trust_remote_code=False,
    log_level='info',
    vllm_model_impl='vllm',
    reasoning_parser='',
    enable_reasoning=None,
    enable_lora=False,
    max_lora_rank=64,
    max_loras=2,
    worker_extension_cls=None,
)

Arguments for vLLM’s native OpenAI server as used by TRL trainers

Functions

Name	Description
do_vllm_serve	Starts the VLLM server for serving LLM models used for online RL

do_vllm_serve

cli.vllm_serve.do_vllm_serve(config, cli_args)

Starts the VLLM server for serving LLM models used for online RL

Args :param config: Parsed dict of the YAML config :param cli_args: dict of additional command-line arguments of type VllmServeCliArgs

Returns

Name	Type	Description
		None; the process is replaced by the native vLLM server