monkeypatch.trainer.trl_vllm

monkeypatch.trainer.trl_vllm

Monkeypatches for TRL’s vLLM integration and trainer utils.

Adds: - VLLMClient.batch_update_named_params: weight sync over vLLM’s native NCCL weight-transfer engine, inside a single weight update, with lazy communicator init - extract_logprobs: NaN→0.0 fix (prevents downstream NaN propagation) - split_tensor_dict / shuffle_sequence_dict: scalar type handling (int/float/bool passthrough) - LLM (colocate): forward vllm: engine args TRL has no config field for (enforce_eager, …)

Functions

Name	Description
colocate_vllm_engine_kwargs	vllm: options that only reach a colocated engine via the LLM() call itself.
patch_trl_vllm	Apply all TRL vLLM monkeypatches.
patch_vllm_colocate_engine_kwargs	Merge extra engine kwargs into the LLM() call VLLMGeneration makes in colocate mode.

colocate_vllm_engine_kwargs

monkeypatch.trainer.trl_vllm.colocate_vllm_engine_kwargs(vllm_cfg)

vllm: options that only reach a colocated engine via the LLM() call itself.

patch_trl_vllm

monkeypatch.trainer.trl_vllm.patch_trl_vllm()

Apply all TRL vLLM monkeypatches.

patch_vllm_colocate_engine_kwargs

monkeypatch.trainer.trl_vllm.patch_vllm_colocate_engine_kwargs(engine_kwargs)

Merge extra engine kwargs into the LLM() call VLLMGeneration makes in colocate mode.

TRL hardcodes that call, so this is the only way for e.g. enforce_eager (needed to avoid vLLM’s custom all-reduce CUDA errors on some setups) to reach the engine.