utils.vllm_lora_sync

utils.vllm_lora_sync

Publish LoRA adapters to a vLLM server through its native adapter routes.

Functions

Name	Description
publish_lora_adapter	Register `<sync_dir>/v<version>` under a fresh versioned adapter name.

publish_lora_adapter

utils.vllm_lora_sync.publish_lora_adapter(
    vllm_client,
    sync_dir,
    version,
    timeout,
    alias=None,
)

Each version gets its own name because vLLM re-reads an adapter reloaded in place from disk on every request and keys its prefix cache on the name alone. In-flight requests are drained first so at most the previous version can still be referenced once generation resumes.

alias is a fixed name that is re-pointed at every version, for clients that cannot follow the versioned name. It should be the served base model name: requests arriving between the unload and the reload then fall back to the base model instead of failing.