Skip to main content

core.trainers.mixins.lora_merge_check

core.trainers.mixins.lora_merge_check

Merge-rounding diagnostic for LoRA weight sync to vLLM.

Classes

Name Description
LoraMergeCheckMixin Measure how much of the LoRA update a merged vLLM weight sync keeps.

LoraMergeCheckMixin

core.trainers.mixins.lora_merge_check.LoraMergeCheckMixin(*args, **kwargs)

Measure how much of the LoRA update a merged vLLM weight sync keeps.

Runs on the main process at the first sync after training starts and then every vllm_merge_check_interval optimizer steps, logging lora_merge/retained and lora_merge/changed_fraction.