cli.utils.lora_merge

cli.utils.lora_merge

Functions

Name	Description
adapter_has_trainable_token_deltas	Return whether a PEFT adapter carries selectively trainable token rows.
copy_non_model_files	Copy all non-model files to the output directory.
find_lora_weights	Find corresponding LoRA A and B weights for a given key.
get_model_shards	Find all model shards in the given path.
merge_lora_sharded_efficient	Memory-efficient LoRA merging that processes shards individually

adapter_has_trainable_token_deltas

cli.utils.lora_merge.adapter_has_trainable_token_deltas(lora_adapter_path)

Return whether a PEFT adapter carries selectively trainable token rows.

copy_non_model_files

cli.utils.lora_merge.copy_non_model_files(input_path, output_path, model_shards)

Copy all non-model files to the output directory.

Parameters

Name	Type	Description	Default
input_path	Path	Source directory	required
output_path	Path	Destination directory	required
model_shards	list[Path]	List of model shard files to skip	required

find_lora_weights

cli.utils.lora_merge.find_lora_weights(lora_state, key, weight_renamings=None)

Find corresponding LoRA A and B weights for a given key.

Also tries keys after applying weight renamings (from transformers v5 conversion mappings) in case the checkpoint key names differ from the runtime model key names used by the LoRA adapter.

get_model_shards

cli.utils.lora_merge.get_model_shards(model_path)

Find all model shards in the given path.

merge_lora_sharded_efficient

cli.utils.lora_merge.merge_lora_sharded_efficient(
    base_model_path,
    lora_adapter_path,
    output_path,
    device='cpu',
    safe_tensors=True,
    simulate_nf4=False,
    simulate_nf4_experts=False,
    nf4_blocksize=None,
    nf4_double_quant=True,
    nf4_backend='bitsandbytes',
    nf4_skips=None,
    nf4_dtype=None,
    bnb_skip_modules=None,
    staged_nf4=False,
    trust_remote_code=False,
    dequant=False,
    override_quantizer=False,
    revision=None,
    shard_sink=None,
    merge_compile=False,
    merge_compile_precision='strict',
    lora_merge_aware=None,
)

Memory-efficient LoRA merging that processes shards individually without loading the full model into memory.

if True, dequantize every quantized weight and write a bf16 checkpoint (strips

quantization_config). Default False = FORMAT-PRESERVING: LoRA-targeted quantized weights are dequantized, the delta folded, then re-quantized back to the SAME format (fp8 stays fp8, nvfp4 stays nvfp4), so a large quantized base does not double in size.

override_quantizer: proceed despite a quantizer-identity mismatch on a merge-aware adapter (encoder version drift); see _resolve_nvfp4_scale_mode.

Parameters

Name	Type	Description	Default
merge_compile	bool	Compile strict FP32 merge arithmetic for merge-aware NVFP4 expert chunks. Other merge paths remain eager.	False
merge_compile_precision	str	“strict” validates compiled chunks against PEFT; “relaxed” uses compiler defaults without numerical equivalence checks.	'strict'
shard_sink	Optional[Callable[[dict[str, torch.Tensor], Path, dict], None]]	Optional writer receiving each merged CPU shard, its destination, and safetensors metadata. Replaces the default shard write.	None
nf4_backend	str	NF4 implementation used for the training base.	'bitsandbytes'
nf4_skips	Optional[set[str]]	Resolved module exclusions from nf4_skip_modules. When provided, weights are selected the way the staged loader selected them (Linear weights and fused experts outside the exclusions); when None on bitsandbytes, transformers’ own Linear4bit selection is reproduced.	None
nf4_dtype	Optional[torch.dtype]	Training weight dtype used before quantization.	None
bnb_skip_modules	Optional[list[str]]	llm_int8_skip_modules the bitsandbytes training load used; None means transformers’ default skips.	None
staged_nf4	bool	Training used CPU-staged NF4, which has no legacy merge fallback.	False
simulate_nf4	bool	Apply NF4 roundtrip to eligible base weight tensors (for QLoRA)	False
simulate_nf4_experts	bool	Apply NF4 roundtrip only to MoE expert tensors (for quantize_moe_experts). Expert tensors are identified by having “expert” in the key name and ndim >= 3.	False
trust_remote_code	bool	Whether to trust remote code when loading model config for layer-type introspection. Defaults to False for safety.	False
lora_merge_aware	Optional[str]	The lora_merge_aware mode the adapter was trained with; its merged half-precision weights are then exactly what training optimized, so their rounding is reported without a warning.	None