loaders.adapter

loaders.adapter

Adapter loading functionality, including LoRA / QLoRA and associated utils

Functions

Name	Description
setup_quantized_meta_for_peft	Replaces quant_state.to with a dummy function to prevent PEFT from moving quant_state to meta device
setup_quantized_peft_meta_for_training	Replaces dummy quant_state.to method with the original function to allow training to continue

setup_quantized_meta_for_peft

loaders.adapter.setup_quantized_meta_for_peft(model)

Replaces quant_state.to with a dummy function to prevent PEFT from moving quant_state to meta device

setup_quantized_peft_meta_for_training

loaders.adapter.setup_quantized_peft_meta_for_training(model)

Replaces dummy quant_state.to method with the original function to allow training to continue