integrations.llm_compressor.utils

integrations.llm_compressor.utils

Utilities for llmcompressor integration with axolotl.

Functions

Name	Description
save_compressed_model	Synchronize processes, apply compression hooks, and save the model.

save_compressed_model

integrations.llm_compressor.utils.save_compressed_model(
    model,
    output_dir,
    trainer,
    save_compressed=False,
)

Synchronize processes, apply compression hooks, and save the model.

Parameters

Name	Type	Description	Default
model	PreTrainedModel	The model to be saved.	required
output_dir	str or bytes	Path where the model files will be written.	required
trainer	Trainer	Hugging Face Trainer for process synchronization.	required
save_compressed	bool	Write compressed tensors if True.	False