monkeypatch.relora
monkeypatch.relora
Implements the ReLoRA training procedure from https://arxiv.org/abs/2307.05695, minus the initial full fine-tune.
Classes
| Name | Description |
|---|---|
| ReLoRACallback | Callback to merge LoRA weights into the base model and save full-weight checkpoints |
ReLoRACallback
monkeypatch.relora.ReLoRACallback(cfg)Callback to merge LoRA weights into the base model and save full-weight checkpoints