core.trainers.mixins.checkpoints
core.trainers.mixins.checkpoints
Checkpoint save handling: unsavable FSDP optimizers and retries after disk errors.
Classes
| Name | Description |
|---|---|
| CheckpointSaveMixin | Mixin to handle saving the optimizer and scheduler if they are not savable. |
| CheckpointWriteRetryMixin | Retry checkpoint file writes that fail with a full disk, quota or I/O error. |
CheckpointSaveMixin
core.trainers.mixins.checkpoints.CheckpointSaveMixin()Mixin to handle saving the optimizer and scheduler if they are not savable.
CheckpointWriteRetryMixin
core.trainers.mixins.checkpoints.CheckpointWriteRetryMixin()Retry checkpoint file writes that fail with a full disk, quota or I/O error.
Functions
| Name | Description |
|---|---|
| with_checkpoint_write_retry | Run a _save_checkpoint override inside the trainer’s write-retry scope. |
with_checkpoint_write_retry
core.trainers.mixins.checkpoints.with_checkpoint_write_retry(method)Run a _save_checkpoint override inside the trainer’s write-retry scope.