Skip to main content

utils.trainer

utils.trainer

Module containing the Trainer class and related functions

Functions

Name Description
add_pose_position_ids use the PoSE technique to extend the context length by randomly skipping
apply_plugin_distributed_env Merges every plugin’s get_distributed_env and applies it with
create_weighted_mask Weight each contiguous run of unmasked (!= -100) labels so it sums to 1.
setup_trainer Helper method for instantiating and building a (causal or RLHF) trainer.

add_pose_position_ids

utils.trainer.add_pose_position_ids(
    sample,
    max_context_len=32768,
    split_on_token_ids=None,
    chunks=2,
)

use the PoSE technique to extend the context length by randomly skipping positions in the context. We only want to skip right before tokens in the split_on_token_ids list. We should attempt to randomly distribute the skips, but we don’t need the final position_ids to be the full context_len. There may be multiple turns in the context, so we want to make sure we take into account the maximum possible number of skips remaining in each sample.

apply_plugin_distributed_env

utils.trainer.apply_plugin_distributed_env(cfg)

Merges every plugin’s get_distributed_env and applies it with os.environ.setdefault, warning if a process group already exists. Returns the merged dict.

create_weighted_mask

utils.trainer.create_weighted_mask(labels)

Weight each contiguous run of unmasked (!= -100) labels so it sums to 1.

setup_trainer

utils.trainer.setup_trainer(
    cfg,
    train_dataset,
    eval_dataset,
    model,
    tokenizer,
    processor,
    total_num_steps,
    model_ref=None,
    peft_config=None,
)

Helper method for instantiating and building a (causal or RLHF) trainer.

Parameters

Name Type Description Default
cfg DictDefault Axolotl config object containing training parameters. required
train_dataset Dataset Dataset to use for training. required
eval_dataset EvalDataset Dataset to use for evaluation. required
model PreTrainedModel The model to train. required
tokenizer PreTrainedTokenizer Tokenizer for processing text input. required
processor ProcessorMixin | None Processor for data preparation. required
total_num_steps int | None The total number of training steps. required
model_ref PreTrainedModel | None Optional reference model for RLHF training. Default is None. None
peft_config PeftConfig | None Optional PEFT (Parameter-Efficient Fine-Tuning) configuration. Default is None. None

Returns

Name Type Description
Trainer A trainer built by HFRLTrainerBuilder or HFCausalTrainerBuilder from the provided parameters.