integrations.lm_eval.cli
integrations.lm_eval.cli
axolotl CLI for running lm_eval tasks
Functions
| Name | Description |
|---|---|
| get_model_path | Determine which model path to use for evaluation. |
| lm_eval | use lm eval to evaluate a trained language model |
get_model_path
integrations.lm_eval.cli.get_model_path(cfg)Determine which model path to use for evaluation.
Priority order (highest to lowest): 1. lm_eval_model - Explicit model path override 2. hub_model_id - Model pushed to HuggingFace Hub 3. None - Falls back to output_dir in build_lm_eval_command
Returns
| Name | Type | Description |
|---|---|---|
| str | None | Model path string or None to use output_dir fallback |
lm_eval
integrations.lm_eval.cli.lm_eval(config, cloud=None)use lm eval to evaluate a trained language model