loaders.patch_manager
loaders.patch_manager
Patch manager class implementation to complement axolotl.loaders.ModelLoader.
Applies pre- and post-model load patches for various fixes and optimizations.
Classes
| Name | Description |
|---|---|
| PatchManager | Manages the application of patches during the model loading process. |
PatchManager
loaders.patch_manager.PatchManager(cfg, model_config, inference=False)Manages the application of patches during the model loading process.
Attributes
| Name | Description |
|---|---|
| has_flash_attn | Check if flash attention is installed. |
Methods
| Name | Description |
|---|---|
| apply_post_model_load_patches | Apply patches that require the model instance. |
| apply_post_plugin_pre_model_load_patches | Apply post plugin-pre_model_load load patches based on config. |
| apply_pre_config_load_patches | Apply patches that must be set up before config loading. |
| apply_pre_model_load_patches | Apply pre-model load patches based on config. |
| apply_pre_tokenizer_load_patches | Apply patches that must be set up before tokenizer loading. |
apply_post_model_load_patches
loaders.patch_manager.PatchManager.apply_post_model_load_patches(model)Apply patches that require the model instance.
apply_post_plugin_pre_model_load_patches
loaders.patch_manager.PatchManager.apply_post_plugin_pre_model_load_patches()Apply post plugin-pre_model_load load patches based on config.
apply_pre_config_load_patches
loaders.patch_manager.PatchManager.apply_pre_config_load_patches(cfg)Apply patches that must be set up before config loading. This is for patches that intercept remote code loading from HuggingFace, which needs to be in place before AutoConfig.from_pretrained() is called.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| cfg | DictDefault | Configuration dictionary with model and training settings. | required |
apply_pre_model_load_patches
loaders.patch_manager.PatchManager.apply_pre_model_load_patches()Apply pre-model load patches based on config.
apply_pre_tokenizer_load_patches
loaders.patch_manager.PatchManager.apply_pre_tokenizer_load_patches(cfg)Apply patches that must be set up before tokenizer loading. This is for patches that intercept remote code loading from HuggingFace, which needs to be in place before AutoTokenizer.from_pretrained() is called.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| cfg | DictDefault | Configuration dictionary with model and training settings. | required |