utils.optimizers.qgalore

utils.optimizers.qgalore

Helpers for the Q-GaLore optimizer integration.

Functions

Name Description
build_qgalore_param_groups Two param-groups: 2D weights matching target_modules get the Q-GaLore
patch_q_galore_for_modern_bnb q-galore-torch==1.0 targets the pre-0.44 bnb optimizer API; adapt it to

build_qgalore_param_groups

utils.optimizers.qgalore.build_qgalore_param_groups(
    model,
    target_modules,
    *,
    rank,
    update_proj_gap,
    scale,
    proj_type,
    proj_quant,
    proj_bits,
    proj_group_size,
    cos_threshold,
    gamma_proj,
    queue_size,
)

Two param-groups: 2D weights matching target_modules get the Q-GaLore projection keys; everything else (norms, biases, embeddings) is plain AdamW.

patch_q_galore_for_modern_bnb

utils.optimizers.qgalore.patch_q_galore_for_modern_bnb()

q-galore-torch==1.0 targets the pre-0.44 bnb optimizer API; adapt it to the installed bnb:

  • bnb >=0.44 inserted (beta3, alpha) into optimizer_update_8bit_blockwise and optimizer_update_32bit: re-emit q_galore’s legacy positional layout.
  • bnb >=0.50 removed the non-blockwise 8-bit path: F.optimizer_update_8bit and F.percentile_clipping are gone, get_config() lost the percentile_clipping/block_wise keys, and — silently worst — Optimizer2State.__init__ lost those two params so q_galore’s positional super().__init__ call lands them on max_unorm/skip_zeros.

No-op on bnb <0.44.