utils.optimizers.qgalore

utils.optimizers.qgalore

Helpers for the Q-GaLore optimizer integration.

Functions

Name	Description
build_qgalore_param_groups	Two param-groups: 2D weights matching target_modules get the Q-GaLore
patch_q_galore_for_modern_bnb	bnb >=0.44 inserted (beta3, alpha) into optimizer_update_8bit_blockwise

build_qgalore_param_groups

utils.optimizers.qgalore.build_qgalore_param_groups(
    model,
    target_modules,
    *,
    rank,
    update_proj_gap,
    scale,
    proj_type,
    proj_quant,
    proj_bits,
    proj_group_size,
    cos_threshold,
    gamma_proj,
    queue_size,
)

Two param-groups: 2D weights matching target_modules get the Q-GaLore projection keys; everything else (norms, biases, embeddings) is plain AdamW.

patch_q_galore_for_modern_bnb

utils.optimizers.qgalore.patch_q_galore_for_modern_bnb()

bnb >=0.44 inserted (beta3, alpha) into optimizer_update_8bit_blockwise and optimizer_update_32bit; q-galore-torch==1.0 still calls the legacy positional layout. Swap q_galore’s bnb handle for one that re-emits the modern layout. No-op on older bnb.