models.mamba.configuration_mamba

models.mamba.configuration_mamba

HF Transformers MambaConfig

Classes

Name	Description
MambaConfig	modeling configuration for state space model/mamba

MambaConfig

models.mamba.configuration_mamba.MambaConfig(
    vocab_size=50280,
    d_model=2560,
    n_layer=64,
    rms_norm=True,
    residual_in_fp32=True,
    fused_add_norm=True,
    pad_vocab_size_multiple=8,
    pad_token_id=50277,
    bos_token_id=0,
    eos_token_id=0,
    tie_word_embeddings=False,
    **kwargs,
)

modeling configuration for state space model/mamba