monkeypatch.models.recurrent_support

monkeypatch.models.recurrent_support

Which recurrent / linear-attention architectures can train on packed sequences.

A packed row concatenates documents, and attention isolates them through the mask or cu_seqlens. Recurrent mixers (Mamba, GatedDeltaNet, KDA, short convolutions) carry state along the row instead, so each one needs a boundary signal (seq_idx or cu_seqlens) threaded into its kernel, and each rank under context parallelism needs the previous rank’s final state. Neither failure raises: the loss just quietly trains on cross-document state.

Functions

Name	Description
validate_recurrent_model_config	Raise when a recurrent architecture would silently train on leaked state.

validate_recurrent_model_config

monkeypatch.models.recurrent_support.validate_recurrent_model_config(cfg)

Raise when a recurrent architecture would silently train on leaked state.