integrations.context_parallel.plugin

integrations.context_parallel.plugin

Context-parallel plugin backed by the standalone ringmaster package.

Switches attention to a ringmaster wrapper and installs batch-sharding hooks while preserving Trainer’s accumulation-window loss normalization.

Setup runs in post_trainer_create because the accelerate device mesh (the cp dim FSDP2 reduces gradients over) only exists once the trainer’s Accelerator has been constructed.

Classes

Name	Description
ContextParallelPlugin	Long-context attention (Ulysses/Ring/USP) via ringmaster.

ContextParallelPlugin

integrations.context_parallel.plugin.ContextParallelPlugin()

Long-context attention (Ulysses/Ring/USP) via ringmaster.