Skip to main content
Home
Getting Started
Quickstart
Which Fine-Tuning Method Should I Use?
Installation
Inference and Merging
Support Matrix
Model Guides
Mistral Large 4
Beam
K2-Horizon
Nex-N2.5
MiniCPM5
Ling 3.0
Muse Glimmer
North Micro Vision Instruct
Shieldstral
Kimi Linear
Plano Orchestrator
MiMo
InternVL 3.5
PaddleOCR-VL
OLMo 3
Trinity
Arcee AFM
Mistral Medium 3.5
Mistral Small 4
Ministral3
Ministral3
Ministral 3 Thinking
Ministral 3 Vision
Magistral
Magistral
Magistral Thinking
Magistral Vision
Ministral
Mistral Small 3.1/3.2
Voxtral
Devstral
Mistral 7B
Llama 4
Llama 2
Qwen 3.8 Flash Next
Qwen 3.5
Qwen 3 Next
Qwen 3
Gemma 4
Gemma 3n
GLM-5.3-Flash
GLM-4.7-Flash
GLM-4.6V
GLM-4.5-Air
Apertus
GPT-OSS
Seed-OSS
Phi
SmolVLM 2
Granite 4
Liquid Foundation Models 2
Hunyuan
Jamba
Orpheus
Command Line Interface (CLI)
Telemetry
Config Reference
API Reference
Dataset Formats
Pre-training
Instruction Tuning
Conversation
Stepwise Supervised Format
Template-Free
Custom Pre-Tokenized Dataset
Deployments
Docker
Offline and Air-Gapped Deployment
Multi-GPU
Multi Node
Ray Train
SkyPilot
Hugging Face Jobs
Nebius Serverless Jobs
CoreWeave
Runcrate
AMD GPUs on HPC Systems
Mac M-series
How To Guides
MultiModal / Vision Language Models (BETA)
RLHF
Native diffusion language-model training
GRPO Training
EBFT Training
vLLM Serving for GRPO Training
Reward Modelling
Learning Rate Groups
LoRA
LoRA Optimizations
Dataset Loading
Quantization Aware Training (QAT)
Quantization with torchao
Exporting to GGUF
NVFP4 MoE LoRA
1.58-bit Finetuning
Optimizations Guide
Core Concepts
Batch size vs Gradient accumulation
Dataset Preprocessing
Streaming Datasets
Multipack (Sample Packing)
Mixed Precision Training
Optimizers
Attention
Advanced Features
FSDP + QLoRA
PyTorch ao
Custom Integrations
Context Parallelism
Gradient Checkpointing, Activation Offloading, and Layer Offloading
N-D Parallelism (Beta)
MoE Expert Quantization
Troubleshooting
FAQ
Training Stability & Debugging
Debugging
NCCL
On this page
integrations.grokfast.optimizer
integrations.grokfast.optimizer
integrations.grokfast.optimizer