PaddleOCR-VL

PaddleOCR-VL-1.6 is a compact document parsing vision-language model from PaddlePaddle for OCR, table, formula, chart, seal, and spotting tasks.

This guide shows how to fine-tune PaddleOCR-VL with Axolotl’s multimodal SFT path.

Getting Started

  1. Install Axolotl following the installation guide.

  2. Run one of the fine-tuning examples:

    axolotl train examples/paddleocr-vl/paddleocr-vl-1_6-qlora.yaml
    axolotl train examples/paddleocr-vl/paddleocr-vl-1_6-full-finetune.yaml

Tips

  • The model uses its bundled chat template through processor_type: AutoProcessor; no explicit chat_template is needed.
  • Do not set trust_remote_code for this example; Axolotl’s pinned Transformers version includes the PaddleOCR-VL model and processor implementation.
  • Do not enable Liger or Cut Cross Entropy; neither path currently patches PaddleOCR-VL’s multimodal ForConditionalGeneration class.
  • PaddleOCR-VL task prompts include OCR:, Table Recognition:, Formula Recognition:, Chart Recognition:, Seal Recognition:, and Spotting:.
  • Dataset rows should use Axolotl’s multimodal messages format with image content in the user turn and the parsed text or markup in the assistant turn.
  • The QLoRA example targets the language decoder, vision encoder, and multimodal projector with LoRA adapters.