Progressive LLM Training

Progressive training for LLMs with 8-GPU support for 松尾研LLMコンペ2025.

Quick Start

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Setup project
git clone <repository-url>
cd progressive-llm-training
uv sync

# Start training
uv run scripts/train_progressive.py --config config/training_config_gemma3_1b.yaml
./scripts/train_gemma3_1b_8gpu.sh --strategy deepspeed

Training Stages

1. basic_cot - Basic reasoning
2. math_reasoning - Math with OpenR1-Math-220k
3. complex_reasoning - Complex reasoning with Mixture-of-Thoughts

Commands

uv sync                          # Install dependencies
uv run scripts/train_progressive.py --config config/training_config_gemma3_1b.yaml     # Single GPU
./scripts/train_gemma3_1b_8gpu.sh --strategy deepspeed                                 # 8 GPUs
uv run pytest                    # Run tests

Key Files

• config/training_config_gemma3_1b_8gpu_deepspeed.yaml - 8-GPU config
• scripts/train_progressive.py - Main training script
• scripts/train_gemma3_1b_8gpu.sh - 8-GPU launcher
• src/progressive_model.py - Core model implementation

Ready to train! 🚀