LLM Training and Tuning

1. pretraining
   1. filtered for quality
   2. include instruction tuning data
   3. synthetic data
   4. weighted sampling from different sources / categories
2. long context training
3. annealing with high quality data
4. supervised finetuning
5. RLHF / DPO

• Common LLM Settings - Google Sheets

• torchtune: Easy and Accessible Finetuning in Native PyTorch - Evan Smothers, Meta - YouTube

Optimizations

Quantized Optimizers

Fused Ops

• GitHub - unslothai/unsloth: Finetune Llama 3.2, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
• GitHub - linkedin/Liger-Kernel: Efficient Triton Kernels for LLM Training

Compile

FlexAttention

Block causal mask to pack samples

Distributed

Pretraining

Finetuning

Post Training / Alignment

Alignment and Post Training