Maximal Update Parametrization (μP)

• GitHub - microsoft/mup: maximal update parametrization (µP)
• The Practitioner’s Guide to the Maximal Update Parameterization | EleutherAI Blog
• µTransfer: A technique for hyperparameter tuning of enormous neural networks - Microsoft Research
• [2404.05728] A Large-Scale Exploration of $\mu$-Transfer
• x.com