Small Foundational Models

• SmolLM - blazingly fast and remarkably powerful
• [2402.14905] MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
• [2404.06395] MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
• [2410.11190] Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilities
• [2312.09241] TinyGSM: achieving >80% on GSM8k with small language models

Vision Language Models

• [2412.01822v1] VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Models
• Introducing Moondream 0.5B: The World’s Smallest Vision-Language Model