Ring Attention

• [2310.01889] Ring Attention with Blockwise Transformers for Near-Infinite Context

• ML Scalability & Performance Reading Group Session 4: Ring Attention - YouTube

• Ring Attention Explained | Coconut Mode