RMAAT Paper Accepted at ICLR 2026!
🎉 Thrilled to share that our paper “RMAAT: Astrocyte-Inspired Memory Compression and Replay for Efficient Long-Context Transformers” has been accepted at ICLR 2026!
We introduce a novel architecture inspired by astrocyte-glial cells to tackle the quadratic complexity of Transformer self-attention. Our method demonstrates competitive accuracy and substantial efficiency gains on the Long Range Arena benchmark.
Authors: Md Zesun Ahmed Mia, Malyaban Bal, Abhronil Sengupta
| 🔗 OpenReview | arXiv |