Researchers at Prefrontal AI, in collaboration with ETH Zurich, have developed Latent Sparse Attention (LSA), an attention mechanism designed to handle ultra-long context windows in large language models more efficiently. The new method, detailed in a paper published on arXiv, addresses the computational expense of full attention at context lengths exceeding one million tokens.LSA operates by hierarchically clustering keys within a learned latent space. Instead of a query evaluating every individual key, it scores this hierarchy of clusters. This approach aims to reduce the computational cost associated with prefill, decode, and training phases when models process extended contexts.The researchers propose two variants of LSA: Distilled-LSA (D-LSA) and Native-LSA (N-LSA). D-LSA can be applied to models that have already been trained through a distillation process. N-LSA is designed for native training, where it learns solely on the next-token loss.In evaluations, D-LSA demonstrated substantial speed improvements on models such as GLM-5.3-FLASH and QWEN-3.8-27B. At a context length of 10 million tokens, D-LSA accelerated a single attention layer's prefill by 22.7 times on GLM-5.3-FLASH and 72.4 times on QWEN-3.8-27B, compared to their original sparse attention and full attention, respectively. Decode speeds also increased, by 4.6 times for GLM-5.3-FLASH and 42.5 times for QWEN-3.8-27B, while maintaining comparable accuracy across various tasks.N-LSA, when trained natively, achieved accuracy levels similar to full attention. It also offered a training speedup of 1.6 times per layer at a context size of 128,000 tokens.The development of LSA responds to the increasing demand for models capable of managing long-horizon tasks, such as open-ended scientific research, which benefit from extensive context windows. Existing sparse attention methods often incur quadratic costs with increasing context size or experience accuracy degradation at ultra-long contexts. The authors of the paper note that at 10 million tokens, 98% of the prefill time in a sparse attention layer is spent on indexing, indicating that the cost of finding relevant keys dominates the overall expense.Latent space clustering, a foundational concept for LSA, involves partitioning data by revealing hidden geometric structures in low-dimensional representations. This technique is applied across various fields, including computer vision, natural language processing, and network analysis, to group complex data efficiently and meaningfully. The "latent space" itself represents a compressed, lower-dimensional data representation learned by a model, where similar items are positioned closer together.The research team behind LSA includes Ales Kubicek, Andrea Caciolai, Liangzu Peng, Sukrit Sumant, Benjamin Hoover, Fraser Greenlee, Ruairidh M. Battleday, and James C.R. Whittington from Prefrontal AI, along with Radim Urban and Torsten Hoefler from ETH Zurich.