Pluralis Research
Decentralized model-parallel training protocol enabling low-bandwidth collaboration and unmaterializable models
Performances
RD Popularity Index
RD Growth Index
Comparison
Fundraising
Follow Updates
Follow Lists
People
About Pluralis Research
Pluralis Research studies and builds Protocol Learning, in which a foundation model is trained by many participants and no single one ever holds the full weights. Its Agora system slices a model into pipeline stages by Transformer layer and spreads them across heterogeneous nodes — home GPUs, cloud instances, marketplace capacity — with several devices replicating each stage. Activations crossing stage boundaries are compressed roughly 100-fold by Subspace Networks, peers in the same stage exchange sparse parameter slices through asynchronous averaging, and a metadata store handles registration, progress and matchmaking, so peers can join or leave without halting a run.
A run has three kinds of participants: nodes supplying compute and bandwidth, a coordinator sequencing stages, and contributors submitting data and evaluations. The model is cut into pipeline stages by Transformer layer, each stage replicated across devices, activations compressed by Subspace Networks as they cross stages, and peers within a stage exchanging parameter slices through asynchronous sparse averaging; the metadata store tracks registration, progress and matchmaking, and replication groups redistribute work as peers join or leave. In May 2026 an 8B model completed a public run of 531.8 billion tokens, and the September 13B run moved to peer-to-peer microbatch routing.
Pluralis Research was founded in January 2024 by Alexander Long, a former FAANG researcher, with the goal of developing true open-source AI through decentralized training. In March 2025, the company announced a $7.6 million seed round co-led by USV and CoinFund, with participation from Balaji Srinivasan and HuggingFace co-founder Clem Delangue. The company pioneered Protocol Learning, a method allowing models to be trained across distributed networks without any single party holding full weights. A key milestone was a public training run of a 7.5B-parameter OLMo-style model over the internet using 300+ globally distributed consumer GPUs for over three weeks, demonstrating the feasibility of decentralized training at scale. This validated their approach and laid groundwork for future open model development.
In 2026 Pluralis published the Agora technical report and results from Pluralis-8B, an 8.6 billion parameter model trained on 500 billion tokens across 330 contributor nodes connected only by ordinary internet links, with no participant ever holding the full weights. The team released details of its multi-party training stack in January 2026 and replicated a full post-training pipeline on a single node in March 2026.
Pluralis Research's work reached ICLR, ICML and NeurIPS in 2026: Factored Gossip DiLoCo replaces exact outer synchronization with approximate gossip, NuMuon constrains update nuclear norms to improve post-compression quality, and AsyncMesh makes data and pipeline parallelism fully asynchronous. In July 2026 the team post-trained an 8 billion parameter mixture-of-experts model with GRPO on 14 consumer Macs and one B200, lifting held-out pass@1 on agentic search from 29 to 63 percent.
Pluralis Research's Future strategy and roadmap: Pluralis will keep running open multi-party training and advance Node0 and Agora fault tolerance and compression, making decentralized training at the scale of hundreds of contributing nodes more robust. After papers like Factored Gossip DiLoCo, NuMuon and AsyncMesh reached top conferences, the team plans to turn protocol learning into a production system where contributors earn model-ownership shares proportional to compute supplied rather than mere rewards. The next step is to package the GRPO post-training flow for 8B-class mixture-of-experts models into a reusable training stack open to more research labs and developers.
Pluralis Research was founded in early 2024 by Alexander Long, who serves as founder and CEO. Long holds a doctorate in computer science and spent more than three years as an AI engineer at Amazon.
He assembled seven other researchers from Amazon and academia. Sameera Ramasinghe focuses on communication-efficient model parallelism, Ajanthan Thalaiyasingam on asynchronous pipelines and optimisation, while Gil Avraham and Yan Zuo handle systems and evaluation.