DeepMind’s PEER scales language models with millions of tiny experts

mixture of millions of expertsParameter-Efficient Expert Retrieval (PEER) is a technique that allows LLMs to scale to millions of experts and remain resource efficient.Read More