News from the AI & ML world
DeeperML - #alphaevolve
|
@www.marktechpost.com
//
Large Language Models (LLMs) are facing significant challenges in handling real-world conversations, particularly those involving multiple turns and underspecified tasks. Researchers from Microsoft and Salesforce have recently revealed a substantial performance drop of 39% in LLMs when confronted with such conversational scenarios. This decline highlights the difficulty these models have in maintaining contextual coherence and delivering accurate outcomes as conversations evolve and new information is incrementally introduced. Instead of flexibly adjusting to changing user inputs, LLMs often make premature assumptions, leading to errors that persist throughout the dialogue.
These findings underscore a critical gap in how LLMs are currently evaluated. Traditional benchmarks often rely on single-turn, fully-specified prompts, which fail to capture the complexities of real-world interactions where information is fragmented and context must be actively constructed from multiple exchanges. This discrepancy between evaluation methods and actual conversational demands contributes to the challenges LLMs face in integrating underspecified inputs and adapting to evolving user needs. The research emphasizes the need for new evaluation frameworks that better reflect the dynamic and iterative nature of real-world conversations. In contrast to these challenges, Google's DeepMind has developed AlphaEvolve, an AI agent designed to optimize code and reclaim computational resources. AlphaEvolve autonomously rewrites critical code, resulting in a 0.7% reduction in Google's overall compute usage. This system not only pays for itself but also demonstrates the potential for AI agents to significantly improve efficiency in complex computational environments. AlphaEvolve's architecture, featuring a controller, fast-draft models, deep-thinking models, automated evaluators, and versioned memory, represents a production-grade approach to agent engineering. This allows for continuous improvement at scale.
Share:
References :
Classification:
@Google DeepMind Blog
//
Google DeepMind has announced AlphaEvolve, an AI coding agent that enhances the problem-solving abilities of large language models (LLMs) like Gemini. Unveiled in May 2025, AlphaEvolve utilizes a blend of LLMs' creative generation capabilities with genetic algorithms, marking a significant advancement in AI's capacity to automate and optimize coding processes. This system is not merely a code generator but a self-evolving AI designed to discover novel and efficient algorithms through automated code generation and testing.
AlphaEvolve operates through a process akin to natural selection for software. It begins with an initial code, potentially a "skeleton" provided by a human, and iteratively refines it. The system intelligently crafts prompts for the underlying Gemini LLM, instructing it to act as a world-class expert in a specific domain, utilizing context from previous attempts. The LLM then generates a diverse pool of candidate solutions, exploring different approaches to the problem at hand. The AI agent has already demonstrated its prowess by improving Google’s data center energy efficiency by 0.7%, optimizing matrix multiplications to speed up Gemini’s training by 23%, and co-designing a portion of Google’s next AI chip. Furthermore, AlphaEvolve has tackled over 50 open problems in mathematics, even discovering a new lower bound for the 300-year-old kissing number problem in 11-dimensional space, surpassing the progress of expert mathematicians. This represents a significant stride toward production-grade AI agent engineering.
Share:
References :
Classification:
@Google DeepMind Blog
//
Google DeepMind has unveiled AlphaEvolve, an AI agent powered by Gemini, that is revolutionizing algorithm discovery and scientific optimization. This innovative system combines the creative problem-solving capabilities of large language models (LLMs) with automated evaluators to verify solutions and iteratively improve upon promising ideas. AlphaEvolve represents a significant leap in AI's ability to develop sophisticated algorithms for both scientific challenges and everyday computing problems, expanding upon previous work by evolving entire codebases rather than single functions.
AlphaEvolve has already demonstrated its potential by breaking a 56-year-old mathematical record, discovering a more efficient matrix multiplication algorithm that had eluded human mathematicians. The system leverages an ensemble of state-of-the-art large language models, including Gemini Flash and Gemini Pro, to propose and refine algorithmic solutions as code. These programs are then evaluated using automated metrics, providing an objective assessment of accuracy and quality. This approach makes AlphaEvolve particularly valuable in domains where progress can be clearly and systematically measured, such as math and computer science. The impact of AlphaEvolve extends beyond theoretical breakthroughs, with algorithms discovered by the system already deployed across Google's computing ecosystem. Notably, AlphaEvolve has enhanced the efficiency of Google's data centers, chip design, and AI training processes, including the training of the large language models underlying AlphaEvolve itself. It has also optimized a matrix multiplication kernel used to train Gemini models and found new solutions to open mathematical problems. By optimizing Google’s massive cluster management system, Borg, AlphaEvolve recovers an average of 0.7% of Google’s worldwide computing resources continuously, which translates to substantial cost savings.
Share:
References :
Classification:
|
BenchmarksBlogsResearch Tools |