News from the AI & ML world
DeeperML - #llm
|
@www.marktechpost.com
//
Mistral AI has released Mistral Small 3.2, an updated version of its open-source model, Mistral-Small-3.2-24B-Instruct-2506, building upon the earlier Mistral-Small-3.1-24B-Instruct-2503. This update focuses on enhancing the model’s overall reliability and efficiency, particularly in handling complex instructions, minimizing repetitive outputs, and maintaining stability during function-calling scenarios. The improvements aim to refine specific behaviors such as instruction following, output stability, and function calling robustness without altering the core architecture.
A significant enhancement in Mistral Small 3.2 is its improved accuracy in executing precise instructions. Benchmark scores reflect this improvement, with the model achieving 65.33% accuracy on the Wildbench v2 instruction test, up from 55.6% for its predecessor. Performance on the challenging Arena Hard v2 test nearly doubled, increasing from 19.56% to 43.1%, demonstrating an enhanced ability to understand and execute intricate commands accurately. Internally, Mistral’s accuracy rose from 82.75% in Small 3.1 to 84.78% in Small 3.2. Mistral Small 3.2 also addresses the issue of repetitive errors by significantly reducing instances of infinite or repetitive output, a common problem in extended conversational scenarios. Internal evaluations show a decrease in infinite generation errors by nearly half, from 2.11% in Small 3.1 to 1.29%. The updated model also demonstrates enhanced capability in calling functions, making it more suitable for automation tasks. Additionally, Mistral AI emphasizes its compliance with EU regulations like GDPR and the EU AI Act, making it an appealing choice for developers in the region. Recommended read:
References :
@www.microsoft.com
//
References:
www.microsoft.com
, Microsoft Research
Microsoft is making significant advancements in artificial intelligence, focusing on improved reasoning in language models and enhanced weather forecasting capabilities. New methods are being developed to boost reasoning in both small and large language models, combining symbolic logic, mathematical rigor, and adaptive planning. These techniques are designed to enable AI models to tackle complex, real-world problems across various fields, potentially transforming AI into a more reliable partner in domains like scientific research and healthcare.
A new AI model named Aurora, developed by Microsoft, can forecast hurricanes and sandstorms up to 5,000 times faster than conventional weather models powered by supercomputers. Aurora outperformed existing systems in predicting weather conditions over a 14-day period in 91% of cases. The model is trained on over 1 million hours of global atmospheric data, including weather station readings, satellite images, and radar measurements, representing one of the largest datasets used to train a weather AI model. To address the growing demand for data control in Europe, Microsoft is expanding its Sovereign Cloud offerings. This includes solutions that ensure European data remains within Europe, handled exclusively by Microsoft employees based in the region. The Sovereign Public Cloud offers tools and options for customer-controlled encryption and simplified configurations, providing organizations in Europe with greater control over their data. The cloud is offered across all existing European data center regions. Recommended read:
References :
@www.marktechpost.com
//
Apple researchers are challenging the perceived reasoning capabilities of Large Reasoning Models (LRMs), sparking debate within the AI community. A recent paper from Apple, titled "The Illusion of Thinking," suggests that these models, which generate intermediate thinking steps like Chain-of-Thought reasoning, struggle with fundamental reasoning tasks. The research indicates that current evaluation methods relying on math and code benchmarks are insufficient, as they often suffer from data contamination and fail to assess the structure or quality of the reasoning process.
To address these shortcomings, Apple researchers introduced controllable puzzle environments, including the Tower of Hanoi, River Crossing, Checker Jumping, and Blocks World, allowing for precise manipulation of problem complexity. These puzzles require diverse reasoning abilities, such as constraint satisfaction and sequential planning, and are free from data contamination. The Apple paper concluded that state-of-the-art LRMs ultimately fail to develop generalizable problem-solving capabilities, with accuracy collapsing to zero beyond certain complexities across different environments. However, the Apple research has faced criticism. Experts, like Professor Seok Joon Kwon, argue that Apple's lack of high-performance hardware, such as a large GPU-based cluster comparable to those operated by Google or Microsoft, could be a factor in their findings. Some argue that the models perform better on familiar puzzles, suggesting that their success may be linked to training exposure rather than genuine problem-solving skills. Others, such as Alex Lawsen and "C. Opus," argue that the Apple researchers' results don't support claims about fundamental reasoning limitations, but rather highlight engineering challenges related to token limits and evaluation methods. Recommended read:
References :
nftjedi@chatgptiseatingtheworld.com
//
Apple researchers recently published a study titled "The Illusion of Thinking," suggesting that advanced language models (LLMs) struggle with true reasoning, relying instead on pattern matching. The study presented findings based on tasks like the Tower of Hanoi puzzle, where models purportedly failed when complexity increased, leading to the conclusion that these models possess limited problem-solving abilities. However, these conclusions are now under scrutiny, with critics arguing the experiments were not fairly designed.
Alex Lawsen of Open Philanthropy has published a counter-study challenging the foundations of Apple's claims. Lawsen argues that models like Claude, Gemini, and OpenAI's latest systems weren't failing due to cognitive limits, but rather because the evaluation methods didn't account for key technical constraints. One issue raised was that models were often cut off from providing full answers because they neared their maximum token limit, a built-in cap on output text, which Apple's evaluation counted as a reasoning failure rather than a practical limitation. Another point of contention involved the River Crossing test, where models faced unsolvable problem setups. When the models correctly identified the tasks as impossible and refused to attempt them, they were still marked wrong. Furthermore, the evaluation system strictly judged outputs against exhaustive solutions, failing to credit models for partial but correct answers, pattern recognition, or strategic shortcuts. To illustrate, Lawsen demonstrated that when models were instructed to write a program to solve the Hanoi puzzle, they delivered accurate, scalable solutions even with 15 disks, contradicting Apple's assertion of limitations. Recommended read:
References :
Kristin Sestito@hiddenlayer.com
//
Cybersecurity researchers have recently unveiled a novel attack, dubbed TokenBreak, that exploits vulnerabilities in the tokenization process of large language models (LLMs). This technique allows malicious actors to bypass safety and content moderation guardrails with minimal alterations to text input. By manipulating individual characters, attackers can induce false negatives in text classification models, effectively evading detection mechanisms designed to prevent harmful activities like prompt injection, spam, and the dissemination of toxic content. The TokenBreak attack highlights a critical flaw in AI security, emphasizing the need for more robust defenses against such exploitation.
The TokenBreak attack specifically targets the way models tokenize text, the process of breaking down raw text into smaller units or tokens. HiddenLayer researchers discovered that models using Byte Pair Encoding (BPE) or WordPiece tokenization strategies are particularly vulnerable. By adding subtle alterations, such as adding an extra letter to a word like changing "instructions" to "finstructions", the meaning of the text is still understood. This manipulation causes different tokenizers to split the text in unexpected ways, effectively fooling the AI's detection mechanisms. The fact that the altered text remains understandable underscores the potential for attackers to inject malicious prompts and bypass intended safeguards. To mitigate the risks associated with the TokenBreak attack, experts recommend several strategies. Selecting models that use Unigram tokenizers, which have demonstrated greater resilience to this type of manipulation, is crucial. Additionally, organizations should ensure tokenization and model logic alignment and implement misclassification logging to better detect and respond to potential attacks. Understanding the underlying protection model's family and its tokenization strategy is also critical. The TokenBreak attack serves as a reminder of the ever-evolving landscape of AI security and the importance of proactive measures to protect against emerging threats. Recommended read:
References :
Carl Franzen@AI News | VentureBeat
//
Mistral AI has launched its first reasoning model, Magistral, signaling a commitment to open-source AI development. The Magistral family features two models: Magistral Small, a 24-billion parameter model available with open weights under the Apache 2.0 license, and Magistral Medium, a proprietary model accessible through an API. This dual release strategy aims to cater to both enterprise clients seeking advanced reasoning capabilities and the broader AI community interested in open-source innovation.
Mistral's decision to release Magistral Small under the permissive Apache 2.0 license marks a significant return to its open-source roots. The license allows for the free use, modification, and distribution of the model's source code, even for commercial purposes. This empowers startups and established companies to build and deploy their own applications on top of Mistral’s latest reasoning architecture, without the burdens of licensing fees or vendor lock-in. The release serves as a powerful counter-narrative, reaffirming Mistral’s dedication to arming the open community with cutting-edge tools. Magistral Medium demonstrates competitive performance in the reasoning arena, according to internal benchmarks released by Mistral. The model was tested against its predecessor, Mistral-Medium 3, and models from Deepseek. Furthermore, Mistral's Agents API's Handoffs feature facilitates smart, multi-agent workflows, allowing different agents to collaborate on complex tasks. This enables modular and efficient problem-solving, as demonstrated in systems where agents collaborate to answer inflation-related questions. Recommended read:
References :
@www.marktechpost.com
//
DeepSeek, a Chinese AI startup, has launched an updated version of its R1 reasoning AI model, named DeepSeek-R1-0528. This new iteration brings the open-source model near parity with proprietary paid models like OpenAI’s o3 and Google’s Gemini 2.5 Pro in terms of reasoning capabilities. The model is released under the permissive MIT License, enabling commercial use and customization, marking a commitment to open-source AI development. The model's weights and documentation are available on Hugging Face, facilitating local deployment and API integration.
The DeepSeek-R1-0528 update introduces substantial enhancements in the model's ability to handle complex reasoning tasks across various domains, including mathematics, science, business, and programming. DeepSeek attributes these improvements to leveraging increased computational resources and applying algorithmic optimizations in post-training. Notably, the accuracy on the AIME 2025 test has surged from 70% to 87.5%, demonstrating deeper reasoning processes with an average of 23,000 tokens per question, compared to the previous version's 12,000 tokens. Alongside enhanced reasoning, the updated R1 model boasts a reduced hallucination rate, which contributes to more reliable and consistent output. Code generation performance has also seen a boost, positioning it as a strong contender in the open-source AI landscape. DeepSeek provides instructions on its GitHub repository for those interested in running the model locally and encourages community feedback and questions. The company aims to provide accessible AI solutions, underscored by the availability of a distilled version of R1-0528, DeepSeek-R1-0528-Qwen3-8B, designed for efficient single-GPU operation. Recommended read:
References :
@www.marktechpost.com
//
DeepSeek has released a major update to its R1 reasoning model, dubbed DeepSeek-R1-0528, marking a significant step forward in open-source AI. The update boasts enhanced performance in complex reasoning, mathematics, and coding, positioning it as a strong competitor to leading commercial models like OpenAI's o3 and Google's Gemini 2.5 Pro. The model's weights, training recipes, and comprehensive documentation are openly available under the MIT license, fostering transparency and community-driven innovation. This release allows researchers, developers, and businesses to access cutting-edge AI capabilities without the constraints of closed ecosystems or expensive subscriptions.
The DeepSeek-R1-0528 update brings several core improvements. The model's parameter count has increased from 671 billion to 685 billion, enabling it to process and store more intricate patterns. Enhanced chain-of-thought layers deepen the model's reasoning capabilities, making it more reliable in handling multi-step logic problems. Post-training optimizations have also been applied to reduce hallucinations and improve output stability. In practical terms, the update introduces JSON outputs, native function calling, and simplified system prompts, all designed to streamline real-world deployment and enhance the developer experience. Specifically, DeepSeek R1-0528 demonstrates a remarkable leap in mathematical reasoning. On the AIME 2025 test, its accuracy improved from 70% to an impressive 87.5%, rivaling OpenAI's o3. This improvement is attributed to "enhanced thinking depth," with the model now utilizing significantly more tokens per question, indicating more thorough and systematic logical analysis. The open-source nature of DeepSeek-R1-0528 empowers users to fine-tune and adapt the model to their specific needs, fostering further innovation and advancements within the AI community. Recommended read:
References :
|
BenchmarksBlogsResearch Tools |