News from the AI & ML world
DeeperML
@www.marktechpost.com
//
Mistral AI has released Mistral Small 3.2, an updated version of its open-source model, Mistral-Small-3.2-24B-Instruct-2506, building upon the earlier Mistral-Small-3.1-24B-Instruct-2503. This update focuses on enhancing the model’s overall reliability and efficiency, particularly in handling complex instructions, minimizing repetitive outputs, and maintaining stability during function-calling scenarios. The improvements aim to refine specific behaviors such as instruction following, output stability, and function calling robustness without altering the core architecture.
A significant enhancement in Mistral Small 3.2 is its improved accuracy in executing precise instructions. Benchmark scores reflect this improvement, with the model achieving 65.33% accuracy on the Wildbench v2 instruction test, up from 55.6% for its predecessor. Performance on the challenging Arena Hard v2 test nearly doubled, increasing from 19.56% to 43.1%, demonstrating an enhanced ability to understand and execute intricate commands accurately. Internally, Mistral’s accuracy rose from 82.75% in Small 3.1 to 84.78% in Small 3.2.
Mistral Small 3.2 also addresses the issue of repetitive errors by significantly reducing instances of infinite or repetitive output, a common problem in extended conversational scenarios. Internal evaluations show a decrease in infinite generation errors by nearly half, from 2.11% in Small 3.1 to 1.29%. The updated model also demonstrates enhanced capability in calling functions, making it more suitable for automation tasks. Additionally, Mistral AI emphasizes its compliance with EU regulations like GDPR and the EU AI Act, making it an appealing choice for developers in the region.
References :
A significant enhancement in Mistral Small 3.2 is its improved accuracy in executing precise instructions. Benchmark scores reflect this improvement, with the model achieving 65.33% accuracy on the Wildbench v2 instruction test, up from 55.6% for its predecessor. Performance on the challenging Arena Hard v2 test nearly doubled, increasing from 19.56% to 43.1%, demonstrating an enhanced ability to understand and execute intricate commands accurately. Internally, Mistral’s accuracy rose from 82.75% in Small 3.1 to 84.78% in Small 3.2.
Mistral Small 3.2 also addresses the issue of repetitive errors by significantly reducing instances of infinite or repetitive output, a common problem in extended conversational scenarios. Internal evaluations show a decrease in infinite generation errors by nearly half, from 2.11% in Small 3.1 to 1.29%. The updated model also demonstrates enhanced capability in calling functions, making it more suitable for automation tasks. Additionally, Mistral AI emphasizes its compliance with EU regulations like GDPR and the EU AI Act, making it an appealing choice for developers in the region.
Share:
References :
- Simon Willison: Blogged too much today and had to send it all out in a newsletter - it's a pretty fun one, covering Gemini 2.5 and Mistral Small 3.2 and the fact that most LLMs will absolutely try and murder you given the chance (and a suitably contrived scenario)
- www.marktechpost.com: Mistral AI Releases Mistral Small 3.2: Enhanced Instruction Following, Reduced Repetition, and Stronger Function Calling for AI Integration
- AI News | VentureBeat: Mistral just updated its open source Small model from 3.1 to 3.2: here’s why
- Simon Willison: Mistral Small 3.2 was released today - I used a 15GB quantized model from Hugging Face (via Ollama) running on my Mac and got it to draw me a pretty decent SVG of a pelican riding a bicycle model (considering the model size)
Classification:
- HashTags: #MistralAI #OpenSource #LLM
- Company: Mistral AI
- Target: AI Developers
- Product: Mistral Small 3.2
- Feature: Instruction Following
- Type: ProductUpdate
- Severity: Informative