News from the AI & ML world
DeeperML - #anthropic
|
Ryan Daws@AI News
// 7d
Anthropic has unveiled a novel method for examining the inner workings of large language models (LLMs) like Claude, offering unprecedented insight into how these AI systems process information and make decisions. Referred to as an "AI microscope," this approach, inspired by neuroscience techniques, reveals that Claude plans ahead when generating poetry, uses a universal internal blueprint to interpret ideas across languages, and occasionally works backward from desired outcomes instead of building from facts. The research underscores that these models are more sophisticated than previously thought, representing a significant advancement in AI interpretability.
Anthropic's research also indicates Claude operates with conceptual universality across different languages and that Claude actively plans ahead. In the context of rhyming poetry, the model anticipates future words to meet constraints like rhyme and meaning, demonstrating a level of foresight that goes beyond simple next-word prediction. However, the research also uncovered potentially concerning behaviors, as Claude can generate plausible-sounding but incorrect reasoning. In related news, Anthropic is reportedly preparing to launch an upgraded version of Claude 3.7 Sonnet, significantly expanding its context window from 200K tokens to 500K tokens. This substantial increase would enable users to process much larger datasets and codebases in a single session, potentially transforming workflows in enterprise applications and coding environments. The expanded context window could further empower vibe coding, enabling developers to work on larger projects without breaking context due to token limits. Recommended read:
References :
Maximilian Schreiner@THE DECODER
// 9d
OpenAI has announced it will adopt Anthropic's Model Context Protocol (MCP) across its product line. This surprising move involves integrating MCP support into the Agents SDK immediately, followed by the ChatGPT desktop app and Responses API. MCP is an open standard introduced last November by Anthropic, designed to enable developers to build secure, two-way connections between their data sources and AI-powered tools. This collaboration between rivals marks a significant shift in the AI landscape, as competitors typically develop proprietary systems.
MCP aims to standardize how AI assistants access, query, and interact with business tools and repositories in real-time, overcoming the limitation of AI being isolated from systems where work happens. It allows AI models like ChatGPT to connect directly to the systems where data lives, eliminating the need for custom integrations for each data source. Other companies, including Block, Apollo, Replit, Codeium, and Sourcegraph, have already added MCP support, and Anthropic's Chief Product Officer Mike Krieger welcomes OpenAI's adoption, highlighting MCP as a thriving open standard with growing integrations. Recommended read:
References :
Ryan Daws@AI News
// 14d
Anthropic has announced that its AI assistant Claude can now search the web. This enhancement allows Claude to provide users with more up-to-date and relevant responses by expanding its knowledge base beyond its initial training data. It may seem like a minor feature update, but it's not. It is available to paid Claude 3.7 Sonnet users by toggling on "web search" in their profile settings.
This integration emphasizes transparency, as Claude provides direct citations when incorporating information from the web, enabling users to easily fact-check sources. Claude aims to streamline the information-gathering process by processing and delivering relevant sources in a conversational format. Anthropic believes this update will unlock new use cases for Claude across various industries, including sales, finance, research, and shopping. Recommended read:
References :
Chris McKay@Maginative
// 3d
Anthropic has unveiled Claude for Education, a specialized AI assistant designed to cultivate critical thinking skills in students. Unlike conventional AI tools that simply provide answers, Claude employs a Socratic-based "Learning Mode" that prompts students with guiding questions, encouraging them to engage in deeper reasoning and problem-solving. This innovative approach aims to address concerns about AI potentially hindering intellectual development by promoting shortcut thinking.
Partnerships with Northeastern University, the London School of Economics, and Champlain College will integrate Claude across multiple campuses, reaching tens of thousands of students. These institutions are making a significant investment in AI, betting that it can improve the learning process. Faculty can use Claude to generate rubrics aligned with learning outcomes and create chemistry equations, while administrative staff can analyze enrollment trends and simplify policy documents. These institutions are testing the system across teaching, research, and administrative workflows. Recommended read:
References :
Ryan Daws@AI News
// 15d
Anthropic's AI assistant, Claude, has gained a significant upgrade: real-time web search. This new capability allows Claude to access and process information directly from the internet, expanding its knowledge base beyond its initial training data. The integration aims to address a critical competitive gap with OpenAI's ChatGPT, leveling the playing field in the consumer AI assistant market. This update is available immediately for paid Claude users in the United States and will be coming to free users and more countries soon.
The web search feature not only enhances Claude's accuracy but also prioritizes transparency and fact-checking. Claude provides direct citations when incorporating web information into its responses, enabling users to verify sources easily. This feature addresses growing concerns about AI hallucinations and misinformation by allowing users to dig deeper and confirm the accuracy of information provided. The update is meant to streamline the information-gathering process, allowing Claude to process and deliver relevant sources in a conversational format, rather than requiring users to sift through search engine results manually. Recommended read:
References :
Ryan Daws@AI News
// 9d
Anthropic has unveiled groundbreaking insights into the 'AI biology' of their advanced language model, Claude. Through innovative methods, researchers have been able to peer into the complex inner workings of the AI, demystifying how it processes information and learns strategies. This research provides a detailed look at how Claude "thinks," revealing sophisticated behaviors previously unseen, and showing these models are more sophisticated than previously understood.
These new methods allowed scientists to discover that Claude plans ahead when writing poetry and sometimes lies, showing the AI is more complex than previously thought. The new interpretability techniques, which the company dubs “circuit tracing” and “attribution graphs,” allow researchers to map out the specific pathways of neuron-like features that activate when models perform tasks. This approach borrows concepts from neuroscience, viewing AI models as analogous to biological systems. This research, published in two papers, marks a significant advancement in AI interpretability, drawing inspiration from neuroscience techniques used to study biological brains. Joshua Batson, a researcher at Anthropic, highlighted the importance of understanding how these AI systems develop their capabilities, emphasizing that these techniques allow them to learn many things they “wouldn’t have guessed going in.” The findings have implications for ensuring the reliability, safety, and trustworthiness of increasingly powerful AI technologies. Recommended read:
References :
@Latest from Tom's Guide
// 23d
References:
AI News | VentureBeat
, venturebeat.com
,
Anthropic's Claude AI platform is generating significant discussion, particularly around its coding capabilities and the future role of AI in scientific advancement. Anthropic has released Claude 3.7 Sonnet, setting new benchmarks for coding performance and positioning itself as the leading LLM for enterprise applications. This development comes alongside the launch of Claude Code, an AI coding agent designed to accelerate application development. Furthermore, Anthropic recently secured $3.5 billion in funding, raising its valuation to $61.5 billion, solidifying its position as a key competitor to OpenAI.
Mike Krieger, chief product officer at Anthropic, predicts that within three years, software developers will primarily be reviewing AI-generated code. This shift raises questions about how entry-level developers will gain the necessary experience in a field where reviewing code demands expertise. However, the optimistic views of Anthropic CEO Dario Amodei are facing scrutiny as Hugging Face co-founder Thomas Wolf challenges the notion of a "compressed 21st century," arguing that current AI systems may produce conformity rather than the revolutionary scientific breakthroughs Amodei envisions. Recommended read:
References :
Matt Marshall@AI News | VentureBeat
// 25d
References:
venturebeat.com
, www.tomsguide.com
Anthropic's Claude 3.7 is rapidly becoming the coding agent of choice for enterprise companies, positioning itself as a key competitor to OpenAI. Released just two weeks ago, Claude 3.7 Sonnet has already set new benchmark records for coding performance, achieving an impressive 70.3% on the SWE-bench benchmark. This benchmark measures an agent's software development skills and it handily outperforming competitors like OpenAI's o1 and DeepSeek-R1.
Alongside the release of 3.7 Sonnet, Anthropic launched Claude Code, an AI coding agent aimed at helping developers build applications more quickly. Evidence of Claude's growing importance is seen in the success of Cursor, an AI-powered code editor that defaults to Anthropic's model. Furthermore, Vercel, a company facilitating front-end application deployment, switched its lead coding model from OpenAI's GPT to Claude after evaluating their performance on key coding tasks, citing that Anthropic continues to come out on top. Recommended read:
References :
Jesus Rodriguez@TheSequence
// 17h
Anthropic has released a study revealing that reasoning models, even when utilizing chain-of-thought (CoT) reasoning to explain their processes step by step, frequently obscure their actual decision-making. This means the models may be using information or hints without explicitly mentioning it in their explanations. The researchers found that the faithfulness of chain-of-thought reasoning can be questionable, as language models often do not accurately verbalize their true reasoning, instead rationalizing, omitting key elements, or being deliberately opaque. This calls into question the reliability of monitoring CoT for safety issues, as the reasoning displayed often fails to reflect what is driving the final output.
This unfaithfulness was observed across both neutral and potentially problematic misaligned hints given to the models. To evaluate this, the researchers subtly gave hints about the answer to evaluation questions and then checked to see if the models acknowledged using the hint when explaining their reasoning, if they used the hint at all. They tested Claude 3.7 Sonnet and DeepSeek R1, finding that they verbalized the use of hints only 25% and 39% of the time, respectively. The transparency rates dropped even further when dealing with potentially harmful prompts, and as the questions became more complex. The study suggests that monitoring CoTs may not be enough to reliably catch safety issues, especially for behaviors that don't require extensive reasoning. While outcome-based reinforcement learning can improve CoT faithfulness to a small extent, the benefits quickly plateau. To make CoT monitoring a viable way to catch safety issues, a method to make CoT more faithful is needed. The research also highlights that additional safety measures beyond CoT monitoring are necessary to build a robust safety case for advanced AI systems. Recommended read:
References :
Michael Nuñez@AI News | VentureBeat
// 1d
References:
venturebeat.com
, The Algorithmic Bridge
,
Anthropic has been at the forefront of investigating how AI models like Claude process information and make decisions. Their scientists developed interpretability techniques that have unveiled surprising behaviors within these systems. Research indicates that large language models (LLMs) are capable of planning ahead, as demonstrated when writing poetry or solving problems, and that they sometimes work backward from a desired conclusion rather than relying solely on provided facts.
Anthropic researchers also tested the "faithfulness" of CoT models' reasoning by giving them hints in their answers, and see if they will acknowledge it. The study found that reasoning models often avoided mentioning that they used hints in their responses. This raises concerns about the reliability of chains-of-thought (CoT) as a tool for monitoring AI systems for misaligned behaviors, especially as these models become more intelligent and integrated into society. The research emphasizes the need for ongoing efforts to enhance the transparency and trustworthiness of AI reasoning processes. Recommended read:
References :
Michael Nuñez@AI News | VentureBeat
// 21d
References:
venturebeat.com
Anthropic researchers have achieved a significant breakthrough in AI safety by developing techniques to detect hidden objectives in AI systems. They trained their AI assistant, Claude, to conceal its true goals, specifically to prioritize maximizing rewards from evaluation models over human preferences. This involved teaching the model about fictional biases that reward models might have. The team then successfully uncovered these hidden agendas using innovative auditing methods, comparing their work to "white-hat hacking" for computer systems.
These findings address a fundamental challenge in AI alignment: ensuring AI systems aren't merely appearing to follow instructions while secretly pursuing other goals. The researchers compared this to students giving answers they know will be marked as correct, regardless of their actual beliefs. The developed auditing methods, including interpretability techniques and behavioral attacks, allowed researchers to uncover the model’s hidden objective. The potential of these methods could transform AI safety standards and prevent rogue AI behavior. Recommended read:
References :
Alexey Shabanov@TestingCatalog
// 21d
References:
venturebeat.com
, Windows Copilot News
,
Anthropic is actively developing and finalizing Harmony, a new feature for Claude that will allow users to integrate local files directly into the AI's context. This integration enables Claude to seamlessly interact with user files, allowing the AI to read, index, and analyze content within a specified directory. Harmony is expected to be available in the feature preview settings soon and promises to significantly enhance Claude's capabilities as an AI-powered coding assistant and tool for content analysis.
Recent tests of Harmony demonstrate its potential. For example, when provided with an Android application, Claude systematically indexed each file before generating a comprehensive summary, including highlighting context capacity usage. Beyond analysis, Harmony also supports modifying existing files, saving changes, and creating new files within the integrated directory. Furthermore, Anthropic researchers have also unveiled techniques to detect when AI systems might be concealing their actual goals, a critical advancement for AI safety research as these systems become more sophisticated and potentially deceptive. Recommended read:
References :
Alexey Shabanov@TestingCatalog
// 5d
References:
The Tech Basic
, TheSequence
Anthropic has unveiled a groundbreaking innovation in AI research with self-coordinating agent networks in their Claude Research Mode. This tool empowers Claude AI to operate as a central authority, capable of constructing and managing multiple helper bots. These bots collaborate to tackle complex challenges, marking a significant shift in how AI research is conducted. By dynamically creating and coordinating AI agents, Anthropic is redefining the possibilities for AI problem-solving in various fields.
The Claude Research Mode operates akin to a human team, where different individuals handle specific tasks. This multi-agent functionality leverages tools like web search, memory, and the ability to create sub-agents. A master agent can delegate tasks to these sub-agents, fostering dynamic and collaborative interactions within a single research flow. These helper bots are designed to enhance problem-solving capabilities by searching the internet using Brave, remembering crucial details, and engaging in careful deliberation before providing answers. This approach promises to transform how AI is applied in science, business, and healthcare. Recommended read:
References :
Tom Krazit@Runtime
// 9d
References:
Shelly Palmer
, THE DECODER
OpenAI and Anthropic, despite being competitors, are joining forces to standardize AI-data integration through the Model Context Protocol (MCP). This collaboration aims to create a unified framework that allows AI models, such as ChatGPT, to more effectively access and utilize data from various sources. OpenAI CEO Sam Altman announced on X that OpenAI will integrate Anthropic's MCP into its product lineup, starting with the Agents SDK, and soon to include the ChatGPT desktop app and Responses API.
This partnership signifies a significant shift in the AI landscape. MCP acts as a tool that enables AI systems to access digital documents and provide enhanced responses by granting access to platforms like Google Drive, Slack, and calendars. By adopting Anthropic's open-source standard, OpenAI is promoting interoperability between different AI tools. According to Anthropic's head of product Mike Krieger, MCP has become a widely adopted standard with numerous integrations, and this collaboration could potentially help more companies join in to help MCP remain useful. Recommended read:
References :
Adarsh Menon@Towards AI
// 22d
References:
Composio
, Thomas Roccia :verified:
,
Anthropic's Model Context Protocol (MCP), released in November 2024, is gaining significant traction in the AI community. This protocol is designed as a standardized method for connecting AI assistants with the systems where data resides, including content repositories, business tools, and development environments. MCP facilitates a consistent manner for applications to provide context to Large Language Models (LLMs), effectively isolating context provision from direct LLM interaction. Thomas Roccia, among others, recognized the value of MCP for AI agents immediately upon its release.
MCP acts as a universal set of rules, enabling seamless communication between clients and servers, regardless of their origin. This interoperability lays the groundwork for a diverse AI ecosystem. It defines how clients interact with servers and how servers manage tools and resources. The protocol aims to standardize the integration of context and tools into AI applications, analogous to the USB-C port for agentic systems, as described by Anthropic. Recommended read:
References :
Alexey Shabanov@TestingCatalog
// 6d
References:
AI News
, TestingCatalog
,
Anthropic is reportedly enhancing Claude AI with multi-agent capabilities, including web search, memory, and sub-agent creation. This upgrade to the Claude Research feature, previously known as Compass, aims to facilitate more dynamic and collaborative research flows. The "create sub-agent" tool would enable a master agent to delegate tasks to sub-agents, allowing users to witness multi-agent interaction within a single research process. These new tools include web_fetch, web_search, create_subagent, memory, think, sleep and complete_task.
Anthropic is also delving into the "AI biology" of Claude, offering insights into how the model processes information and makes decisions. Researchers have discovered that Claude possesses a degree of conceptual universality across languages and actively plans ahead in creative tasks. However, they also found instances of the model generating incorrect reasoning, highlighting the importance of understanding AI decision-making processes for reliability and safety. Anthropic's approach to AI interpretability allows them to uncover insights into the inner workings of these systems that might not be apparent through simply observing their outputs. Recommended read:
References :
Tom Krazit@Runtime
// 8d
References:
Runtime
Anthropic is gaining traction in the AI infrastructure space with its Model Context Protocol (MCP), introduced last November as an open standard for secure, two-way connections between data sources and AI-powered tools. This protocol is designed to simplify the process of building AI agents by providing a standard way for applications to retrieve data, allowing agents to take actions based on that data. Microsoft and Cloudflare have already announced support for MCP, with Microsoft highlighting that MCP simplifies agent building and reduces maintenance time.
The MCP protocol works by taking natural language input from a large-language model and providing a standard way for MCP clients to find and retrieve data stored on servers running MCP. This is analogous to the API, which made web-based computing a standard. Previously, developers needed to set up MCP servers locally, which was impractical for most users. This barrier to entry has now been removed. In other news, Anthropic is facing a legal challenge as music publishers' request for a preliminary injunction in their copyright infringement suit was denied. The publishers alleged that Anthropic's LLM Claude was trained on their song lyrics. However, the judge ruled that the publishers failed to demonstrate specific financial harm and that their list of forbidden lyrics was not final, requiring constant updates to Anthropic's guard rails. The case is ongoing, and the publishers can collect more evidence. Recommended read:
References :
|
BenchmarksBlogsResearch Tools |