News from the AI & ML world
DeeperML
Carl Franzen@AI News | VentureBeat
//
Sakana AI, co-founded by former Google AI scientists, has introduced a new AI model architecture called Continuous Thought Machines (CTM). This innovative design aims to bridge the gap between artificial intelligence and human cognition. Unlike traditional AI models that process inputs in fixed, parallel layers, CTMs incorporate neural timing as a fundamental representation, allowing the model to manage tasks like image recognition, problem-solving, and reinforcement learning with greater flexibility and adaptability. The core concept behind CTM is to enable AI to "think" through problems step-by-step, mimicking the way human brains operate.
The Continuous Thought Machine architecture introduces two key innovations inspired by nature. First, it equips each artificial neuron with a short-term memory of its previous activity, enabling the neuron to decide when to activate again. Second, the CTM allows computation to unfold over steps within each artificial neuron. These neurons operate on their own internal timeline, adjusting their reasoning duration dynamically based on the complexity of the input. This time-based architecture enables CTMs to reason progressively, taking a variable number of "ticks" depending on the challenge at hand. Sakana AI points out that researchers still primarily use a single output from a neuron, which tells us how it’s firing, but “neglects the precise timing of when the neuron is firing in relation to other neurons.”
The CTM architecture distinguishes itself from existing AI models, particularly Transformer-based language models, by allowing each neuron to operate on its internal timeline and make activation decisions based on a short-term memory of its previous states. This unique approach offers the potential for AI to reason more like human brains, which is seen as vital. While CTMs are still primarily a research architecture and not yet production-ready, they represent a significant step towards creating more trustworthy and efficient AI technology. Sakana AI has made their research available, including a paper on arXiv, a microsite, and a GitHub repository.
ImgSrc: venturebeat.com
References :
The Continuous Thought Machine architecture introduces two key innovations inspired by nature. First, it equips each artificial neuron with a short-term memory of its previous activity, enabling the neuron to decide when to activate again. Second, the CTM allows computation to unfold over steps within each artificial neuron. These neurons operate on their own internal timeline, adjusting their reasoning duration dynamically based on the complexity of the input. This time-based architecture enables CTMs to reason progressively, taking a variable number of "ticks" depending on the challenge at hand. Sakana AI points out that researchers still primarily use a single output from a neuron, which tells us how it’s firing, but “neglects the precise timing of when the neuron is firing in relation to other neurons.”
The CTM architecture distinguishes itself from existing AI models, particularly Transformer-based language models, by allowing each neuron to operate on its internal timeline and make activation decisions based on a short-term memory of its previous states. This unique approach offers the potential for AI to reason more like human brains, which is seen as vital. While CTMs are still primarily a research architecture and not yet production-ready, they represent a significant step towards creating more trustworthy and efficient AI technology. Sakana AI has made their research available, including a paper on arXiv, a microsite, and a GitHub repository.
ImgSrc: venturebeat.com
Share:
References :
- bdtechtalks.com: An overview of Sakana AI's Continuous Thought Machine (CTM).
- AI News | VentureBeat: The article discusses Sakana AI's advancements in AI architecture, specifically their continuous thought machines.
- bdtechtalks.com: Sakana AI's Continuous Thought Machine enhances AI's alignment with human cognition, promising a future of more trustworthy and efficient technology.
Classification:
- HashTags: #AI #NeuralDynamics #ContinuousThoughtMachine
- Company: Sakana AI
- Target: AI Models
- Product: Continuous Thought Machines
- Feature: Continuous Thought Machine
- Type: Research
- Severity: Medium