DeeperML - #Music

Google's Magenta project has unveiled Magenta RealTime (Magenta RT), an open-weights live music model designed for interactive music creation, control, and performance. This innovative model builds upon Google DeepMind's research in real-time generative music, providing opportunities for unprecedented live music exploration. Magenta RT is a significant advancement in AI-driven music technology, offering capabilities for both skill-gap accessibility and enhancement of existing musical practices. As an open-weights model, Magenta RT is targeted towards eventually running locally on consumer hardware, showcasing Google's commitment to democratizing AI music creation tools.

Magenta RT, an 800 million parameter autoregressive transformer model, was trained on approximately 190,000 hours of instrumental stock music. It leverages SpectroStream for high-fidelity audio (48kHz stereo) and a newly developed MusicCoCa embedding model, inspired by MuLan and CoCa. This combination allows users to dynamically shape and morph music styles in real-time by manipulating style embeddings, effectively blending various musical styles, instruments, and attributes. The model code is available on Github and the weights are available on Google Cloud Storage and Hugging Face under permissive licenses with some additional bespoke terms.

Magenta RT operates by generating music in sequential chunks, conditioned on both previous audio output and style embeddings. This approach enables the creation of interactive soundscapes for performances and virtual spaces. Impressively, the model achieves a real-time factor of 1.6 on a Colab free-tier TPU (v2-8 TPU), generating two seconds of audio in just 1.25 seconds. This technology unlocks the potential to explore entirely new musical landscapes, experiment with never-before-heard instrument combinations, and craft unique sonic textures, ultimately fostering innovative forms of musical expression and performance.


References :

• Magenta: Today, we’re happy to share a research preview of Magenta RealTime (Magenta RT), an open-weights live music model that allows you to interactively create, control and perform music in the moment.
• THE DECODER: Google has released Magenta RealTime (Magenta RT), an open-source AI model for live music creation and control. The article appeared first on The Decoder.
• the-decoder.com: Google has released Magenta RealTime (Magenta RT), an open-source AI model for live music creation and control. The article appeared first on .
• github.com: Magenta RealTime: An Open-Weights Live Music Model
• aistudio.google.com: Magenta RealTime: An Open-Weights Live Music Model
• huggingface.co: Sharing a research preview of Magenta RealTime (Magenta RT), an open-weights live music model that allows you to interactively create, control and perform music in the moment
• Magenta: Magenta RealTime: An Open-Weights Live Music Model
• Magenta: Magenta RT is the latest in a series of models and applications developed as part of the Magenta Project.
• www.marktechpost.com: Google Researchers Release Magenta RealTime: An Open-Weight Model for Real-Time AI Music Generation
• Simon Willison's Weblog: Fun new "live music model" release from Google DeepMind: Today, we’re happy to share a research preview of Magenta RealTime (Magenta RT), an open-weights live music model that allows you to interactively create, control and perform music in the moment.
• MarkTechPost: Google’s Magenta team has introduced Magenta RealTime (Magenta RT), an open-weight, real-time music generation model that brings unprecedented interactivity to generative audio.

Classification:

• HashTags: #AI #MusicAI #MagentaRT
• Company: Google Magenta
• Target: Musicians, Artists
• Product: Magenta RealTime
• Feature: Music Generation
• Type: AI
• Severity: Informative