OpenAI is working on a new tool that will allow users to generate music using text descriptions and audio recordings. This development will enable users to create musical accompaniment for videos, as well as add, for example, guitar accompaniment to vocal tracks.
This is reported by Business • Media
Collaboration with music academies and the working principle
To enhance the quality of the model, OpenAI is involving students from the Juilliard School—a renowned music institution—who are engaged in annotating scores. This collaboration helps train the tool to better understand the structure and composition of musical works.
“The company already has experience in sound generation, but after the launch of ChatGPT, it focused on speech synthesis models. The new project could mark OpenAI’s first major return to the field of music AI.”
Competitors and prospects of the new product
The tool will allow for the integration of music into existing videos and vocal recordings. It is still unclear whether this service will be a standalone product or become part of ChatGPT or the Sora video model. The release date and final format have not yet been determined.
OpenAI has experience in sound generation; however, in recent years, it has primarily focused on language technologies. Nevertheless, the new music tool could be a key step for the company in developing artificial intelligence specifically for music.
In the music generation market, Google and the startup Suno are already actively working, with the latter having integrated its model into Microsoft Copilot. Experts believe that if OpenAI successfully integrates text and audio models, it could create a competitive tool that complements the company’s ecosystem of products at the intersection of technology and creativity.
As a reminder, it was previously reported that OpenAI plans to create an AI banker.