This year at Google I/O 2022, Google showcased many breakthroughs in new technologies and AI security. One of those cool advancements from a few years ago made its way onto the stage in the pre-show. It’s called Tone Transfer from Team Magenta at Google, and it lets you turn normal instruments or sounds into something completely different.
What is Google Tone Transfer?
Developed by the Magenta team at Google and announced in October 2020, Tone Transfer is a pretty neat tool that allows artists to input sounds and get something completely different out of them. For example, you can take the sound of people singing, the chirping of birds, or even the high-pitched thuds of pots and pans, and turn that sound into a flute or a fiddle.
This project is based on a machine learning process called Differentiable Digital Signal Processing (DDSP). In one example, the team configured the model with a 10-minute recording of a violin for them to process and run in their system. After that, the team began monitoring the progress of the program and its ability to recreate that sound. After an hour of practice in the example, the recreated sound is similar to a synth being played without emotion. After 10 hours of training, the model is able to reproduce the sound of a violin almost indistinctly. The reason is that the model, at this point, is able to control several different audio signatures provided by the instrument.
Overall, Google’s Magenta Tone Transfer project is incredibly interesting and shows how far machine learning has come as a whole. AI being able to develop and reproduce musical instruments on its own is an incredible feat.
How to create your own sounds in Tone Transfer
Magenta at Google has made Tone Transfer available for the public to try out for themselves. You can take a sample provided in the program, or you can upload your own. After that, you can choose what that sound turns into. Here are the available end result sounds:
Simply choose a source sound and click the button play icon below the sound signature to get an idea of what the source sounds like. After that choose an output sound. To click player again to hear the sound recreated and transformed into a new instrument.
If you want to upload your own sound sample, you can click Add yours under the source example section. If your device is compatible, you can record a sound or upload a file from your computer. When you’re done recording or downloading, tap Fact and you can listen to your recording. If you’re happy with it, press Transform and Tone Transfer will start processing the sample.
Tip: If you want to get a good result, be sure to choose a sound that only has one tone at a time. In this way, the ML model can differentiate and better create an end result.
While playing you can hit the cursor icon to set the final result. You can adjust the mix, octave and loudness as you like.
Overall, Tone Transfer from Google and the agent team is pretty cool. The model demonstrates how far much of this technology has advanced and provides a good example of what it can be used for. We look forward to what Magenta is working on in the future.
FTC: We use income-generating automotive affiliate links. More.
Check out 9to5Google on YouTube for more info: