🥳 New publication announcement!
Marco Pasini solved the problem of error accumulation in continuous autoregressive models (CAMs), making it possible to generate sequences without the need for prior tokenization. Say goodbye to RVQ codecs (use music2latent 😉).
@SonyCSLMusic
✨ Train language models directly on continuous data - without tokenization ✨
We propose an easy way to train GPT-style autoregressive models on continuous data, without error accumulation.
We test it on audio 🔊, but this method can easily work with other modalities 🎆
👇🧵