PhD student @PrincetonCS audiovisual ML. @CCRMA/@Stanford, @BrownUniversity @dbraun.bsky.social

Joined November 2016
174 Photos and videos
🎛️Audio ML friends, I made a transpiler from Faust code to JAX! Faust is a powerful language for sound synthesis, so connecting it to JAX while supporting differentiability and learnable parameters is a huge research opportunity. 🧵
1
16
127
David Braun (dbraun.bsky.social) retweeted
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument. MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency. Open weights. Open source inference engine. Suite of apps and plugins. Hear what it can do and try it out for yourself below đź§µ
21
95
492
109,156
Happy to release "DAC-JAX: A JAX Implementation of the Descript Audio Codec." This can reuse PyTorch weights of all model sizes, and it includes a device-parallel training script. It uses the standard JAX libraries: Flax, Optax, Orbax, and CLU. github.com/DBraun/DAC-JAX
1
6
29
2,091
I benchmarked the chunked compression/decompression speeds. These are the functions you would use on long files or streaming. For a hop size of 8.2 ms, JAX performs compression in 7.1 ms and decompression in 4.3 ms. PyTorch performs compression in 8.3 ms, decompression in 6.3 ms.
1
2
397
In other words, if I wanted to explore usage of a real-time, 8.2 ms latency, 44.1 kHz DAC model, JAX might be faster. Of course, more analysis and testing are welcome. More details in the paper linked at github.com/DBraun/DAC-JAX
1
313
New talk and workshop on Faust JAX, this time with parameter *automation*. With simple SGD and L1 time-domain loss over the input audio and ground truth, we recover the parameter automation of a lowpass filter's cutoff frequency. youtube.com/watch?v=046Gi7Wh…
1
5
761
Another example shows a differentiable polyphonic wavetable synth. The wavetables (2048 sample arrays) are learnable as well as the "Wavetable Position" which blends between them. Notice that the middle plot is a blend of a sine and triangle, but all the wavetables are learnable!
1
1
356
DawDreamer now has a demo script for using multiprocessing to efficiently create one-shots from a synthesizer! Use all the cores! github.com/DBraun/DawDreamer…

6
875
Last week I presented my work integrating the Faust audio language and machine learning framework JAX youtu.be/AnWBSbC8vU8 This video goes over many nuances of the Google Colabs which I shared in October. I think it’s a great roadmap for future research in audio ML.

1
1
582
🎛️Audio ML friends, I made a transpiler from Faust code to JAX! Faust is a powerful language for sound synthesis, so connecting it to JAX while supporting differentiability and learnable parameters is a huge research opportunity. 🧵
1
16
127
If you’re a Faust or JAX user, I’d love to get your feedback. I’d be interested in working with anyone whose company wants to do a larger audio ML experiment 🤖🎸📊.
1
1
@StephaneLetz, an original author of Faust, is now on Twitter! Thanks for reviewing and merging these features in!
Inspired by @jeonghopark's SketchSynth and a graphic score by Mark Applebaum, I'm sharing a "port" to #TouchDesigner with Faust as the audio engine. Download: github.com/DBraun/TD-Faust/
4
25