PhD student @PrincetonCS audiovisual ML. @CCRMA/@Stanford, @BrownUniversity @dbraun.bsky.social
Joined November 2016
- Tweets 248
- Following 441
- Followers 1,895
- Likes 1,645
174 Photos and videos
Pinned Tweet
20 Oct 2022
🎛️Audio ML friends, I made a transpiler from Faust code to JAX! Faust is a powerful language for sound synthesis, so connecting it to JAX while supporting differentiability and learnable parameters is a huge research opportunity. 🧵
1
16
127
David Braun (dbraun.bsky.social) retweeted
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
21
95
492
109,156
David Braun (dbraun.bsky.social) retweeted
21 May 2024
``DAC-JAX: A JAX Implementation of the Descript Audio Codec,'' David Braun, ift.tt/FzlXUdq
1
12
1,437
19 May 2024
Happy to release "DAC-JAX: A JAX Implementation of the Descript Audio Codec." This can reuse PyTorch weights of all model sizes, and it includes a device-parallel training script. It uses the standard JAX libraries: Flax, Optax, Orbax, and CLU. github.com/DBraun/DAC-JAX
1
6
29
2,091
19 May 2024
I benchmarked the chunked compression/decompression speeds. These are the functions you would use on long files or streaming. For a hop size of 8.2 ms, JAX performs compression in 7.1 ms and decompression in 4.3 ms. PyTorch performs compression in 8.3 ms, decompression in 6.3 ms.
1
2
397
19 May 2024
In other words, if I wanted to explore usage of a real-time, 8.2 ms latency, 44.1 kHz DAC model, JAX might be faster. Of course, more analysis and testing are welcome. More details in the paper linked at github.com/DBraun/DAC-JAX
1
313
New talk and workshop on Faust JAX, this time with parameter *automation*. With simple SGD and L1 time-domain loss over the input audio and ground truth, we recover the parameter automation of a lowpass filter's cutoff frequency. youtube.com/watch?v=046Gi7Wh…
1
5
761
Another example shows a differentiable polyphonic wavetable synth. The wavetables (2048 sample arrays) are learnable as well as the "Wavetable Position" which blends between them. Notice that the middle plot is a blend of a sine and triangle, but all the wavetables are learnable!
1
1
356
The afternoon workshop is here youtube.com/watch?v=VIlCY7wR… Thanks to the Programmable Audio Workshop (@Inria @insadelyon @Grame_Lyon) for having me!
1
276
DawDreamer now has a demo script for using multiprocessing to efficiently create one-shots from a synthesizer! Use all the cores! github.com/DBraun/DawDreamer…
6
875
Last week I presented my work integrating the Faust audio language and machine learning framework JAX youtu.be/AnWBSbC8vU8 This video goes over many nuances of the Google Colabs which I shared in October. I think it’s a great roadmap for future research in audio ML.
1
1
582
20 Oct 2022
🎛️Audio ML friends, I made a transpiler from Faust code to JAX! Faust is a powerful language for sound synthesis, so connecting it to JAX while supporting differentiability and learnable parameters is a huge research opportunity. 🧵
1
16
127
20 Oct 2022
If you’re a Faust or JAX user, I’d love to get your feedback. I’d be interested in working with anyone whose company wants to do a larger audio ML experiment 🤖🎸📊.
1
1
22 Oct 2022
@StephaneLetz, an original author of Faust, is now on Twitter! Thanks for reviewing and merging these features in!
11 Jun 2022
Inspired by @jeonghopark's SketchSynth and a graphic score by Mark Applebaum, I'm sharing a "port" to #TouchDesigner with Faust as the audio engine. Download: github.com/DBraun/TD-Faust/
4
25