engineer and researcher. @ACEStudio_en @acemusicAI
Joined October 2022
- Tweets 291
- Following 2,435
- Followers 934
- Likes 4,938
38 Photos and videos
Gong Junmin retweeted
Jun 11
Meet LiveBand: a real-time AI jamming companion! 🎸
It generates live music accompaniments with zero perceived latency ⚡️
It runs locally on Macbooks, can generate any instrument (more than one at a time), is wildly robust, and is trained from scratch on a single GPU! 🧵👇
8
13
93
4,624
Jun 10
we've reached the point where "new SOTA model" generates less dopamine than a good meme
jaded isn't even the word👀
1
6
394
Jun 10
Here is the answer
x.com/junmingong/status/2064…
Jun 10
we've reached the point where "new SOTA model" generates less dopamine than a good meme
jaded isn't even the word👀
2
111
Gong Junmin retweeted
Jun 4
Built ACE-Step UI in public (4.1k⭐).
Shipped LocalMusic AI on Mac App Store.
Now bootstrapping a web agency back home.
Same engineering bar, smaller scope.
Following along is fun: @AmbsdOP
1
2
210
Gong Junmin retweeted
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
21
95
492
109,099
Gong Junmin retweeted
May 30
Release soon on the Unity Store!
discussions.unity.com/t/3d-m…
Music by AceStep 😍 Hey @junmingong ! 😀 If you knew how happy I am to be able to make my own music...
#unity3d #indie #indiedev #gamedev #madewithunity #indiegames #assetstore #unity #unitydeveloper
1
2
10
330
Gong Junmin retweeted
May 29
To top off this week, there's now a paper about DEMON published on @arxiv.
A lot of great insights there - especially for researchers from the AI audio-gen domain.
Here's the video covering some basics, and the paper is in the tweet below.
Next week, on Tuesday (Jun 2), we are hosting our webinar with @RyanOnTheInside, author of the paper, so you can ask any questions.
1
3
10
511
May 28
Built on ACE-Step, this lets you perform AI-generated music with synth hardware, almost like playing an instrument
Hardware knobs shape DiT's initial noise, and the music is generated in real time through streaming
A new kind of music is emerging
Live music is the future
4
20
182
9,794
May 27
awesome!
May 27
I just released this open source project built on @ACEStep_Music. DEMON: Diffusion Engine for Musical Orchestrated Noise. It lets you play ACEStep like a musical instrument, remixing songs and loops with feedback that approaches real-time.
Its essentially StreamDiffusion but instead of Stable Diffusion it is ACEStep1.5, and instead of images it is full songs. It runs on 30/40/5090. Built with @DaydreamLiveAI team, testing, and building the demo. We are hosting it if you want to try it without installing. For full details, links, and writeup please see the pinned project page.
1
15
844
Gong Junmin retweeted
May 27
🚀 DiffSynth-Studio now supports training DiT-based musical models! To kick things off, we’re dropping 4 Instrument-Enhancement LoRAs for ACE-Step-v1.5-XL:modelscope.cn/collections/Di…
Differential LoRA Training to boost target instruments with high fidelity:🎸 Guitar | 🎹 Piano | 🥁 Drums | 🎼 Accompaniment🎧
Listen to the Drum demo below & try it out for yourself: github.com/modelscope/DiffSy…
4
9
91
7,290
May 22
streaming is a good idea
May 22
Can we transform offline audio diffusion into real-time streaming interactive instruments?
Yes!
Presenting Live Music Diffusion Models: a new paradigm for taking your favorite open models into live performance, right on your own laptop! 🎵🎵
🧵
1
13
1,058
May 20
It’s amazing to see an open-source model surpass ACE-Step 1.5 so soon — smaller in size, faster in speed, and truly built for musicians. Good job!
May 20
Stable Audio 3, explained in 5 figures.
It’s a family of open-weight models for generating instrumental music and sound effects.
The models are fast, support editing, and are trained on licensed and Creative Commons audio.
👾 artintech.substack.com/p/sta…
🏋️♂️github.com/Stability-AI/stab…
3
34
2,791
Gong Junmin retweeted
May 20
Stable Audio 3, explained in 5 figures.
It’s a family of open-weight models for generating instrumental music and sound effects.
The models are fast, support editing, and are trained on licensed and Creative Commons audio.
👾 artintech.substack.com/p/sta…
🏋️♂️github.com/Stability-AI/stab…
5
22
117
43,612
May 16
Khala 1.0 just dropped — a music generation model from the Central Conservatory of Music in Beijing. Paper, code, weights, and demo all open-sourced.
I gave a talk there recently on ACE-Step and got an early look at Khala. Excited to see it officially out. Open-source music gen is thriving.
💻 github.com/Khala-Music-AI/Kh…
📝 arxiv.org/abs/2605.01790
🎧 khala-music-ai.github.io/Kha…
15
77
463
26,313
May 12
Diffusers 0.38.0 now supports ACE-Step 1.5 🔥
May 12
We released Diffusers 0.38.0, and it's packed with new pipelines and several library-related improvements 🔥
A bunch of new pipelines, including audio 🎼
* Ace-Step 1.5
* LongCat-AudioDiT
* Ernie-Image
And more!
Next up, we added support for:
* Flash Attention 4
* Loading with FlashPack
* Ring Anything as a new backend for context parallelism
Last but not least, we added an example on how to profile a DiffusionPipeline and potentially improve its performance.
Enjoy 🧨
19
1,212