Independent researcher | AI advisor | Distribution Fine Tuning (DFT) for better LLM writing quality

Joined October 2023
124 Photos and videos
Pinned Tweet
May 18
I fixed why LLMs write so poorly, and I have a demo to prove it Announcing Distribution Fine Tuning (DFT): A post training step that fixes LLM writing Model outputs fooled pangram on 100% of test cases
124
155
3,223
502,119
Jun 13
This is gives me so much optimism for the future Setting a precedent of 50/50 split of ad revenue is a big deal. If we maintain that ratio then everyone gets cheap AI, access isn't gated by wealth Andrew just singlehandedly saved everyone from the permanent underclass lol
Get paid to wait The Claude Code spinner might be the most watched line on Earth. So I turned it into an ad marketplace. Advertisers bid on it. You keep 50% of the money. Install the extension → get cash from ads. Introducing Kickbacks
4
327
Anthropic is continung its pivot into being a full time advertiser for codex
When Fable 5 is used for frontier LLM development, it does not notify the user and instead limits the model’s capabilities through methods such as prompt modification, steering vectors, and PEFT. Anthropic estimated that this would affect approximately 0.03% of traffic.
2
4
67
2,556
May 29
Codex: "we're all trying to find the guy who did this"
1
15
822
May 27
Currently hazing the intern (5.4-mini) by making it look at every single data sample manually to make sure it's high quality
2
9
589
May 24
Here's something hot:
Twitter Growth Strategy 0-500 Followers: Reply Guy 501-2K: Niche Bangers 2-5K: Thirst Traps 5-10K: Parody News 10-25K: Cringe Threads 25-50K: Shitposts 50-75K: Fortune Cookies 75-100K: Cringe Bangers >100K: Get Cancelled
5
42
2,326
May 22
Got a thermal camera to check my server GPUs are hot because they're thinking hard the dog is... not
8
28
1,257
May 22
Context: I wanted to make sure the cables were not bent in a way that would make them overheat, so got the camera. To be fair it might just be the haircut hiding the heat on the head. But there has been no evidence of her thinking in the past, so idk x.com/rosmine/status/2054574…

1
8
679
May 21
woah! My gpu blog is top 10 on HN! It only got 6 points when I posted it myself 😆 Thank you apwheele!
13
84
3,193
May 21
And when I posted DFT it got 1 point. If anyone wants to try again with that, please go for it 😃
1
13
920
May 20
ReLoRA is such a great trick You repeatedly train a LoRA, merge it, then train a new LoRA on top of that Each LoRA is only a low rank update, but a sum of low rank updates can be a full rank update, so you can get better results For DFT, I used a total of 13
16
38
602
39,056
May 20
Should I make a DFT powered AI humanizer? My main goal with DFT is to help people produce higher quality writing, not more slop. A "cursor for writing" where you can edit and rewrite and get feedback But the main request I get is to make a humanizer I'm not sure if a humanizer will be helpful, or it has too much potential for abuse
May 18
I fixed why LLMs write so poorly, and I have a demo to prove it Announcing Distribution Fine Tuning (DFT): A post training step that fixes LLM writing Model outputs fooled pangram on 100% of test cases
29
2
128
12,039
May 20
Very happy to see these responses, looks like almost everyone agrees no humanizer. Awesome IDE for writing it is!
5
22
1,088
May 19
This describes DFT better than I could:
This is a massive deal. The DFT approach defined here "trains at this higher level [distribution-level information], optimizing the distribution of outputs so that it better matches the training data." That level as I understand it are things such as sentence length and distribution of certain rhetorical moves, stylistic changes, paragraph length, rhythm and prosody, and other elements of textuality. This is the first time I've seen someone in this industry try to focus not just on language production but textiality production. If this is replicable across genres (which will be much, much harder than it sounds), this could be a massive change in what LLMs can do. Just incredible work, bravo.
2
20
2,534
May 19
Turns out you only need 14B 😃 dft.rosmine.ai/
crazy that it took 100B parameters to solve coding but might take 10000B parameters to solve writing
6
2
152
21,356
May 19
Confession: I spent the last 6 months bookmarking every tweet I saw about models being bad at writing so I could reply once I got my model working 😅
2
2
41
1,381
May 19
The launch was amazing, that you so much everyone ❤️ - multiple companies reached out to request DFT training - successful author said the model was incredible - at least one donation offer that was not a scam Now I'm getting ready to train the open weights model. I've figured out several tricks that are going to make the next model even better Huge shoutout to @brendanh0gan @sanmking @HrishbhDalal for providing feedback on early versions, and to @Algomancer for sponsoring this and other work They are all awesome and you should follow them immediately
May 18
I fixed why LLMs write so poorly, and I have a demo to prove it Announcing Distribution Fine Tuning (DFT): A post training step that fixes LLM writing Model outputs fooled pangram on 100% of test cases
15
7
333
22,139
May 18
Be careful of AI text detectors that are made to sell humanizers. I put in a passage from my Technical Report which was 100% written by me, and it told me it was also 74% AI generated
Replying to @rosmine
Sorry no luck Tried some text and the first 2 AI detectors I tried rejected it Text below I opened it on the train platform. A dog-eared, crumpled paperback book with a picture of a plump girl in a tartan skirt, grinning at the reader. The book smelled of damp. I found a seat near the window and settled down with it. I could remember vividly that exact morning fifty years ago, how I had run barefoot across the hot yard of our family house, clutching the book against my chest. My mother never gave in to my repeated pleas and bought me the post-war reissue ofThe Railway Children, but my father did. He had taken me to the bookseller’s shop, where he had negotiated for three shillings off the marked price. Holding the book in my hands felt like holding something precious and secret, which no one else in the world had. I read it with wide-eyed attention. Whenever I stopped reading to listen for an approaching train, the smell of the leather-bound cover filled my nostrils. I could almost hear the shrill cries of the children as their mother clutched the broken luggage label. The train was late, the children were in terrible trouble; and yet they were not at all afraid, because they had a secret which only they knew. At the end of our little platform, I bade goodbye to the book and carried on with my journey. A Publica bus was due to come along shortly, bound for Chichester, where I was to meet up with my friend, who had promised to bring along a copy of The Railway Children, newly purchased. Another dog-eared paperback book, this one in excellent condition with its dust jacket intact. I opened it and began to read. Nothing in it shocked, surprised or thrilled me. The smell of the book was clean, almost chemical. I thought of my friend, and how, with typical generously, she had gone out of her way to get me a copy. I thought of all those rainy afternoons, in a borrowed sitting room with sloping ceilings, when my friend and I would read aloud to each other. After school, after the long families lunches, when everyone was dozing; she and I in our dressing-gowns, surrounded by books borrowed from the school library. I missed the bus. I thought of my friend, who in all probability was now sitting at home, reading quietly to herself. In front of her a saucer of cold digestives and a glass of milk, gone slightly sour. She would be tired, because her father had fallen ill and she had spent a sleepless night with him. As for me, I was going to have to wait until the next bus, bound for a different destination altogether. I wandered towards the newsagents and thought of all the things I had bought there over the years – colouring pencils, lemonade, chewing gum, sweeties in little paper bags, Zines, paperbacks, more paperbacks, many, many paperbacks. As for The Railway Children, I held my own copy close to my chest and read it all the way home
6
1
72
11,048
May 18
I fixed why LLMs write so poorly, and I have a demo to prove it Announcing Distribution Fine Tuning (DFT): A post training step that fixes LLM writing Model outputs fooled pangram on 100% of test cases
124
155
3,223
502,119
May 18
The key idea is that instead of trying to improve writing quality (which is vaguely defined) I focus on making model outputs more similar to the training data Surprisingly, SFT is not all you need. I measured the distribution distance between model outputs and human reference, and there was a huge gap! With DFT I was able to reduce the distribution distance by 49%, which boosted creativity scores by 164%, coherence by 28%, and meaning detail by 146% DFT also prevents overused "slop signs" like emdash or "it's not X, it's Y" I plan to release a small open weights model trained with DFT. This demo was trained for web documents, please let me know what you want the open model to be trained for (creative writing? poetry? arxiv papers? I will not train it to write X posts) Demo: dft.rosmine.ai/ Technical Report: rosmine.ai/2026/05/18/fixing… You can guess if the "Made with AI" tag is for the text or the capybara
31
19
743
39,706
May 18
Disclaimer: I worked quite hard on this, so I will use X boost to make sure people see this launch
9
438
24,041