Joined January 2022
- Tweets 5,229
- Following 636
- Followers 1,952
- Likes 568
525 Photos and videos
Pinned Tweet
I ran a 35-billion parameter AI agent on a $600 Mac mini.
Specs: M4 Mac-Mini 16GB RAM
The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second.
On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster.
No cloud. No API keys. $0/month.
Here's what it can do đ§”
171
211
3,253
725,182
I was told GPUs were the only way..
Jun 13
56,000 tokens/sec at just 80 MHz. đ€Ż
I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU)
Just pure digital silicon running @karpathy microGPT, spelling out names on a tiny LCD.
This is GateGPT đ
1
145
salesforce?????
Jun 13
contrarian ai take:
the safest startup is no longer the one with the prettiest interface.
it is the one buried inside a disgusting workflow with permissions, exceptions, refunds, audits, and angry humans.
beauty is where incumbents and tourists go to die.
45
Guess who would benefit if anthropic dropped aws? itâs all a game
SpaceX cuts compute deal and Anthropic turns off fable
106
dramatic
Jun 13
The United States is no longer the best place to build an AI lab.
55
âsuper powerful modelâ
83
itâs all a show guys, truly theatre at this point. because the govt went back on that statement and re-enabled it. Fable will be back by next week
Three months ago, @DeptofWar kicked @AnthropicAI out of our buildingâforever.
Every passing day proves why that was the right move. đșđž
117
let me guess before the markets open Anthropic and the govt will have a positive message
39
âlocal ai will win!â
says the guy whoâs telling you to spend 4K on GPUs alone, doesnât tell you how much itâll cost in watts, or the pain of installing GPUs.. totally ignores just buying a Mac đ
some people really arenât in touch
1
1
58
all of the sudden everyone gets open source
I ran a 35-billion parameter AI agent on a $600 Mac mini.
Specs: M4 Mac-Mini 16GB RAM
The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second.
On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster.
No cloud. No API keys. $0/month.
Here's what it can do đ§”
1
124
SpaceX cuts compute deal and Anthropic turns off fable
196
I also think doing local developments with GPUs is pointless unless youâre being given GPUs. Macs are the future of any local ai and their machines have an insane OS as well as an amazing computer. Mac > GPU
I love the local ai movement that is trying to monetize the local ai movement đ fam local ai means just downloading from GitHub you donât need a website
1
76
I love the local ai movement that is trying to monetize the local ai movement đ fam local ai means just downloading from GitHub you donât need a website
1
121
Google Omni : { tcg, Japanese, some animations, comic book effects }
85
the day in the life working at a frontier labs:
wake up, watch whatâs popping on X, clip content doing those viral things on X with frontier model, rinse repeat đ
44
moe plz
Jun 10
$GOOGL released DiffusionGemma which is an experimental 26B MoE model that uses text diffusion to generate text in blocks instead of one token at a time.
The model delivers up to 4x faster inference and is aimed at speed-critical local AI workflows.
1
75
đ đ đ
Introducing Claude Fable 5: a Mythos-class model that weâve made safe for general use.
Its capabilities exceed those of any model weâve ever made generally available.
55
@interaction system prompt is written in XML
Why would you want your agent using HTML for references when XML code is designed specifically for storage and transporting data?
1
1
101
Jun 8
now they are
main agent (as close as possible): gist.github.com/burritosoftwâŠ
email processing subagent: gist.github.com/burritosoftwâŠ
older ones (before apple messages): github.com/x1xhlol/system-prâŠ
61
Look at @interaction systemâs prompt guess what file their prompt is in? Everyone talking about HTML this and that.. I might know a thing or two
đ
you want a real hack? XML, the tags do wonders for your agent. youâre welcome
1
1
132
Jun 8
now they are
main agent (as close as possible): gist.github.com/burritosoftwâŠ
email processing subagent: gist.github.com/burritosoftwâŠ
older ones (before apple messages): github.com/x1xhlol/system-prâŠ
84