Here for the research papers
Joined November 2012
- Tweets 101
- Following 290
- Followers 1,132
- Likes 1,258
4 Photos and videos
rewon retweeted
29 Jun 2023
Excited to announce that we’ve raised $1.3B to build one of the largest clusters in the world and turbocharge the creation of Pi, your personal AI.
forbes.com/sites/alexkonrad/…
142
307
2,541
929,026
rewon retweeted
29 Jun 2023
It’s a big week! We’ve raised $1.3 billion and are building the world’s largest AI cluster (22k H100s).
We’re grateful for our investors and new funding that will help us accelerate our mission to make personal AI available to every person in the world. inflection.ai/inflection-ai-…
68
212
1,466
2,768,081
rewon retweeted
22 Jun 2023
We have amazing results to announce! Inflection-1 is our new best-in-class LLM powering Pi, outperforming GPT-3.5, Llama and PALM-540B on major benchmarks commonly used for comparing LLMs. inflection.ai/inflection-1
22
86
520
170,731
rewon retweeted
29 Jun 2021
Kullback-Leibler divergence is not the same as Leibler-Kullback divergence
50
281
3,188
rewon retweeted
10 Mar 2021
I've ported @rewonfc's very deep VAE (arxiv.org/abs/2011.10650) from PyTorch to JAX/Flax! Hope other JAX users find this SOTA VAE useful as a forkable baseline... github.com/j-towns/vdvae-jax.
2
10
93
rewon retweeted
5 Mar 2021
My post on SotA image generative models was released 🥳
Featured 7 notable recent papers with emphasis on:
- VD-VAE
- VAE discriminator (e.g. VQGAN, DC-VAE)
- Diffusion models (e.g. DDPMv2)
Plus some notes on scaling (e.g. DALL-E) and evaluation.
arankomatsuzaki.wordpress.co…
1
56
295
rewon retweeted
5 Jan 2021
IMO, best empirical proof to date that AI can be creative. After this sinks in, will there be any naysayers left?
5 Jan 2021
"The images are preprocessed to 256x256 resolution during training. [...] each image is compressed to a 32x32 grid of discrete latent codes using a discrete VAE that we pre-trained using a continuous relaxation."
GPT VAE scale = impressive results!
openai.com/blog/dall-e/
8
6
128
Congrats to Aditya and the rest of the team for an awesome release!
14
1 Dec 2020
Happy to announce our new work on score-based generative modeling: high quality samples, exact log-likelihoods, and controllable generation, all available through score matching and Stochastic Differential Equations (SDEs)!
Paper: arxiv.org/abs/2011.13456
6
rewon retweeted
6 Oct 2020
It breaks my 💚 when researchers tell me that VAEs don't work. My first typical question is "did you try hierarchial VAE or vanilla VAE?", the answer is usually vanilla VAE.
VAEs work much better with hierarchical structures. NVAEs and this work take this to the extreme!
Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
“Very Deep VAEs” achieve higher likelihoods, use fewer parameters, generate samples 1000x faster, and are more easily applied to hi-res images, compared to PixelCNN.
openreview.net/forum?id=RLRX…
8
87
565
rewon retweeted
9 Sep 2020
Posted my first paper on arXiv💥🙌
GPT-f is a Transformer-based automated theorem prover. We show that Transformer Search is suitable to formal reasoning and continuous self-improvement 🦾
arxiv.org/abs/2009.03393
17
185
866
Thanks if you came to our ICML poster on Distribution Augmentation. The zoom discussion was way more fun/interesting than I expected! TLDR of our work: use powerful data aug in your generative model by conditioning it on the aug. Improves samples likelihoods considerably.
2
9
48
Our base model is a Sparse Transformer. If we make it bigger and train for a while with this augmentation, it results in both very high likelihoods (2.55-2.65 bpd on CIFAR-10) and also samples equal/better than most GANs (as measured by FID). Code here: github.com/openai/distributi…
1
2
9
rewon retweeted
15 Jul 2020
I keep seeing all kinds of crazy reports about people's experiences with GPT-3, so I figured that I'd collect a thread of them.
33
847
2,871
rewon retweeted
17 Jun 2020
We found that just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. openai.com/blog/image-gpt/
3
27
98
rewon retweeted
30 Apr 2020
Hot Tub Christmas (with GPT-2 lyrics selected by @rewonfc ) is my own favorite -- even though the model can't figure out what's going on in the completely terrible intro. Our weird but fun new holiday tradition! soundcloud.com/openai_audio/…
1
8
27
One of my favorites from @OpenAI's jukebox: 'Lose Yourself' re-rendered by Kanye
soundcloud.com/openai_audio/…
1
6
53