a little saturday morning lite expo writing
New Post:
Of Assholes and Obsessives
on the distorted intensities of ego and conviction
tdevane.substack.com/p/of-as…
I’m excited to finally reveal Near Horizon to the world!
In early 2023, @mfk , @mjacobstein and I were frustrated at how great solo founders were told to “go find a co-founder” by traditional VC firms ...
...so set out to create a new kind of early stage venture firm!
1/
OpenAI API compatibility shipped for 100 models on @togethercompute API.
Replace GPT calls with Mixtral or Llama-70B, get faster responses and for less $$
🚀🚀🚀
We are thrilled to introduce our Chief Scientist @tri_dao! Tri joins from @StanfordAILab where he’s focused on efficient training of AI models.
Today he released #FlashAttention2—a significant breakthrough to speed up LLM training & inference. Read more: together.ai/blog/tri-dao-fla…
The era of sub-quadratic LLMs is about to begin. At @togethercompute we've been building next gen models with large space state architectures and training them on very long sequences and the results from the recent builds are... incredible. Will share more as we get closer to releases!
A Lido V2 Audit Update:
Security is a top priority. To this end Lido DAO dedicated significant effort to 9 independent V2 audits.
These audits have uncovered important findings, all of which have been either acknowledged or fixed.
🧵
The first RedPajama models are here! The 3B and 7B models are now available under Apache 2.0 license, including instruction-tuned and chat versions!
This project demonstrates the power of the open-source AI community with many contributors ... 🧵 together.xyz/blog/redpajama-…
RedPajama-7B performs better at 440B tokens than all the best models trained on Pile, and continues to get better. More information on experiment design in the blog post and will keep you all posted as this converges further!
Training our first RedPajama 7B model is going well! Less than half way through training (after 440 billion tokens) the model achieves better results on HELM benchmarks than the well-regarded Pythia-7B trained on the Pile.
Details at together.xyz/blog/redpajama-…
Llamas in Red Pajamas will be everywhere soon! In the meantime here’s the inimitable Rae Sremmurd for your weekend listening 😁🦙youtu.be/HqiqVZ8PJ2Q@togethercompute
Today, I decided to do a deep-dive into the age-old gas saving trick:
Using i instead of i .
You may have seen this trick and asked yourself how a change this trivial and inconsequential could result in a difference in gas usage.
Well folks, here's the full explanation 🧵:
Introducing GPT-JT, a 6B parameter open source model that can outperform many 100B parameter models and was trained over slow (1Gbps) internet links.
together.xyz/research/releas…
.@github @GitHubHelp haven't been able to login to my account for weeks, what's going on: "There have been several failed attempts to sign in from this account or IP address. Please wait a while and try again later." Nothing from the support ticket either... Pro user too. :(
Current Alameda borrow on WAVES looks to be 631,763 paying ($30M ) 12.5% APR
api.vires.finance/user/3PHkZ…
While WAVES-PERP on FTX is at -120% APR, Binance at -80% APR