In this report we describe the 2025 Apple Foundation Models ("AFM"). We also introduce the new Foundation Models framework, which gives app developers direct access to the on-device AFM model. machinelearning.apple.com/re…

1/ KV Prediction for Improved Time to First Token Tired of waiting forever for your on-device LLM to begin outputting tokens? I know I am. In our latest preprint, we investigate a method for improving time to first token (TTFT) for on-device models by predicting a model’s KV cache using a small Auxiliary model. We demonstrate improvements in TTFT of up to 2x at a fixed accuracy. Full details: arxiv.org/pdf/2410.08391 . Code at github.com/apple/corenet/tre… . Work done with @awk_ai , Chenfan Sun, Yanzi Jin, @sacmehtauw @morastegari Moin Nabi. #LLM #Apple #Research #TTFT #Ondevice #OpenELM #Corenet #KVPrediction

@awk_ai @HannaHajishirzi Can’t run billion-level LLMs efficiently? Take a look at our work: APT. We are excited to share our #ICML2024 oral paper, “APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference”. Paper: shorturl.at/xabJl

Thank you Akari for the help!

Like OpenELM, CatLIP is also "Open" github.com/apple/corenet

Apple presents OpenELM An Efficient Language Model Family with Open-source Training and Inference Framework The reproducibility and transparency of large language models are crucial for advancing open research, ensuring the trustworthiness of results, and

I'll be moderating an Efficiency in #NLProc Panel at the #SustaiNLP 🌿 workshop at #ACL2023 this Thursday. We have top leaders from Academia and Industry: @DrorRotem, @myleott, @sysnlp, @sarahookr Any questions you would like to ask? Post here or join the panel!

Don't miss our #ACL2023 tutorial on Retrieval-based LMs and Applications this Sunday! acl2023-retrieval-lm.github.… with @sewon__min, @ZexuanZhong, @danqi_chen We'll cover everything from architecture design and training to exploring applications and tackling open challenges! [1/2]

Introducing💃AdANNS: A Framework for Adaptive Semantic Search🕺 TL;DR: Up to 90× faster nearest neighbor retrieval and 2× lower memory cost for web-scale search. Applies to vector search at scale & improves all "retrieval" augmented models! arxiv.org/abs/2305.19435 [1/8]

4-bit QLoRA is here to equalize the playing field for LLM exploration. You can now fine-tune a state-of-the-art 65B chatbot on one GPU in 24h. Paper: arxiv.org/abs/2305.14314 Code and Demo: github.com/artidoro/qlora

Looking for valuable insights and carefully curated pointers on efficient NLP research directions? 🔎 Check out our updated survey covering an extensive range of topics in the classic NLP pipeline 🚀: arxiv.org/abs/2209.00099 Product of an amazing collaborative team effort! 🤝

How can NLP be more efficient? We can't blindly scale forever. This survey I'm lucky to be part of presents current efficiency research at many stages of the NLP process, and actionable advice to practitioners for making their NLP more efficient. arxiv.org/abs/2209.00099 #nlproc

I'll be at #NAACL2022 this week, my first ever in-person #nlproc conf since I joined the @uwnlp group at the University of Washington. I'm happy to chat about efficient #nlproc (QA, retrieval, vision-language models, etc.) or postdoc at UW, life in Seattle.

@lal_yash is presenting this in Bavaro 4 right now! Stop by and have a chat @emnlpmeeting #EMNLP2021