🚀 Paper Spotlight from
@iclr_conf #ICLR2025 🚀
Our Research Lead, Anthony Soronnadi (
@tonysoronnadi ), is live at the paper presentation showcasing 4 of our (
@dsn_ai_network accepted research work while also interacting with other amazing works like the:
ReALLM: A Unified Framework for Low-Bit Quantization and Fine-Tuning of LLMs introduces a groundbreaking approach for compressing and adapting pre-trained LLMs to ultra-low memory budgets (<4 bits)! By decomposing matrices into high-precision low-rank parts and vector-quantized representations, ReALLM achieves state-of-the-art results — even with just 2 to 3 bits of storage.
ReALLM sets a new standard for 2-bit LLM quantization with an autoencoder residuals pipeline. It unifies and outperforms previous PTQ methods while also speeding up inference!
Congratulations to Lisa Bedin, Louis Leconte, Van Minh N., and Eric Moulines from École Polytechnique, Huawei, Math and Science Algo. Sciences Lab for pushing the frontier of efficient AI!
At EqualyzAI, we're excited about innovations like ReALLM that align with our vision to build small, powerful, and accessible LLMs, especially for African languages.
Read Full Paper Here -
researchgate.net/publication…
#equalyzai #iclr2025 #researchpaper #smallllms #iclr