#NewPaper Have you been wondering how your favorite LLM, e.g. Llama, Mistral, or Gemma performs on materials property prediction? We have just released LLM4Mat-Bench, an extensive benchmark for materials property prediction with LLMs! LLM4Mat-Bench has unique features: ☀️It spans 10 data collections, containing more than 2.6 Million data points. ☀️It covers 45 distinct material properties. ☀️It covers three different material representations: CIF, text description, and composition. ☀️It provides baseline results from different types and sizes of LLMs, e.g. Llama, Mistral, Gemma, MatBERT, and LLM-Prop. With materials data scattered everywhere, we believe LLM4Mat-Bench represents a unified data source for driving research on leveraging LLMs for materials science. The benchmark will be maintained and we look forward to your task and data contributions. Our @andre_niyongabo will present the paper at the AI4Mat #NeurIPS2024 workshop this December. Paper: arxiv.org/abs/2411.00177 Code: github.com/vertaix/LLM4Mat-B… Authors: Andre Niyongabo Rubungo (@andre_niyongabo), Kangming Li (@KangmingLi_), Jason Hattrick-Simpers, and Adji Bousso Dieng (@adjiboussodieng) #AI4Materials #MatSci #NLP4Science #Benchmarks #LLMs #Vertaix