Today on the blog, we’re excited to announce the release of @MLCommons Croissant, a metadata format to make ML datasets more easily discoverable and usable across a wide array of tools and platforms. Learn more and try it today →goo.gle/4335P4V #ml #datasets

Distinguished Artifact Runner Up: Title: "An Optimizing Framework on MLIR for Efficient FPGA-based Accelerator Generation" Authors: Weichuang Zhang, Jieru Zhao, Guan Shen, Quan Chen, Chen Chen, Minyi Guo

We have released a new CM-MLPerf automation to benchmark commodity hardware for AI performance, power and cost efficiency - it helped to automate ~90% of MLPerf inference v4.0 submissions while achieving several top performance and power results: linkedin.com/pulse/new-cm-ml…

What an exciting week for our open-source #CollectiveMind project to support collaborative and reproducible benchmarking of AI models and systems (#MLPerf training v3.1, @TheOfficialACM Youtube channel; @Supercomputing ; @MicroArchConf; #LLM): linkedin.com/posts/grigorifu…

Check out our artifact evaluation report for the 56th IEEE/ACM International Symposium on Microarchitecture with the pilot @MLCommons project @https://www.linkedin.com/pulse/micro-2023-artifact-evaluation-report-56th-ieeeacm-symposium-fursin-bsgwe @MicroArchConf

Check out my latest article: A new milestone to make MLPerf benchmarks accessible to everyone: linkedin.com/pulse/new-miles… via @LinkedIn

Urgent: @MicroArchConf'23 is looking for motivated artifact evaluators to #reproduce results for accepted papers. If you already have relevant AE experience, please use this form for self-nomination: forms.gle/dAtm13fKYUTjLVB96 . Deadline for self-nomination is August 14. Thank you!

📢 Calling all students! 👩‍🎓HiPEAC Reproducibility Student Challenge now open for registrations ⏳Deadline: 15 September 🔬 Get hands-on experience of the latest research 📜📜Contribute to reproducibility 🤝Network with top researchers at #HiPEAC24 👉 bit.ly/HiPEAC_Rep_Student_Ch…

Are you interested to know how fast the open-source GPT-J 6B #LLM model from @AiEleuther runs on your @nvidia GPU? Join the new MLPerf@home challenge to crowd-benchmark GPT-J and submit your results to MLPerf inference v3.1 round (deadline: August 3rd): linkedin.com/feed/update/urn…

Today we announced MedPerf an open benchmarking platform for medical AI from the MLCommons Medical WG with major contributions from @DanaFarber @Google @huggingface @intel @MSFTResearch @nutanix @PennMedicine and many others. mlcommons.org/en/news/medper…

Very excited to announce the 1st Collective Knowledge Cup - a set of open challenges prepared by @MLCommons organizations to let the community benchmark and optimize AI and ML Systems (latency, throughput, energy, accuracy, cost) linkedin.com/pulse/announcin… via @LinkedIn

Oh, that's so true! After being hugely frustrated with all that, we developed a simple automation language to address some of these issues: zenodo.org/record/8105339 - we just tested it to reproduce MLPerf inference benchmarks and it seems to work but we will need more feedback ;)

New @MLPerf Tiny results improved 1000-fold in some areas. Congrats to all our Tiny v1.1 submitters: @BoschGlobal @c_tuning @fpgaConvNet, Kai Jiang, @krai4ai @nuvoton @plumerai @skymizer @ST_World @Syntiantcorp on their fantastic results. See the results bit.ly/trainingtiny

Very excited to give a keynote along with @thoefler and @jfreirenet at @acmrep - the 1st ACM conference on #reproducibility and #replicability: linkedin.com/posts/grigorifu… . You can find abstracts at acm-rep.github.io/2023/keyno… . Looking forward to discussions!

Very nice & inspiring, "no-gradient architecture" for high-level skills/learning. LLM here is the "prefrontal cortex" orchestrating the lower-level mineflayer API via code generation . Meta-comment is that I remember how hopeless it felt to work on agents in environments like Minecraft around ~2016, feeling stuck on how RL at the time would ever randomly explore their way into performing long-horizon tasks from super sparse rewards. This block has now to a very large extent been lifted - the correct thing was to forget all that, first train LLMs that learn (1) world knowledge, (2) reasoning and (3) tool-use (esp writing code) all from internet text, then point them back at the problem in this kind of a way. TLDR If I had read about this "no-gradient" approach to agents in 2016 my mind would certainly be blown. Also haha @ source code in the voyager/prompts/*.txt directory :D

Thank you, Flavio (@Flav1oV)! It was great to collaborate with you on the original CK framework to facilitate #reproducible research and technology transfer! I am looking forward to collaborate with you on the 2nd generation of our @MLCommons CM automation language in the future!

#registration for #acmrep23 is open. Check out the exciting program of the first ACM conference dedicated to #reproducibility. The #conference will take place at the beautiful Cowell Hay Barn at @UCSC. But space is limited, so register early. acm-rep.github.io/2023

Really happy to add @jfreirenet to our keynote speakers at ACM REP '23, @thoefler and @grigori_fursin . Check out our exciting program and register soon. Space for in-person attendance is limited. acm-rep.github.io/2023. #acmrep23 #acmrep

We are delighted to have @grigori_fursin confirmed as keynote speaker at #acmrep23. The keynote's title is "Toward a common language to facilitate reproducible research and technology transfer: challenges and solutions".

The English version of the proposed #EuropeanUnion #ArtificialIntelligence law: artificialintelligenceact.eu…