AI can help nonprofits scale impact. Is your organization ready? Learn how to build responsible AI policies that align with your mission. Register by August 5: tmu-emarketplace.paymytuitio… #nonprofit #impact #digitalskills

Why we need orgs like @HumanFeedbackIO and open source benchmarks to measure and track these changes in model behavior and performance.

Had a great time talking about evaluations for language and embodied agents at the Paper Club by @AITinkerers . Thank you to all the people who tuned in and asked great questions! @HumanFeedbackIO paperclub.aitinkerers.org/p/…

RT @Prolific: 🇨🇦 From Toronto to London 🇬🇧 We’ve had another month full of exciting discussions with AI/ML builders at @AITinkerers! We he…

Proud to help launch RAISE — a first-of-its-kind program to help Canadian nonprofits adopt AI responsibly. Led by @HumanFeedbackIO w/ @creativedlab & support from @DTSupercluster. A big step forward for #ResponsibleAI #SocialImpact. 🔗 humanfeedback.io/ #AIforGood

CDL, @HumanFeedbackIO, and @daisTMU are proud to jointly announce the launch of the Responsible AI Adoption for Social Impact (RAISE) pilot program, a national initiative to help nonprofit organizations across Canada adopt AI responsibly and effectively. globenewswire.com/news-relea…

📊 Introducing Prolific’s AI user experience leaderboard—the most reliable framework for evaluating AI against human preferences. We’re thrilled to share our benchmark on @huggingface, which assesses how well language models handle real-world tasks based on user experiences. 👇

This evaluation approach means we can tune #AI output to human preferences for better accuracy, alignment, and advancement. Special thanks to @huggingface, @HumanFeedbackIO, @MLCommons, @collect_intel, and @FactoredAI for supporting the development of this benchmark.

Glad to see @communicable here!

OpenAI's CPO Kevin Weil: Why You Need Evals to Build AI Products "The easiest way to think about it is as a quiz or a test to gauge how well a model knows a certain set of subject material, or how good it is at responding to a set of questions. Evals basically perform as benchmarks for how smart and capable the model is." @kevinweil with @lennysan

Loved the @lennysan ep with @rauchg . Some non-obvious insights/themes that stood out to me: Idea velocity going wild: AI lets you test, iterate, and fail way faster. Roles are blurring, silos collapsing: AI means PMs, Designers, Marketers can actually build stuff, fewer hand-offs needed. Open ecosystems are powerful: Shared code (LLM training data) shared ideas (v0 Community) = faster innovation. Engineering grounds AI: Best AI tools spit out reliable, scalable, production-level code and practical results. Feedback loops power AI: Direct user feedback literally trains the AI, making products get better quickly. Human intent still king: AI is a superpowered co-pilot, but human creativity, judgment, and vision are crucial. Tools for builders = force multiplier: Building tools that help others build stuff creates exponential impact.

Ok this kind of exploded, in a good way. Our very first large meetup was hosted by @Shopify youtube.com/watch?v=PVOwXCO1… Happy to be back for our next event on April 24th toronto.aitinkerers.org/p/ai… Supporting AI/ML builders in this city and beyond via @HumanFeedbackIO x @AITinkerers 🚀

At my org @HumanFeedbackIO we: - Produce Paper Club feat. AI researchers doing talks on their papers for AI/ML builders - Build AI Tinkerers Toronto community via meetups, tech workshops, etc - Collaborate with leading universities on AI in Production mentorship program

Where are AI agents deployed today? USA. shared via research by @StephenLCasper on @AITinkerers Paper Club call just now @HumanFeedbackIO @composio @KaranVaidya6

Join our next Paper Club session @AITinkerers @HumanFeedbackIO - almost 700 people registered for this online session with the authors. @communicable - I just added @Paper2Audio to the page so you can listen to a summary of the paper.

747 registered for this week! Paper Club - @AITinkerers @HumanFeedbackIO - come hang with @StephenLCasper and learn about the AI Agent Index ✈️

This awesome team of @UofT students is about to demo the AI Tutor Chatbot they created over the course of the last couple of months as part of their course. I’m very proud of the effort you put in guys 👏🎉 @HumanFeedbackIO @AITinkerers #TorontoAI @mozilla

If you want to influence the future, now is the time to release a really good benchmark. x.com/jamescham/status/18944…

One reason I wish more humanities-oriented people would engage with AI is that models are writers, trained on words, producing words. There are strengths & weaknesses in the models that can only be seen if you engage deeply with them as writers. They do not show up in benchmarks

How about applying to the automated production of reproducibility reports of existing papers? This should be easier than producing entirely new papers, and be quite useful in that this is a type of work that we probably don't do enough as a community.