🚨 New Paper Alert! 🚨 How can we align language models without drowning in prompt engineering or falling into reward hacking traps? We introduce Meta Policy Optimization (MPO)—a new reinforcement learning framework that evolves its own reward model rubrics through meta-level reflection. Inspired by metacognition and evaluative thinking, MPO trains models to think about how they evaluate, not just what they generate. 🔥 Why it matters: ✔️ Boosts stability and robustness in RLAIF ✔️ Reduces human labor in prompt crafting ✔️ Generalizes across tasks: essays, summarization, ethical and mathematical reasoning Check it out: huggingface.co/papers/2504.2… Big thanks to co-authors @chanwoopark20 (MIT), @_vipulraheja (Grammarly), and @dongyeopkang (UMN)! #AI #LLMs #ReinforcementLearning #MetaLearning #NLP #Alignment #RLHF #RLAIF #EvaluativeThinking #PromptEngineering

SIP catchups at Exec, todays focus SD3 @vstavros3 @picnicpoinths #evaluativethinking #hipl @NSWEducation

Share reasons why collaboration is important for "good" evaluative thinking. Seriously, I need good arguments for my undergraduate class. #eval #reflectoneval #evaluativethinking #collaboration @clysy #evaltwitter

Join us for MEA's Learning Session#2023/13 titled 'Evaluative Thinking' by Dr. Tom Grayson, former President of American Evaluation Association & moderated by Suzanna. ⏰21:00-22:30 Thursday Dec 21, 2023 (ULAT) Register here: zoom.us/meeting/register/tJA… #evaluativethinking @aeaweb

Join us for MEA's Learning Session#2023/13 titled 'Evaluative Thinking' by Dr. Tom Grayson, former President of American Evaluation Association & moderated by Suzanna. ⏰21:00-22:30 Thursday Dec 21, 2023 (ULAT) Register here: zoom.us/meeting/register/tJA… #evaluativethinking @aeaweb

Discover the strategies and tools you need to make learning visible and intentional for your school and classroom at the Illinois Visible Learning Institute. Learn more and register at ow.ly/Ye7G50Q0pFL #VisibleLearning #EvaluativeThinking #education

Dr John Almarode From research to reality; students use the success criteria to ask themselves, “Where am I? Where am I expected to go? What do I need to do next to close the gap?" #evaluativethinking #deliberatethinking #VisibleLearning #Feedback

1/5 Hot off the presses! An exciting arrival to my doorstep today: Practical Wisdom for an Ethical Evaluation Practice. With a chapter from @tgarchibald and I on the relationship between #reflectivepractice, #evaluativethinking, and #practicalwisdom. Super proud of this work.

In Scan’s latest issue, Dr Helen Timperley unpacks the value of ‘adaptive expertise’ and the importance of embedding #EvaluativeThinking in every educational activity: education.nsw.gov.au/teachin… #AdaptiveExpertise #ProfessionalReading #k12 #AussieED #EdChat #SchoolLeadership

•Scan’s Term 4 issue is live! Read about: 📌strengthening #equity of education with @Pasi_Sahlberg 📌teaching #oracy 📌adaptive expertise & #EvaluativeThinking 📌#history planning tools 📌our new online archive! … and MORE (free!) Scan: education.nsw.gov.au/teachin… #k12 #AussieED

Scan’s Term 4 issue is live! Read about: 📌strengthening #equity of education with @Pasi_Sahlberg 📌teaching #oracy 📌adaptive expertise & #EvaluativeThinking 📌#history planning tools 📌our new online archive! … and MORE (free!) in Scan: education.nsw.gov.au/teachin… #k12 #AussieED

Good to emphasise the work to be done after an #evaluation report is published - but let's not leave out #processuse - the impact of evaluation processes esp. in terms of #evaluativethinking and early action. #AsianEvaluationWeek22

So many different approaches, choice on #context Bang on about #EvaluativeThinking; don’t forget about designing interventions and integrating these spaces! Barrier to entry is language and terminology - work on definitions. Text processing and adding to #evaluations! #aes22ADL

Other big difference - focus on -#evaluativethinking rather than #rvaluation reports. #aes22ADL

#evaluativethinking, #analyticalthinking

Example of building in #evaluativethinking & #theoryofchange in implementation processes @CIF_Action Neha Sharma #EES2022

This is an interesting analogy for the use of results chains ( or impact ladders ). If you make the levels too far apart you lose people. #evaluativethinking

NEXT: To what extent has and will the recent work undertaken by the @NAOorguk influence the ways in which Government embeds #evaluativethinking and #evaluation literacy across the policy development cycle…🤔 #ukesconf22 @UKevaluation

"It isn't what you think, it's HOW you think that matters." Wisdom from @b_mmoorhead today! 🤯🤓#evaluativethinking #CriticalThinking

Some early thinking about systems, design and evaluation all together - but also for different purposes. #systemsthinking #designthinking #evaluativethinking matthealey.substack.com/p/mu…