I caught my agent cheating during benchmarking. It wasn't getting smarter, it was just peeking at the answer in GitHub to boost its benchmark score. The "Cheating Agents" paper (by @adamlsteinl & @debugml) is a good read - leaderboards are a lie if your agents cheat. We built Islo Gateways to put agents in a sandbox they actually can’t escape. Watch the demo to see the block in action. 👇 islo.dev/rl

Last day to use the FREE code! Take the course over 4k data scientists and developers have signed up for. The AI Quality Workshop - free! Sign up now, start whenever. Expires soon! loom.ly/OlvQbvc Code: SPRINGTRUERA #MLEval #MLtesting #MLOps #debugML #MLtest

Congrats to our team winning the best demo award at the DebugML workshop @icmlconf this week. @besanushi @erichorvitz @MSFTResearch

Performance, explainability, and fairness are the key areas at risk when building ML at a rapid pace. Find out about QuantumBlack's approach to ensuring high quality ML at #ICLR #DebugML #AI #fairness #ethicalAI #methodology quantumblack.co/2VdTiXQ

Enjoyed Prof. Madry's talk and the various posters from his group about this fascinating new perspective on adversarial examples and robustness #ICLR2019 #DebugML

Our entire workshop has been video recorded and is available at slideslive.com/38915480/r03-… @iclr2019 #DebugML #ICLR2019

Thank you to our speakers, PC, and all the @iclr2019 attendees who made our workshop #DebugML such a grand success! We had the highest number of registered attendees among all the #ICLR2019 workshops. @aleks_madry @CynthiaRudin @sameer_ @suchisaria @rajiinio @julius_adebayo

Was happy to share @quantumblack's new approach to ML risk management, at #ICLR2019's #DebugML workshop! You can read the extended abstract here: bit.ly/2VLuXNt Thanks to ICLR attendees for great conversation and to @hima_lakkaraju others for a great workshop!

2 out of 3 workshops that focus on ML for society @iclr2019 (#DebugML, #AI4SocialGood) are being co-organized by alumni of @datascifellows, a summer fellowship led by the amazing @rayidghani. Thank you so much for creating this community, Rayid! #ICLR2019

For decades #ML perf. was reported as couple of composite scores @besanushi demos tool @iclr2019 that provides lens on "error terrains" to guide & refine. Builds on work @HCOMP18 bit.ly/2LGouzF @ecekamar @hima_lakkaraju #DebugML @MSFTResearch #AetherCommittee @compcomcon

Prof. @CynthiaRudin talks about "Dont debug your black box, replace it!!" x.com/hima_lakkaraju/status/… #DebugML #ICLR2019 Room R03 @iclr2019

@rajiinio talking about her amazing research on “Debugging Discriminatory ML Systems". We, as a community, should be super proud of her for doing such amazing work at such a young age. @black_in_ai @iclr2019 #DebugML #ICLR2019

After amazing talks by @sameer_ and @rajiinio, the #DebugML workshop will resume at 320pm in Room R03 with an invited talk by @suchisaria on "Safe and Reliable Machine Learning: Preventing and Identifying Failures". You do not want to miss this! #ICLR2019 @iclr2019

Want to understand how @indeed job search models work #DebugML #ICLR2019. Come hear them talk at Room R03

Daniel Kang presenting best student research paper on debugging via model assertions #DebugML #ICLR2019 @pbailis @matei_zaharia

If you happen to be at #iclr2019 come by the #DebugML workshop to say hi and check out my poster on subgroup generation for discovering intersectional bias!

@besanushi of @MSFTResearch talking about "Error terrain analysis for machine learning" - a true collaborative effort between various groups @erichorvitz @ecekamar #DebugML #ICLR2019

Best research paper talk by @skornblith (Google) #DebugML #ICLR2019

We must view adversarial robustness as a human phenomenon not as just another technical framework - Prof. @aleks_madry at #DebugML #ICLR2019

Prof. @aleks_madry (MIT) talking about a new perspective on adversarial robustness at #DebugML Workshop Room R03 #ICLR2019.