New ACL 2022 System Demo paper! It used to take a lot of technical effort to set up custom AI tasks, evaluate models, and collect crowdworker data with models in-the-loop. We’ve added a new framework to @DynabenchAI that aims to help: Dynatask. arxiv.org/abs/2204.01906 1/3

Yes, exactly - in the Dynatask validation setup: some right, some wrong, and some we expect will get flagged.

Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks abs: arxiv.org/abs/2204.01906

@samlightstone Software Engineer at @facebookai will introduce and demo a new open-source paradigm for AI benchmarking called #DynaTask that uses dynamic adversarial data collection to evaluate #AI models & assess how easily an AI can be fooled by humans Know more at #TMLS2021!

Facebook AI Unveils Dynatask, A New Paradigm For Benchmarking AI, Enabling Custom NLP Tasks For AI Community Quick Read: marktechpost.com/2021/09/24/… #AI #ArtificialIntelligence #NLP #BigData #DataScience @facebookai

Super exciting news: You can now create your own Dynabench tasks using Dynatask!