If you want an AI system to emulate the decisions of a human expert, then you need to give it all the key information that the human uses during their decision-making process. That's been our philosophy at @_MathAcademy_, and we're about to start taking it to a whole new level. Currently, we have our curriculum organized into a gigantic, super information-rich knowledge graph of - thousands of topics, - tens of thousands of knowledge points, and - tens of thousands of "edge" relationships between them (e.g., A is a prerequisite of B, but also other kinds of relationships). And to determine the most optimal tasks for a student to work on, we 1. overlay every single one of their answers onto the knowledge graph, 2. have the information "flow" through the graph, 3. determine the student's mastery & memory state at each topic, 4. identify topics in need of review (sufficiently low memory state) and topics on the student's "knowledge frontier" (not yet learned but all prerequisites mastered), and 5. compress those into the smallest set of tasks that "knocks out" all the due review while simultaneously maximizing the rate at which the student learns new material and pushes their knowledge frontier forward. (More details here: mathacademy.com/how-our-ai-w…) This process works great and it's enabled us to drastically speed up a student's pace of learning by charting an efficient path through the curriculum that is tailored to their needs! However, there's another piece of the puzzle that's still needed: micro-behavior coaching. We get the student learning/reviewing exactly the right topics, working on exactly the right exercises -- but we still need to get the student engaging in exactly the right behaviors that are productive for learning. In other words: even if students are working on exactly the right things, they need to be working exactly the right way to capture the most learning from their time spent working. Productive learning behavior can be hard for some students to manage on their own because it usually involves balancing a tradeoff between two extremes. For instance: - Read the instructional material carefully before you attempt to solve the problems -- but don't spend too much time on it, because the problem-solving is where the learning happens! - When you solve a problem, try to minimize your reliance on reference material. Instead, retrieve information from memory. But if you're really stuck and can't remember what to do despite your best effort to recall, don't just sit there! Peek back at the worked example to refresh your memory. But then try to solve the rest of the problem unassisted -- don't solve the problem side-by-side with the worked example! - When you work on math, focus fully on the math! Don't continually get distracted and interrupt your train of thought. When you're engaging in deliberate practice at the edge of your ability, 30 minutes of full focus moves the needle much further than 60 minutes of half-focus where your train of thought is continually being interrupted. Now, I could go on for a while listing micro-behaviors that are necessary to support learning... But even with a fully comprehensive list of micro-behaviors, few students in need of it would actually read it and put it into practice. What's needed is just-in-time feedback: detect when a student is engaging in unproductive behavior and tell them how to fix it. (And incentivize them to fix it -- so that regardless of whether the student is intrinsically interested in understanding the material deeply, or they just want to earn enough XP to keep their parent/teacher off their back, the quickest path to reward is to work productively and maximize their learning.) That's my next big modeling task -- and as @exojason describes below, we now have the data to make it happen.

4 days until IES Single-Case Research Design and Analysis Training Institute and my office looks like it! Sometimes the PI is also head of IT dept, but @ssequinn is my everything at the Madison Institute including my back up brain.