Sure! Here's a step-by-step explanation of the "Agentic Knowledgeable Self-awareness" framework (KnowSelf) based on the provided visuals and context:
1. Concept Overview:
KnowSelf enables agents to dynamically assess situations and adapt their behavior by switching between three modes of thinking:
Fast Thinking: For simple tasks the agent can handle directly.
Slow Thinking: For moderately challenging tasks requiring reflection.
Knowledgeable Thinking: For complex tasks where external knowledge is needed.
2. Framework Steps:
Step 1: Self-awareness Data Construction
Agents explore their environment and log actions, observations, and reflections.
They classify situations using a heuristic criterion into fast, slow, or knowledgeable thinking.
Example: If an egg is dirty, the agent reflects on how to clean it or finds a clean one instead.
Step 2: Self-awareness Learning
The agent undergoes a two-stage training process:
Supervised Fine-Tuning (SFT): Teaches initial self-awareness patterns.
Reinforcement with RPO Loss: Optimizes long-term planning by refining decision-making strategies.
Step 3: Self-awareness Inference
During task execution, the agent identifies situations in real time and generates special tokens (e.g., <reflection>, <knowledge>).
Example: When tasked to put two newspapers in a drawer, it reflects on errors and incorporates knowledge to prioritize correct actions.
3. Performance Highlights:
KnowSelf was tested on ALFWorld tasks using Llama-8B and Gemma-2B models.
It outperformed baseline methods by efficiently balancing reflection and knowledge use, achieving higher task success rates with minimal resource usage.
Let me know if you'd like more details about any specific part! 😊