Just because you can measure something doesn’t mean it matters
Big Tech may be learning this lesson the hard way with AI.
Amazon mandated that 80% of developers use AI weekly and built dashboards tracking token consumption.
So employees adapted.
They're running AI agents on personal tasks, auto-drafting emails nobody reads, using sub-agents to analyze Slack threads for fun. One Amazonian openly bragged about burning tokens to roast his PM. Other Big Tech companies also have these dashboards in place. Everyone is measuring token usage, as if the tokens are the productivity.
Once a metric becomes a target, people start optimizing for the metric rather than the outcome.
This is the McNamara Fallacy in the digital age.
In the Vietnam War, U.S. Secretary of Defense Robert McNamara counted dead bodies as a metric for whether the U.S. was winning the war, because bodies were easily countable. Morale, legitimacy, the will of the enemy were ignored because they were harder to measure.
In the end America won every metric and lost the war.
Sociologist Daniel Yankelovich summarized the trap:
"Measure what's easy, dismiss what isn't, then pretend the unmeasurable doesn't exist."
The tech industry has been falling into the same trap for decades. We count the effort, not the result:
→ Lines of code written, not problems actually solved. → Tests run, not real risks caught.
→ Tasks automated, not reliability gained.
→ Tokens burned, not increased quality of work.
A metric that exists just to show a number isn't measuring anything.
So before you roll out a dashboard, ask yourself:
Will this metric just measure output, or will it actually drive a better outcome?