A country of geniuses is a country with disagreements, status, and hierarchy. This is inherent to the creative process in the face of resource constraints.
In other words AI will have politics.
It cannot be engineered out without destroying creative output. These are things that are not inherent to humans.
It’s not an accident Ancient Athens had such a chaotic political life with vibrant characters & competing personalities.
While I love the idea of a country of geniuses or an intelligence explosion, I think it misses the point that at no time in history have we had this many smart people with access to education and the freedom to pursue their ideas.
Yet when you look at the output it sure doesn’t seem like output stacks linearly with intelligence deployed. It’s not clear that our physics or math research is very much better than it was 50 years ago. You may argue that there is less low hanging fruit but that is precisely a point in favor of what I’m arguing.
Some of what makes great advances possible is courage, primarily the courage to take a different perspective and also creativity. Creativity is difficult to measure and therefore to hill climb. But both courage and intelligence seem very weakly correlated with intelligence.
Creativity is necessary to think of something new to pursue. Courage because most new things fail and may have institutional friction and usually resistance in implementing them. Courage also allows you to pursue something that *seems* wrong for a long time.
Ancient Athens had competing characters with different perspectives and resulted in one of the most colorful characters (Socrates) being sentenced to death. It seems great leaps and advancements require a creative destruction and fierce level of disagreement and independence that we may find intolerable in digital intelligence as we find it barely tolerable in human societies.