Why does it matter?
More autonomous AI would make control and oversight problems harder to catch before deployment.
Direct quote
The latter is a real concern. Agent-3 is not smarter than all humans. But in its area of expertise, machine learning, it is smarter than most, and also works much faster. What Agent-3 does in a day takes humans several days to double-check.
AI 2027