Why does it matter?
Clusters that expensive would make frontier AI dependent on state-scale financing, energy, and supply chains.
Direct quote
Before the decade is out, we will likely see individual training clusters costing hundreds of billions of dollars. The trillion-dollar cluster—+4 OOMs from the GPT-4 cluster, or roughly 10^29 FLOP—will be within reach.
Leopold Aschenbrenner