Brett Winton
@wintonARK · Oct 7, 2022
The world’s largest computer does 1.1 exaflops
Tesla’s exapod is designed to achieve 1.3
(Not 🍎 to 🍏 since high-performance compute runs at 64 bit whereas Tesla is tuning to 16 bit; divide the exapod by 4 though and it would be the world’s 3rd largest computer...)
Brett Winton@wintonARK· Oct 7, 2022On a performance basis Tesla is anticipating roughly the same performance advance off of Dojo ~4x by mid-23’ that it would have gotten if had stuck with Nvidia (upgrading from A100 to H100)
Big difference in cost and customization to Tesla’s workloads tho…
Elon Musk
@elonmusk
Dojo uses bf16, so actual performance difference vs 64 bit is much less than it would seem, since you truncate so many useless zeroes & round from there
01:52 AM · October 8, 2022 · 0 views
342
257
4.8K