Jim Fan
@DrJimFan · Nov 23, 2023
It’s pretty obvious that synthetic data will provide the next trillion high-quality training tokens. I bet most serious LLM groups know this. The key question is how to SUSTAIN the quality and avoid plateauing too soon.
The Bitter Lesson by @RichardSSutton continues to guide AI
Elon Musk
@elonmusk
Yeah. It’s a little sad that you can fit the text of every book ever written by humans on one hard drive (sigh).
Synthetic data will exceed that by a zillion.
02:27 AM · November 23, 2023 · 127.1K views
97
88
1.4K