Best Quote by Jeff Dean
“Some things are easier to parrellelize than others. It's pretty easy to train up 100 models and pick the best one. If you want to train one big model but do it on hundreds of machines, that's a lot harder to parallelize.”
About This Quote
Source Talk: AI scaling strategies, Jeff Dean, 2020
Parallelizing many small models is easier than scaling a single large model across many machines, highlighting challenges in distributed training and resource coordination.
In simple terms: Training many small models is easier than scaling one huge model across machines.
Key Takeaway
Choose the right scale for your resources.
Themes
scalability
efficiency
distributed computing
Mood
analytical
strategic
Type
technical
insightful
When to use this quote
- machine learning research
- cloud computing
- model selection
Key Concepts
parallelism
resource allocation
Questions to Reflect On
- When is ensemble training preferable?
- What bottlenecks arise in massive model scaling?
A Different Perspective
Large models may still outperform ensembles in some tasks.
![“I kind of pride myself on coming onto things that are well-oiled machines and finding a way to bring what I bring and fit in, and raise it to another level if I can. And this [Code Black ] has been another one of those really fun success stories, much like Parks And Recreation, although obviously…” quote by Rob Lowe](https://sms4smile.com/quotes/img-quotes/ea3/rob-lowe-855_black_i-kind-of-pride-myself-on-coming-onto-th_6557.jpg)
