When we recently heard about the history of YouTube’s growth strategy from CEO Chad Hurley’s point of view, he described it as “hanging onto a rocket.” But an engineer’s take is always going to be a bit less rose-colored and a bit more about the terrifying situations you brained your way out of. So we were particularly interested to tune in to a talk at YouTube’s developer conference Thursday by Cuong Do, an early software engineer who’s now manager of the site’s Core Product Engineering group.
Do’s talk was titled “Behind the Scenes: A Look Into YouTube’s Infrastructure,” and he didn’t disappoint, with harrowing tales of outages; gory details about the specific languages, architectures, and tools YouTube uses; and a flow-chart level view on the way the site handles uploads and video delivery while undergoing the massive usage it sees on a daily basis.
“One of the key phrases we had in the early days was ‘These are good problems to have,’” Do said. “And after a while we’re like, ‘I’m going to kill the next person who says that.’”
YouTube promised it would post video from the talk on its site eventually, but I don’t see it there yet, so check out the version from my handheld camera.
I believe this is the same as this video:
http://video.google.com/videoplay?docid=-6304964351441328559
That’s actually a different talk by the same guy, but cool!
Thanks for the video. If you get news on the youtube version(more clarity), please let us know.
Nice video.. very nice to know what is youtube running on ..
Cheers, Nag
I love GigaOm and the scope of the reporting, but you guys suck at video.
Perhaps you could invest in a shotgun mic that picks up distant audio more clearly; a portable monopod that steadies the shot; and sit far enough back so that we can see the entire screen.
Don’t get me wrong, I really appreciate being able to see the video, but you need to improve your technique, otherwise instead of being a ‘value-added resource’, it becomes an annoyance.
Thanks.
PS
Anyone know if Scribd or anyone else has a copy of his PPT?
Wow that is pretty amazing. Cool stuff to know.
http://www.FireMe.To/udi
I enjoyed how he can’t pronounce MySQL lol.
Cuong Do Cuong had already had an excellent presentation about the YouTube architecture at the Seattle Conference on Scalability last year.
A Google video of the presentation can be found here:
http://video.google.com/videoplay?docid=-6304964351441328559
Well worth watching.
For all the right reasons, YouTube launched on with solid, configurable open source components:
Apache / Lighttpd
MySql
Python
…until user demand crushed them, and they had to move to Google distribution tech such as bigtable, and database shards.
An articulate alpha geek with terrible jokes :-)
Very interesting talk.
@anon
That’s how it’s pronounced in the business/tech world.
Really good video. I posted a transcript at http://mukulblog.blogspot.com/2008/07/early-youtube-engineer-talks-about.html
Thanks,
Mukul.
Hey Will, you’re right about sucking at video. I was just psyched to be present for the talk and figured more people would want to see it too so I pulled out the camera in my purse. But I promise we will get better in the future!
Hey Liz,
Thanks for the note back. We all learn as we go, so it’s something that, with more advanced planning, can only get better for events like this that we all appreciate having the GigaOm team be our ‘fly on the wall’ for.
PS,
Thanks to Mukul for going to the trouble of posting a transcript for everyone. That’s what makes GigaOm’s community of users/readers a nice ‘family’ to be a part of.