Google: What does it use?

[Kevin O'Gorman]

On Thu, 6 May 2004, Michael Hipp wrote:

> Joel Hammer wrote:
> > Can someone tell me what Google uses for its server OS? I
> > think they use linux. If so, the following should be
> > of interest.
> > 
> > Note: This editorial says 100,000 servers. I wonder if
> > they got the decimal place wrong.
> 
> 100,000 commodity servers running Linux is consistent with everything 
> I've read. They also have a support staff/server ratio that is a 
> fraction of the typical number.
> 
> But they're fairly secretive about all this.
> 
> Michael

About three years ago, when I was still in grad school, someone
from Google made a presentation to us, and mentioned that there
were 7500 live servers, and another 7500 devoted to testing software,
doing the web crawling and building the next version of the database.
And they were still in a single location, making plans for geographical
diversification.

Accordingly, and considering the new services and increased coverage
they've claimed since then, I'm inclined to think 100,000 may be a bit
low.

Interestingly, at that point they were mostly using Linux, but had
a number of MS NT boxen in their overall mix.  They commented that the
solution they took to NT stability issues was to simply reboot all NT
boxes every 2 hours automatically, on a rotating schedule.  Smart.

They already had a pretty pragmatic attitude about reliability.  They
bought motherboards with all components built in, because connectors
are a major point of failure.  They were buying celerons a bit 
slower than the bleeding edge because that was about the best 
price/performance point.  Moveover, they were aware that with over
15,000 boxen, simple arithmetic indicated that at any given time
some of the boxes would be down with probability near 1.0.  If the
MTBF is 10,000 hours and you have more than 15,000 machines ...

This had a big effect on how they designed software.  

++ kevin


-- 
Dr. Kevin O'Gorman  (805) 756-2986  mailto:kogorman at calpoly.edu
Home Page: http://www.csc.calpoly.edu/~kogorman