06/03/2008

Holy Hadoop!!

I'm still trying to decide if I mean holy as in holy grail or holy crap cause they are both pretty applicable.

Yahoo launched Haadop in production today and I have to say the statistics are bloody unbelievable. Whomever questioned Microsoft's reasons for wanting to buy this company may either be a financial analyst or partially crazy.

Statistics:
  • Number of links between pages in the index: roughly 1 trillion links
  • Size of output: over 300 TB, compressed!
  • Number of cores used to run a single Map-Reduce job: over 10,000
  • Raw disk used in the production cluster: over 5 Petabytes


No comments: