Google cluster architecture
Download as PPTX, PDF11 likes11,441 views
Google's search infrastructure uses a cluster architecture with thousands of machines to handle high query volumes. It employs techniques like PageRank to measure page importance, replicates data for reliability and availability, uses low-cost commodity hardware, and manages power usage. Key systems include the Google File System for storage and MapReduce for distributed processing of large datasets.
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to Google cluster architecture (20)
Ad
Recently uploaded (20)
Google cluster architecture
- 1. Cluster ArchitectureWeb Search for a Planet and much more….Abhijeet [email protected]
- 2. 2
- 3. Google Query Serving InfrastructureElapsed time: 0.25s, machines involved: 1000s+3
- 4. PageRankPageRankTM is the core technology to measure the importance of a pageGoogle's theory – If page A links to page B # Page B is important # The link text is irrelevant – If many important links point to page A # Links from page A are also important4
- 5. 5
- 6. Key Design PrinciplesSoftware reliabilityUse replication for better request throughput and availability Price/performance beats peak performanceUsing commodity PCs reduces the cost of computation 6
- 7. The Power ProblemHigh density of machines (racks) – High power consumption 400-700 W/ft2 # Typical data center provides 70-150 W/ft2 # Energy costs – Heating # Cooling system costsReducing power – Reduce performance (c/p may not reduce!) – Faster hardware depreciation (cost up!)7
- 8. ParallelismLookup of matching docs in a large index --> many lookups in a set of smaller indexes followed by a merge stepA query stream --> multiple streams (each handled by a cluster)Adding machines to a pool increases serving capacity8
- 9. Hardware Level Consideration Instruction level parallelism does not helpMultiple simple, in-order, short-pipeline coreThread level parallelismMemory system with moderate sized L2 cache is enoughLarge shared-memory machines are not required to boost the performance9
- 10. GFS (Google File System) Design Master manages metadata
- 11. Data transfers happen directly between clients/chunk servers
- 12. Files broken into chunks (typically 64 MB)10
- 13. GFS Usage @ Google• 200+ clusters• Many clusters of 1000s of machines• Pools of 1000s of clients• 4+ PB Filesystems• 40 GB/s read/write load – (in the presence of frequent HW failures)11
- 14. The Machinery12
- 15. Architectural view of the storage hierarchy13
- 16. Clusters through the years“Google” Circa 1997 (google.stanford.edu)Google (circa 1999)14
- 17. Clusters through the yearsGoogle (new data center 2001)Google Data Center (Circa 2000)3 days later15
- 18. Current Design• In-house rack design• PC-class motherboards• Low-end storage and networking hardware• Linux• + in-house software16
- 22. Comparison Between Custom built & High-end Servers 20
- 23. Implications of the Computing EnvironmentStuff Breaks• If you have one server, it may stay up three years (1,000 days)• If you have 10,000 servers, expect to lose ten a day• “Ultra-reliable” hardware doesn’t really help• At large scale, super-fancy reliable hardware still fails, albeit less often – software still needs to be fault-tolerant – commodity machines without fancy hardware give better performance/$• Reliability has to come from the software• Making it easier to write distributed programs21
- 24. Infrastructure for Search SystemsSeveral key pieces of infrastructure: – GFS – MapReduce – BigTable22
- 25. MapReduce• A simple programming model that applies to many large-scale computing problems• Hide messy details in MapReduce runtime library: – automatic parallelization – load balancing – network and disk transfer optimizations – handling of machine failures – robustness – improvements to core library benefit all users of library!23
- 26. Typical problem solved by MapReduce• Read a lot of data• Map: extract something you care about from each record• Shuffle and Sort• Reduce: aggregate, summarize, filter, or transform• Write the results• Outline stays the same, map and reduce change to fit the problem24
- 27. ConclusionsFor a large scale web service system like Google – Design the algorithm which can be easily parallelized – Design the architecture using replication to achieve distributed computing/storage and fault tolerance – Be aware of the power problem which significantly restricts the use of parallelism25
- 28. References1. Luiz André Barroso , Jeffrey Dean , UrsHölzle, Web Search for a Planet: The Google Cluster Architecture, IEEE Micro, v.23 n.2, p.22-28, March 2003 [doi>10.1109/MM.2003.1196112]2. S. Brin and L. Page, “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” Proc. Seventh World Wide Web Conf. (WWW7), International World Wide Web Conference Committee (IW3C2), 1998, pp. 107-117.3. “TPC Benchmark C Full Disclosure Report for IBM eserverxSeries 440 using Microsoft SQL Server 2000 Enterprise Edition and Microsoft Windows .NET Datacenter Server 2003, TPC-C Version 5.0,” http://www.tpc.org/results/FDR/TPCC/ibm.x4408way.c5.fdr.02110801.pdf.4. D. Marr et al., “Hyper-Threading Technology Architecture and Microarchitecture: A Hypertext History,” Intel Technology J., vol. 6, issue 1, Feb. 2002.5. L. Hammond, B. Nayfeh, and K. Olukotun, “A Single-Chip Multiprocessor,” Computer, vol. 30, no. 9, Sept. 1997, pp. 79-85. 6. L.A. Barroso et al., “Piranha: A Scalable Architecture Based on Single-Chip Multiprocessing,” Proc. 27th ACM Int’l Symp. Computer Architecture, ACM Press, 2000, pp. 282-293.7. L.A. Barroso, K. Gharachorloo, and E. Bugnion, “Memory System Characterization of Commercial Workloads,” Proc. 25th ACM Int’l Symp. Computer Architecture, ACM Press, 1998, pp. 3-14. 26