Asia Pacific SharePoint Capacity Planning by Joel Oleson

Capacity Planning and Performance Planning Joel Oleson http://blogs.msdn.com/joelo Sr. Technical Product Manager Microsoft Corporation

Agenda Addressing Capacity Boundaries and Considerations Workload Considerations Collaboration Web Publishing Search Capacity planning Performance High Availability Demo: Perfmon and SharePoint Demo: Caching and Tuning 06/06/09

Capacity Planning The art of evaluating a technology against the needs of an organization, and making an educated decision about the procurement of HW to meet the demands specific to a system being installed

Collaboration Business Intelligence Enterprise Content Management Forms and Business Processes Search Web Content Management

Common Questions How much hardware do we need? Should we implement a server farm? Do we need SQL Server? How much data can we store? How many users can our environment support? How many sites can we run on our servers? How do we validate our design? What tools can we use to measure performance?

Components Software Boundaries Throughput Targets Data Capacity Hardware Phases Plan for Software Boundaries Estimate Performance and Capacity Requirements Plan Hardware and Storage Requirements Test Your Design Performance and capacity planning: The process of mapping your solution design to a farm size and set of hardware that will support your business goals. Framework Planning for Capacity and Performance

Object Categories Software Scalability vs. Hardware Scalability RTM Test Results, Findings, and Recommendations from the Product Group Test Environment Test Results Recommendations Other Considerations

Site Objects Site Collections, Web sites, documents, document libraries, list items, document file size, etc. People Objects User profiles, security principals, etc. Search Objects Search indexes, Indexed documents Logical Architecture Objects Shared Services Providers, Site Collections, Content Databases, Zones, etc. Physical Objects Servers: Index, WFE, Database, Application, etc.

Site Collection Max and Recommended Sizes Database Max and Recommended Sizes Backup and Operations Considerations Server Throughput Considerations User Type Considerations Authentication Considerations

Software scalability Recommendations for acceptable performance based on software behavior and characteristics Hardware scalability Does not change/modify software behavior or characteristics…but can increase overall throughput of a server farm and might be necessary to achieve acceptable performance as the number of objects approach recommended limits

Hardware Specifications: Network: Gigabit Ethernet (one billion bits/sec) Farm Configurations Tested : Computer Role Specifications Stand-alone computer 1 dual core Intel Xeon 2.0 GHz 64-bit processor, 2 GB RAM Web Front-End computer 2 dual core Intel Xeon 2.8 GHz 64-bit processors, 4 GB RAM Database (SQL Server) computer 4 dual core Intel Xeon 2.8 GHz 64-bit processors, 32 GB RAM Client computers Pentium III 1.2 GHz processor, 1 GB RAM DB’s 1 WFE 2 WFE 3 WFE 4 WFE 5 WFE 6 WFE 7 WFE 8 WFE 1 X X X X X X X X

Throughput vs. Number of Site Collections in One Content Database

Throughput differences between flat document library vs. document library with folders

For all recommendations, visit “Plan for software boundaries http://technet2.microsoft.com/Office/en-us/library/6a13cd9f-4b44-40d6-85aa-c70a8e5c34fe1033.mspx Object Recommended Maximum Scope Object Category Site collection 50,000 per web application Web application Logical architecture Content database 100 per Web application Web application Logical architecture Web site 250,000 per site collection Site collection Site object Document 5 million per library (2,000 per nested folder) Library Site object User profile 5 million per farm Farm People object Indexed document 50 million per search index (1 index per index server, 1 index server per SSP) SSP Search object Web server / database server ratio 8 Web servers per database server (hard limit) Farm Physical object

Throughput vs. number of Web servers Test findings showed plateau at 5:1 Perform tests in your environment Other Recommendations Carefully plan your site hierarchy and design Minimize # Web applications and application pools Limit # of Shared Service Providers Plan for database growth Follow data and feature best practices and suggested limits…

Growth patterns Server performance High availability Applications Data growth Search 06/06/09 MOSS

Global Intranet Considerations Centralized or Distributed Global teams or local teams Where to put the sites End user experience Bandwidth/Latency Operational costs Search Offline Network accelerators Security ACLs 06/06/09

Web Publishing Characteristics Fewer content creators (may be external agencies) Large number of viewers (authenticated or not) Small number of central sites for content publishing Approval workflow Staging and deployment Database Characteristics Publishing databases mainly read Design considerations Design for end to end performance Page size matters! (html, .js,.css,….) More memory than CPU intensive (caching) Navigation and search ability 06/06/09

Large/complex environments Content deployments between staging production Not everything is deployable via content deployment Page sizes Compression Supporting files .js files .css files Web sites with member areas Authentication Authorization Caching strategy ACLs TTL 06/06/09

Search Characteristics Multiple sources Security trimming Indexing Characteristics Network/CPU intensive Querying Characteristics CPU/Memory/IO Intensive Database Characteristics Property store in SQL Design considerations 50 M items / catalog 06/06/09

Determine Usage Profile Usage profile == User community’s behavior Distribution of requests across content Operation types and frequency Existing solution in place? Mine IIS logs Leverage usage profiles provided in configurations tested by Product Group as starting point: Configuration tested by Product Group Windows SharePoint Services collaboration environments Portal collaboration environments Search environments Internet environments InfoPath Forms Services environments Determine resource requirements to support Excel Services

Operation Percentage of throughput Get home page 15.00 Get cached document 15.00 Get static document 15.00 Get list page (HTML) 10.00 Get list page (grid) 10.00 Get list form 7.00 404 errors 5.00 Insert list item 2.00 Edit list item 2.00 Delete list item 2.00 Insert document 2.00 Synchronize with Outlook 2.00 Delete document 2.00 RSS (Really Simple Syndication) 1.00 Start workflow 0.75 Workflow task completion 0.75

Estimating Throughput Targets User response time, concurrency Throughput target rule of thumb: 1 RPS == 100 concurrent users. * Note, this is consistent with SPS 2003, WSS 2.0 Throughput targets (in RPS) at various concurrency rates (recommended response time of 1 – 2 seconds) Warning: Plan for Peak Concurrency Total Users 5% 10% 15% 25% 50% 75% 100% 500 0.25 0.5 0.75 1.25 2.5 3.75 5.0 1,000 0.5 1.0 1.5 2.5 5.0 7.5 10.0 5,000 2.5 5.0 7.5 12.5 25.0 37.5 50.0 10,000 5.0 10.0 15.0 25.0 50.0 75.0 100.0 20,000 10.0 20.0 30.0 50.0 100.0 150.0 200.0 50,000 25.0 50.0 75.0 125.0 250.0 375.0 500.0 100,000 50.0 100.0 150.0 250.0 500.0 750.0 1,000

Other configuration factors that can influence throughput targets Indexing (schedule indexing window off-hours) Caching enabled? Output Caching and Cache Profiles Object Caching Disk-based Caching for Binary Large Objects Page customizations Custom Web parts

Latency components Server processing SQL processing, # SQL round trips, AJAX processing, security trimming Client processing Javascript, CSS, AJAX requests, HTML load, Client machine specs Wire transfer Bandwidth, size of download Recommendations Primary cause of latency problems: custom web parts Watch for: SQL round trips, unnecessary data, excessive client side script Re-use existing client code versus adding more Design code for performance – (Use HTML and .Net best practices) Profile your solutions

Plan Hardware and Storage Phase 3 – How SharePoint Scales Designed to grow with organization needs Server resources: x32, x64, CPU, RAM, HDD Recommend 64-bit for back end services (SQL) which can leverage additional addressable memory SQL: HDD configuration critical Server Farm Topology restrictions removed WFE, Query, Index, Excel Calc, Project, SQL Adopted WSS adage: content only limited by HW capability* Sites: In WSS 3.0, Portals sites are "just another WSS site”

Plan Hardware and Storage Phase 3 – 64-bit vs. 32-bit Hardware WSS 3.0 and MOSS 2007 can work on both 64-bit Hardware Recommended 32-bit can directly address only a 4GB Memory Address Space 64-bit supports up to 1,024 GB Memory (Physical and/or Addressable) 32-bit will look better and perform better with 2GB RAM, but we recommend 64-bit for future investments and scale Larger # of Processors 64-bit HW Prioritization SQL Server  Index  Excel  Search  WFE *64-bit hardware can be mixed within a farm * Adobe does not have PDF Ifilter for x64

Plan Hardware and Storage Phase 3 – Single Server Example One Server Configured as: Web Front-End Server Role Application Server Role Database Server Role Appropriate for limited use-scenarios including the following: Installing Office SharePoint Server 2007 for evaluation purposes. Deploying only Microsoft Windows SharePoint Services 3.0. Deploying a subset of the Office SharePoint Server 2007 features. Deploying Office SharePoint Server 2007 for a limited purpose (such as for a single department) or for a limited number of users. Server Type RAM Disk Processors Stand-alone 2 GB NTFS file system–formatted partition with 3 GB of free space plus adequate free space for your Web sites (Typically at least 100GB) Dual processors that are each 3 GHz or faster

Plan Hardware and Storage Phase 3 – Multi-Server Farm Example Optimizes performance of web servers Increases redundancy and reduces points of failure Redundancy at WFE and Database server roles Determine configuration based on your business needs and goals Determine config of other Application roles (Excel Services, Index, Forms, etc) Web Servers Application Server Clustered SQL Server

Plan Hardware and Storage Phase 3 – Storage Considerations Primary Metric: Document Storage Plan for 1.2 – 1.5 x file system size for SQL Server note: metric is closely tied to RAID level used on SQL disks Secondary Metric: Index Size Index Server: ~10% of total size of all indexable content Query Server: 2+ x index size (master merge)

Phases Phase 1: Plan for Software Boundaries Phase 2: Estimate Performance and Capacity Requirements Phase 3: Plan Hardware and Storage Requirements

Tools Visual Studio Team System Test Edition (VSTT) WSSDW.exe (on Codeplex.com/sptdatapop) When? Initial deployments New hardware configurations Impact of scaling across or up Validating custom solutions Debugging performance related issues

Data for testing (Sites, Search terms, lists, documents etc) A test farm with enough servers to mirror a topology you are testing for A VSTT controller machine (2-4 GB RAM); preferably with SQL Server 2005 to store the test results VSTT clients machines; preferably physical; with at least 2GB of RAM to support many concurrent connections Create VSTT Test Project (Web and Load)

Establish a purpose for testing. What are you testing for? Usage Profiles Create a good test plan with mix of different types of test Setting up the VSTT rig with Controller, Clients and Database Create Test data (Sites, Content etc) Can use SharePoint 2007 Test Data Population Tool on CodePlex ( http://www.codeplex.com/sptdatapop ) Uses a xml Configuration file to load or delete data Create VSTT web tests for each scenario identified Customize Web test as needed to read variable data from an external data source Determining what your peak loads are. Create Load test profile, think times, new users and capturing counters ,Page hit frequency, Web test frequency, Test run times, effect of Async jobs Configure other test properties like warm up time, Connection type etc VSTT automatically collects data from all client machines and aggregates them Analyzing test data remotely

Server Role Performance Counter All Servers Memory\Available Megabytes All Servers Memory\% Committed Bytes in Use All Servers Memory\Pages/second All Servers System\Processor Queue Length All Servers Processor\%Processor Time All Servers Network\Network Queue Length Web Front-End Server Process (w3wp)\%Processor Time Web Front-End Server Process (w3wp)\%Private Bytes Web Front-End Server Web Service\Connection Attempts/second Web Front-End Server Requests Per Second SQL Server Process (sqlserver)\% Processor Time SQL Server Process (sqlserver)\Private Bytes

Setup your Rig Create your content (Data Load Tool or Manually) Create your tests (web, load) Configure the test parameters Run. Measure. Rinse. Repeat.

Understanding Performance Using Perfmon with SharePoint

SharePoint sends down a relatively large payload when you visit the site; even though it is improved from SPS 2003 This payload comes from jScript files, IE Behaviors, and images, in addition to the text content Some elements continue to come down Multiple times on a single page Download again during same browser session on the same site Example…

Files requested on first hit of the home page of a just created publishing site /Pages/Default.aspx 107,128 /_layouts/1033/core.js 54,367 /_layouts/1033/ie55up.js 20,508 /_layouts/images/gosearch.gif 19,933 /_layouts/1033/init.js 15,732 /_layouts/1033/styles/core.css 13,596 /PublishingImages/newsarticleimage.jpg 10,710 /WebResource.axd 8,272 /WebResource.axd 5,373 /_layouts/1033/search.js 5,092 /_layouts/images/navshape.jpg 2,924 /_layouts/1033/EditingMenu.js 2,735 /WebResource.axd 2,482 /_layouts/1033/styles/controls.css 1,448 /_layouts/1033/styles/HtmlEditorTableFormats.css 1,317 /_layouts/images/titlegraphic.gif 1,299 /_layouts/images/icongo01.gif 1,171 /_layouts/images/pagebackgrad.gif 1,082 /_layouts/images/stsicon.gif 1,070 /_layouts/images/itgen.gif 1,050 /_layouts/images/itdisc.gif 1,049 /_layouts/images/itlink.gif 1,043 /_layouts/images/ittask.gif 1,039 /_layouts/images/itcontct.gif 1,036 /_layouts/images/helpicon.gif 1,025 /_layouts/images/recycbin.gif 1,004 /_layouts/portal.js 954 /_layouts/1033/styles/HtmlEditorCustomStyles.css 642 /_layouts/images/itevent.gif 619 /_layouts/images/itdl.gif 582 /WebResource.axd 224 /WebResource.axd 218 /_layouts/images/siteTitleBKGD.gif 209 /_layouts/images/tvplus.gif 209 /_layouts/images/tvminus.gif 207 /_layouts/images/quickLaunchHeader.gif 148 /_layouts/images/topnavselected.gif 146 /_layouts/images/siteactionsmenugrad.gif 146 /_layouts/images/topnavunselected.gif 92 /_layouts/images/menudark.gif 68 /_layouts/images/whitearrow.gif 68 /_layouts/images/Menu1.gif 68 /_layouts/images/pageTitleBKGD.gif 63 /_layouts/images/navBullet.gif 58 /_layouts/images/tvblank.gif 56 /_layouts/images/lstbulet.gif 56 /_layouts/images/blank.gif 43 Total 288,361 Bytes (nearly 300K)

HTTP Compression is enabled and turned on by default for Static files like .js, .css etc Aspx pages need to be added Compression improves latency and decreases payload size Compression affects up to 15% throughput

Caching Output caching and Cache Profiles: At an Individual Page Level Disk-based caching: Individual binary large object (BLOB) level and caches images, sound, movies, and code <BlobCache location="C:\blobCache" path="\.(gif|jpg|png|css|js)$" maxSize="10" max-age="86400" enabled="false"/> in the web.config file for a web application Object Caching: Individual Web Part control, field control, and content level

Troubleshooting Poor throughput SQL – use SQL best practices for perf; especially disk perf Asynchronous operations and timer job conflicts Resource contention: # web apps, app pools, DBs, etc. Check ANY custom code for SQL round trips and payload Look for mis-configured network components (NICs, routers, etc.) Search consumes multiple resources Poor end user perceived latency Page payload OOB 1st page download ~ 200 KB; should not be significantly higher Use page compression where possible Caching strategy: Turn on BLOB and Output caching whenever possible Client machine specs Mis-configured network (see above)

HP Performance Tests HP recommends X64 2-socket dual-core servers with at least 4GB memory. More than 4GB may be required depending upon tuning parameters and strategies that leverage memory against CPU utilization – some HP testing was able to exhaust 4GB memory on X64-based WFE servers with various memory consumptive tuning schemes. As memory is a relatively less expensive component than CPUs this trade-off can be beneficial, and the solution designer should consider configuring increased memory where the situation would warrant it, such as when using server-side caches.

Read-only workload WFE Server RPS WFE CPU% SQL CPU% Net KB/sec BL20p G3 SC X86 60 85 10 6700 BL20p G4 DC X86 148 85 32 10000 BL20p G4 DC X64 140 85 32 10000 BL25p G1 DC X86 132 85 28 8900 BL25p G1 DC X64 124 85 24 7900 DL385 G2 DC X64 * 138 85 30 9800 DL385 G2 DC X64 ** 100 75 14 7000 DL385 G2 DC X64 *** 85 65 12 6000 DC= Dual Core

Table 2. Read/write workload WFE Server RPS WFE CPU% SQL CPU% Net KB/sec BL20p G4 DC X86 126 85 34 9000 BL20p G4 DC X64 120 85 34 9000 BL25p G1 DC X86 112 85 31 8200 BL25p G1 DC X64 105 85 30 7700 DL385 G2 DC X64 118 85 33 8700 * Dual Core is nearly double performance of single core and quad core is anticipated 1.8 X performance of dual core.

HP Learnings Office SharePoint Server 2007 sites, assume a per-WFE server throughput maximum of 100-120 requests/second If the Query Search service will be running on the WFE servers, use 85 requests/second. For the 2-server SMB configuration (all Office SharePoint Server services running on a single server), assume a maximum throughput of 75-100 requests/second For a single-server SMB configuration (All Office SharePoint Server services and SQL Server running on one server), assume a maximum throughput of 50-75 requests/second. Assume a single active X64 2-socket dual-core SQL Server can support between 4 and 6 WFE/application 2-socket dual-core servers of the same type/power, depending on expected workload

Best practices for capacity planning Start big enough but not too big Do proper monitoring in order to know what/when to grow Go 64 bit where possible PDF iFilter is currently only available in 32bit Plan End to End user experience Bandwidth Latency Payload 06/06/09

Related Content Planning and Architecture for Office SharePoint Server 2007 Plan for Performance and Capacity: http://technet2.microsoft.com/Office/en-us/library/eb2493e8-e498-462a-ab5d-1b779529dc471033.mspx Design Server Farms and Topologies: http://technet2.microsoft.com/Office/en-us/library/651be1bf-c354-4f6d-965d-d3385cea0a2d1033.mspx Watch the Blogs: http://blogs.msdn.com/sharepoint http://blogs.msdn.com/joelo HP Performance Whitepaper http://h71019.www7.hp.com/ActiveAnswers/cache/497613-0-0-0-121.html Dell Case Study http://blogs.msdn.com/sharepoint/archive/2007/05/04/dell-case-study-showcase-of-consolidation-manageability-and-scale.aspx

Asia Pacific SharePoint Capacity Planning by Joel Oleson

More Related Content

What's hot (20)

Viewers also liked (12)

Similar to Asia Pacific SharePoint Capacity Planning by Joel Oleson (20)

More from Joel Oleson (20)

Recently uploaded (20)

Asia Pacific SharePoint Capacity Planning by Joel Oleson

Editor's Notes