WSPDC June 2013: Working with Large Lists in SharePoint 2010 by Zewdi Solomon
2 likes4,863 views
The document discusses techniques for working with large lists in SharePoint 2010 that contain more than 5,000 items, as operations on large lists can cause performance issues if not properly managed. It provides tips for optimizing large lists such as indexing columns, using filtered views, organizing items into folders, and leveraging search to avoid hitting the list view threshold. The key recommendations are to plan carefully for lists with millions of items, reduce unique permissions, and use daily time windows for resource-intensive operations.
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to WSPDC June 2013: Working with Large Lists in SharePoint 2010 by Zewdi Solomon (20)
Ad
Recently uploaded (20)
WSPDC June 2013: Working with Large Lists in SharePoint 2010 by Zewdi Solomon
- 1. Working with large lists in SharePoint 2010
- 2. About Me • Software Developer at Avaya Government Solutions • Executive office of immigration review (EOIR) an agency within DOJ • Have been developing for 10+ years • SharePoint Development since 2007 • Married to my college sweetheart and mother of twins boys • Bikram Yoga rocks!!
- 3. Introduction • When is a list/library considered large? • >5000 items • It is not recommended to query or run any kind of operation that would require to scan more than 5000 items at a time • Database Contention • Performance Degradation • Service or page load timeout • SharePoint supports list with millions of items but must be carefully planned.
- 5. Resource throttling • Introduced in SharePoint 2010 • To prevent operations that adversely affect farm performance • Configured in Central Administration >Manage Web Applications >General Setting – Resource Throttling
- 8. List View Threshold • Prevent operations that will involve more than 5000 items (query, indexing, calculated fields etc..) • Microsoft recommends NOT to increase this limit
- 9. List View threshold for auditors and administrators • Used by certain service • Search Query Account • Object cache super-reader and super-writer accounts
- 10. Allow object model override • Privileged developers can programmatically override list view threshold for • A web application globally • SPWebAppication.MaxItemsPerThrottledOperation • An individual list • Splist.EnableThrottling property to False • the Query Throttling • SpQueryThrottleOption
- 11. Daily time window • A database operation or query started within the daily time window continues until completion even if it does not finish with the specified time window
- 12. Unique permissions • Default 50,000 • Recommended to be reduced to 5000 • Performance decreases as the number of unique permissions increases • The throughput difference for operations on a list with 0 – 1000 permissions is around 20 percent • Avoid fine-grained permissions • Every broken permission inheritance adds one to the unique permission count
- 13. Lookup columns and list view • Causes a join with another table • Increases complexity of metadata navigation and list view queries • Workflow status also count
- 14. Operations Classifications • List exceeding the list view threshold • Add/remove/update a list Column • Add/remove/update a list Content type • Create/remove an index • Container exceeding the list view threshold • Delete/copy/rename a folder • Queries that filter on non-indexed columns • Set fine-grained security permissions
- 16. Techniques and Tips • Indexed Columns • Filtered views based on Column indexes • Organizing items into folders • Using the daily time windows • Using Search
- 17. Indexing • Up to 20 columns • Columns Filtered or Sorted by • Consumes extra resources in the database • Adds overhead to every operation to maintain the index • Columns that cannot be indexed • Multiple lines of text • Choice (multi-valued) • Calculated • Hyperlink or Pictures • For a complete list please visit Enforcing Uniqueness in Column Values
- 18. How to Create an index 1. Go to the library settings for the list 2. Under the Columns section, click on indexed columns 3. Create a new index
- 19. How to Create an index 4. In the Primary Column section, under Primary column for this index, select your column 5. Click Create
- 20. Filtered Views • First filter column must be index • First filter column must not return more items than the threshold limit • Check recycle bin for deleted items
- 21. How to Create Filtered View 1. In the Managed Views group, click create view 2. Click on Standard View 3. Enter your view name
- 22. How to Create Filtered View 4. In the Filter section, click on Show items only when the following is true 5. choose your indexed columns and enter value
- 23. CAML Query
- 24. Organizing items into Folders • Internal index gets created • Effectively using internal index to access data • Can contain more items than the list view threshold • Show all items without folders • Default view show all available folders
- 25. Daily time window • Use the Daily time window for operations like: • Adding/deleting an index • Adding Calculated field • Deleting/copying folders
- 26. Using Search • Has its own indexing mechanisms • Not subject to List View Threshold • Create Scope • Keyword or phrase search • Refinement Panel
- 27. SharePoint 2013 • This behavior has not changed in SharePoint 2013. • http://technet.microsoft.com/en- us/library/ff382641.aspx
- 28. References • Query Throttling and Indexing • http://msdn.microsoft.com/en- us/library/ff798465.aspx • Manage lists and Libraries with many items • http://office2010.microsoft.com/en-us/sharepoint- server-help/manage-lists-and-libraries-with-many- items-HA010378155.aspx?redir=0 • Designing Large lists and maximizing list performance (SharePoint Server 2010) • http://technet.microsoft.com/en- us/library/cc262813.aspx
- 29. Contact • Email: [email protected] • Twitter: @zewdiyemane • LinkedIn:http://www.linkedin.com/pub/zewdi -solomon-mcp-mcts/17/324/2bb
Editor's Notes
- #8: These default has been tested and carefully chosen.The computer administrator of the front-end computer and the central administrator site owner accessing a list or library with many items are not subject to these limits. Specially during testing it might look that it is working fine
- #9: You can temporarily change the EnableThrottling on an individual list
- #11: You should only change this as a temporary solution or last resortQuery throttle Option:Default: Enumeration value of Default is 0. If the user is a local administrator on the server no throttling limits will apply to the query. Otherwise throttling limits on the number of items and the number of Lookup, Person/Group, and Workflow Status fields will apply to the query.Override:Enumeration value of Override is 1. If the user is a local administrator on the server no throttling limits will apply to the query. If the web application security policy grants the user Full Read or Full Control permissions the throttling limit for Auditors and Administrators will apply to the number of items involved in the query and no throttling will apply to the number of Lookup, Person/Group, and Workflow Status fields. Otherwise the [Microsoft.SharePoint.SPQueryThrottleOption.SPQueryThrottleOption.Default] setting will apply to the query. Strict:Enumeration value is 2. Throttling for both the number of items and for number of Lookup, Person/Group, and Workflow Status fields will apply to the query regardless of user permissions.