The Alfresco ECM 1 Billion Document Benchmark on AWS and Aurora - Benchmark details and scalability recommendations
Download as PPTX, PDF5 likes11,767 views
Infrastructure, use cases and performance considerations for an Enterprise Grade ECM implementation up to 1B documents on AWS (Amazon Web Services EC2 and Aurora) based on the Alfresco (http://www.alfresco.com) Platform, leading Open Source Enterprise Content Management system.
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to The Alfresco ECM 1 Billion Document Benchmark on AWS and Aurora - Benchmark details and scalability recommendations (20)
Ad
More from Symphony Software Foundation (8)
Recently uploaded (20)
The Alfresco ECM 1 Billion Document Benchmark on AWS and Aurora - Benchmark details and scalability recommendations
- 1. Alfresco 1 Billion document benchmark Infrastructure, use cases and performance considerations for an Enterprise Grade ECM implementation
- 2. Gabriele Columbro Sr. Product Manager, Core Platform / API
- 3. ECMUsecases 5.1 Disclaimer The following information is based on an development version of the unreleased Alfresco 5.1. Performance data is provisional and subtle to change based on testing the final Alfresco 5.1 release.
- 4. Alfresco reaches the 1B document mark on AWS • 10 Alfresco 5.1 nodes, 20 Solr 4 nodes in Sharding mode, 1 Aurora DB • Loaded 1B documents at 1000 docs / sec – 86M per day • Indexed 1B documents in 5 days – > 2000 docs / sec • No degradation in ingestion or content access upon content growth • Tested up to 500 Share concurrent users and 200 CMIS concurrent sessions “We applaud Alfresco’s ability to leverage Amazon Aurora to address business requirements of the modern digital enterprise, and enable a more agile and cost-effective content deployments.” Anurag Gupta, Vice President, Database Services, Amazon Web Services, Inc. – 2015 October 6th 4 Highlights Press release
- 5. 5 ECMUsecases Systems of record at scale Enterprise Document Library Loans & Policies Claims & Case Processing Transaction & Logistics Records Research & Analysis Real-time Video Internet of Things Medical & Personnel Records Government Records & Archives Discovery & Litigation
- 6. ECMUsecases Systems of engagement use cases at scale Document Library Image Management File Sync & Share Search & Retrieval Business Process Management Records Management Case Management Media Management Information Archiving
- 7. Accelerate user adoption Freedom to innovate SIMPLE SMART OPEN Drive digital transformation Connect people, content, and processes to accelerate digital transformation ECM BPM
- 8. Content in context Consumer-like search & usability Secure & mobile collaboration Modular & scalable architecture Effortless Information Governance SIMPLE SMART OPEN Cloud integration & sync Highly extensible & open source ECM BPM Powerful metadata, rules & relationships Easy process (app) creation & analysis
- 9. 9 Divideetimpera Decomposing the problem of Alfresco Scalability Alfresco Index Server Alfresco Repository Search ServicesContent Services Database Storage Network Customizations / Applications Share or Bespoke
- 10. Sizing Area Collaboration Headless Content Platform Search Search is usually just a small portion of the operations percentage (around 10%) In most of the cases especially for very large repositories there wont be full text indexing/search. Permissions Permission control happens at Alfresco layer. User authority structure will be complex. With users belonging to many groups in average. Most of the times permission control is happening elsewhere. Authority structures will be in general fairly simple. Ingestion Ingestion rates are usually not important uploads are normally manually driven. Injection rates are usually very important. Dedicated layers/nodes may be needed. Repository Size Repository Sizes are usually of small (hundreds of thousands) or intermediate (millions) size. Repository sizes are usually quite big (tens of millions to billions). Customizatio n Level of customization will vary but in most cases will concentrate at the front end (Share). Customizations are usually important, typically on the repository side. Custom solution code may live external to Alfresco by using CMIS, public APIs, etc. Architecture Architecture options will be in general the standard ones provided by Alfresco (cluster, dedicated index/transformation layers, etc). Architecture options may vary considerably with more high scale and availability solutions being used: proxies, cluster and un-clustered layers, multi-repositories options of Alfresco repository, etc. Concurrency Concurrent users will possibly be many, with average and peak values important to be considered. Concurrent users will be in general few but think times will be much smaller than for collaboration Interfaces You may expect mostly the Share interface to be used, but also it will be very common SPP, CIFS, IMAP, WebDAV and other public interfaces (CMIS) for other interfaces (mobile). Most of the load should concentrate around public API (CMIS) and custom developed REST API (Webscripts). Batch Batch operations should mostly be around human interaction workflows and the standard Alfresco jobs. Batch operations will usually have a considerable importance, including content injection processes (bulk import), custom workflows and scheduled jobs. 10 ECMScenarios ECM is no one size fits all.
- 11. 11 BechmarkResults Introducing the 1B documents benchmark • Repository Layout – 10k sites; 2 levels deep; 10 folders per level; 1000 files per folder – 100 kb avg plain text files with varying content complexity (for indexing purpose) – Default content model • Scenarios – Share interaction (Enterprise Collaboration) • First focused on the Repository, no Search • Then with Search, including Solr4 Sharding – CMIS interaction (Headless Content Platform) • Transactional Metadata Query testing • AWS Fully cloud environment (provisioned by chef-alfresco) – Alfresco 5.1 + Share 5.1 (development code, unreleased) – AWS EC2 / Aurora (Mysql compatible and Alfresco supported) – Ephemeral for Index storage / EBS for content storage (spoofed)
- 12. 12 Cloudstack 1.2B documents execution environment UI Test x 20 m3.2xlarge Simulate 500 Users • Selenium / Firefox • 1 hour constant load • 10 sec think time UI Test UI Test Alfresco Alfresco Alfresco x 10 c3.2xlarge Alfresco Repo and Share Solr x 20 m3.2xlargeSolr Solr Aurora x 1 db.r3.xlarge ELB Sharded Solr 4 sites folders files transactions dbSize GB 10,804 1,168,206 1,168,206,000 15,475,064 3,185 EBS Ingestion (in place) EBS
- 13. 13 Cloudscaletesting How did we test it? • Repository Loaded using bm- dataload (with file spoofing option) • 1B document benchmark AKA BM-0004 - Testing Repository Limits base on bm-share • Scalability & Sizing testing on Enterprise Collaboration Scenario (bm-share) and Headless Content Platform (bm-cmis) https://wiki.alfresco.com/wiki/Benchmark_Testing_with_Alfresco https://github.com/derekhulley/alfresco-benchmark Benchmark Server Tomcat 7 Rest API MongoDB Config Data Services MongoDB Test Data UI Benchmark Driver (xN) Benchmark Driver (xN) Benchmark Driver Tomcat 7 Extras (Selenium) Servers / APIs Servers / APIs Load Balancer Servers / APIs Test Services Rest API
- 14. 14 BenchmarkResults Getting to 1B documents • Ingestion – With 10 nodes, 1000 documents / second (3 million per hour, 86M per day, 12 days for the full repo) – spoofed content comparable to in place BFSIT loading – Load rate consistent even beyond 1B documents – Throughput grew linearly by adding ingestion nodes (100 docs / sec per node) – Adding additional loading nodes likely to raise ingestion throughput – as Aurora was only at 50% CPU • Indexing – Index distributed over 20 Alfresco Index Servers, sharding on ACLs (good for site based repository), with Alfresco dedicated tracking instance – Each shard holds approx (in excess of) 50M nodes – Re-Indexing completed in about 5 days (each node tracks a sub-set of the 1B) – Dynamic sharding autoconfiguration (5.1 feature) NOTE: requires Alfresco tracking nodes to be in the cluster
- 15. 1515 BechmarkResults Testing Alfresco on 1b docs • Repository Only (500 Share users) test – Sub-second login times and good, linear responses for other actions • Open Library: 4.5s / Page Results: 1s / Navigate to Site: 2.3 – CPU loads: • Database: 8-10% / Alfresco (each of 10 nodes): 25-30% • Shows room for growth up to 1000 concurrent users • Repository + Search (100 Share users) – Metadata and full text search ~ 5s (on 1B documents) – 1.2 searches / sec hitting the 20 shards • TMDQ queries (database only, no index) via CMIS – IN_FOLDER (sorted, limited) ~ 160ms at CMIS interface – CMIS:NAME (=, LIKE) ~ 20ms at CMIS interface
- 16. 16 1Bdocstests Repository – Performances at 1B docs 500 concurrent Share users – no search NOTE: Minor repo changes between 5.0.1 and 5.1 – performance are comparable 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Arithmetic Mean (ms) Standard Deviation (ms) Avg response time (ms) Std deviation (ms)
- 17. 17 Recommendations Lessons Learned • A single Alfresco repository can grow to 1B documents on AWS without notable issues, especially with a scalable DB like AWS Aurora • As for the index, Shard, Shard, Shard – Shard to cope with content growth • Single Solr instance tuned for 50M docs / 32GB – Shard for performance / SLA • Improve performance of search on large scale repositories to hit SLA requirements – Shard for operational reasons • Improve reindexing time (1B docs re-index in 5 days with 20 shards) – NOTE: Sharding has a cost of results post-ranking. Use reasonably. • No indications of any size-related bottlenecks with 1.1 Billion Documents • DB Indexes optimized (no index scans) even at a 3.2TB Aurora DB • Low
- 18. 18 FolderSizematters Limiting the number of files in a folder is a good best practice Avg response time (ms) – 1000 docs/folder Avg response time (ms) – 5000 docs/folder
- 19. 19 Outofscope 1B document Benchmark – Requires further testing • The following items were out of scope for the benchmark and will be tested in the future. Keep this into account when using this info for sizing. • Content Store I/O – File were spoofed, so not on the filesystem (bm-dataload allows to store them) – What does it mean from a scalability standpoint? • For ingestion, comparable to an in-place ingestion of content with BFSIT • For indexing, no difference, Alfresco provides Solr with on-the-fly generated content • For performance testing, difference in download, negligible with large files • Transformation server / subsystem • All files are plain text files • Can be added to testing at later stage, as it’s a separate dimension • Trying to keep the problem ‘testable’
- 20. 20 Conclusions Conclusions • Alfresco can power Enterprise Grade deployments of several ECM use cases in a fully AWS best-of-breed cloud environment • Alfresco Repository can ingest and serve 1B documents without bottlenecks or notable performance issues • The Alfresco Index Server, as of 5.1, will leverage sharding to support large distributed, high performance indices • Using Alfresco in conjunction with AWS Aurora is a powerful combination to reach high scalability without operational complexity • Alfresco is investing in provisioning technologies like chef-alfresco to ensure a seamless experience for DevOps deploying Enterprise Grade architectures in the cloud • This data is based on Alfresco: further testing is undergoing to provide additional data and provide Alfresco 5.1 final sizing & scalability guidelines
- 21. 21 5.1 Key Alfresco 5.1 scalability items to look forward to • Alfresco Solr Sharding – On ACL – Tested up to 80M documents per shard and 20 shards • Improved Transactional metadata queries – Boolean, Double and OR construct • Easy deployment and scaling in AWS using provisioning technologies like chef-alfresco • Alfresco support for Amazon Aurora (also available in Alfresco 5.0) • Updated field collaterals – Scalability Blueprint for Alfresco 5.1 – Sizing Guide for Alfresco 5.1 – AWS Reference architecture, implementation guide and CloudFormation template for Alfresco 5.0 and 5.1
- 22. 22 Wrapup Questions? • Please send feedback to: – [email protected] – Twitter: @mindthegabz • Participate to the Alfresco Research process: Help us help you. Our products are better with your input and thoughts. Sign up for research at: http://bit.ly/alfresco-research-signup There are many ways to help: – Research Surveys – Remote or in person interviews – Investigative workflow conversations or online design exercises
Editor's Notes
- #4: More traditional transactional use cases
- #6: More traditional transactional use cases
- #7: More traditional transactional use cases
- #9: This slides extends on each of those elements
- #10: Start bottom up, tuning lower layers. Then understand the use case and based on the use case and customizations there might be different scaling techniques.
- #11: This slide is hidden, but can be used as a backup / help to qualify customer answers and establish the best use case to map onto from a performance standpoint.
- #19: With 5000 docs / folder performances drastically degraded, when compared to 1000 docs/folder.
- #20: These are out of scope dimensions, not tested in the 1B benchmark. Keep these in mind when presenting results, it’s hidden by default but you might decide to show it to more technical audiences