Storage Networking Solutions for High Performance Databases by QLogic
0 likes511 views
The document outlines the challenges faced by data warehouse systems due to the exponential growth of data transactions, emphasizing the need for higher bandwidth, improved availability, and dynamic I/O provisioning. It describes QLogic's solutions, which focus on optimizing I/O throughput, enhancing system CPU efficiency, and ensuring guaranteed quality of service (QoS) to manage data warehouse operations effectively. Key aspects include high-performance interconnect characteristics, resource allocation, and the benefits of 10GbE technology in meeting the demands of high I/O-intensive database deployments.
Storage Networking Solutions for High Performance Databases by QLogic
- 1. Technology Brief Industry Challenges The exponential growth of data transactions and analysis is generated by a multitude of data sources. Business intelligence analytics creates I/O-intensive requests on the data warehouse ecosystem. It has contributed to ever-increasing demands in the enterprise for the following: •• Higher bandwidth •• Improved availability •• Dynamic I/O provisioning In addition to resolving the challenges presented by data growth, data warehouse systems require guaranteed Quality of Service (QoS) for consistent, end-to-end performance to meet Service Level Agreements (SLAs), even under varying workload conditions. Storage Networking Solutions for High Performance Databases Optimized Performance and I/O Efficiency for Data Warehouse Deployments
- 2. TB12001 SN0330974-00 Rev. B 02/13 2 Technology Brief Figure 1. Data Flow Data Warehouse Deployments Primarily used in analyzing queries for business intelligence, data warehouse activities include the following: extract, transform, load (ETL), backup and recovery, analytical processing, management services, and traffic bursts. Large volumes of data sets are extracted and transported for further data transformation and analysis. Data sources stationed in singular or distributed locations across multiple nodes must be moved into the data warehouse for subsequent analytical processing. ETL type of processing with massive volumes of data in execution workloads rely heavily on the CPU and I/O system resources. Typical data flow through the data warehouse environment is shown in Figure 1. The I/O performance for the data warehouse is expected to be balanced for providing end-to-end, consistent bandwidth. The elements of I/O connectivity include Fibre Channel Host Bus Adapters and 10GbE Converged Network Adapters that connect servers, switches, and storage. The scalability of data warehouse workloads and I/O throughput performance is typically linear and therefore scalability is essential for managing end-to-end system resource allocation. Resource starvation is observed when bottlenecks are created by unmet needs of increasing workloads for dynamic throughput provisioning or additional I/Os to meet network requests. Because the shared resources are limited, it is important to maximize the efficiency of the system, improving CPU and I/O utilization and thereby eliminating the bottlenecks. Table 1 is a checklist of the characteristics of a high-performance, scalable I/O interconnect required to complement a high-performance database transaction infrastructure. Table 1: Characteristics of a High-Performance I/O Interconnect Essential for Complementing High-Performance Databases Technology Benefits Faster Business Analytics High I/O bandwidth for performance scale out •• More data warehouse transactions •• Faster throughput and response •• Faster backup and restore Improve Service Up Time High availability •• Prevent single point of failure •• Improve I/O reliability for transactions Reduce Infrastructure Footprint (Lower CAPEX) Improve I/O bandwidth efficiency •• Reduce underutilization of I/O resources •• Avoid overutilization of I/O resources Simplify System Management, Tools, and Methodology (Lower OPEX) Dynamic provisioning balances I/O workloads •• Consistent end-to-end performance •• Allocate I/O resources where required •• Switch-agnostic •• Flexible I/O allocation •• Operating system (OS) agnostic •• Flexible SAN and LAN functions •• Single-pane-of-glass management Higher Application Scale Out CPU efficiency saves system CPU cycles •• Fewer CPU cycles used for I/O •• More room for application scalability •• Run more data warehouse operations Meet SLA Compliance QoS provides guaranteed I/O provisioning •• Minimum and maximum metering •• Fine grain QoS Meet Unexpected Demands Oversubscription leverages unused bandwidth reserve •• Manage unexpected I/O usage spikes •• Use same hardware for additional I/O Executive Dashboard Operational & Analytical Reports Interactive Query & Reporting Export to Excel, PDF etc. for Offline Viewing SOURCE DATA LAYER DATA TRANSFORMATION LAYER DATA WAREHOUSE LAYER REPORTING LAYER Extract, Transform, Load (ETL) OPERATIONS LAYER METADATA LAYER
- 3. TB12001 SN0330974-00 Rev. B 02/13 3 Technology Brief QLogic Solution Data center consolidation driven by server virtualization and storage network convergence is accelerating the transition to 10GbE technology to complement the needs of high throughput and I/O-intensive database deployments. ConvergeFlex® and VMflex™ solutions from QLogic® support QoS to extend the benefits of virtualization by reducing TCO on cables, ports, PCIe® slots and improving CPU and I/O utilization efficiency for end-to-end system performance. High-performance data warehouse systems require I/O throughput optimization (higher bandwidth), system CPU efficiency (improved availability), and system I/O utilization efficiency (dynamic provisioning). All of these factors are critical for meeting industry challenges and achieving business goals. The QLogic solution, designed for high I/O throughput performance and fine grain QoS with dynamic bandwidth provisioning, is well suited for data warehouse deployments. Primarily used in analyzing queries for business intelligence, data warehouse activities—extract, transform, load (ETL), backup and recovery, analytical processing, management services, and traffic bursts—use large volumes of data sets. The data sets are extracted and transported using high performance and network bandwidth. Data sources that are stationed in singular or distributed locations across multiple nodes are moved into the data warehouse for subsequent data transformation and analytical processing. Executing ETL processing with massive volumes of data in parallel execution creates significant demand on the CPU and I/O system resources. Data warehouse transactions typically are characterized by the following model: •• Sequential I/O access •• Large I/O transactions •• I/O size range: 64k to 1MB •• DB block size: 8KB (or larger) •• Parallel execution for scaling (if not limited by CPU or I/O) Efficient I/O utilization in the hardware is achieved by allocating the precise amount of bandwidth resources to the applications running on the virtualized or native host. The technology significantly improves resource availability, reduces the possibility of I/O underutilization, and prevents wasted bandwidth.An administrator can change the I/O function’s personality between storage or networking on the same hardware. The 10GbE converged solutions provide the best storage and networking capabilities to match the sustained throughput requirements. Large block and I/O transactions take up a bigger slice of the bandwidth. The I/O off- load technology (SAN or LAN) frees up the system CPU cycles, especially during parallel execution of the workloads when the demand on the system CPU is further stressed to process the data warehouse transactions. Fine grain QoS allows minimum guaranteed bandwidth allocation and maximum metering. The resource allocation can be dynamically managed to align and respond to varying degrees of bandwidth workload conditions. The I/O bottlenecks are removed by providing sufficient bandwidth when and where it is required. Oversubscription can be used for meeting unexpected I/O workload spikes beyond the minimum guaranteed bandwidth. Numerous operations in the data warehouse ecosystem are executed in parallel, such as accessing and processing large volumes of data for analytics, updates, inserts, and deletes. The performance of I/O-intensive operations relies on the overall efficiency of the system’s performance and the availability of guaranteed scalable bandwidth. QLogic’s solution helps mitigate the overutilization and underutilization of resources that can otherwise lead to general system performance degradation. QLogic provides a simple, dynamically configurable, high availability solution with scalable bandwidth to meet the I/O performance and efficiency demands of data warehouse deployments. Figure 2. The QLogic Solution in the Database Warehouse Related Video: QLogic Adapter of Choice for Database Performance Click on the Image to Watch a YouTube Video Hypervisor Mgmt. DR Log Files Database Transactions Database Clients Mobile Access QLogic 8200 Series Converged Network Adapters ENTERPRISE DATABASE WAREHOUSE Data Access Server Executive Dashboard Operational & Analytical Reports Interactive Query & Reporting Export to Excel, PDF etc. for Offline Viewing Business Intelligence Remote Office Cloud Storage LAN REMOTE SAN (off-site) NIC NIC NIC FCoE Data Warehousing Data Mart Data Mining KEY ATTRIBUTES OF QLOGIC SOLUTION • 10Gb Solutions for High Bandwidth • Flexible Personality for Improved Availability in LAN and SANs • QoS for Dynamic Provisioning Guaranteed Provisioning SAN QLogic 5800 Series Switch with FCF FC Storage QLogic 2500 Series Fibre Channel Adapters 10GbE Switch NAS Storage QLogic 3200 Intelligent Ethernet Adapters QLogic ISR6200
- 4. TB12001 SN0330974-00 Rev. B 02/13 4 Technology Brief www.qlogic.com Follow us: Share: Corporate Headquarters QLogic Corporation 26650 Aliso Viejo Parkway Aliso Viejo, CA 92656 949-389-6000 International Offices UK | Ireland | Germany | France | India | Japan | China | Hong Kong | Singapore | Taiwan © 2013 QLogic Corporation. Specifications are subject to change without notice. All rights reserved worldwide. QLogic, the QLogic logo, ConvergeFlex, and VMflex are trademarks or registered trademarks of QLogic Corporation. PCIe is a registered trademark of PCI-SIG Corporation. All other brand and product names are trademarks or registered trademarks of their respective owners. Information supplied by QLogic Corporation is believed to be accurate and reliable. QLogic Corporation assumes no responsibility for any errors in this brochure. QLogic Corporation reserves the right, without notice, to make changes in product design or specifications.