Information Storage and Management notes ssmeena
- 1. Introduction to Information Storage and Management Why Information Storage “Digital universe – The Information Explosion” o 21st Century is information era o Information is being created at ever increasing rate o Information has become critical for success We live in an on-command, on-demand world:- that means we need information when and where it is required. o Example: Social networking sites, e-mails, video and photo sharing website, online shopping, search engines etc Information management is a big challenge Organization seek to Store, Protect, Optimize and Leverage the information optimally. What is Data : “Collection of raw facts from which conclusions may be drawn” Most data is being converted into a digital format Driven by user demand list of some of the factors that have contributed to the growth of digital data: o Increase in data processing capabilities o New and cheaper peripherals o Lower cost and increased speed of storage o Affordable and faster networks
- 2. Who creates data? Individuals Businesses Categories of Data Data can be categorized as either structured or unstructured data Structured: o Data Bases o Spread Sheets Unstructured o Forms o Images o Audio o Movies Over 80% of enterprise Information is unstructured Define Information What do individuals/businesses do with the data they collect? o They turn it into “information” o “Information is the intelligence and knowledge derived from data” Businesses analyze raw data in order to identify meaningful trends o For example: o Buying habits and patterns of customers o Health history of patients
- 3. Virtuous cycle of information o Information begets information Value of Information to a Business Storage Data created by individuals/businesses must be stored for further processing
- 4. Type of storage used is based on the type of data and the rate at which it is created and used Examples: Individuals: Digital camera, Cell phone, DVD’s, Hard disk Businesses: Hard disk, external disk arrays, tape library Storage model: An evolution Centralized: mainframe computers Decentralized: Client – server model (Data spread across many servers) Centralized: Storage Networking (Hugh repositories) Storage Technology and Architecture Evolution Historically, organizations had centralized computers (mainframe) and information storage devices (tape reels and disk packs) in their data center. The proliferation (very fast growth) of departmental servers in an enterprise resulted in unprotected, unmanaged, fragmented islands of information and increased operating cost. There were very limited policies and processes for managing these servers and the data created. To overcome these challenges, storage technology evolved from non-intelligent internal storage to intelligent networked storage (see Figure 1-4).
- 5. Data Center Infrastructure Organizations maintain data centers to provide centralized data processing capabilities across the enterprise. Data centers store and manage large amounts of mission-critical data. The data center infrastructure includes computers, storage systems, network devices, dedicated power backups, and environmental controls (such as air conditioning and fire suppression). Core Elements Five core elements are essential for the basic functionality of a data center: • Applications An application is a computer program that provides the logic for computing operations. Applications, such as an order processing system, can be layered on a database, which in turn uses operating system services to perform read/write operations to storage devices. • Databases – Database Management System (DBMS) and the physical and logical storage of data • Servers/Operating systems A computing platform that runs applications and databases. • Networks (LAN and SAN) A data path that facilitates communication between clients and servers or between servers and storage. • Storage arrays A device that stores data persistently for subsequent use.
- 6. Figure 1-5 shows an example of an order processing system A customer places an order through the AUI of the order processing application software located on the client computer. The client connects to the server over the LAN and accesses the DBMS located on the server to update the relevant information such as the customer name, address, payment method, products ordered, and quantity ordered. The DBMS uses the server operating system to read and write this data to the database located on physical disks in the storage array. The Storage Network provides the communication link between the server and the storage array and transports the read or write commands between them. The storage array, after receiving the read or write commands from the server, performs the necessary operations to store the data on physical disks. 1 2 3 4 5 Optimal Order Processing Example The application should be optimized for fast interaction with the DBMS The tables in the database should be constructed with care so the number of read/write operations can be minimized The server should have sufficient CPU and memory resources to satisfy application and DBMS needs The different networks should provide fast communication between client and server, as well as server and storage array The storage array should service the read/write requests from the server for optimal performance When the DBMS receives a request from the application: It first searches the server memory; if data is found there, the operation takes about a millisecond If not, it uses the operating system to request the data from the storage array Dedicated high speed networks transport this request to the storage array Intelligent storage arrays can deliver the requested data within a few milliseconds, and are typically configured to protect data in the event of drive failures Key Requirements for Data Center Elements Uninterrupted operation of data centers is critical to the survival and success of a business. It is necessary to have a reliable infrastructure that ensures data is accessible at all times.
- 7. • Availability: All data center elements should be designed to ensure accessibility. The inability of users to access data can have a significant negative impact on a business. • Security: Polices, procedures, and proper integration of the data center core elements that will prevent unauthorized access to information must be established. In addition to the security measures for client access, specific mechanisms must enable servers to access only their allocated resources on storage arrays. • Scalability: Data center operations should be able to allocate additional processing capabilities or storage on demand, without interrupting business operations. Business growth often requires deploying more servers, new applications, and additional databases. The storage solution should be able to grow with the business. • Performance: All the core elements of the data center should be able to provide optimal performance and service all processing requests at high speed. • Data integrity: Data integrity refers to mechanisms such as error correction codes or parity bits which ensure that data is written to disk exactly as it was received. • Capacity: Data center operations require adequate resources to store and process large amounts of data efficiently. When capacity requirements increase, the data center must be able to provide additional capacity without interrupting availability, or, at the very least, with minimal disruption. • Manageability: A data center should perform all operations and activities in the most efficient manner. Manageability can be achieved through automation and the reduction of human (manual) intervention in common tasks. Managing Storage Infrastructure Managing a modern, complex data center involves many tasks. Key management activities include: • Monitoring is the continuous collection of information and the review of the entire data center infrastructure. The aspects of a data center that are monitored include security, performance, accessibility, and capacity. • Reporting is done periodically on resource performance, capacity, and utilization. Reporting tasks help to establish business justifications and chargeback of costs associated with data center operations. • Provisioning is the process of providing the hardware, software, and other resources needed to run a data center. o Provisioning activities include capacity and resource planning. Capacity planning ensures that the user’s and the application’s future needs will be addressed in the most cost-effective and controlled manner.
- 8. Resource planning is the process of evaluating and identifying required resources, such as personnel, the facility (site), and the technology. Resource planning ensures that adequate resources are available to meet user and application requirements. Challenges in Managing Information Some Constraints to Meeting the Requirements Constraints include: • Cost • Physical environment • Maintenance and support • Compliance – regulatory and legal • Hardware and software infrastructure • Interoperability and compatibility Information Lifecycle The information lifecycle is the “change in the value of information” over time. The value of the information is highest when a company receives a new sales order and processes it to deliver the product. After order fulfillment, the customer or order data need not be available for real-time access. The company can transfer this data to less expensive secondary storage with lower accessibility and availability requirements unless or until a warranty claim or another event triggers its need. After the warranty becomes void, the company can archive or dispose of data to create space for other high-value information.
- 9. Information Lifecycle Management ILM Implementation
- 10. ILM Benefits