Front cover

IBM Scale Out

Network Attached
Storage Concepts
Introduces IBM Scale Out NAS
concepts

Details hardware and software

architecture

Provides configuration
considerations

Mary Lovelace
Vincent Boucher
Shradha Radhakrishna Nayak
Curtis Neal
Lukasz Razmuk
John Sing
John Tarella

ibm.com/redbooks
International Technical Support Organization

IBM Scale Out Network Attached Storage Concepts

November 2010

SG24-7874-00
 Note: Before using this information and the product it supports, read the information in “Notices” on
page vii.

First Edition (November 2010)

This edition applies to IBM Scale Out Network Attached Storage (SONAS) 1.1.1.

© Copyright International Business Machines Corporation 2010. All rights reserved.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.
Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
The team who wrote this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Stay connected to IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Chapter 1. Introduction to Scale Out Network Attached Storage . . . . . . . . . . . . . . . . . . 1

1.1 Marketplace requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Understanding I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 File I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.2 Block I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.3 Network Attached Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 IBM Scale Out Network Attached Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 SONAS architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3.2 SONAS scale out capability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3.3 SONAS Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.4 High availability design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Chapter 2. SONAS features, function, and value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.1 Lifecycle of a file from creation to deletion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Logical diagram of SONAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3 Creating, writing, and reading files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.1 Creating and writing a file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.2 Scaling out high performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.3 Reading a file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.4 Scaling out high concurrency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4 Managing storage centrally and automatically . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.4.1 Integrated storage pools for tiered storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.4.2 Policy managed storage and policy engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4.3 High performance scan engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.4.4 High performance physical data movement for ILM / HSM. . . . . . . . . . . . . . . . . . 33
2.4.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.5 Hierarchical storage management, backup/restore to external storage . . . . . . . . . . . . 35
2.5.1 Requirements for high performance external HSM and backup restore . . . . . . . . 35
2.5.2 SONAS high performance HSM and backup/restore with Tivoli Storage Manager 36
2.5.3 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Chapter 3. SONAS architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.1 Hardware architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.1.1 Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.1.2 Interface nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.1.3 Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.1.4 External ports: 1 GbE / 10 GbE on the interface nodes . . . . . . . . . . . . . . . . . . . . 49
3.1.5 Storage pods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.1.6 SONAS storage controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

© Copyright IBM Corp. 2010. All rights reserved. iii

 3.1.7 SONAS storage expansion unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.1.8 Connection between hardware components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.1.9 Available SONAS configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.1.10 Various drive types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.1.11 SONAS scale out capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.2 Software architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3 SONAS data access layer: File access protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.3.1 File export protocols: CIFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.3.2 File export protocols: NFS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.3.3 File export protocols: FTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.3.4 File export protocols: HTTPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.4 SONAS Cluster Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.4.1 SONAS Cluster Manager and interface node workload allocation . . . . . . . . . . . . 64
3.4.2 Principles of SONAS workload allocation to interface nodes . . . . . . . . . . . . . . . . 66
3.4.3 Interface node failover and failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.4.4 Storage node failover and failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.4.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.4.6 SONAS Cluster Manager manages multi-platform concurrent file access . . . . . . 70
3.4.7 Distributed metadata manager for concurrent access and locking . . . . . . . . . . . . 72
3.4.8 SONAS Cluster Manager summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.5 SONAS authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.5.1 Considerations for authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.5.2 SONAS authentication: ID mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.5.3 SONAS Active Directory authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.5.4 SONAS directory authentication with SFU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.5.5 SONAS with LDAP authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.6 Data repository layer: SONAS file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.6.1 SONAS file system scalability and maximum sizes . . . . . . . . . . . . . . . . . . . . . . . 77
3.6.2 Introduction to SONAS file system parallel clustered architecture . . . . . . . . . . . . 77
3.6.3 SONAS file system performance and scalability. . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.7 SONAS data management services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.7.1 SONAS: Using the central policy engine and automatic tiered storage . . . . . . . . 84
3.7.2 Using and configuring Tivoli Storage Manager HSM with SONAS basics . . . . . . 87
3.8 SONAS resiliency using snapshots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.8.1 SONAS Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.8.2 Integration with Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.9 SONAS resiliency using asynchronous replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.10 SONAS and Tivoli Storage Manager configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.10.1 Tivoli Storage Manager terminology and operational overview . . . . . . . . . . . . . 95
3.10.2 General Tivoli Storage Manager and SONAS guidelines . . . . . . . . . . . . . . . . . . 96
3.10.3 Tivoli Storage Manager software licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
3.11 SONAS system management services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.11.1 Management GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.11.2 Health Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.11.3 Command Line Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.12 Summary: SONAS hardware and software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.13 SONAS goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

Chapter 4. SONAS usage cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

4.1 Types of network attached storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.1.1 Classical NAS storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.1.2 Scalable NAS storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.1.3 Scale out NAS storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

iv IBM Scale Out Network Attached Storage Concepts

4.2 Usage cases for scale out NAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.2.1 General NAS consolidation and streamlining of operations . . . . . . . . . . . . . . . . 110
4.2.2 Industry usage cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.2.3 Digital media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.2.4 Modern analytics applications for competitive advantage. . . . . . . . . . . . . . . . . . 122
4.2.5 Deep computing and high performance computing . . . . . . . . . . . . . . . . . . . . . . 129
4.2.6 Healthcare. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
4.3 Usage cases for scale out NAS in cloud storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.3.1 The transition to cloud service delivery models . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.3.2 What is cloud computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.3.3 Delivery models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.3.4 SONAS as foundation for cloud storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.3.5 IBM cloud storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.3.6 Sharing the storage cloud: Tenancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Chapter 5. Next steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

5.1 Where to go from here . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
5.1.1 Other SONAS Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
5.1.2 Websites about IBM SONAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
5.1.3 ISV software supported. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
5.1.4 More information about IBM SONAS components . . . . . . . . . . . . . . . . . . . . . . . 145
5.1.5 Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.1.6 IBM Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.1.7 White papers on SONAS ISV software certification . . . . . . . . . . . . . . . . . . . . . . 147
5.1.8 TCOnow! for SONAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.2 Contact your IBM or IBM Business Partner representative. . . . . . . . . . . . . . . . . . . . . 148

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
How to get Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

Contents v
vi IBM Scale Out Network Attached Storage Concepts
Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area.
Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product, program, or service that does
not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not give you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION
PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.

Any references in this information to non-IBM websites are provided for convenience only and do not in any
manner serve as an endorsement of those websites. The materials at those websites are not part of the
materials for this IBM product and use of those websites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without
incurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs in source language, which illustrate programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs.

© Copyright IBM Corp. 2010. All rights reserved. vii

Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines
Corporation in the United States, other countries, or both. These and other IBM trademarked terms are
marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US
registered or common law trademarks owned by IBM at the time this information was published. Such
trademarks may also be registered or common law trademarks in other countries. A current list of IBM
trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml

The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
AIX® HACMP™ Smarter Planet™
DB2® IBM® System p5®
Enterprise Storage Server® pSeries® Tivoli®
FlashCopy® Redbooks® xSeries®
GPFS™ Redbooks (logo) ® z/OS®

The following terms are trademarks of other companies:

Snapshot, and the Network Appliance logo are trademarks or registered trademarks of Network Appliance,
Inc. in the U.S. and other countries.

Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States,
other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

viii IBM Scale Out Network Attached Storage Concepts

Preface

IBM® Scale Out Network Attached Storage (IBM SONAS) is a Scale Out NAS offering
designed to manage vast repositories of information in enterprise environments requiring
very large capacities, high levels of performance, and high availability.

SONAS provides a range of reliable, scalable storage solutions for a variety of storage
requirements. These capabilities are achieved by using network access protocols such as
NFS, CIFS, HTTP, and FTP.

Utilizing built-in RAID technologies, all data is well protected with options to add additional
protection through mirroring, replication, snapshots, and backup. These storage systems are
also characterized by simple management interfaces that make installation, administration,
and troubleshooting uncomplicated and straightforward.

In this IBM Redbooks® publication, we discuss the marketplace requirements that led to the
SONAS stand-alone storage system. We introduce storage architectures and file systems.
We then introduce the reader to SONAS and describe the hardware and software that make
up the SONAS appliance.

The team who wrote this book

This book was produced by a team of specialists from around the world working at the
International Technical Support Organization, San Jose Center.

Mary Lovelace is a Consulting IT Specialist at the International Technical Support

Organization. She has more than 20 years of experience with IBM in large systems, storage,
and storage networking product education, system engineering and consultancy, and
systems support. She has written many Redbooks publications about IBM Tivoli® Storage
Productivity Center, Tivoli Storage Manager, and IBM z/OS® storage products.

Vincent Boucher is an IT Specialist as a member of the EMEA Products and Solutions

Support Center (PSSC) of Montpellier France. His role within the Storage Benchmark team is
to demonstrate the efficiency of IBM solutions and their added value to customers. Vincent
holds an Engineering degree in Mathematics and Computer Science from the ENSEEIHT
Engineering schools in Toulouse. His areas of expertise include Linux®, IBM systems x,
mid-range IBM Storage, and IBM General Parallel File System (IBM GPFS™) from both his
past experience with High Performance Computing and new Storage benchmarks.

Shradha Radhakrishna Nayak is a Staff Software Engineer working with IBM India software
Labs in Pune, India. She holds a Bachelor of Computer Science Engineering degree and has
over six years of experience. She has been working in the storage domain since and has
good expertise in Scale out File Service (SoFS) and Scale Out Network Attached Storage
(SONAS). Prior to this, she worked as a Level-3 developer for Distributed File Service (DFS)
and also worked for IBM AFS. Shradha is focusing on storage products and cloud storage
and is currently part of the Level-3 developers teams for SONAS. Being a part of the SONAS
developing and testing team, she has developed a thorough knowledge of SONAS with its
components and functions. In this book, she has mainly focused on installation, configuration,
and administration of SONAS. Shradha is also interested in social media and social
networking tools and methodologies.

© Copyright IBM Corp. 2010. All rights reserved. ix

 Curtis Neal is an Executive IT Specialist working for the System Storage Group in San Jose,
California. He has over 25 years of experience in various technical capacities, including
mainframe and open system test, design, and implementation. For the past eight years, he
has led the Open Storage Competency Center, which helps customers and IBM Business
Partners with the planning, demonstration, and integration of IBM System Storage Solutions.

Lukasz Razmuk is an IT Architect at IBM Global Technology Services in Warsaw, Poland.

He has six years of IBM experience in designing, implementing, and supporting solutions in
IBM AIX®, Linux, IBM pSeries®, virtualization, high availability, General Parallel File System
(GPFS), SAN Storage Area Network, Storage for Open Systems, and IBM Tivoli Storage
Manager. Moreover, he acts as a Technical Account Advocate for Polish clients. He holds a
Master of Science degree in Information Technology from the Polish-Japanese Institute of
Information Technology in Warsaw as well as many technical certifications, including IBM
Certified Advanced Technical Expert on IBM System p5®, IBM Certified Technical Expert
pSeries, IBM HACMP™, Virtualization Technical Support, and Enterprise Technical Support
AIX 5.3.

John Sing is an Executive IT Consultant with IBM Systems and Technology Group. John has
specialties in large Scale Out NAS, in IT Strategy and Planning, and in IT High Availability and
Business Continuity. Since 2001, John has been an integral member of the IBM Systems and
Storage worldwide planning and support organizations. He started in the Storage area in
1994 while on assignment to IBM Hong Kong (S.A.R. of China), and IBM China. In 1998,
John joined the IBM Enterprise Storage Server® Planning team for PPRC, XRC, and IBM
FlashCopy®. He has been the marketing manager for these products, and in 2002, began
working in Business Continuity and IT Strategy and Planning. Since 2009, John has also
added focus on IT Competitive Advantage strategy, including Scale Out NAS and Cloud
Storage. John is the author of three Redbooks publications on these topics, and in 2007,
celebrated his 25th anniversary of joining IBM.

John Tarella is a Senior Consulting IT Specialist who works for IBM Global Services in Italy.
He has 25 years of experience in storage and performance management in mainframe and
distributed environments. He holds a degree in Seismic Structural Engineering from
Politecnico di Milano, Italy. His areas of expertise include IBM Tivoli Storage Manager and
storage infrastructure consulting, design, implementation services, open systems storage,
and storage performance monitoring and tuning. He is presently focusing on storage
solutions for business continuity, information lifecycle management, and infrastructure
simplification. He has written extensively on z/OS DFSMS, IBM Tivoli Storage Manager,
SANs, storage business continuity solutions, content management, and ILM solutions. John
is currently focusing on cloud storage delivery. He also has an interest in Web 2.0 and social
networking tools and methodologies.

Thanks to the following people for their contributions to this project:

Sven Oehme
Mark Taylor
Alexander Saupp
Mathias Dietz
Jason Auvenshine
Greg Kishi
Scott Fadden
Leonard Degallado
Todd Neville
Warren Saltzman
Wen Moy
Tom Beglin
Adam Childers

x IBM Scale Out Network Attached Storage Concepts

 Frank Sowin
Pratap Banthia
Dean Hanson
Everett Bennally
Ronnie Sahlberg
Christian Ambach

Now you can become a published author, too!

Here's an opportunity to spotlight your skills, grow your career, and become a published
author - all at the same time! Join an ITSO residency project and help write a book in your
area of expertise, while honing your experience using leading-edge technologies. Your efforts
will help to increase product acceptance and customer satisfaction, as you expand your
network of technical contacts and relationships. Residencies run from two to six weeks in
length, and you can participate either in person or as a remote resident working from your
home base.

Find out more about the residency program, browse the residency index, and apply online at:
ibm.com/redbooks/residencies.html

Comments welcome
Your comments are important to us!

We want our books to be as helpful as possible. Send us your comments about this book or
other IBM Redbooks publications in one of the following ways:
򐂰 Use the online Contact us review Redbooks publications form found at:
ibm.com/redbooks
򐂰 Send your comments in an email to:
[email protected]
򐂰 Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400

Preface xi
Stay connected to IBM Redbooks publications
򐂰 Find us on Facebook:
http://www.facebook.com/IBMRedbooks
򐂰 Follow us on twitter:
http://twitter.com/ibmredbooks
򐂰 Look for us on LinkedIn:
http://www.linkedin.com/groups?home=&gid=2130806
򐂰 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks
publications weekly newsletter:
https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm
򐂰 Stay current on recent Redbooks publications with RSS Feeds:
http://www.redbooks.ibm.com/rss.html

xii IBM Scale Out Network Attached Storage Concepts

 1

Chapter 1. Introduction to Scale Out

Network Attached Storage
SONAS is designed to address the new storage challenges posed by the continuing
explosion of data. Utilizing mature technology from the IBM High Performance Computing
experience, and based upon the IBM General Parallel File System (GPFS), SONAS is an
easy-to-install, turnkey, modular, scale out Network Attached Storage (NAS) solution that
provides the performance, clustered scalability, high availability, and functionality that are
essential to meeting strategic Petabyte Age and cloud storage requirements.

In this chapter, we consider how a high-density, high-performance SONAS system can help
organizations consolidate and manage data affordably, reduce crowded floor space, and
reduce management expense associated with administering an excessive number of
disparate storage systems. With its advanced architecture, SONAS virtualizes and
consolidates multiple filers into a single, enterprise-wide file system, which can translate into
reduced total cost of ownership, reduced capital expenditure, and enhanced operational
efficiency.

© Copyright IBM Corp. 2010. All rights reserved. 1

1.1 Marketplace requirements
There are various factors driving the need for a new way of looking at information and the way
we make decisions based on that information. Today, the changes in our world—the
instrumentation, interconnectedness, and intelligence of our environments—combine to
produce a massive glut of new information, from new sources, with new needs to utilize it.
These pressures exacerbate a few of the challenges that we have been dealing with for
awhile now, but on a whole new scale.

There is an explosion in the amount of data, of course, but also there are shifts in the nature
of data (Figure 1-1). Formerly, virtually all the information available to be “processed” was
authored by someone. Now that kind of data is being overwhelmed by machine-generated
data, spewing out of sensors, RFID, meters, microphones, surveillance systems, GPS
systems, and all manner of animate and inanimate objects. With this expansion of the
sources of information comes a large variance in the complexion of the available data—very
noisy, lots of errors—and no time to cleanse it in a world of real-time decision making.

Also, consider that today’s economic times require corporations and governments to analyze
new information faster and make timely decisions for achieving business goals. As the
volume, variety, and velocity of information and decision making increases, this places a
larger burden on organizations to effectively and efficiently distribute the right information, at
the right time, to the people, processes, and applications that are reliant upon that information
to make better business decisions.

All of these situations are creating challenges, while providing an excellent opportunity for
driving an information-led transformation.

Figure 1-1 Explosion of data demands and information-led transformation

2 IBM Scale Out Network Attached Storage Concepts

Today’s businesses are demanding the ability to create, manage, retrieve, protect, and share
business and social digital content or large rich media files over a broadband Internet that
reaches to every corner of the globe (Figure 1-2). Users are creating and using data that is
redefining our business and social world in real time. Unlike traditional IT data, this rich digital
content is almost entirely file-based or object-based, and it is growing ever larger in size, with
highly diverse and unpredictable usage patterns.

Figure 1-2 Today’s workloads demand a new approach to data access

Innovative applications in business analytics, digital media, medical data, and cloud storage
are creating requirements for data access rates and response times to individual files that
were previously unique to high-performance computing environments, and all of this is driving
a continuing explosion of business data.

While many factors are contributing to data growth, these trends are significant contributors:
򐂰 Digital representation of physical systems and processes
򐂰 Capture of digital content from physical systems and sources
򐂰 Deliveries of digital content to a global population

Additional trends are driven by the following kinds of applications:

򐂰 Product Life Cycle Management (PLM) systems, which include Product Data
򐂰 Management systems and mechanical, electronic, and software design automation
򐂰 Service Life Cycle Management (SLM) systems
򐂰 Information Life Cycle Management (ILM), including email archiving
򐂰 Video on demand: Online, broadcast, and cable
򐂰 Digital Video Surveillance (DVS): Government and commercial
򐂰 Video animation rendering
򐂰 Seismic modeling and reservoir analysis
򐂰 Pharmaceutical design and drug analysis
򐂰 Digital health care systems
򐂰 Web 2.0 and service-oriented architecture

Chapter 1. Introduction to Scale Out Network Attached Storage 3

 When it comes to traditional IT workloads, traditional storage will continue to excel for the
traditional applications for which they were designed. But solutions such as Network Attach
Storage (NAS) were not intended to scale to the high levels and extremely challenging
workload characteristics required by today’s Internet-driven, Petabyte Age applications.

1.2 Understanding I/O

A major source of confusion regarding NAS is the concept of File I/O versus Block I/O.
Understanding the difference between these two forms of data access is crucial to realizing
the potential benefits of any SAN-based or NAS-based solution.

1.2.1 File I/O

When a partition on a hard drive is under the control of an operating system (OS), the OS will
format it. Formatting of the partition occurs when the OS lays a file system structure on the
partition. This file system is what enables the OS to keep track of where it stores data. The file
system is an addressing scheme that the OS uses to map data on the partition. Now, when
you want to get to a piece of data on that partition, you must request the data from the OS that
controls it.

For example, suppose that Windows® 2000 formats a partition (or drive) and maps that
partition to your system. Every time you request to open data on that partition, your request is
processed by Windows 2000. Because there is a file system on the partition, it is accessed by
File I/O. Additionally, you cannot request access to just the last 10 KB of a file. You must open
the entire file, which is another reason that this method is referred to as File I/O.

Using File I/O is like having an accountant. Accountants are good at keeping up with your
money for you, but they charge you for that service. For your personal checkbook, you
probably want to avoid that cost. On the other hand, for a corporation where many kinds of
requests are made, having an accountant is a good idea, so that wrongful checks do not get
written.

A file I/O specifies the file. It also indicates an offset into the file (Figure 1-3). For instance, the
I/O might specify “Go to byte ‘1000’ in the file (as if the file were a set of contiguous bytes),
and read the next 256 bytes beginning at that position.” Unlike block I/O, there is no
awareness of a disk volume or disk sectors in a file I/O request. Inside the NAS appliance, the
operating system keeps track of where files are located on disk. It is the NAS OS which
issues a block I/O request to the disks to fulfill the client file I/O read and write requests that it
receives.

By default, a database application that is accessing a remote file located on a NAS device is
configured to run with File System I/O. It cannot utilize raw I/O to achieve improved
performance.

4 IBM Scale Out Network Attached Storage Concepts

 Figure 1-3 File I/O

1.2.2 Block I/O

Block I/O (raw disk) is handled differently (Figure 1-4). There is no OS format done to lay out
a file system on the partition. The addressing scheme that keeps up with where data is stored
is provided by the application using the partition.

An example of this might be IBM DB2® using its tables to keep track of where data is located
rather than letting the OS do that job. We do not mean to say that DB2 cannot use the OS to
keep track of where files are stored. It is just more efficient for the database to bypass the cost
of requesting the OS to do that work.

Figure 1-4 Block I/O

Chapter 1. Introduction to Scale Out Network Attached Storage 5

 When sharing files across a network, something needs to control when writes can be done.
The operating system fills this role. It does not allow multiple writes at the same time, even
though many write requests are made. Databases are able to control this writing function on
their own, so in general, they run faster by skipping the OS, although this depends on the
efficiency of the implementation of file system and database.

1.2.3 Network Attached Storage

Storage systems that optimize the concept of file sharing across the network have come to be
known as Network Attached Storage (NAS). The NAS solutions utilize the mature Ethernet IP
network technology of the LAN. Data is sent to and from NAS devices over the LAN using
TCP/IP protocol.

One of the key differences in a NAS appliance, compared to direct attached storage (DAS) or
other network storage solutions such as SAN or iSCSI, is that all client I/O operations to the
NAS use file level I/O protocols. File I/O is a high level type of request that, in essence,
specifies only the file to be accessed, but does not directly address the storage device. This is
done later by other operating system functions in the remote NAS appliance.

By making storage systems LAN addressable, the storage is freed from its direct attachment
to a specific server, and any-to-any connectivity is facilitated using the LAN fabric. In
principle, any user running any operating system can access files on the remote storage
device. This is done by means of a common network access protocol; for example, NFS for
UNIX® servers and CIFS for Windows servers. In addition, a task such as backup to tape can
be performed across the LAN using software such as Tivoli Storage Manager, enabling
sharing of expensive hardware resources (for example, automated tape libraries) between
multiple servers.

NAS file system access

Network access methods such as NFS and CIFS can only handle file I/O requests to the
remote file system. This is located in the operating system of the NAS device. I/O requests
are packaged by the initiator into TCP/IP protocols to move across the IP network. The
remote NAS file system converts the request to block I/O and reads or writes the data to the
NAS disk storage. To return data to the requesting client application, the NAS appliance
software repackages the data in TCP/IP protocols to move it back across the network.

A storage device cannot just attach to a LAN. It needs intelligence to manage the transfer and
the organization of data on the device. The intelligence is provided by a dedicated server to
which the common storage is attached. It is important to understand this concept. NAS
comprises a server, an operating system, and storage that is shared across the network by
many other servers and clients. So a NAS is a specialized server or appliance, rather than a
network infrastructure, and shared storage is attached to the NAS server.

NAS file system administration

However, NAS filers today do not scale to high capacities. When one filer was fully utilized, a
second, third, and more filers were installed. The result was that administrators found
themselves managing “silos” of filers. It was not possible to share capacity on individual filers.
Various filers were heavily accessed, while others were mostly idle.

6 IBM Scale Out Network Attached Storage Concepts

 Managing the many filers adds complexity to the administrator's job. When adding more
storage capacity to filers, we cannot add more performance to a particular file than what is
possible with the single disk drive or controller that the filer typically uses. In other words,
there is limited parallelism (typically, perhaps one, two, or a few controllers) for serving an
individual file. Figure 1-5 shows a summary of traditional NAS limitations.

Figure 1-5 Network Attached Storage limitations

This situation is compounded by the fact that at hundreds of terabytes or more, conventional
backup of such a large storage farm is difficult, if not impossible. There is also the issue that
even though one might be using incremental only backup, scanning hundreds of terabytes to
identify the changed files or changed blocks might in itself take to long, with too much
overhead.

More issues include the possibility that there might not be any way to apply file placement,
migration, deletion, and management policies automatically from one centrally managed,
centrally deployed control point. Doing manual management of tens or hundreds of filers was
proving to be neither timely nor cost-effective, and effectively prohibited any feasible way to
globally implement automated tiered storage.

1.3 IBM Scale Out Network Attached Storage

IBM Scale Out Network Attached Storage (SONAS) is designed to address the new storage
challenges posed by the continuing explosion of data. IBM recognizes that a critical
component of future enterprise storage is a scale-out architecture that takes advantage of
industry trends to create a truly efficient and responsive storage environment, eliminating the
waste created by the proliferation of scale-up systems and providing a platform for file server
consolidation. That is where SONAS comes in, as shown in Figure 1-6.

Chapter 1. Introduction to Scale Out Network Attached Storage 7

 Utilizing mature technology from the IBM High Performance Computing experience, and
based upon the IBM flagship General Parallel File System (GPFS), SONAS is an
easy-to-install, turnkey, modular, scale out NAS solution that provides the performance,
clustered scalability, high availability, and functionality that are essential to meeting strategic
Petabyte Age and cloud storage requirements.

Simply put, SONAS is a scale out storage system combined with high-speed interface nodes
interconnected with storage capacity and GPFS, which enables organizations to scale
performance alongside capacity in an integrated, highly-available system. The high-density,
high-performance SONAS can help your organization consolidate and manage data
affordably, reduce crowded floor space, and reduce management expense associated with
administering an excessive number of disparate storage systems.

Figure 1-6 IBM SONAS overview

1.3.1 SONAS architecture

The SONAS system is available in as small a configuration as 30 terabytes (TB) in the base
rack, up to a maximum of 30 interface nodes and 60 storage nodes in 30 storage pods. The
storage pods fit into 15 storage expansion racks. The 60 storage nodes can contain a total of
7200 hard-disk drives when fully configured and you are using 96-port InfiniBand switches in
the base rack. With its advanced architecture, SONAS virtualizes and consolidates multiple
filers into a single, enterprise-wide file system, which can translate into reduced total cost of
ownership, reduced capital expenditure, and enhanced operational efficiency. For details, see
Chapter 3, “SONAS architecture” on page 41.

8 IBM Scale Out Network Attached Storage Concepts

Figure 1-7 provides a high level overview of the SONAS architecture.

Figure 1-7 SONAS architecture

Assuming 2 TB SATA disk drives, such a system has 14.4 petabytes (PB) of raw storage and
billions of files in a single large file system (up to 2 PB as standard support; larger is possible
by request to IBM). You can have as few as eight file systems in a fully configured 14.4 PB
SONAS system or as many as 256 file systems. It provides an automated policy-based file
management that controls backups and restores, snapshots, and remote replication. It also
provides:
򐂰 A single global namespace with logical paths that do not change because of physical data
movement
򐂰 Support for Serial Attached SCSI (SAS) and Serial Advanced Technology Attachment
(SATA) drives
򐂰 Superior performance per price
򐂰 High-availability and load-balancing
򐂰 Centralized management
򐂰 Centralized backup
򐂰 An interconnected cluster of file-serving and network-interfacing nodes in a redundant
high-speed data network
򐂰 Virtually no capacity limits
򐂰 Virtually no scalability limits
򐂰 IBM Call Home trouble reporting and IBM Tivoli Assist On Site (AOS) remote support
capabilities

Chapter 1. Introduction to Scale Out Network Attached Storage 9

 򐂰 Enhanced support for your Tivoli Storage Manager Server product, with a preinstalled
Tivoli Storage Manager client
򐂰 Support for the cloud environment. A controlled set of end users, projects, and
applications can perform the following functions:
– Share files with other users within one or more file spaces
– Control access to their files using access control lists (Microsoft Windows clients) and
user groups
– Manage each file space with a browser-based tool

Global namespace
SONAS provides a global namespace that enables your storage infrastructure to scale to
extreme amounts of data, from terabytes to petabytes. Within the solution, centralized
management, provisioning, control, and automated information life-cycle management (ILM)
are integrated as standard features to provide the foundation for a truly cloud storage enabled
solution.

Interface nodes
The high-performance interface nodes provide connectivity to your Internet Protocol (IP)
network for file access and support of both 1-gigabit Ethernet (GbE) and 10-GbE connection
speeds. Each interface node can connect to the IP network with up to eight separate
data-path connections. Performance and bandwidth scalability are achieved by adding
interface nodes, up to the maximum of 30 nodes, each of which has access to all files in all
file systems.

You can scale out to thirty interface nodes. Each interface node has its own cache memory,
so you increase caching memory and data paths in your file-serving capacity by adding an
interface node. Of course, you also increase file-serving processor capacity. If raw storage
capacity is the prime constraint in the current system, the SONAS system scales out to as
much as 14.4 petabytes (PB) with 2 terabytes (TB) SATA drives, with up to 256 file systems
that can each have up to 256 file-system snapshots. Most systems that a SONAS system
typically displaces cannot provide clients with access to so much storage from a single
file-serving head. Every interface node has access to all of the storage capacity in the
SONAS system.

1.3.2 SONAS scale out capability

SONAS provides extreme scale out capability, a globally clustered NAS file system built upon
IBM GPFS. The global namespace is maintained across the entire global cluster of multiple
storage pods and multiple interface nodes. All interface nodes and all storage nodes share
equally in the cluster to balance workloads dynamically and provide parallel performance to
all users and storage, while also assuring high availability and automated failover.

SONAS is a scalable virtual file storage platform that grows as data grows. It meets
demanding performance requirements as new processors can be added independently or as
storage capacity is added, eliminating a choke point found in traditional scale-up systems.
SONAS is designed for high availability 24x7 environments with a clustered architecture that
is inherently available and, when combined with the global namespace, allows for much
higher utilization rates than found in scale-up environments.

10 IBM Scale Out Network Attached Storage Concepts

1.3.3 SONAS Software
In this section we discuss the features and benefits of SONAS Software.

Storage management features

SONAS Software utilizes a powerful cross-platform access to the same files with locking for
data integrity. In addition, SONAS provides high availability Linux, UNIX, CIFS (Windows)
sessions with no client side changes (Figure 1-8). Deploying SONAS allows users to reduce
the overall number of disk drives and file storage systems that need to be housed, powered,
cooled, and managed relative to scale-up systems.

Figure 1-8 SONAS Software

Storage management benefits

SONAS also provides integrated support of policy-based automated placement and
subsequent tiering and migration of data. Customers can provision storage pools and store
file data according to its importance to the organization. For example, a user can define
multiple storage pools with various drive types and performance profiles. They can create a
higher performance storage pool with SAS drives and define a less expensive (and lower
performance) pool with SATA drives.

Rich, sophisticated policies built into SONAS can transparently migrate data between pools
based on many characteristics, such as capacity threshold limits and age of the data. This
helps to address business critical performance requirements. Leveraging automated storage
tiering, users can finally realize the cost savings and business benefits of information lifecycle
management (ILM) at an immense scale.

Chapter 1. Introduction to Scale Out Network Attached Storage 11

1.3.4 High availability design
SONAS provides a NAS storage platform for global access of your business critical data. Your
business critical data can be secured with both information protection and business continuity
solutions, giving you a high level of business continuity assurance.

In the event of data corruption or an unexpected disaster that might harm corporate data,
SONAS helps you to recover and quickly resume normal enterprise and data center
operations (Figure 1-9).

SONAS supports large enterprise requirements for remote replication, point-in-time copy
(file system-level snapshots), and scalable automated storage tiering, all managed as a single
instance within a global namespace. SONAS asynchronous replication is specifically
designed to cope with connections that provide low bandwidth, high latency, and low
reliability. The async scheduled process will pick up the updates on the source SONAS
system and write them to the target SONAS system, which can be thousands of miles away.

Security and information protection are enhanced in a consolidated SONAS environment.

For example, users considering the implementation of offerings such as Symantec security
and protection solutions are concerned about maintaining data availability as systems scale,
which is a key design point for SONAS. Its clustered architecture is designed for high
availability at scale and 24 x 7 x Forever operation, complementing Symantec’s security and
protection solutions to provide an always-on information infrastructure.

Figure 1-9 High availability and disaster recovery design

1.3.5 Summary
SONAS is a highly desired choice for organizations seeking to better manage their growing
demand for file-based storage. SONAS is designed to consolidate data that is scattered in
multiple storage locations and allows them to be efficiently shared and managed. The
solution helps improve productivity by providing automated ILM, automatic storage allocation,
user management by storage quota, and universal reporting and performance management.

12 IBM Scale Out Network Attached Storage Concepts

 2

Chapter 2. SONAS features, function,

and value
In this chapter we provide an overview of the various features, functions, and values that IBM
SONAS can bring to your IT organization.

We illustrate these benefits by following the lifecycle of a file from creation to deletion. We
demonstrate how files are written, centrally managed and migrated, backed up, archived,
remotely replicated, and finally deleted from the IBM SONAS file system.

In the process, we also consider how SONAS achieves these characteristics:

򐂰 High performance
򐂰 Scalability to petabytes of storage
򐂰 Central policy engine for storage management
򐂰 Automated tiered storage
򐂰 Hierarchical storage management to external storage (disk or tape)
򐂰 High performance scan engine
򐂰 High availability and resiliency, including snapshots and remote replication
򐂰 Concurrent high performance access to files from multiple platforms
򐂰 Performing all of these operations at the scale of petabytes of storage

© Copyright IBM Corp. 2010. All rights reserved. 13

2.1 Lifecycle of a file from creation to deletion
We start by previewing the contents of this chapter. In Figure 2-1 we illustrate the topics to be
discussed.

IBM Scale Out NAS:

The life of a file from creation to deletion
• Using Automatic Tiered storage
• Storage pool – group of LUNs
Protocols
• Fileset - define subtrees of a file system
CIFS
NFS • Policies – for rule based management of
HTTP files inside the storage pools
FTP
• Delivering:
Management
• Scalability - billions of files
Central • One global file system name space
Administration across independent logical storage
Monitoring
File Mgmt pools
• Files in the same directory can be in
different pools
• Files placed in storage pools at create
Availability time using policies
Data Migration
Replication • Files moved between pools via
Backup automated policy-driven tiered storage
• Hierarchical storage based on files
• Hierarchical to both disk and tape
• Allows classification of data according to
Service Level Agreements

Figure 2-1 IBM SONAS: Lifecycle of a file from creation to deletion

In this chapter, we provide an overview of IBM SONAS Software functionality, by examining

the lifecycle of a file, from creation to deletion. Along the way, we explain how SONAS offers
the toolset to provide a scale out architecture for high performance, high availability, centrally
managed NAS storage. We consider what is meant by storage pools, and discuss SONAS file
sets, the role of the policy engine, how automatic tiered storage is implemented, and how
data and files are physically moved.

In the following topics, the graphics show how SONAS physically manages files, without
changing the logical appearance to the users. We can see how files that logically appear to be
in the same directory might physically be in separate pools; how these files are created in
particular physical pools by a central SONAS policy engine; and how SONAS manages and
moves these files from creation to deletion, according to central policies.

14 IBM Scale Out Network Attached Storage Concepts

2.2 Logical diagram of SONAS
First, let us examine the logical diagram of IBM SONAS (Figure 2-2).

Logical
Logical Diagram of IBM SONAS
/home/appl/data/web/important_big_spreadsheet.xls
/home/appl/data/web/important_big_spreadsheet.xls
/home
/home/appl/data/web/big_architecture_drawing.ppt
/ho me/appl/data/web/big_architecture_drawing.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/ho me/appl/data/web/unstructured_big_video.mpg
/web

IBM Scale Out NAS Physical

Global Namespace
Policy Engine
Inte rfa c e
node s … Inter fac e
nodes … Interfa ce
node s … > scale
out

Stor age
node s
….. Stor age
node s
….. St ora ge
nodes
... > scale
out

Storage Pool (Tier 1): SAS drives Storage Pool (Tier 2) with 1TB SATA Storage Pool (Tier 3) with 2TB SATA

Figure 2-2 Logical Diagram of IBM SONAS

In the top half of this diagram, we see the logical file directory structure as seen by the users.
SONAS presents and preserves this same logical appearance to the users, no matter what
we do to physically manage these files (and all files in the SONAS system) from creation to
deletion. The users see only their global namespaces. As we expand and change the physical
data location and supporting physical infrastructure, the users will still have the unchanged
appearance of one single global namespace.

In the lower half of this diagram, we see a physical representation of the SONAS architectural
components. SONAS has interface nodes that serve data to and from the users over the
network. SONAS also has storage nodes, which service the storage for the SONAS clustered
file system.

All SONAS nodes are in a global cluster, connected by Infiniband. All interface nodes have full
read/write access to all storage nodes. All storage nodes have full read/write access to all
interface nodes. Each of the nodes runs a copy of IBM SONAS Software (5639-SN1), which
provides all the functions of SONAS, including a Cluster Manager that manages the cluster
and dispatches workload evenly across the cluster. Also included is the SONAS storage
policy engine, for managing the lifecyle of all files in a centrally deployed, centrally managed
way. The policy engine function is not tied to a particular node, it executes in a distributed
manner across all nodes. Not shown are the SONAS management nodes, which monitor the
health of the SONAS system.

Chapter 2. SONAS features, function, and value 15

 IBM SONAS Software manages the cluster and maintains the coherency and consistency of
the file systems, providing file level and byte level locking, using a sophisticated distributed,
token (lock) management architecture that is derived from IBM General Parallel File System
(GPFS) technology. As we shall see, the SONAS clustered grid architecture provides the
foundation for automatic load balancing, high availability, scale out high performance even
with multiple concurrent writers and readers.

Physical disk drives are allocated to SONAS logical storage pools. Typically, we might
allocate a high performance pool of storage (which uses the fastest disk drives), and a lower
tier of storage for capacity (less expensive, slower spinning drives). In the foregoing example,
we have allocated three logical storage pools.

2.3 Creating, writing, and reading files

Let us begin by creating and writing a file.

2.3.1 Creating and writing a file

When a “create file’” request comes into SONAS, it is directed to the SONAS Software central
policy engine. The policy engine has file placement rules that determine to which of the
logical storage pools the file is to be written. The policy engine does not run on just one node;
it is a distributed function which runs across all nodes.

An interface node is also selected to handle this client. The incoming workload is IP-balanced
equally by the external network Domain Name Server, across all SONAS interface nodes.
SONAS Software works in conjunction with the external Domain Name Server (DNS) to
allocate the interface node. As shown in Figure 2-3, we have selected an interface node.

Create and write file 1.1

/home/appl/data/web/important_big_spreadsheet.xls
/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/home/appl/data/web/big_architecture_drawing.ppt
/ho me/appl/data/web/big_architecture_drawing.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/ho me/appl/data/web/unstructured_big_video.mpg
/web

IBM Scale Out NAS

/home/appl/data/web/important_big_spreadsheet.xls Global Namespace

Policy Engine
Inte rfa c e
node s … Inter fac e
nodes … Inte rfa ce
node s … > scale
out

….. ….. ... > scale

Physica l

Stor age Stor age St ora ge

node s node s nodes
out

Storage Pool (Tier 1): SAS drives Storage Pool (Tier 2) with 1TB SATA Storage Pool (Tier 3) with 2TB SATA

Figure 2-3 Create and write file 1 - step 1 - policy

16 IBM Scale Out Network Attached Storage Concepts

All incoming create file requests pass through the SONAS central policy engine in order to
determine file placement.

The interface node takes the incoming create file request and passes the write request to all
appropriate storage nodes. The storage nodes, in parallel, perform a large data striped write
into the appropriate logical storage pool, across multiple disk drives.

SONAS data writes are done in a wide parallel data stripe write, across all disk drives in the
logical storage pool. SONAS Software architecture aggregates the file write and read
throughput of multiple disk drives, thus providing high performance. SONAS Software will
write the file in blocksize chunks, according to the blocksize specified at the file system level.

This wide data striping architecture determines where the data blocks must go, thus providing
SONAS Software with the ability to automatically tune and load balance all disk drives in the
storage pool. This process is illustrated in Figure 2-4.

Create and write file 1.2

/home/appl/data/web/important_big_spreadsheet.xls
/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/home/appl/data/web/big_architecture_drawing.ppt
/ho me/appl/data/web/big_architecture_drawing.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/ho me/appl/data/web/unstructured_big_video.mpg
/web

IBM Scale Out NAS

Global Namespace
Policy Engine
Inte rfa c e
node s … Inter fac e
nodes … Inte rfa ce
node s … > scale
out

….. ….. ... > scale

Physica l
Stor age Stor age St ora ge
node s node s nodes
/home/a ppl/data/web/important_big_spr eadsheet.xls out

Storage Pool (Tier 1): SAS drives Storage Pool (Tier 2) with 1TB SATA Storage Pool (Tier 3) with 2TB SATA

Figure 2-4 Create and write file 1 - step 2 - wide parallel data stripe write

Now, let us write another file to SONAS. As shown in Figure 2-5, another interface node is
appropriately selected by the Domain Name Server, and the file is passed to that interface
node for writing.

Chapter 2. SONAS features, function, and value 17

 Create and write file 2.1
/home/appl/data/web/important_big_spreadsheet.xls
/home/appl/data/web/ important_big_spreadsheet.xls
/home

Logical
/ho me/appl/data/web/big_architecture_drawing.ppt
/home/appl/data/web/ big_architecture_drawing.ppt
/app l

/data /ho me/appl/data/web/unstructured_big_video.mpg

/home/appl/data/web/ unstructured_big_video.mpg
/web

IBM Scale Out NAS

/home/appl/data/web/big_architecture_drawing.ppt
Global Namespace
Policy Engine
Inte rfa c e
node s … Inter fac e
nodes … Interfa ce
node s … > scale
out

….. ….. ... > scale

Physica l
Stor age Stor age St ora ge
node s node s nodes
out

Storage Pool (Tier 1): SAS drives Storage Pool (Tier 2) with 1TB SATA Storage Pool (Tier 3) with 2TB SATA

Figure 2-5 Create and write file 2 - step 1 - Policy

Notice that another interface node has been chosen; this illustrates the automatic balancing
of the incoming workload across the interface nodes. The interface node is told by the policy
engine that this file is to be written to the 1 TB intermediate pool. In the same manner as
previously described, the file is written in a wide data stripe (Figure 2-6).

Create and write file 2.2

/home/appl/data/web/important_big_spreadsheet.xls
/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/home/appl/data/web/big_architecture_drawing.ppt
/ho me/appl/data/web/big_architecture_drawing.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/ho me/appl/data/web/unstructured_big_video.mpg
/web

IBM Scale Out NAS

Global Namespace
Policy Engine
Inte rfa c e
node s … Inter fac e
nodes … Inte rfa ce
node s … > scale
out

….. ….. ... > scale

Physica l

Stor age Stor age St ora ge

node s node s nodes
/home/appl/data /web/big_ar chitecture _dr awing.ppt out

Storage Pool (Tier 1): SAS drives Storage Pool (Tier 2) with 1TB SATA Storage Pool (Tier 3) with 2TB SATA

Figure 2-6 Create and write file 2 - step 2 - wide parallel data stripe write

18 IBM Scale Out Network Attached Storage Concepts

Finally, we write a third file (Figure 2-7). A third interface node is selected by the Domain
Name Server.

Create and write file 3.1

/home/appl/data/web/important_big_spreadsheet.xls
/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/home/appl/data/web/big_architecture_drawing.ppt
/ho me/appl/data/web/big_architecture_drawing.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/ho me/appl/data/web/unstructured_big_video.mpg
/web

IBM Scale Out NAS

/home/appl/data/web/unst ructured_big_video.mpg
Global Namespace
Policy Engine
Inte rfa c e
node s … Inter fac e
nodes … Inte rfa ce
node s … > scale
out

….. ….. ... > scale

Physica l
Stor age Stor age St ora ge
node s node s nodes
out

Storage Pool (Tier 1): SAS drives Storage Pool (Tier 2) with 1TB SATA Storage Pool (Tier 3) with 2TB SATA

Figure 2-7 Create and write file 3 - step 1 - policy

Chapter 2. SONAS features, function, and value 19

 The SONAS policy engine has specified that this file is to be written into the 2 TB SATA pool.
A wide data stripe parallel write is done, as depicted in Figure 2-8.

Create and write file 3.2

/h
/hom
ome/ap
e/appl/data/web/im
pl/data/web/important_big_spreads
portant_big_spreadshee
heet.xls
t.xls
/home

Logical
/hom
/home/appl/data/web/big_architecture_drawing.ppt
e/appl/data/web/big_architecture_drawing.ppt
/appl

/data /hom
/home/appl/data/web/unstructured_big_video.mpg
e/appl/data/web/unstructured_big_video.mpg
/web

IBM Scale Out NAS

Global Namespace
Policy Engine
Interface
nodes … Interface
nodes … Interface
nodes …> scale
out

….. ….. ... >

Physica l
Storage
nodes
Storage
nodes
Storage
nodes scale out
/home/appl/data/web/unstructured_big_video.mpg

Storage Pool (Tier 1): SAS drives Storage Pool (Tier 2) with 1TB SATA Storage Pool (Tier 3) with 2TB SATA

Figure 2-8 Create and write file 2 - step 2 - wide parallel data stripe write

20 IBM Scale Out Network Attached Storage Concepts

With these illustrations, we can now see how the fundamental components of the SONAS
policy engine, interface nodes, and storage nodes work in concert with each other. This
process is summarized in Figure 2-9.

Create and write files - summary

/home/appl/data/web/important_big_spreadsheet.xls
/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/home/appl/data/web/big_architecture_drawing.ppt
/ho me/appl/data/web/big_architecture_drawing.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/ho me/appl/data/web/unstructured_big_video.mpg
/web
Note: all three files,
in same directory,
but each allocated to
Workload auto- different physical
balanced across all storage pool
IBM Scale Out NAS
interface nodes

Global Namespace
Policy Engine Data striped across
Inte rfa c e
node s … Inter fac e
nodes … Inte rfa ce
node s … > scale
out
all disks in storage
pool.
High performance,
auto-tuning, auto-
load balancing
….. ….. ... > scale

Physica l
Stor age Stor age St ora ge
node s node s nodes
out

Storage Pool (Tier 1): SAS drives Storage Pool (Tier 2) with 1TB SATA Storage Pool (Tier 3) with 2TB SATA

Figure 2-9 Create and write files - summary

In summary, SONAS workload is automatically balanced across all interface nodes. Data is
striped across all disks in the logical storage pool, providing high performance, automatic
tuning, and automatic load balancing. Notice that from the user’s perspective, these three
files all reside in the same logical path and directory. The user does not know that automatic
physical tiered storage has been implemented. The SONAS Software will continue to
maintain this same logical file structure and path, regardless of physical file location changes
due to automatic tiered storage data movement.

Chapter 2. SONAS features, function, and value 21

2.3.2 Scaling out high performance
Now we examine how the SONAS architecture can be utilized to scale out increased
performance. Performance in SONAS can be increased by adding more disk drives to a
SONAS logical storage pool. With more drives, SONAS can write a wider parallel data stripe
that can span multiple storage nodes and storage pods, as illustrated in Figure 2-10.

Increase write performance – scale out more disk drives

/home/appl/data/web/important_big_spreadsheet.xls
/hom e

Logical
/appl

/data

/web

IBM Scale Out NAS

Global Namespace
Policy Engine
Inte rf ac e
node
… Inter fa ce
node
… Inte rf ac e
node
…> scale
out

Physical
Stora ge
node s
….. Stor age
node s
….. St or age
nodes
... > scale
out
/home/appl/data/web/im portant_bi g_spreadsheet.xls

Expand Storage Pool with more disk drives Ti er 3

Figure 2-10 Scale out more disk drives for more write performance

SONAS architecture thus provides the ability to scale out both the number of disk drives and
the number of storage nodes that can be applied to support a higher amount of parallel
physical data writes.

In this way, SONAS architecture is very flexible, providing the ability to expand the scale and
capacity of the system in any direction that is desired. Disk drives and storage nodes can be
added non-disruptively to SONAS. Immediately upon doing so, SONAS automatically starts
to auto-balance and auto-tune new workload onto the additional disks, and automatically
starts taking advantage of the additional resources.

2.3.3 Reading a file

Reading data in SONAS applies the same principles of exploiting the wide data stripe for
aggregating the performance of reading the data in parallel across multiple disk drives
(Figure 2-11).

22 IBM Scale Out Network Attached Storage Concepts

 Read files – one user
/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/appl

/data

/web

Parallel read of file,

IBM Scale Out NAS performance is
aggregate all disk
drives in that storage
Global Namespace pool.

Policy Engine
Interface
nodes … Interface
nodes … Interface
nodes …> scale
out

Physical
Storage
nodes
….. Storage
nodes
….. Storage
nodes
... > scale
out
/home/appl/data/web/important_big_spreadsheet.xls

Tier 1 Tier 2 Tier 3

Figure 2-11 Read files - aggregate parallel data reads

Furthermore, the interface node is designed to utilize advanced algorithms that improve
read-ahead and write-behind file functions, and recognizes and does intelligent pre-fetch
caching of typical access patterns such as sequential, reverse sequential, and random
(Figure 2-12).

Read files 1.2

/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/app l

/data

/web

Interface node
performs read-ahead
caching, intelligent
pre-fetch
IBM Scale Out NAS

Global Namespace
Policy Engine
Inte rfa c e
node s … Inter face
nodes … Inte rfa ce
node s … > scale
out
Physica l

Stor age
node s
….. Stor age
node s
….. St ora ge
nodes
... > scale
out

Tier 1 Tier 2 Tier 3

Figure 2-12 Read files - read-ahead caching, intelligent pre-fetch

Chapter 2. SONAS features, function, and value 23

 Just as write performance can be enhanced by simply adding more disk drives to the logical
storage pool, read performance can be enhanced in the same way (Figure 2-13).

Read files – one user – scale out more disk drives

/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/app l
Interface node
/data
performs read-ahead
/web caching, intelligent
pre-fetch

Read performance is aggregate

IBM Scale Out NAS of all disk drives in storage pool.

More disks and storage pods

provides more performance
Policy Engine
Inter face
node s … Int er fac e
nodes … Inter fac e
nodes … > scale
out

Physica l
Stor age
node s
….. Stor age
node s
….. S tora ge
nodes
... > scale
out
/ho me/app l/d ata /we b/impor ta nt _big_ s pr ea ds he et .xls\

Expand Storage Pool with more disk drives Tier 3

Figure 2-13 Scale out more disk drives for read performance - parallel data stripe read

In summary, as requirements for NAS storage capacity or performance increase, SONAS

scale out architecture provides the following linearly scalable, high performance, parallel disk
I/O capabilities:
򐂰 Striping data across multiple disks, multiple storage nodes, and storage pods provides
increased capacity.
򐂰 Reading and writing data in parallel wide data stripes, thus increasing the number of disk
drives in the logical storage pool, can increase the performance.
򐂰 Supporting a large block size, configurable by the administrator to fit I/O requirements,
allows for growth.
򐂰 Utilizing advanced algorithms improves read-ahead and write-behind file functions.
SONAS recognizes typical access patterns such as sequential, reverse sequential, and
random, and optimizes I/O access for these patterns.

The scale-out architecture of SONAS provides superb parallel performance, especially for
larger data objects, and excellent performance for large aggregates of smaller objects.

2.3.4 Scaling out high concurrency

In this section we discuss how SONAS scale out architecture can be utilized to concurrently
service many tens of thousands of users.

As we have seen, SONAS is uniquely able to provide a scale out NAS architecture that can
grow to 30 interface nodes and 60 storage nodes, all in a global active-active
share-everything cluster. SONAS is based on the IBM General Parallel File System, which
has proven the ability to scale to thousands of nodes. IBM is scaling down that capability and
providing access to this capability in an appliance form factor, SONAS.

24 IBM Scale Out Network Attached Storage Concepts

One of the major values of the SONAS clustered node architecture is the ability to provide
scale out capability, meaning that many individual nodes can concurrently service storage
requests in parallel. This capability is depicted in Figure 2-14.

Read files – multiple users – in parallel

/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/home/appl/data/web/big_architecture_design.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/web

IBM Scale Out NAS

Parallel streaming
Global Namespace reads of multiple files
to multiple nodes
Policy Engine
Inter fa ce
node s … Int er fac e
nodes … Inter face
nodes …> scale
out

Physica l
Stor age
node s
….. Stor age
node s
….. S tora ge
nodes
... > scale
out
/home/appl/data/web/important_big_sprea dshee t.xls

Tier 1 Tier 2 Tier 3

Figure 2-14 Concurrent service storage requests in parallel

The value of a scale out architecture is the ability to flexibly and dynamically add nodes as
needed to increase the amount of parallel concurrent users that can be supported. Each
individual node works in parallel to service clients, as shown in Figure 2-15.

Read files – multiple users – in parallel

/home/appl/data/web/important_big_spreadsheet.xls
/home

Logical
/home/appl/data/web/big_architecture_design.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/web

IBM Scale Out NAS All interface nodes

performs read-ahead
caching, intelligent
Global Namespace pre-fetch in parallel
Policy Engine
Inter face
node s … Int er fac e
nodes … Inter fac e
nodes … > scale
out
Physica l

Stor age
node s
….. Stor age
node s
….. S tora ge
nodes
... > scale
out

Tier 1 Tier 2 Tier 3

Figure 2-15 Individual nodes providing service to clients

Chapter 2. SONAS features, function, and value 25

 SONAS has the same operational procedures and read/write file system architectural
philosophy, whether you have a small SONAS with two interface nodes and two storage
nodes, or a larger SONAS with 30 interface nodes and 60 storage nodes.

2.4 Managing storage centrally and automatically

Now that we have seen how SONAS can provide linear scale out performance and capacity
for petabytes of storage, we consider how to manage storage when we are operating at this
level of scale:
򐂰 How can the management of that much storage be automated?
򐂰 What tools does SONAS provide for this automation?
򐂰 Will such tools allow operation at this scale with fewer people and less resources?

In the following topics, we examine how SONAS provides integrated, automated tools to help
accomplish these goals.

2.4.1 Integrated storage pools for tiered storage

SONAS is designed to help you to achieve data lifecycle management efficiencies through
providing integrated policy-driven automation and tiered storage management, all as part of
the base SONAS Software license. SONAS Software provides integrated logical storage
pools, filesets, and user-defined policies to provide the ability to do automated tiered storage,
and therefore more efficiently match the cost of your storage to the value of your data.

SONAS logical storage pools allow you to allocate physical storage hard drives to logical
storage pools within the SONAS file system. Using logical storage pools, you can create tiers
of storage by grouping physical disk storage based on performance, locality or reliability
characteristics. Logical storage pools can span storage nodes and storage pods.

26 IBM Scale Out Network Attached Storage Concepts

You can have multiple logical storage pools, and the size of a storage pool can be as big as
the entire file system. SONAS automatically manages, load-balances, and balances storage
utilization at the level of the entire logical storage pool. As shown in Figure 2-16, multiple
logical storage pools have been set up.

Logical storage pools and filesets in an IBM SONAS SONAS

fileset

/h ome /home/appl/data/web/file_in_storage_pool_1.xls
/ho me/a pp l /d ata /we b/im por tant _big_s pre ads hee t.xls

/ap pl
/home/appl/data/web/file_in_storage_pool_2.ppt
/h om e/ap p l/d ata/w eb /big_ ar chite ctur e _dra wing.ppt
/d ata

/w eb
/home/appl/data/web/file_in_storage_pool_3.mpg
/h om e/ap p l/d ata/w eb /unstr uctur ed_ big_ video.m pg

IBM Scale Out NAS

Global Namespace
Policy Engine scale
In e
t rfa c
e n od es
In e
t rfa c
e n od es
I nt er f a c
e n ode s
I nt er f ac
e no de s
I nt er f ac
e no de s
In e
t r f ac
e no des
In e
t rfa c
e n od es
In e
t rfa c
e n od es …> out

scale
out
Stor ag
e
nodes
Stor ag
e
nodes
t orag
S
e
nodes
St orag
e
nodes
St orag
e
nodes
St orag
e
nodes
St orag
e
nodes
St roag
e
nodes
Stor ag
e
nodes
Stor ag
e
nodes
Stor ag
e
nodes
Storag
e
nodes
St or g
e
nodes
a St orag
e
nodes
St orag
e
node s
St orag
e
nodes
St orag
e
nodes …>
scale
…> out

etc….. etc….. etc… ..

Logical Logical Logical

Storage Storage Storage
Pool 1 Pool 2 Pool 3 External HSM
storage

Figure 2-16 SONAS Logical Storage Pools

Logical storage pool #1 might be high performance SAS disks, and logical storage pool #2
might be more economical SATA 1 TB disk storage. Logical storage pool #3 might be a 2 TB
SATA pool with Hiearchical Storage Management, where the data is intended to be staged in
and out of SONAS, to external tape or data de-deduplication technology. Within the internal
SONAS logical storage pools, all of the data management, from creation to physical data
movement to deletion, is done by SONAS Software.

Attention: If all data movement is within a SONAS system, then an external Tivoli Storage
Manager server is not necessary.

Chapter 2. SONAS features, function, and value 27

 In addition to internal storage pools, SONAS also supports external storage pools, which are
managed through a Tivoli Storage Manager server. When moving data to an external pool,
SONAS handles all the file system scanning for file identification, and then hands the data to
the Tivoli Storage Manager for storage on alternate media, such as tape, tape library, virtual
tape library, or data de-duplication devices. If the data is moved for Hiearchical Storage
Management purposes, a stub file is left on the disk, and this HSM data can be retrieved from
the external storage pool on demand, as a result of an application opening a file. HSM data
can be retrieved in a batch operation.

SONAS Software provides the file management concept of a fileset. A fileset is a sub-tree of
the file system namespace and provides a way to partition the global namespace into smaller,
more manageable units. A fileset is basically a named collection of files that you want to
operate upon or maintain using a fileset name. Filesets provide an administrative boundary
that can be used to set quotas and be specified in a user defined policy, to control initial data
placement or data migration.

As shown in Figure 2-16 on page 27, data in a single fileset can reside in one or more logical
storage pools. As the data is physically migrated according to storage policy, the fileset
grouping continues. Where the file data actually resides, and how and when it is migrated, is
based on a set of rules in a user defined policy, and this action is performed by the central
policy engine, which is described next.

2.4.2 Policy managed storage and policy engine

All files in the SONAS system are managed under the control of an integrated central storage
policy engine. Within the central policy engine are rules that specify all aspects of file
management. There are two types of rules:
򐂰 Initial physical file placement: This rule indicates where the file is initially placed.
򐂰 Physical file management: This rule includes physical movement of the file between tiers
of disk storage, and between disk storage and external tape, virtual tape library, or
de-duplication storage. File management rules can also include backup/restore, global
alteration of file characteristics according to any file system criteria, and deletion of files
that have expired.

File placement policies determine which storage pool file the data is initially placed in. File
placement rules are determined by attributes known when a file is created, such as file name,
user, group, or the fileset. Here are a few examples:
򐂰 ‘place all files that end in .avi onto the silver storage pool’
򐂰 ‘place all files created by the performance critical applications in the gold storage pool’
򐂰 ‘place all files in the fileset ‘development’ in the copper pool’

Files written to SONAS are physically placed according to these rules, which are contained in
a SONAS storage policy. The SONAS administrator writes these rules, which are SQL-like
statements.

28 IBM Scale Out Network Attached Storage Concepts

Figure 2-17 shows a few examples of such rules.

Policy engine and Storage Policies

• Automated Tiered Storage policy statem ent examples:

• Migration policies, evaluated periodically

• rule 'cleangold' migrate from pool ‘TIER1' threshold (90,70) to pool ‘TIER2‘

• rule 'hsm' migrate from pool ‘T IER3' threshold(90,85) weight(current_timestamp –

access_time) to pool ‘HSM' where file_size > 1024kb

• rule 'cleansilver' when day_of_week()=Monday migrate from pool 'silver' to pool 'bronze'
where access_age > 30 days

• Deletion policies, evaluated periodically

• rule 'purgebronze' when day_of_month()=1 delete from pool 'bronze' where access_age>365
days

• There are also P olicies for:

• File based Backup/Archive
• Restore/Retrieve
• Many more options…..

Figure 2-17 Policy engine and storage policies

When files exist in a file system, file management policies allow you to move, change the
replication status, or delete files. You can use file management policies to move data from one
pool to another, without changing the file’s location in the directory structure.

The rules are very flexible; as an example, you can write a rule that says: ‘replicate all files in
/database/payroll which have the extension *.dat and are greater than 1 MB in size to storage
pool 2’. In addition, file management policies allow you to prune the file system, deleting files
as defined by policy rules.

File management policies can use more attributes of a file than file placement policies,
because when a file exists, there is more known about the file. In addition to the file
placement attributes, the policies can now utilize attributes such as last access time, size of
the file or a mix of user and file size. This can result in policy statements such as: ‘delete all
files with a name ending in .temp that have not been accessed in 30 days’, ‘move all files that
are larger than 2 GB to pool2’, or ‘migrate all files owned by GroupID=Analytics that are larger
than 4 GB to the SATA storage pool’.

Chapter 2. SONAS features, function, and value 29

 Rules can include attributes related to a pool instead of a single file, using the threshold
option. Using thresholds, you can create a rule that moves files out of the high performance
pool if it is more than 80% full, for example.

The threshold option comes with the ability to set high low and pre-migrate thresholds,
meaning that SONAS begins migrating data at the high threshold, until the low threshold is
reached. If a pre-migrate threshold is set, SONAS begins copying data until the pre-migrate
threshold is reached, thus allowing the data to continue to be accessed in the original pool
until it is quickly deleted to free up space the next time the high threshold is reached.

Policy rule syntax is based on the SQL 92 syntax standard and supports multiple complex
statements in a single rule enabling powerful policies. Multiple levels of rules can be applied
because the complete policy rule set is evaluated for each file when the policy engine
executes.

2.4.3 High performance scan engine

Storage management rules are applied to all files in the SONAS system. However, as the
numbers of files and storage grows from terabytes into petabytes, storage management now
has a major new requirement: how can we scan the file systems fast enough in order to
identify files that must be:
򐂰 Migrated to another storage pool
򐂰 Propagated to remote sites
򐂰 Backed up
򐂰 Restored
򐂰 Deleted
򐂰 Any other storage management requirements

As the number of files continues to grow, the time required for this scan using the traditional
method of “walking the directory trees” has often become the major obstacle to effective
storage management. Shrinking backup and storage management windows require scan
times to stay small or even shrink, even as the file systems continue grow from hundreds of
terabytes to petabytes. It is increasingly clear that at the level of petabyte storage scalability,
which is becoming more commonplace every day, walking the directory trees to identify files
is no longer feasible; it simply takes too long.

To address this essential scanning requirement, SONAS is specifically designed to provide a

high performance, very high speed scan engine.

The SONAS scan engine is an integrated part of the file system. Also integrated into the
SONAS file system is an internal database of file system metadata, which is specifically
designed for the integrated scan engine. The goal of these two functions is to provide the
ability to scan the file system very quickly, at any scale, extending to billions of files.

30 IBM Scale Out Network Attached Storage Concepts

Let us see how this process works in more detail. To begin a scan to identify files, we submit
a job to the SONAS central policy engine to evaluate a set of policy rules, as shown in
Figure 2-18.

Scan Engine •Scan Engine reads internal SONAS file system metadata
•Does not need to read the file or directory tree
/home
•All nodes can participate in scan of file system

/app l

/data

/web
Central policy
engine starts scan
by reading policies

IBM Scale Out NAS

1. Start scan Global Namespace

Policy Engine
Inter fac e Inte r fac e Int er face
node
Interface nodes node node
2. Read policies
Scale
Out
Stor age nodes Stor age nodes St or a ge node s S torage node s St ora ge node s S tora ge node s

Tier 1 Tier 2 Tier 3

Figure 2-18 High performance scan engine - start scan by reading policies

The SONAS high performance scan engine is designed to utilize the multiple nodes of the
SONAS system in parallel, to scan the internal file system metadata. The multiple nodes
equally spread the policy engine rule evaluation, file scan identification, and subsequent data
movement responsibilities over the multiple nodes in the SONAS cluster.

Chapter 2. SONAS features, function, and value 31

 If greater scan speed is required, more SONAS nodes can be allocated to the scan, and each
node then scans only its equal portion of the total scan. This unique architectural aspect of
SONAS Software provides a very scalable, high performance, scale out rule processing
engine that can provide the solution to petabyte file system scan requirements (Figure 2-19).

Scan Engine •Scan Engine reads internal SONAS file system metadata
•Does not need to read the file or directory tree
/home
•All nodes can participate in scan of file system

/app l

/data

/web Some or all nodes

(both storage and
interface) participate
Parallel metadata scan in parallel scan
engine
Scan > 15 million
IBM Scale Out NAS files/minute

Global Namespace
Policy Engine
Inter fac e Inte r fac e Int er face
node
Interface nodes node node

3. Parallel Scan 3. Parallel Scan Scale

Out
Stor age nodes Stor age nodes St or a ge node s S torage node s St ora ge node s S tora ge node s

Tier 1 Tier 2 Tier 3

Figure 2-19 High performance scan engine - parallel scan of metadata by all nodes

The results of the parallel scan are aggregated, and returned as the actionable list of
candidate files, as shown in Figure 2-20.

Scan Engine •Scan Engine reads internal SONAS file system metadata
•Does not need to read the file or directory tree
/home
•All nodes can participate in scan of file system

/app l

/data
Scan results
/web completed in much
shorter period of
time, compared to
traditional methods
IBM Scale Out NAS

4. Return results of scan

Global Namespace
Policy Engine
Inter fac e Inte r fac e Int er face
node
Interface nodes node node

Scale
Out
Stor age nodes Stor age nodes St or a ge node s S torage node s St ora ge node s S tora ge node s

Tier 1 Tier 2 Tier 3

Figure 2-20 High performance scan engine - return results of parallel scan

32 IBM Scale Out Network Attached Storage Concepts

 Notice that the SONAS scan engine is not limited to just storage. The scan engine can also
be used for the following purposes:
򐂰 Reset file attributes according to policy (change deletions, change storage pool allocation)
򐂰 Run reports on file system usage and user activities
򐂰 Identify changed data blocks for asynchronous replication to a remote site

Now that we have used the scan engine to identify candidate files for automated storage
management, let us see how the parallel grid architecture of SONAS is used to scale out
physical data movement.

2.4.4 High performance physical data movement for ILM / HSM

After the list of candidate files has been identified using the SONAS parallel scan engine,
physical data movement is then performed, also using the multiple nodes of the SONAS
cluster in parallel.

Figure 2-21 shows an example of physically moving a file from “Storage Pool 1” to ‘”Storage
Pool 2”.

High performance internal data movement

/home/appl/data/web/important_big_spreadsheet.xls
/home

/app l
/home/appl/data/web/big_architecture_drawing.ppt

/data
/home/appl/data/web/unstructured_big_video.mpg
/web

6. All nodes (both

storage and
interface) can
IBM Scale Out NAS participate in parallel
data movement
5. Perform results of scan
Global Namespace
Policy Engine
Int er fac e Inte r fac e Int er face
node Interface nodes node node

Scale
Out
Stor age nodes Stor age nodes St or a ge node s S torage node s St ora ge node s S tora ge node s

Tier 1 Tier 2 Tier 3

Figure 2-21 High performance parallel data movement for ILM - pool 1 to pool 2

All files remain online and fully accessible during this physical data movement; the logical
appearance of the file path and location to the user does not change. The user has no idea
that the physical location of their file has moved. This is one of the design objectives of
SONAS.

According to the results of the scan, SONAS continues with other physical file movement.
According to policy, data can be up-staged as well as down-staged, as shown in Figure 2-22.

Chapter 2. SONAS features, function, and value 33

 High performance internal data movement

/home/appl/data/web/important_big_spreadsheet.xls
/home

/app l
/home/appl/data/web/big_architecture_drawing.ppt

/data
/home/appl/data/web/unstructured_big_video.mpg
/web

6. All nodes (both

storage and
IBM Scale Out NAS interface) can
participate in parallel
5. Perform results of scan data movement
Global Namespace
Policy Engine
Int er fac e Inte r fac e Int er face
node Interface nodes node node

Scale
Out
Stor age nodes Stor age nodes St or a ge node s S torage node s St ora ge node s S tora ge node s

Tier 1 Tier 2 Tier 3

Figure 2-22 High performance parallel data movement for ILM - pool 2 to pool 1

As the SONAS system grows in capacity over time, it is a straightforward matter to add
additional nodes to the parallel cluster, thus maintaining the ability to perform and complete
file system scans and physical data movement in a timely manner, even as the file system
grows into hundreds of terabytes and petabytes.

2.4.5 Summary
The SONAS parallel scan engine provides the following functions:
򐂰 Reads policy engine policies
򐂰 Identifies files to be moved within the physically tiered storage or sent to remote sites
򐂰 Enables and makes feasible automated tiered storage at terabyte and petabyte scale

The SONAS high performance scan engine has the following characteristics:
򐂰 Does not need to read the file or directory tree
򐂰 Reads special metadata integrated and maintained by the file system
򐂰 Is designed so that all nodes can participate in a parallel scan of the file system
򐂰 Delivers very high performance scan with minimized impact on concurrent workloads
򐂰 Can perform scans on a frequent basis due to low overhead

As long as the data movement is within the SONAS system, or between SONAS devices,
then all physical data movement is done solely through SONAS Software, and no involvement
of any external servers or external software is involved.

This combination of the internal file system metadata and the SONAS scale out parallel grid
software architecture enables SONAS to provide an architectural solution to scanning the file
systems quickly and efficiently, at the level of millions and billions of files in a short period of
time. The SONAS Software integrated high performance scan engine and data movement
engine work together to make feasible the management of automated tiered storage, with
physical data movement transparent to the users, at the level of hundreds of terabytes to
petabytes in the file systems.

34 IBM Scale Out Network Attached Storage Concepts

2.5 Hierarchical storage management, backup/restore to
external storage
SONAS also supports the ability to extend physical data movement to external storage
outside of the SONAS system. There are two types of operations to external storage:
򐂰 Hierarchical storage management (HSM): migrate inactive files to external storage, while
leaving a stub file on disk.
򐂰 Backup/restore (B/R): back up or restore copies of files, to and from SONAS and external
storage

Traditional software that performs these functions can accomplish these functions on SONAS,
by walking the directory trees, identifying candidate files through normal means, and
performing normal LAN I/O to do data movement. In this case, the normal parameters of file
system scan time will apply.

Tip: IBM has an extensive Independent Software Vendor (ISV) certification program for
SONAS. Enterprises use many ISV applications for their storage management to address
business requirements. IBM has done extensive testing and intends to continue to ensure
interoperability and compatibility of the leading ISV applications with SONAS to reduce
deployment risks. See 5.1.3, “ISV software supported” on page 144.

2.5.1 Requirements for high performance external HSM and backup restore
However, as file systems continue grow from hundreds of terabytes to petabytes, the following
requirements have arisen:
򐂰 The elapsed time is becoming too long for traditional scans to identify files that have to be
moved for HSM or backup/restore purposes. This occurs because of the requirement of
walking the directory trees, and due to the scale of these searches, incurs a very large
amount of small block IOPs.
򐂰 These long scan times are severely inhibiting the ability to manage a large amount of
storage. In many cases, the scan time alone can be longer than the backup window.
򐂰 In order to make feasible automated tiered storage at large scale, we must have a way to
reduce this overly long scan time.
򐂰 In addition, after we do identify the files, the large amount of data that this can represent
often drives the need for very high data rates in order to accomplish the needed amount of
Hierarchical Storage Management or backup/restore data movement, within a desired
(and continually shrinking) time window.

IBM SONAS has specific exploitation and integration with IBM Tivoli Storage Manager to
provide a solution to this problem. To address these requirements, SONAS provides specific
high performance capabilities for hierarchical storage management to external storage,
through integration of IBM Tivoli Storage Manager with the SONAS high performance scan
engine.

Chapter 2. SONAS features, function, and value 35

2.5.2 SONAS high performance HSM and backup/restore with Tivoli Storage
Manager
With IBM Tivoli Storage Manager, the SONAS scan engine works together, combining
SONAS parallel grid architecture with software parallelism in Tivoli Storage Manager, to
significantly enhance the speed, performance, and scale of both HSM and backup/restore
processes.

Specifically, Tivoli Storage Manager does not need to walk directory trees to identify files that
need to be moved to external storage, backed up, or restored. Instead, the SONAS high
performance scan engine is used to identify files to be migrated, and Tivoli Storage Manager
servers are architected to exploit multiple SONAS interface nodes for data movement in
parallel, as shown in Figure 2-23.

Hierarchical Storage Management to tape using TSM

Scan engine results
TSM/HSM
Server

Parallel data streams

Migrate inactive
data to tape, tape lib,
or de-duplication device

IBM Scale Out NAS Stub file is left on

disk, remainder of
file migrated to
external

Figure 2-23 Hiearchial Storage Management to external storage using Tivoli Storage Manager

In the HSM scenario, a stub file is left on disk. allowing the appearance of the file to be active
in the file system. Many operations, such as “list files,” are satisfied by the stub file without any
need for recall. You have flexible control over the HSM implementation, such as specifying the
size of the stub file, the minimum size of file in order to be eligible for migration, and so on.
If the file is requested to be accessed but is resident only on external storage, the file is
transparently auto-recalled from the external storage through the Tivoli Storage Manager
server. Data movement to and from external storage is done in parallel through as many
multiple SONAS interface nodes as desired, maximizing throughput through parallelism. Data
can be pre-migrated, re-staged, and de-staged, according to policy.

In this manner, SONAS provides the ability to support the storing of petabytes of data in the
online file system, yet staging only the desired portions of the file system on the actual
SONAS disk. The external storage can be any Tivoli Storage Manager-supported storage,
including external disk, tape, virtual tape libraries, or data de-duplication devices.

The same SONAS and Tivoli Storage Manager architecture is used for backup and restore
acceleration. The first step in backup / restore is to identify the files that need to be backed up
for Tivoli Storage Manager’s “incremental forever” method of operation. To identify changed
files, rules for backup are included in a SONAS policy engine scan of the file system. The high
performance scan engine locates files that need to be backed up, builds a list of these files,
and then passes the results to the Tivoli Storage Manager server, as shown in Figure 2-24.

36 IBM Scale Out Network Attached Storage Concepts

 Backup/restore acceleration using Tivoli Storage Manager

Scan engine results

Tivoli Storage
Manager
backup ser ver

Parallel data streams

•Any TSM-supported
devices including:

•ProtectTier de-dup
•Virtual Tape Library
•Tape

IBM Scale Out NAS

Figure 2-24 Backup and restore acceleration using Tivoli Storage Manager

Let us now examine how the SONAS system and Tivoli Storage Manager exploitation works
in a little more detail.

The first step in either HSM or backup/restore is to identify the files that need to be migrated.
or backed up. We kick off the SONAS process to perform a central policy engine scan of the
file system, according to centrally managed rules written for the purpose of performing the
HSM or backup/restore. The high performance scan engine then passes these results to the
Tivoli Storage Manager server, as shown in Figure 2-25.

SONAS scan engine identifies files for Tivoli Storage Manager

/home/appl/data/web/important_big_spreadsheet.xls
/home/appl/data/web/important_big_spreadsheet.xls
/home
/home/appl/data/web/big_architecture_drawing.ppt
/ho me/appl/data/web/big_architecture_drawing.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/ho me/appl/data/web/unstructured_big_video.mpg
/web

Scan engine List of files passed

identifies files to be
restaged (up or to Tivoli Storage
down), or to be Manager server
backed
IBM Scale OutupNAS
Tiv oli St ora ge

Global Namespace Mana ger

Policy Engine
Int er fac e
node … Inte rfa ce
node …
2. Pass list of changed
files to TSM

1. Scan to identify
changed files
St or a ge
node
… St ora ge
node
…

Tier 1: SAS drives Tier 2: SATA drives

Figure 2-25 SONAS scan engine and Tivoli Storage Manager

Chapter 2. SONAS features, function, and value 37

 Rather than walking the directory tree, the SONAS scan engine uses multiple SONAS
interface nodes to parallel scan the file system and identifying the list of changed files.
SONAS then passes the list of changed files directly to the Tivoli Storage Manager server. In
this way, we use the SONAS scan engine to avoid the need to walk the directory tree, and to
avoid the associated traditional time-consuming small block directory IOs.

The results of the scan are divided up among multiple interface nodes. These multiple
interface nodes then work in parallel to with the Tivoli Storage Manager servers to initiate the
HSM or backup/restore data movement, creating parallel data streams. The Tivoli Storage
Manager software implements a “virtual node” function that allows the multiple SONAS
interface nodes to stream the data in parallel to a Tivoli Storage Manager server, as illustrated
in Figure 2-26.

Data movement to external storage using Tivoli Storage Manager

/home/appl/data/web/important_big_spreadsheet.xls
/home/appl/data/web/important_big_spreadsheet.xls
/home
/home/appl/data/web/big_architecture_drawing.ppt
/ho me/appl/data/web/big_architecture_drawing.ppt
/app l

/data /home/appl/data/web/unstructured_big_video.mpg
/ho me/appl/data/web/unstructured_big_video.mpg
/web

High perfor mance parallel

data movement to Tivoli
Storage Manager server
IBM Scale Out NAS
Tiv oli St ora ge
Global Namespace Mana ger

Policy Engine
Int er fac e
node … Inte rfa ce
node …
2. Pass list of changed
files to TSM

1. Scan to identify
changed files
St or a ge
node
… St ora ge
node
…

Tier 1: SAS drives Tier 2: SATA drives

3. Parallel data movement using
multiple interface nodes to TSM
server - backup to Tape, Virtual Tape
Lib, or De-duplication

Figure 2-26 Parallel data streams between SONAS and Tivoli Storage Manager

38 IBM Scale Out Network Attached Storage Concepts

 In this way, the SONAS Software and Tivoli Storage Manager work together to exploit the
SONAS scale out architecture to perform these functions at petabyte levels of scalability and
performance. As higher data rates are required, more interface nodes can be allocated to
scale out the performance in a linear fashion, as shown in Figure 2-27.

High performance data movement to external

storage using Tivoli Storage Manager server
/home/appl/data/web/important_big_spreadsheet.xls
/home

/app l
/home/appl/data/web/big_architecture_drawing.ppt

/data
/home/appl/data/web/unstructured_big_video.mpg
/web

TSM
IBM Scale Out NAS s er ve rs
TS M
ser v er s
5. Perform results of scan

Policy Engine
Int er fac e Inte r fac e Int er face
node Interface nodes node node

Scale
Out

Stor age nodes Stor age nodes St or a ge node s S torage node s St ora ge node s S tora ge node s

Tier 1 Tier 2 Tier 3

Stub files left on

disk, auto-recall

Figure 2-27 High performance parallel data movement at scale, from SONAS to external storage

SONAS scale out architecture combined with Tivoli Storage Manager can be applied to
maintain the desired time windows for automated tiered storage, hierarchical storage
management, and backup/restore, even as file systems grow into hundreds of terabytes to
petabytes.

2.5.3 Summary
In this section, we have seen the lifecycle of a file in SONAS:
򐂰 Creation of files
򐂰 Serving of files in a high performance manner
򐂰 Automatic tiered storage management, effecting physical storage movement using central
policy engine
򐂰 Migration of data to external storage for hierarchical storage management and for backup

We have also seen how SONAS provides a rich set of integrated tools to perform centralized
storage and data management at large levels of scalability. We reviewed specific exploition
with Tivoli Storage Manager to extend the scalability of SONAS scale out NAS data
management to external storage, including tape, virtual tape libraries, and data de-duplication
devices.

Existing Tivoli Storage Manager server infrastructure and skills can be used to provide these
Tivoli Storage Manager-based HSM and backup/restore acceleration capabilities to a SONAS
environment.

Chapter 2. SONAS features, function, and value 39

 As shown in Figure 2-28, this is the end result of SONAS automated tiered storage,
centralized data management capability to manage the lifecyle of a file.

End result SONAS automated tiered storage:

/home/appl/data/web/important_big_spreadsheet.xls
/home

/app l
/home/appl/data/web/big_architecture_drawing.ppt

/data
/home/appl/data/web/unstructured_big_video.mpg
/web

Note: all three files,

no change to logical
directory
TSM
IBM Scale Out NAS s er ve rs
TS M
ser ver s
5. Perform results of scan

Policy Engine
Int er fac e Inte r fac e Int er fac e
node Interface nodes node node

Scale
Out

Stor age nodes Stor age nodes St or a ge node s S tora ge node s St ora ge node s S torage node s

Tier 1 Tier 2 Tier 3

Physical data movement Stub files left on

transparent to users disk, auto-recall

Figure 2-28 End result - SONAS automated tiered storage

During all of these physical data movement and management operations, the user logical file
path and appearance remains untouched. The user does not have any idea that this large
scale, high performance physical data management is being automatically performed on their
behalf.

In the remainder of this book, we explore other aspects of SONAS in order to provide the
complete picture, starting with an overview of the SONAS hardware and software.

40 IBM Scale Out Network Attached Storage Concepts

 3

Chapter 3. SONAS architecture

In this chapter we describe the hardware and software components of the SONAS appliance.
The hardware configuration includes various nodes, switches, and storage pods. The
software provides the ability for SONAS to appear as a single global virtual file server for
centrally managed, automated tiered storage management.

© Copyright IBM Corp. 2010. All rights reserved. 41

3.1 Hardware architecture
In this section we discuss the basic hardware structure of the SONAS appliance product.
The configuration consists of a collection of interface nodes that provide file services to
external application machines running standard file access protocols such as NFS or CIFS, a
collection of storage nodes that provide a gateway to the storage, and a management node
that provides a management interface to the configuration. In addition to the nodes, there are
switches and storage pods.

3.1.1 Nodes
The SONAS system consists of three types of server nodes:
򐂰 A set of interface nodes that provide connectivity to your Internet Protocol (IP) network for
file services to external application machines running standard file access protocols such
as NFS or CIFS
򐂰 A management node that provides a management interface to the SONAS configuration
򐂰 Storage nodes that provide a gateway to the SONAS storage

The SONAS nodes and their roles are summarized in Figure 3-1.

IBM Scale Out NAS – Nodes

Interface Nodes
• The interface node provides the connections to the customer’s IP
network for attaching to the SoNAS system for network file serving
capabilities (CIFS, NFS, HTTP, FTP, SCP).
• An IBM Scale Out NAS system contains a minimum of two interface
nodes and a maximum of 30.
• Four PCIe Gen 2.0 x8 adapter slots are provided – 2 for customer use.

Storage Nodes
• The storage node provides the connection to the InfiniBand cluster
interconnect and direct Fibre Channel attachment to the IBM Scale Out NAS
RAID controller.
• Storage nodes are configured in high-availability pairs, connected to one or
two IBM Scale Out NAS RAID controllers.
• Four PCIe adapters attach to the internal network and the Scale Out NAS
storage controller.

Management Node
• The management node provides the user interface for configuring,
administering and monitoring.
• There are two 1 GbE ports for health monitoring and configuration. There
are also two 1 GbE ports that connect to the customer network for the11/14/20 10
SONAS Management GUI and Command Line Interface (CLI) access.

Figure 3-1 SONAS nodes - interface, storage, and management

42 IBM Scale Out Network Attached Storage Concepts

 The management node, the interface nodes, and the storage nodes all run the SONAS
Software product in a Linux operating system. Product software updates to the management
node are distributed and installed on each of the interface nodes and storage nodes in the
system.

The interface nodes, management nodes, and the storage nodes are connected through a
scalable, redundant infiniband fabric allowing data to be transferred between the interface
nodes providing access to the application and the storage nodes with direct attachments to
the storage. Infiniband was chosen for its low overhead and high speed - 20 Gbits/sec for
each port on the switches. The basic SONAS hardware structure is shown in Figure 3-2.

Figure 3-2 Overview of SONAS hardware structure

3.1.2 Interface nodes

The Interface node is a 2U server that provides the TCP/IP data network connectivity and the
file services for the SONAS system. SONAS supports the following file-serving capabilities:
򐂰 Common Internet File System (CIFS)
򐂰 Network File System v2 or v3 (NFSv2 or NFSv3)
򐂰 File Transfer Protocol (FTP)
򐂰 HyperText Transfer Protocol Secure (HTTPS)

Chapter 3. SONAS architecture 43

 The interface node is an enterprise class IBM System x x3650M2 server, configured
specifically to run the SONAS Software and to perform the role of a SONAS interface node.
An overview of the interface node specifications follows. See Figure 3-3.

SONAS Interface Node (2851-SI1)

 IBM System x x3650M2 server FRONT VIEW
– Form:
• 2U

– Processor:
• Dual Quad Core Intel® Xeon® X5530
• 2.26 G Hz, 8 MB L2 cache, 80W
– Memory:
• 32 GB, 64 GB, or 128 G B RAM

– Storage to hold the SONAS Software: REAR VIEW

• 2 – 300G SAS 10K (1-RAID1 pair)

– Network Interfaces:
• Four 1 Gbps NICs
- 2 Customer Network
- 2 Management Network
• Optional additional card with four more 1 GbE ports or
Optional 10 GbE NIC
- (Mutually exclusive)

– Two single port 4X DDR InfiniBand Host

Channel Adapters
• For connection to Infiniband switch and from
there, to s torage nodes IBM x3650M2 rack-mount server

Figure 3-3 SONAS interface node (2851-SI1)

Each SONAS interface node contains two redundant hot-swappable 300 GB 2.5-inch 10K
RPM SAS hard disk drives (HDDs) with mirroring between the two HDDs for high-availability.
The HDDs contain the SONAS system software product, containing the operating system and
all other software needed for an operational SONAS system.

The interface nodes can operate at up to 10 Gbit speeds with the optional 10 GbE Ethernet
adapters, providing extremely fast access to data. All nodes in the SONAS system are in a
global parallel cluster, and are interconnected and cross-communicate by a redundant high
speed InfiniBand data network.

Users can access files in the SONAS file system through each of the SONAS interface nodes.
In conjunction with an external Domain Name Server, incoming workload is IP-address
balanced across all interface nodes. If more interface node capacity is needed, you simply
add more interface nodes, which provides a highly scalable capability. Additional data access
performance can be obtained by adding interface nodes up to the maximum limits of SONAS.

At the time of writing of this book, SONAS supports a minimum of two interface nodes and a
maximum of 30 interface nodes. All interface nodes can access all files and all file systems in
the SONAS system (with proper authentication, of course). The interface nodes request
access to the SONAS file systems, which are resident on file systems served by the storage
nodes, which are gateways to SONAS integrated storage controllers and disks drawers. All
interface nodes can access all storage on all storage nodes. All storage nodes can send data
to any interface node. In 3.2, “Software architecture” on page 58, we see how the SONAS
Software exploits this parallel global clustering capability.

44 IBM Scale Out Network Attached Storage Concepts

Storage nodes
The SONAS storage node is a 2U server that connects the SONAS interface nodes by the
internal InfiniBand network. Each storage node directly connects to the Fibre Channel
attached SONAS storage controllers within the SONAS storage pod.

The specifications of the SONAS storage node are summarized in Figure 3-4.

SONAS Storage Node (2851-SS1)

 IBM System x x3650M2 server
FRONT VIEW
– Form:
• 2U
– Processor:
• Dual Quad Core Intel® Xeon® X5530
• 2.26 GHz, 8 MB L2 cache, 80W
– Memory:
• 8 GB DDR3 RAM
REAR VIEW
– Storage which holds SONAS software:
• 2 – 300G SAS 10K (1-RAID1 pair)
– Network Interfaces:
• Four 1 Gbps NICs
- 2 Management Networks
– Two single port 4X DDR InfiniBand
Host Channel Adapters
• For connection to interface nodes
– Two dual-port 8 Gbps Fibre Channel
Host Bus Adapters (HBA) IBM x3650M2 rack-mount server
• For attaching SONAS integrated storage

Figure 3-4 SONAS storage node 2851-SS1

Storage nodes are incorporated into “building blocks’” of physical SONAS disk storage called
storage pods. A storage pod consists of two storage nodes, which are configured as a
high-availability (HA) pair. The storage nodes are connected to one or two SONAS storage
controllers (2851-DR1). A SONAS storage controller can optionally attach a SONAS Disk
Expansion Unit (2851-DE1).

Chapter 3. SONAS architecture 45

 Figure 3-5 shows a SONAS storage pod with its two storage nodes, and fully populated with
two SONAS storage controllers (2851-DR1, each one with 60 drives) and two SONAS Disk
Expansion Units (2851-DE1, each one with 60 drives), for a maximum number of 240 disk
drives.

SONAS Integrated Storage - High density advanced storage

two SONAS 2851-SS1 • SONAS Storage Pod contains at maximum:

storage nodes
• Two 2851-SS1 Storage Nodes
• Two 2851-DR1 Storage Controllers (60 drives each)
• Two 2851-DE1 Disk Expansion Units (60 drives each)

• Ultra-Dense
• 4U high 2851-DR1 storage controller includes 60
drives
• 4U high 2851-DE disk expansion Unit provides
additional 60 drives

• Flexible
• Intermix SAS and SATA
• A 60-drive drawer must be all SAS or all SATA

• Highly Reliable
• Active/Active Storage Controller Failover
• Data Integrity Validation
• RAID 5 (SAS) or RAID 6 (SATA)
two SONAS 2851-DR1 two SONAS 2851-DE1 • Redundancy Throughout
storage controllers disk expansion units • Battery Backed Cache

Figure 3-5 SONAS storage pod with two storage nodes, fully populated with 240 disk drives

The storage node contains two redundant hot-swappable 300 GB 2.5-inch 10K RPM SAS
HDDs with mirroring between them for high-availability. The hard disk drives contain the
SONAS System Software product (which hosts the SONAS operating system and all other
software needed for an operational SONAS system.

The global clustered architecture of SONAS means that all SONAS interface nodes can
access all storage on all storage nodes. All storage nodes can send data to any interface
node. A SONAS system contains a minimum of two storage nodes. The maximum number of
storage nodes depends on the size of the InfiniBand switches that are ordered - when using
96-port InfiniBand switches in the SONAS base rack, a maximum of 60 storage nodes is
possible. When using 36-port InfiniBand switches in the base rack, the maximum number of
storage nodes is 28.

The two onboard Ethernet ports in the storage node connect to the internal private SONAS
1 GbE management network, and two onboard Ethernet provide a NULL Ethernet connection
to the SONAS Disk Storage Controllers.

Management node
The Management node is a 2U server that provides a central point for the system
administrator to configure, monitor, and manage the operation of the SONAS cluster. The
management node supports both a browser-based graphical user interface (GUI) and a
command line interface (CLI). It also provides a System Health Center for monitoring the
overall health of the system. A single management node is required. The SONAS system will
continue to operate without the SONAS management node being active, but configuration
changes can only be performed from an active management node. The SONAS management
node specifications are shown in Figure 3-6.

46 IBM Scale Out Network Attached Storage Concepts

 SONAS Management Node (2851-SM1)
 IBM System x x3650M2 server FRONT VIEW
– Form:
• 2U
– Processor:
• Dual Quad Core Intel® Xeon® X5530
• 2.26 GHz, 8 MB L2 cache, 80W
– Memory:
• 32 GB DDR3 RAM REAR VIEW
– Storage:
• 2 – 300G SAS 10K (1-RAID1 pair)
• 1 – 300G SAS 10K (log/trace collection)
– Network Interfaces:
• Four 1 Gbps NICs
- 2 Customer Network – for authorized
users to access the mgmt node

- 2 Management Network – for

monitoring all the internal SONAS
IBM x3650M2 rack-mount server
components

Figure 3-6 SONAS management node 2851-SM1

The management node contains two hot-swappable 300 GB 2.5-inch 10K RPM SAS hard
disk drives with mirroring between the two HDDs for high-availability. The hard disk drives
contain the SONAS System Software product, containing the operating system and all other
software needed for an operational SONAS system. The third hot-swappable 300 GB 2.5-inch
10K RPM SAS hard disk drive stores the logging and trace information for the entire SONAS
system.

The management node comes with all of the cables that are required to connect it to the
switches within the base rack. The management node is assumed to be in the SONAS base
rack with the two InfiniBand switches.

3.1.3 Switches
The SONAS system contains internal InfiniBand, internal Ethernet switches and external
customer supplied external Ethernet switches.

Internal Infiniband switch

The nodes of a SONAS system (the interface nodes, storage nodes, and management node)
are interconnected by a high-performance low-latency InfiniBand 4X Double Data Rate
(DDR) fabric.

Two redundant InfiniBand switches are included inside each SONAS system. These two
identical IB switches are configured when you order a SONAS system;- the two InfiniBand
switches are used in order to provide redundancy for each other. For small and medium
configurations, a 1U 36 port 4X DDR InfiniBand switch is available. For larger configurations,
a 7U 96-port 4X DDR InfiniBand switch is available.

Chapter 3. SONAS architecture 47

 Choosing a switch type
At the time of ordering the SONAS system, you must choose between the smaller InfiniBand
36-port switch or the larger 96-port InfiniBand switch. In the case of the 96 port switch, you
can start with just the default 24-port InfiniBand card in each switch and order additional
24-port cards until you scale out the larger InfiniBand switch to the maximum amount of 96
InfiniBand ports in each switch.

Unfortunately, it is not possible to field upgrade the 36-port InfiniBand switch to the 96-port
switch. Therefore, at initial SONAS order time, consider your growth requirements, and order
the appropriate InfiniBand switch.

The SONAS InfiniBand network supports high bandwidth, low latency file system data and
control traffic among the nodes of the SONAS system. The data carried by the InfiniBand
fabric includes low level file system data, as well as the TCP/IP-based locking and
management messaging traffic for the parallel global cluster. The locking and management
traffic occurs on IP over InfiniBand.

Cabling within a SONAS rack is provided as part of the rack configuration and it is performed
by IBM at the plant of manufacture. You must order InfiniBand cable features for inter-rack
cabling after determining the layout of your multi-rack system.

InfiniBand backplane bandwidth

The Infiniband switches have sufficient bandwidth capability to handle a fully configured
SONAS solution. There are two switch sizes; the total backplane bandwidth is as follows:
򐂰 Infiniband switches = 20 Gbit/sec per port (2 GBytes/sec per port)
򐂰 Infiniband 36-port switch backplane = 1.44 Tbits/sec (144 Gigabytes//sec total)
򐂰 Infiniband 96-port switch backplane = 3.84 Tbits/sec (384 Gigabytes//sec total)

Internal private Ethernet switch

Included with the SONAS system is an internal management network, which is used as a
separate independent path to manage and monitor all internal SONAS components. Each
SONAS rack has a pair of 50-port Ethernet management switches, which are interconnected
to form the management network.

The management network is not used for user data or file system data transfer, rather, it only
carries low bandwidth internal messages in support of managing, monitoring, and performing
configuration, health, and monitoring of the SONAS subsystem. All major components of a
SONAS system such as interface nodes, storage nodes, management nodes, and infiniband
switches are connected to internal Ethernet management network.

External customer network l Ethernet switches

You can connect your existing external TCP/IP network infrastructure to the SONAS interface
nodes for your users to access data and files on the SONAS system, using
customer-supplied Ethernet cables, to your standard Ethernet network patch panels or other
connection points.

You can also connect the SONAS management network switches to your external network
However, these connections are only for your SONAS and IT management staff to be able to
securely access the SONAS management node for management GUI and Command Line
Interface. Your external user network infrastructure must support either or both 1 Gb/s or
10 Gb/s Ethernet links, according to the way that you have configured the 1 GbE or 10 GbE
ports on your SONAS interface nodes.

The external customer network and network switches and the cables attaching the SONAS
system to your customer network are not provided as part of the SONAS appliance.

48 IBM Scale Out Network Attached Storage Concepts

3.1.4 External ports: 1 GbE / 10 GbE on the interface nodes
SONAS supports up to 30 interface nodes. Each interface node connects to your Ethernet
network. As SONAS is designed as a parallel grid architecture with the appearance of a
single global namespace to the users. Every interface node is able to share in supporting the
network users.

Each interface node is capable of serving data to all users, you can control which users reach
which interface nodes, by the appropriate Domain Name Server definitions.

The manner in which this is done is described in “Software architecture” on page 58. Each
interface node is a balanced modular network building block, with sufficient main memory and
PCI bus capacity to provide full throughput with the Ethernet adapters configured to that
node.

Each interface node can be configured with either of the following possibilities:
򐂰 One pair of active/passive 10 GbE Ethernet ports or
򐂰 Two or six 1 GbE Ethernet ports, all of which can simultaneously be active.

3.1.5 Storage pods

Each storage pod consists of a pair of storage nodes (2851-SS1). A storage pod must have
at least one storage controller (2851-DR1). A SONAS storage controller also contains one
enclosure of 60 disks.

You can attach one SONAS disk expansion unit (2851-DE1) to the first SONAS storage
controller. A storage pod can be expanded by additional second SONAS storage controller.
To that second storage controller, you can add one additional disk expansion unit. Each
storage controller has 60 drives, and each storage expansion unit has 60 drives. The 60
drives for any individual storage controller or storage expansion unit must all be the same
drive type. You can choose any drive type that you want for this group of 60 drives, as long as
they are all the same.

The storage nodes within each storage pod provides dual paths to all storage controllers, and
each storage controller or storage expansion unit has dual paths to all disks for reliability.
Each storage node and storage controller operate in parallel with each other. Multiple storage
pods operate in parallel with each other.

Note that there is no fixed relationship between the number of interface nodes and the
number of storage pods. Storage pods can be scaled to provide an ability to increase storage
capacity and bandwidth, independently of the interface nodes. Storage capacity can be
added to the SONAS system by either adding disks to existing storage pods, or adding
storage pods. Each storage pod supports a maximum of 240 hard disk drives.

Chapter 3. SONAS architecture 49

3.1.6 SONAS storage controller
The high density SONAS storage controller (2851-DR1) is a 4U enclosure containing dual
redundant active/active RAID controllers, redundant power and cooling modules, and 60 hard
disk drives in an attached enclosure. The SONAS storage controller contains dual redundant
hot-swappable RAID controllers and dual redundant hot-swappable power supplies and
cooling modules. Each SONAS storage controller supports up to 60 3.5-inch SAS or 60 SATA
hard-disk drives.

The storage controller is configured to use RAID 5 with 450 GB or 600 GB SAS hard-disk
drives, and RAID 6 with 1 or 2 terabyte (TB) SATA hard-disk drives. All 60 disk drives in the
storage controller must be the same type, either six 10-packs of 7.2K RPM 1 TB SATA
hard-disk drives or six 10-packs of 2 TB SATA drives, or six 10-packs of 15K RPM 450 GB or
six 10-packs of 15K RPM 600 GB SAS drives. You cannot mix drive types or sizes within an
enclosure. The SONAS Storage Controller and attached disk expansion unit can contain
unique disk types. Each SONAS Storage Controller can attach one high-density SONAS disk
expansion unit (2851-DE1).

3.1.7 SONAS storage expansion unit

The high density SONAS disk expansion unit (2851-DE1) is a 4U enclosure containing
redundant connections, redundant power and cooling modules and 60 hard disk drives. One
optional disk storage expansion unit can be attached to one SONAS storage controller. The
storage controller and the disk expansion unit support both high performance 15K RPM SAS
disk drives and high-capacity 7.2K RPM SATA disk drives, again, all the drives within an
enclosure (tray of 60 drives) have to be the same type.

3.1.8 Connection between hardware components

Now that we have seen the overall components of the IBM SONAS architecture, let us review
the connections between these SONAS components.

The purpose of the component connection determines the type of connection that is used.
There are four connection types that are used in SONAS:
򐂰 External customer network:
The only customer supplied connection is the connection between the SONAS interface
nodes to the external customer network.

The remaining three connections are internal to the SONAS system.

򐂰 Internal data transfer: Infiniband (20 Gbit/sec per port)
򐂰 Direct storage connection to the storage nodes: Fibre Channel
򐂰 Management network: 1 Gbit Ethernet

50 IBM Scale Out Network Attached Storage Concepts

Let us examine a diagram of the connections of the SONAS components in Figure 3-7.

Windows, UNIX, and

other clients
Backup server

Customer IP Network
SONAS System Tap e

Interface Node Min 2

Ethernet Management Interface Node Max 30 Customer-
Node Interface Node Interface
connection for supplied
nodes
Ethernet
management, network
monitoring
Redundant private Ethernet
Redundant private InfiniBand
Management Network Data Network
Infiniband

connects
Storage Node Storage Node
all SONAS
nodes for
internal
data transfer:
High Density High Density
RAID Controller RAID Controller Interface,
Fibre Storage,
Channel Management
connects High Density High Density
storage Disk Enclosure Disk Enclosure
nodes to
storage
Storage Pod

Figure 3-7 SONAS component and outside network connection

From Figure 3-7, you can see that the external network connections to the SONAS system
are provided by the customer. The internal SONAS connections can be summarized as
follows:
򐂰 Each of the internal connections is fully duplexed and redundant. Specifically, each
internal SONAS component has dual connections, one connection each to pair of
switches. This provides provide high availability and automatic failover, in the event of
failure of any internal component connection.
򐂰 For internal data transfer between all nodes in the SONAS cluster, InfiniBand is used. All
data transfer takes place over Infiniband cable connections to a pair of internal Infiniband
switches, taking advantage of Infiniband's very low latency and high 20 Gbits/second/port
data transfer rate.
򐂰 Storage is Fibre Channel attached, directly to the Storage nodes. There is no internal
SAN, and there are no internal SAN switches inside the SONAS system.
򐂰 Finally, all SONAS components are connected to the Management node by 1 Gbit/sec
Ethernet. Over this connection, the Management node monitors the health of the entire
cluster and all components, and manages the cluster. Because the nature of this
connection is commands and health information, 1 GbE is more than adequate.

SONAS features a very flexible, modular architectural approach, as discussed in the following
topics

Chapter 3. SONAS architecture 51

3.1.9 Available SONAS configurations
Now that we know the components and how they are connected, we can consider how
SONAS is packaged.

Rack types: Choosing the correct rack for your solution

A SONAS system can consist of one or more racks into which the components of the system
are installed. A 42U enterprise class rack is available. Note that installation of SONAS
components in customer-supplied racks is not permitted.

There are three types of SONAS racks:

򐂰 Base rack
򐂰 Interface expansion rack
򐂰 Storage expansion rack

Base rack
The SONAS system always includes a base rack that contains the management node, the
InfiniBand switches, a minimum of two interface nodes, and a keyboard, video, and mouse
(KVM) unit. The capacity of the SONAS system that you order affects the number of racks in
your system and the configuration of the base rack.

Figure 3-8 shows the three types of base SONAS racks.

Figure 3-8 SONAS base rack options

52 IBM Scale Out Network Attached Storage Concepts

There are three available feature codes for the SONAS base rack: 2851-RXA feature code
9003, 9004, and 9005. Your first base rack will depend on how you are going to scale out the
SONAS system in the future.

SONAS base rack feature code 9003

The SONAS base rack feature code 9003 (left rack in Figure 3-8 on page 52) is the base rack
with two gigabit Ethernet (GbE) switches at the top of the rack, a management node, two
smaller 36-port InfiniBand switches, and at least two interface nodes. There is no storage with
this rack, as the intent is that there will be a large number of interface nodes, and thus you can
add storage in storage expansion racks. You can add to this rack, one additional interface
expansion rack.

With the feature code 9003 rack, you are configuring the 36 Infiniband port switches. Note
that the InfiniBand switches cannot be expanded or exchanged. The feature code 9003
SONAS based rack can have up to 14 interface nodes. You can attach one additional
interface expansion rack and as many storage expansion racks as you see fit, up to the
maximum attachability of the 36 port InfiniBand switches.

SONAS base rack feature code 9004

The SONAS base rack feature code 9004 (middle rack in Figure 3-8 on page 52) is the base
rack with two gigabit Ethernet (GbE) switches at the top of the rack, a management node, two
larger 96-port InfiniBand switches, and at least two interface nodes. There is no storage with
this rack, the assumption is that you will add storage in another storage expansion rack. You
can add to this rack, one additional interface expansion rack. The feature code 9004 base
SONAS rack can have up to eight interface nodes.

The purpose of the feature code 9004 rack is to allow you to scale to maximum scalability,
because it has the large 96-port InfiniBand switches. You can attach one additional interface
expansion rack and as many storage expansion racks as you see fit, up to the maximum
attachability of the 96 port InfiniBand switches.

SONAS base rack feature code 9005

The SONAS base rack feature code 9005 (right rack in Figure 3-8 on page 52) is the base
rack with two gigabit Ethernet (GbE) switches at the top of the rack, a management node, two
smaller 36-port InfiniBand switches, at least two interface nodes, and at least one storage pod
that consists of two storage nodes and a RAID controller.

The purpose of the feature code 9005 rack is to support a combination of the interface nodes
and storage nodes within the base rack. This rack can have up to 6 interface nodes and one
storage pod. A fully populated storage pod in this base rack has 240 disk drives (configured in
groups of 60 as previously discussed). You can attach one additional interface expansion rack
and as many storage expansion racks as you see fit, up to the maximum attachability of the
36 port InfiniBand switches.

Chapter 3. SONAS architecture 53

 Interface expansion rack
The SONAS interface expansion rack extends the number of interface nodes in an already
existing base rack by providing up to 20 additional interface nodes. The total number of
interface nodes cannot exceed 30. The two 50-port 10/100/1000 Ethernet switches are
mandatory, as well as at least one interface node per interface expansion rack. Figure 3-9
shows the interface expansion rack.

Figure 3-9 Interface Expansion Rack

54 IBM Scale Out Network Attached Storage Concepts

Storage expansion rack
The SONAS storage expansion rack extends the storage capacity of an already existing base
rack, by adding up to four additional storage nodes, up to four additional storage controllers,
and up to four disk storage expansion units. The eight possible disk storage expansion and
controller units can hold a maximum of 480 hard-disk drives. Up to 30 storage pods can exist
in a SONAS system. Figure 3-10 shows the storage expansion rack.

Figure 3-10 Storage expansion rack

There is a SAS verses SATA configuration consideration when configuring SONAS storage
expansion racks.

Note that SAS drives consume more power than SATA drives. The power supplies in the
SONAS storage expansion rack can support a maximum of the following drives:
򐂰 960 SATA drives (in groups of 60)
򐂰 720 SAS drives, because that amount can consume the power supply capability of the
storage expansion rack

Chapter 3. SONAS architecture 55

 You can have an intermix of SAS and SATA drives (in 60 drive groups) in a storage expansion
rack. The IBM or IBM Business Partner SONAS configuration tools are aware of these
considerations regarding power and number of drives, and configure the SONAS storage
expansion racks appropriately.

3.1.10 Various drive types

The lowest hardware storage component of SONAS is a physical disk, which are grouped in
sets of 10 disks. Each set of disks consists of a single type of physical disk drive: either six
10-packs of 7.2K RPM 1 TB SATA hard-disk drives or six 10-packs of 2 TB SATA drives; or six
10-packs of 15K RPM 450 GB SAS drives or six 10-packs of 15K RPM 600 GB SAS drives.

An enclosure holds six 10-packs (60 drives), and all drives within an enclosure must be of the
same type (you cannot mix drive types or sizes within an enclosure). The SONAS storage
controller uses RAID 5 with 450 GB and 600 GB SAS hard-disk drives, and RAID 6 with 1 TB
and 2 TB SATA hard-disk drives. Table 3-1 shows a summary of possible configurations.,
which are preconfigured and cannot be changed.

Table 3-1 Drive types configuration summary

Drive type Drive RAID Total Data Parity Spare
capacity array drives drives drives drives

SATA 1 TB or RAID 6 60 48 12 0
2 TB

SAS 450 GB or RAID 5 60 48 6 6

600 GB

SONAS supports drives intermix at the level of the enclosure (a group of 60 drives). Each of
the four possible enclosures within a storage pod might be another drive type. Thus, it is
possible to have within the same storage pod, or attached to the same, a storage controller,
an enclosure with high performance SAS disks and another storage enclosure with high
capacity SATA disks.

As we can see in 3.2, “Software architecture” on page 58, SONAS Software provides
automated tiered storage, so that Hierarchical Storage Management can be performed on
physical data and storage between the various types of physical drives. Using this capability,
data can be automatically migrated according to centrally managed and deployed storage
policies within SONAS from faster, but smaller and more expensive SAS disks, to slower, but
larger and less expensive SATA disks, and even migrated out to external storage.

3.1.11 SONAS scale out capabilities

One of the primary values of the SONAS system is the ability to scale out the system in either
or both interface nodes or storage nodes, to a level that encompasses petabytes of data.

After you have decided on which base SONAS rack you will start with, you can then expand
the SONAS system independently in either interface node expansion rack or storage node
expansion racks as shown in Figure 3-11.

56 IBM Scale Out Network Attached Storage Concepts

 IBM SONAS Scale Out Expansion Options
Switches Swi tches Interface Node Expansion
Interface Node Storage Node
Interface Node Storage Node
Interface Node
60 Disks
Interface Node • Up to 30 interface nodes
Interface Node
60 Disks • High speed extremely low latency (20 Gbps) private
Interface Node
Interface Node
60 Disks Infiniband cluster data network
Interface Node
Interface Node
60 Disks
Interface Node
Interface Node Storage Node
Storage Node
Storage Pod Expansion
Interface Node
Interface Node
60 Disks
Interface Node
Interface Node
60 Disks
• Up to 30 storage pods, each with up to 240 HDD’s
Interface Node
Interface Node • Up to 2 storage pods per storage expansion rack
60 Disks
Interface Node
Interface Node
• Scalable to 14.4 PB of raw storage in single system
60 Disks
Interface Node using 2 TB drives
Interface node Storage Expansion • Max of 7,200 hard drives
Expansion Rack
Rack

Figure 3-11 SONAS scale out expansion options

Using these expansion racks, you can build a SONAS system to the level of scalability that
you want. Figure 3-12 shows the mid-point of the SONAS expansion capability. The example
SONAS system has 3360 disks, out of the 7200 maximum possible.

IBM SONAS Scale Out Expansion Options

Switches Switches Switches Switches Switches Switches Switches Switches Switches

Interface Node Storage Node Storage Node Storage Node Stora ge Node Storage Node Stora ge Node Storage Node
Interface Node Mgmt Node Storage Node Storage Node Storage Node Stora ge Node Storage Node Stora ge Node Storage Node
Interface Node
60 Disk s 60 Disks 60 Disks 60 Disks 60 D isks 60 Disks 6 0 Disks
Interface Node Infiniband swi tch
Interface Node
60 Disk s 60 Disks 60 Disks 60 Disks 60 D isks 60 Disks 6 0 Disks
Interface Node
Infiniband swi tch
Interface Node
60 Disk s 60 Disks 60 Disks 60 Disks 60 D isks 60 Disks 6 0 Disks
Interface Node
Interface Node
60 Disk s 60 Disks 60 Disks 60 Disks 60 D isks 60 Disks 6 0 Disks
Interface Node Monitor/KVM
Interface Node Interface Node Storage Node Storage Node Storage Node Storage Node Storage Node Storage Node Storage Node
Interface Node Interface Node Storage Node Storage Node Storage Node Storage Node Storage Node Storage Node Storage Node
Interface Node Interface Node
60 Disk s 60 Disks 60 Disks 60 Disks 60 D isks 60 Disks 6 0 Disks
Interface Node Interface Node
Interface Node Interface Node
60 Disk s 60 Disks 60 Disks 60 Disks 60 D isks 60 Disks 6 0 Disks
Interface Node Interface Node
Interface Node Interface Node
60 Disk s 60 Disks 60 Disks 60 Disks 60 D isks 60 Disks 6 0 Disks
Interface Node Interface Node
Interface Node Interface Node
60 Disk s 60 Disks 60 Disks 60 Disks 60 D isks 60 Disks 6 0 Disks
Interface Node Interface Node

Example SONAS with 30 interface nodes, 7 disk expansion racks, 3360 disks

Figure 3-12 SONAS at the midpoint of its storage scalability

Chapter 3. SONAS architecture 57

 The diagram in Figure 3-13 shows a maximum SONAS configuration.

IBM SONAS Scale Out Expansion Options

Sw ti ch es
S w ti ch es S w ti c hes S w i t che s S wic
t he s S w i t che s S wic
t he s S w i t che s S w i t che s SwSw
i t iche
t chses Sw i t che s Sw i t che s Sw i t ch es Sw i t che s Sw i t ch es Sw i t ch es S w ti c hes

In e
t r fa ce N od e S t orag e N ode St o a
r ge N od e S t or a ge N od e St o a
r ge N od e S t or a ge N od e St o a
r ge N od e St o a
r ge N od e S tS
or to
a ge
r agNeod
Noede S to r age No de St o r age N o de S to rag e No de St o r age N o de S to r ag e No de S to rage No de S t or a ge N ode

S t orag e N ode St o a
r ge N od e S t or a ge N od e St o a
r ge N od e S t or a ge N od e St o a
r ge N od e St o a
r ge N od e S tS
or to
a ge
r agNeod
Noede S to r age No de St o r age N o de S to rag e No de St o r age N o de S to r ag e No de S to rage No de S t or a ge N ode
In e
t r fa ce N od e M g m t No de

In e
t r fa ce N od e
60 D i sks 60 D i sks 60 D i sks 60 Di sk s 60 D i sks 60 Di sk s 60 Di sk s 6 0
6 0
Di D
skissks 6 0 Di sk s 6 0 Di sk s 60 D i sks 6 0 Di sk s 6 0 D isk s 6 0 D isk s 60 D i sks
In e
t r fa ce N od e I n if n b
i an d sw i c
t h

In e
t r fa ce N od e
60 D i sks 60 D i sks 60 D i sks 60 Di sk s 60 D i sks 60 Di sk s 60 Di sk s 6 0
6 0
Di D
skissks 6 0 Di sk s 6 0 Di sk s 60 D i sks 6 0 Di sk s 6 0 D isk s 6 0 D isk s 60 D i sks
In e
t r fa ce N od e

I n if n b
i an d sw i c
t h
In e
t r fa ce N od e
60 D i sks 60 D i sks 60 D i sks 60 Di sk s 60 D i sks 60 Di sk s 60 Di sk s 6 0
6 0
Di D
skissks 6 0 Di sk s 6 0 Di sk s 60 D i sks 6 0 Di sk s 6 0 D isk s 6 0 D isk s 60 D i sks
In e
t r fa ce N od e

In e
t r fa ce N od e
60 D i sks 60 D i sks 60 D i sks 60 Di sk s 60 D i sks 60 Di sk s 60 Di sk s 6 0
6 0
Di D
skissks 6 0 Di sk s 6 0 Di sk s 60 D i sks 6 0 Di sk s 6 0 D isk s 6 0 D isk s 60 D i sks
In e
t r fa ce N od e M o ni t or / K VM

In e
t r fa ce N od e In e
t r fa ce N od e So
t r ag e No de S t ora ge N ode S t ora ge N ode S t or a ge N od e S t ora ge N ode S t or a ge N od e S t or a ge N ode S t St
or a
o ge
a
r ge
N od
N od
e e St o a
r ge N od e St o a
r ge N od e S to r age No de St o a
r ge N od e St o a
r ge N od e St o a
r ge N od e S to rag e No de

So
t r ag e No de S t ora ge N ode S t ora ge N ode S t or a ge N od e S t ora ge N ode S t or a ge N od e S t or a ge N ode S t St
or a
o ge
a
r ge
N od
N od
e e St o a
r ge N od e St o a
r ge N od e S to r age No de St o a
r ge N od e St o a
r ge N od e St o a
r ge N od e S to rag e No de
In e
t r fa ce N od e In e
t r fa ce N od e

In e
t r fa ce N od e
In e
t r fa ce N od e
60 D i sks 60 D i sks 60 D i sks 60 Di sk s 60 D i sks 60 Di sk s 60 Di sk s 6 0
6 0
Di D
skissks 6 0 Di sk s 6 0 Di sk s 60 D i sks 6 0 Di sk s 6 0 D isk s 6 0 D isk s 60 D i sks
In e
t r fa ce N od e
In e
t r fa ce N od e

In e
t r fa ce N od e In e
t r fa ce N od e
60 D i sks 60 D i sks 60 D i sks 60 Di sk s 60 D i sks 60 Di sk s 60 Di sk s 6 0
6 0
Di D
skissks 6 0 Di sk s 6 0 Di sk s 60 D i sks 6 0 Di sk s 6 0 D isk s 6 0 D isk s 60 D i sks
In e
t r fa ce N od e In e
t r fa ce N od e

In e
t r fa ce N od e In e
t r fa ce N od e

60 D i sks 60 D i sks 60 D i sks 60 Di sk s 60 D i sks 60 Di sk s 60 Di sk s 6 0

6 0
Di D
skissks 6 0 Di sk s 6 0 Di sk s 60 D i sks 6 0 Di sk s 6 0 D isk s 6 0 D isk s 60 D i sks
In e
t r fa ce N od e
In e
t r fa ce N od e

In e
t r fa ce N od e
In e
t r fa ce N od e
60 D i sks 60 D i sks 60 D i sks 60 Di sk s 60 D i sks 60 Di sk s 60 Di sk s 6 0
6 0
Di D
skissks 6 0 Di sk s 6 0 Di sk s 60 D i sks 6 0 Di sk s 6 0 D isk s 6 0 D isk s 60 D i sks
In e
t r fa ce N od e In e
t r fa ce N od e

Maximum SONAS with 30 interface nodes, 15 disk expansion racks, 7200 disks
Figure 3-13 SONAS maximum configuration

As we can see, the SONAS hardware architecture provides the foundation for petabyte level
scale out storage.

3.2 Software architecture

In this section, we examine the SONAS Software, which provides the ability for the SONAS
system to appear as a single global virtual file server, providing high performance and
centrally managed, automated tiered storage management across this entire large global
virtual file server. We first review the concepts of the SONAS Software, providing an overview
of the SONAS Software functionality stack, file access protocols, SONAS Cluster Manager,
parallel file system, central policy engine, scan engine, automatic tiered storage, workload
allocation, availability, administration, snapshots, asynchronous replication, and system
management services.

The functionality of IBM SONAS is provided by the IBM SONAS Software licensed program
product (5639-SN1). Each node of the IBM SONAS system is licensed and pre-installed with
one copy of SONAS Software. According to the role of the node (interface node, storage
node, or management node), the appropriate functions are called upon out of the common
software code load. Each node and each copy of SONAS Software together in a parallel grid
cluster architecture, working in parallel to provide the functions of SONAS.

58 IBM Scale Out Network Attached Storage Concepts

SONAS Software provides multiple elements and integrated components that work together
in a coordinated manner to provide the functions shown in the diagram in Figure 3-14.

CIFS NFS FTP HTTPS future

SONAS Cluster Manager

HSM and ILM

Parallel Monitoring Agents
File System
Backup & Restore GUI/CLI mgmt
Policy Engine Interfaces
Snapshots and Scan Engine
Security
Replication

Enterprise Linux

IBM Servers

Figure 3-14 SONAS Software functional components

In this section we overview each of the following components:

򐂰 SONAS data access layer: File access protocols supported (CIFS, NFS, FTP, HTTPS)
򐂰 SONAS Cluster Manager: Workload allocation and high availability
򐂰 SONAS data repository layer: parallel clustered file system
򐂰 SONAS data management services:
– Automated Hierarchical Storage Management (HSM) and Information Lifecycle
Management (ILM)
– Backup and restore data protection and Hierarchical Storage Management, using
integration with Tivoli Storage Manager,
– Snapshot™ for local data resiliency
– Remote async replication for remote recovery
򐂰 SONAS system management services:
– GUI, Health Center, CLI, and management interfaces
– Monitoring agents, security, and access control lists

Chapter 3. SONAS architecture 59

3.3 SONAS data access layer: File access protocols
We begin by examining the SONAS data access layer, and the supported file access
protocols that are supported today, as shown in Figure 3-15.

Network
Storage
Users

Ethernet

CIFS NFS FTP HTTPS future

SONAS Cluster Manager

HSM and ILM Monitoring Agents

Parallel
File System
Backup & Restore GUI/CLI mgmt
Policy Engine Interfaces

Snapshots and Scan Engine

Security
Replication

Enterprise Linux

IBM Servers

Figure 3-15 SONAS Software - file access protocols

The file access protocols that are supported by SONAS today are CIFS, NFS, FTP, and
HTTPS. Following this section, we then discuss the role the SONAS Cluster Manager plays in
concurrent access to a file from multiple platforms (that is, concurrently access a file from
both CIFS and NFS, for example).

3.3.1 File export protocols: CIFS

SONAS provides Common Internet File System (CIFS) support for data access from CIFS
clients. These typically are Windows clients, although they can also be any operating system
using CIFS, such as Linux with SMBClient or Mac OS X.

The base SONAS file system is a POSIX-compliant UNIX/Linux-style file system. SONAS
communicates with Microsoft Windows clients by emulating Windows file system behavior
over this POSIX-compliant SONAS file system. The SONAS Cluster Manager (see “SONAS
Cluster Manager” on page 63) provides the mapping of the CIFS semantics onto the
UNIX/Linux-based SONAS file system. The SONAS Cluster Manager also maps UNIX/Linux
Access Control Lists (ACLs) to Windows security semantics.

60 IBM Scale Out Network Attached Storage Concepts

A multitude of appropriate concurrency and cross-platform mapping functions are done by the
SONAS Software Cluster Manager. This support includes the following capabilities to allow
Windows users to interact transparently with the SONAS file system:
򐂰 The full CIFS data access and transfer capabilities are supported with normal locking
semantics.
򐂰 User authentication is provided through Microsoft Active Directory or through LDAP
򐂰 NTFS Access Control Lists (ACLs) are enforced on files and directories; they can be
modified using the standard Windows tools
򐂰 Semi transparent failover, if the CIFS application supports network retry;
for more information, see 3.4.3, “Interface node failover and failback” on page 68.
򐂰 Consistent central Access Control Lists (ACLs) enforcement across all platforms:
ACLs are enforced on files and directories, and they are modified (with proper authority
and ownership) using the standard Windows tools
򐂰 Share Modes:
Supports the win32 share modes for opening and creating files
򐂰 Support for case insensitive file lookup.
򐂰 Support for DOS attributes on files and directories
򐂰 Archive bit, ReadOnly bit, System bit, other semantics not requiring POSIX file attributes
򐂰 MS-DOS / 16 bit Windows short file names
򐂰 Supports generation of 8.3 character file names
򐂰 Notification support of changes to file semantics to all clients in session with the file
򐂰 Provides consistent locking across multiple other platforms, including NFS, FTP, HTTPS
Supports mandatory locking mechanisms and strict locking
򐂰 Opportunistic locks / leases
Supports lease management for enabling client side caching
򐂰 Off-line / de-staged file support
– Windows files that have been de-staged to external tape storage (using the SONAS
Hierarchical Storage Management function through Tivoli Storage Manager) will be
displayed as “off-line” within the Windows Explorer, marked with an hourglass symbol
(off-line bit).
– Users (and applications) can see in advance that a file is off-line
– Recall to disk is transparent, so no additional operation beside the file open is needed.
– Directory browsing using the Windows Explorer supports file property display without
the need to recall off-line / migrated files
򐂰 Support of SONAS Snapshot integrated into the Windows Explorer Volume Shadow
Services (VSS) interface, allowing users with proper authority to recall files from SONAS
Snapshots. This file version history support is for versions created by SONAS Snapshots.
򐂰 The standard CIFS timestamps are made available:
– Created Time stamp: Indicates the time when the file is created in the current directory.
When the file is copied to a new directory, a new value will be set.
– Modified Time stamp: Indicates the time when the file is last modified. When the file is
copied to elsewhere, the same value will be carried over to the new directory.

Chapter 3. SONAS architecture 61

 – Accessed Time stamp: Indicates the time when the file is last accessed. This value is
set by the application program that sets or revises the value (this is application
dependent; unfortunately certain applications do not revise this value).

CIFS locks set by Windows clients are stored both in the SONAS Cluster Manager interface
node cluster-wide database, and by mapping to POSIX locks in the SONAS file system. This
mapping ensures that NFS clients see relevant CIFS locks as POSIX advisory locks, and
NFS clients honor these locks.

3.3.2 File export protocols: NFS

NFSv2 and NFSv3 are supported by SONAS. NFSv4 is not currently supported by SONAS;
this is a known requirement. The following characteristics apply to the NFS export:
򐂰 Normal NFS data access functions are fully supported with NFS consistency guarantees
򐂰 Authorization / ACL support
򐂰 Supports client machine authorization through NFS host lists
򐂰 Supports enforcement of Access Control Lists (ACLs). Supports reading and writing of the
standard NFSv3 / POSIX bits
򐂰 Supports the NFSv3 advisory locking mechanism
򐂰 Semi transparent node failover (application must support network retry);
for more information, see 3.4.3, “Interface node failover and failback” on page 68.

Note that the SONAS Software file system implements NFSv4 Access Control Lists (ACLs)
for security, regardless of the actual network storage protocol used. This provides the
strength of the NFSv4 ACLs even to clients that access SONAS by the NFSv2, NFSv3, CIFS,
FTP, and HTTPS protocols.

3.3.3 File export protocols: FTP

SONAS provides FTP support from any program supporting the FTP protocol. The following
characteristics apply:
򐂰 Data transfer to/from any standard FTP client.
򐂰 Supports user authentication through Microsoft Active Directory and through LDAP.
򐂰 Supports enforcement of Access Control Lists (ACLs). Supports retrieval of POSIX
attributes. ACLs can not be modified using FTP (note that there is no support for the chmod
command).
򐂰 On node failover, SONAS supports FTP resume (application must support network retry).
򐂰 Characters for file names and directory names are UTF 8 encoded.

3.3.4 File export protocols: HTTPS

SONAS supports simple read only data access to files through the HTTPS protocol from any
web browser. The following features are supported through this protocol:
򐂰 Read access to appropriately formatted files.
򐂰 Supports enforcement of Access Control Lists (ACLs). ACLs cannot be modified or viewed
using this protocol.
򐂰 Supports user authentication through Microsoft Active Directory or LDAP.

62 IBM Scale Out Network Attached Storage Concepts

 򐂰 On node failure during a file transfer, the transfer is cancelled and must be retried at
another node. Partial retrieve is supported, which minimizes duplicate transfers in a
failover situation.
򐂰 Characters for file names and directory names are UTF 8 encoded.

The Apache daemon provides HTTPS access to the SONAS file system. SONAS supports
secure access only, so that the credentials will always be SSL encrypted. SONAS uses HTTP
aliases as a vehicle to emulate the share concept. For example, share XYZ will be accessible
by https://server.domain/XYZ.

Note that WebDAV1 and the REST2 API are not supported at this time in SONAS; they are
known requirements.

3.4 SONAS Cluster Manager

Next, we examine the SONAS Cluster Manager, as diagrammed in Figure 3-16.

CIFS NFS FTP HTTPS future

SONAS Cluster Manager

HSM and ILM Monitoring Agents

Parallel
File System
Backup & Restore GUI/CLI mgmt
Policy Engine Interfaces

Snapshots and Scan Engine

Security
Replication

Enterprise Linux

IBM Servers

Figure 3-16 SONAS Cluster Manager

1
WebDAV is “Web-based Distributed Authoring and Versioning”
2 REST API is “Representational State Transfer”

Chapter 3. SONAS architecture 63

 The SONAS Cluster Manager has the following responsibilities:
1. It provides file serving functions for CIFS, NFS, FTP, and HTTPS access, and provides the
mapping of the various network storage access protocols onto the SONAS parallel file
system. The CIFS file serving function maps of CIFS semantics and security onto the
POSIX-based parallel file system and NFSv4 Access Control Lists.
2. It provides the clustered implementation and management of the interface nodes,
including tracking and distributing record updates across the interface nodes in the cluster.
3. It provides control of the interface nodes in the cluster. SONAS Cluster Manager controls
the public IP addresses used to publish the NAS services, and moves them as necessary
between nodes. By monitoring scripts, SONAS Cluster Manager monitors and determines
the health state of each individual interface node. If an interface node becomes unhealthy,
SONAS Cluster Manager will dynamically migrate affected public IP addresses and
in-flight workloads to healthy interface nodes, and use “tickle-ack” technology with the
affected user clients, so that they reestablish connection to their new interface node.

The SONAS Cluster Manager also provides advanced functions such as byte-range capable
locking available in the SONAS parallel file system; managing the interface nodes and
multiple protocols to work in conjunction with the SONAS parallel file system base technology
for concurrent access; and parallel read and write access, for multiple protocols and multiple
platforms, across multiple SONAS interface nodes. It is able to do all of these tasks with full
data integrity to all files, anywhere within the file system.

3.4.1 SONAS Cluster Manager and interface node workload allocation

In this section we discuss how interface node workload allocation is done among the multiple
interface nodes, and the role played within that by the SONAS Cluster Manager.

SONAS global cluster for workload allocation and high availability

IBM SONAS provides high availability through a sophisticated implementation of a global
cluster of active-active peer nodes. Each SONAS node is a peer to all other SONAS nodes,
each SONAS node is in an active-active relationship with all other SONAS nodes, and
incoming workload is evenly distributed among all SONAS nodes of the same type. If a node
fails, the SONAS Software will automatically fail over the workload to another healthy node of
the same type. We consider the following operations:
򐂰 The operational characteristics and types of SONAS nodes in the global cluster
򐂰 Clustered node failover/failback for both CIFS and NFS
򐂰 Dynamic insertion/deletion of nodes into the cluster

64 IBM Scale Out Network Attached Storage Concepts

Let us start by reviewing the SONAS architecture, as depicted in Figure 3-17.

HTTP
HTTP NFS
NFS CIFS
CIFS FTP
FTP Other
Other
Clients
Clients Clients
Clients Clients
Clients Clients
Clients Clients
Clients

IP Network

Global Namespace

Management
Management
Node
Node
Interface
Interface
Node
Node ... Interface
Interface
Node
Node .... Interface
Interface
Node
Node

IP Mgmt. Network Infiniband Data Network Tape

Storage Pod Storage Pod

Storage
Storage Node
Node Storage
Storage Node
Node Storage
Storage Node
Node Storage
Storage Node
Node

Storage
controller &
disk
Storage
controller &
disk
... Storage
controller &
disk
Storage
controller &
disk

Storage Storage Storage Storage

Expansion Expansion Expansion Expansion

Figure 3-17 SONAS architecture

Node types
There are three types of nodes in a SONAS system. The nodes are divided and configured
according to one of three roles. All nodes are in a global cluster, and a copy of SONAS
Software runs on each of the nodes. A node performs only one of the following three roles:
򐂰 Interface node: This node provides the connections to the customer IP network for file
serving. These nodes establish and maintain the connections to CIFS, NFS, FTP, or HTTP
users, and serve the file requests. All four of these protocols can and do co-exist on the
same interface node. Each interface node can talk to any of the storage nodes.
򐂰 Storage node: This node acts as a storage servers, and reads and writes data to and from
the actual storage controllers and disks. Each storage node can talk to any of the interface
nodes. A storage node serves file and data requests from any requesting interface node.
SONAS Software writes data in a wide stripe across multiple disk drives in a logical
storage pool. If the logical storage pool is spans multiple storage nodes and storage pods,
the data striping will also span storage nodes and storage pods.
򐂰 Management node: This node monitors and manages the SONAS global cluster of nodes,
and provides Command Line Interface management and GUI interface for administration.
Command Line Interface commands come into the SONAS system through the
Management node.

Notice that SONAS is a two-level architecture, there are multiple clustered interface nodes,
and there are multiple clustered storage nodes. This is an important aspect of the design, as
it allows independent scalability of interface nodes (user throughput) from storage pods and
storage nodes (storage capacity).

Chapter 3. SONAS architecture 65

 Each SONAS node is an IBM System x® commercial enterprise class 2U server, and each
node runs a copy of IBM SONAS Software licensed program product (5639-SN1). SONAS
Software manages the global cluster of nodes, provides clustered auto-failover, and provides
the following functions:
򐂰 The IBM SONAS Software manages and coordinates each of these nodes running in a
peer-peer global cluster, sharing workload equitably, striping data, running the central
policy engine, and performing automated tiered storage.
򐂰 The cluster of SONAS nodes is an all-active clustered design, based upon proven
technology derived from the IBM General Parallel File System (GPFS).
򐂰 All interface nodes are active and serving file requests from the network, and passing
them to the appropriate storage nodes. Any interface node can talk to any storage node.
򐂰 All storage nodes are active and serving file and data requests from any and all interface
nodes. Any storage node can respond to a request from any interface node. SONAS
Software will stripe data across disks, storage RAID controllers, and storage pods.
򐂰 SONAS Software also coordinates automatic node failover and failback if necessary.
򐂰 From a maintenance or failover / failback standpoint, any node can be dynamically deleted
or inserted into the global cluster. Upgrades or maintenance can be performed by taking a
node out of the cluster, upgrading it if necessary, and re-inserting it into the cluster. This is
a normal mode of operation for SONAS, and this is the manner in which rolling upgrades
of software and firmware are performed.

SONAS Software is designed with the understanding that over time, various generations and
speeds of System x servers will be used in the global SONAS cluster. SONAS Software
understands this and is able to distribute workload equitably, among various speed interface
nodes and storage nodes within the cluster.

3.4.2 Principles of SONAS workload allocation to interface nodes

In order to cluster SONAS interface nodes so that they can serve the same data, the interface
nodes must coordinate their locking and recovery. This coordination is done through the
SONAS Cluster Manager. It is the SONAS Cluster Manager’s role to manage all aspects of
the SONAS interface nodes in the cluster.

Clusters usually cannot outperform a stand-alone server to a single client, due to cluster
overhead. At the same time, clusters can outperform stand-alone servers in aggregate
throughput to many clients, and clusters can provide superior high availability.

SONAS is a hybrid design that provides the best of both of these approaches.

66 IBM Scale Out Network Attached Storage Concepts

From an incoming workload allocation standpoint, SONAS uses the Domain Name Server
(DNS) to perform round-robin IP address balancing, to spread workload equitably on an IP
address basis across the interface nodes, as shown in Figure 3-18.

SONAS.virtual.com SONAS.virtual.com

Client I Client II Client n

DNS Server
(name resolution)

SONAS.virtual.com
10.0.0.10
10.0.0.11
10.0.0.10 10.0.0.12 10.0.0.14 10.0.0.12
10.0.0.11 10.0.0.13 10.0.0.15 10.0.0.13
10.0.0.14
10.0.0.15

Figure 3-18 SONAS interface node workload allocation

SONAS allocates a single user network client to a single interface node, to minimize cluster
overhead. SONAS Software does not rotate a single client’s workload across interface nodes.
That is not only unsupported by DNS or CIFS, but can also decrease performance, because
caching and read-ahead is done in the SONAS interface node, and it is for this reason that
any one individual client is going to be assigned, for the duration of their session, to one
interface node at the time they authenticate and access the SONAS.

At the same time, the workload from multiple users (numbering into the thousands or more)
is equitably spread across as many SONAS interface nodes as are available. If more user
network capacity is required, you simply add more interface nodes. SONAS scale out
architecture thus provides linear scalability as the numbers of users grow.

Agnostically to the application or the interface nodes, SONAS Software will always stripe data
across disks, storage RAID controllers, and storage pods, thus providing wide data striping
performance and parallelism to any file serving requests, by any interface node. This is
shown in Figure 3-19.

Chapter 3. SONAS architecture 67

 Single
connection
write file write file for ease of attach

Parallelism
for high
performance

agnostic to
application

Figure 3-19 SONAS interface node workload allocation - parallelism at storage level

SONAS provides a single high performance NFS, CIFS, FTP, or HTTPS connection for any
one individual network clients. In aggregate, multiple users are IP-balanced equitably across
all the interface nodes, thus providing scale out capability; the more interface nodes, the more
user capacity that is available.

SONAS was designed to make the connection a standard CIFS, NFS, FTP, or HTTP
connection, in order to allow attachability by as wide a range of standard clients as possible,
and to make unnecessary the installation of any client side code.

3.4.3 Interface node failover and failback

In the event that the redundancy within an interface node might fail, if there is a fatal error
(or if the interface nodes simply need to be upgraded or maintained), interface nodes can be
dynamically removed from and later re-inserted into the SONAS cluster. The normal method
of upgrade, or repair of an interface node, is to take the interface node out of the cluster. The
SONAS Cluster Manager also manages the failover of the workload to the remaining healthy
interface nodes in the SONAS cluster workload. The offline interface node can then be
upgraded or repaired, then re-inserted into the SONAS cluster, and the workload will then be
automatically rebalanced across the interface nodes in the SONAS.

When an interface node is removed from the cluster, or if there is an interface node failure,
healthy interface nodes take over the load of the failed node, and the SONAS Software
Cluster Manager will automatically perform the following actions:
򐂰 Terminate old network connections and move the network connections to a healthy
interface node. IP addresses are automatically re-allocated to a healthy interface node.
– Session and state information that was kept in the Cluster Manager is used to support
re-establishment of the session and maintaining IP addresses and ports.

68 IBM Scale Out Network Attached Storage Concepts

 – This state and session information and metadata for each user and connection is
stored in memory in each node in a high performance clustered design, along with
appropriate shared locking and any byte-range locking requests, as well as other
information needed to maintain cross-platform coherency between CIFS, NFS, FTP,
and HTTP users.
򐂰 Notification technologies are used to “tickle” the application and cause a reset of the
network connection.

This process is illustrated in Figure 3-20.

SONAS.virtual.com SONAS.virtual.com

Client I Client II Client n

DNS Server
(name resolution)

SONAS.virtual.com
10.0.0.10
10.0.0.13 10.0.0.12 10.0.0.11
10.0.0.10 10.0.0.14 10.0.0.12
10.0.0.11 10.0.0.15 10.0.0.13
10.0.0.14
10.0.0.15

Figure 3-20 SONAS interface node failover

At the time of the failover of the node, if the session or application is not actively in a
connection transferring data, the failover can usually be transparent to the client. If the client
is transferring data, depending on the protocol and application, the application service failover
might be transparent to the client, depending on nature of the application, and depending on
what is occurring at the time of the failover.

In particular, if the client application, in response to the SONAS failover and SONAS
notifications, automatically does a retry of the network connection, then it is possible that the
user will not see an interruption of service. Examples of software that does this can include
many NFS-based applications, as well as Windows applications that do retries of the network
connection, such as the Windows XCOPY utility.

If the application does not do automatic network connection retries, or the protocol in question
is stateful (for example, CIFS), then a client side reconnection might be necessary to
re-establish the session. Unfortunately for most CIFS connections, this will be the likely case.

Chapter 3. SONAS architecture 69

3.4.4 Storage node failover and failback
While we are discussing interface node failover and failback in SONAS, let us mention that a
similar principle is operational for storage node failover and failback. The SONAS Cluster
Manager does not directly participate in storage node failover and failback (the SONAS
parallel file system manages that).

If a storage node fails, the remaining healthy storage node in the storage pod takes over the
load of the failed storage node. An individual storage node is very high in capacity and
throughput, to allow good operation in the event of a failed storage node.

Logical storage pools in SONAS typically are defined as spanning disks, storage RAID
controllers, and storage pods. Furthermore, the data striping means that files are spread in
blocksize “chunks” across these components in order to achieve parallel performance and
equalized utilization of storage hardware.

One of the purposes of this dispersion of SONAS data is to mitigate the effect of a failed
storage node. Files in SONAS are spread across multiple storage nodes and storage pods,
with the intent that only a small portion of any file is affected by a storage node failure.

Just as with interface nodes, storage nodes can be dynamically removed and re-inserted into
a cluster. Similar to the interface node methodology, which is the same, the method of
upgrade or repair of a storage node is to take the storage node out of the cluster. The
remaining storage node in the storage pod dynamically assumes the workload of the pod.
The offline storage node can then be upgraded or repaired, and then re-inserted into the
cluster. When this is done, workload will then be automatically rebalanced across the storage
nodes in the storage pod.

During all of these actions, the file system and file access stays online and available to the
users.

3.4.5 Summary
We have seen that the IBM SONAS provides equitable workload allocation to a global cluster
of interface nodes, including high availability through clustered auto-failover:
򐂰 All SONAS nodes operate in a global cluster.
򐂰 Workload allocation to the interface nodes is done in conjunction with external Domain
Name Servers.
򐂰 The global SONAS cluster offers dynamic failover/failback, and if the application supports
network connection retries, can provide transparent failover of the interface nodes.
򐂰 Normal upgrade and maintenance for SONAS nodes is by dynamic removal and insertion
of nodes into and out of the cluster.

Let us now discuss in more detail the SONAS Software components that provide the
functionality to support these capabilities.

3.4.6 SONAS Cluster Manager manages multi-platform concurrent file access

One of the primary functions of the SONAS Cluster Manager is to support concurrent access
from concurrent NAS users, spread across multiple network protocols and platforms to many
files. SONAS Software also supports, with proper authority, concurrent read and write access
to the same file, including byte-range locking.

70 IBM Scale Out Network Attached Storage Concepts

In Figure 3-21, we see that all file accesses from the users to the SONAS parallel file system
will logically traverse the SONAS Cluster Manager.

CIFS NFS FTP HTTP

SONAS Cluster
Manager

concurrent_access_to_a_file

SONAS File System

uses NFSv4 Access Control Lists

Figure 3-21 All file accesses traverse the SONAS Cluster Manager including concurrent accesses

The SONAS Cluster Manager is logically positioned in the file access path-length to provide
the mapping of the multiple protocols onto the SONAS parallel file system, and to manage the
necessary locking and data integrity across all interface nodes. The SONAS Cluster Manager
works together with the SONAS parallel file system to provide concurrent file access from
multiple platforms in the following way:
򐂰 SONAS Cluster Manager: Provides the mapping and concurrent access control across
multiple interface nodes, and when multiple protocols access the same file, and provides
locking across users across all interface nodes.
򐂰 SONAS parallel file system: Provides the file system concurrent access control at the level
of the physical file management, provides ability to manage and perform parallel access,
provides the NFSv4 access control list security, and provides the foundational file system
data integrity capabilities.

SONAS Software Cluster Manager functionality supports multiple exports or shares of the file
system, over multiple interface nodes, by providing distributed lock, share, and lease support.
An individual interface node can simultaneously be serving CIFS, NFS, FTP, and HTTPS;
the multiple file access protocols are not limited or exclusive to a particular interface node.
The SONAS Cluster Manager is transparent to the NFS, CIFS, FTP, and HTTP clients; these
clients do not need to know that the SONAS Cluster Manager is servicing and managing
multiple protocols concurrently.

The base SONAS file system is a full POSIX-compliant file system, a UNIX/Linux-style file
system. SONAS communicates with Microsoft Windows clients by emulating Windows file
system behavior over this POSIX-compliant SONAS file system.

Chapter 3. SONAS architecture 71

 When sharing files and directories, SONAS reflects changes made by one authorized user, to
all other users that are sharing the same files and directories. As an example, if a SONAS
resident file is renamed, changed, or deleted, other users who are affected by this fact will
immediately be reflected properly to all SONAS-attached clients on other platforms, including
those using other protocols, as shown in Figure 3-22.

delete SONAS User1 SONAS User2

files

01:00
12
9 3
6

01:01

12
9 3
6 Other
users see
files are
deleted

Figure 3-22 SONAS concurrent access to a shared directory from multiple users

SONAS Software employs sophisticated distributed cluster management, metadata

management, and a scalable token management system to provides data consistency while
supporting concurrent file access from thousands of users. All read and write locking types
are kept completely coherent between NFS and CIFS clients, globally, across the cluster.

3.4.7 Distributed metadata manager for concurrent access and locking

In order to assure data consistency, SONAS Software provides both a sophisticated
multi-platform interface node locking capability that works in conjunction with a sophisticated
token (lock) management capability in the file system that is derived from the IBM General
Parallel File System. This capability coordinates a shared-everything global access from any
and all interface nodes, to any and all disk storage, thus assuring the consistency of file
system data and metadata when various nodes access the same file.

72 IBM Scale Out Network Attached Storage Concepts

 SONAS Software is designed to provide a flexible, multi-platform environment (Figure 3-23).

Access files from multiple platforms – in parallel

/home Windows/home/appl/data/web/writing_reading_the_file.dat

Logical
/app l
Unix-Linux/home/appl/data/web/writing_reading_the_file.dat
/data
Any/home/appl/data/web/writing_reading_the_file.dat
/web

CIFS NFS
IBM Scale Out NAS SONAS file system
provides ability multiple
concurrent readers/writers
Global Namespace from multiple platforms
Policy Engine

Inter fa ce
node s … Int er fac e
nodes … Inter face
nodes …> scale
out

Physica l
Stor age
node s
….. Stor age
node s
….. S tora ge
nodes
... > scale
out

Tier 1 Tier 2 Tier 3

Figure 3-23 SONAS Software and file system provides advanced concurrent access and locking

SONAS Software has multiple facilities to provides scalability. These include the distributed
ability for multiple nodes to act as token managers for a single file system. SONAS Software
also provides scalable metadata management by providing for a distributed metadata
management architecture, thus allowing all nodes of the cluster to dynamically share in
performing file metadata operations while accessing the file system.

This capability distinguishes SONAS from other cluster NAS filer architectures, which might
have a centralized metadata server handling fixed regions of the file namespace. A
centralized metadata server can often become a performance bottleneck for metadata
intensive operations and can represent a scalability limitation and single point of failure.

SONAS solves this problem by managing metadata at the node that is using the file, or in the
case of parallel access to the file, at a dynamically selected node which is using the file.

3.4.8 SONAS Cluster Manager summary

In summary, the scale out NAS architecture of SONAS is served by the following functions
provided by the SONAS Cluster Manager:
򐂰 All user clients are able to connect to any interface node, by DNS.
򐂰 All interface nodes appear to the user as a single large virtual global namespace NAS
server.
򐂰 Users are not aware that their high performance and scale is the result of a parallel file
filesystem, from which all interface nodes can serve out various files or the same set of
files, with parallel high performance.
򐂰 Provides full data integrity across all interface nodes and across all concurrent users and
applications.

Chapter 3. SONAS architecture 73

 򐂰 Interface nodes can fail and clients are transparently reconnected to another interface
node.
򐂰 All file changes are immediately seen on all interface nodes and all other clients accessing
the same file.
򐂰 Latency and cross-communication are minimized so that interface nodes efficiently can
check for proper file and data integrity, yet can scale in a linear, non-disruptive fashion by
simply adding more interface nodes or storage nodes.

The SONAS Cluster Manager provides these capabilities, and provides full active/active
interface node clustering, with the appearance of a single server global namespace instance
spanning all SONAS interface nodes. This clustering occurs transparently to all applications.

3.5 SONAS authentication

SONAS supports the following authentication methods:
򐂰 Microsoft Active Directory:
– Active Directory itself provides Kerberos Infrastructure)
– Active Directory with SFU (Services for UNIX, RFC2307 schema)
򐂰 LDAP (Lightweight Directory Access Protocol), including LDAP with MIT Kerberos
򐂰 Samba PDC / NT4 mode
򐂰 Samba Primary Domain Controller PDC / NT4 mode
򐂰 Network Information Service (NIS) with NFS NetGroup support only for ID mapping
– Note that in SONAS we do not support NIS as an authentication mechanism.
– We use NIS exclusively for netgroup support and ID mapping.

3.5.1 Considerations for authentication

At the current SONAS 1.1.1 release level, a single SONAS system supports only one of the
above authentication methods at a time. In order to access a SONAS system, the user must
be authenticated using the authentication method that is implemented on that particular
SONAS machine.

The authentication server is external to SONAS and must have a proper connectivity to
SONAS. The authentication server is configured externally to the SONAS, using normal
authentication server skills.

All of the SONAS nodes must have access to a network time protocol server that is
synchronized in time to be the same as an authentication server such as Active Directory
Server and/or Kerberos KDC server. An Active Directory domain controller can be used as a
NTP time source.

Only the SONAS interface nodes are configured for authentication by users. SONAS storage
nodes are not part of this configuration. In the current SONAS release, the number of groups
per user is limited to approximately 1000.

Note that the external authentication server is an essential component in the SONAS data
access flow. If the authentication server is unavailable, then the data access is not available.

74 IBM Scale Out Network Attached Storage Concepts

3.5.2 SONAS authentication: ID mapping
SONAS Software is designed to support multiple various platforms and protocols, accessing
the SONAS system concurrently.

To make both worlds work together in SONAS and provide full concurrent access from both
platforms, a mapping between Windows Security Identifiers (SIDs) and UNIX userids /
groupids (UID/GID) is performed.

When accessing SONAS data using UNIX or Linux systems (that is, through NFS), there are
no issues because the user UID/GID directly maps to the underlying SONAS file system UID
and GID.

However, when Windows clients access SONAS, the SONAS Cluster Manager provides the
mapping between the Windows user SID (Security identifier) and the internal file system UID
to identify users. Depending on the type of authentication used, various methods are applied
to solve this UID to SID mapping requirement.

The SONAS user ID mapping is depicted in Figure 3-24.

Windows AD

User- or Groupname  Microsoft Security ID (SID)

CIFS SONAS file system

uses Linux
User- / Groupname
SONAS file System (GPFS

UID / GID

NFS Shared
Id map db
UID / GID

SONAS maps Usernames and

Groups to
Unix User/group IDs
NFS provides UID / GID CLIENT consistently across
level only. This means the mapping all nodes
has to happen on NFS CLIENT level:

- Create user with correct ID’s manually Interface

- Use Microsoft AD Service for Unix (SFU) nodes

Figure 3-24 SONAS authentication - userid mapping

3.5.3 SONAS Active Directory authentication

In case of Microsoft Active Directory (AD), SONAS generates a UID for each SID, using
auto-increment logic. This means that if any new user accesses SONAS, then SONAS
creates a UID at runtime, and it is stored in SONAS in the SONAS Cluster Manager. This
information is kept in sync across all interface nodes.

Chapter 3. SONAS architecture 75

 Therefore, when using AD as authentication for SONAS, be aware that you have the ability to
create the user on the UNIX/Linux machine, which matches the UID created on SONAS.
Note that among the many ways this can be solved, you can also use Network Information
Services (NIS). Many installations keep their UNIX user information in NIS, which can be
used for ID mapping.

3.5.4 SONAS directory authentication with SFU

When using SONAS with AD Services for UNIX (SFU), SONAS uses AD SFU to read the SID
to UID mapping. In this case, when users access SONAS, the SONAS Software fetches this
SID to UID mapping from SFU.

3.5.5 SONAS with LDAP authentication

For SONAS authentication using UNIX/Linux Lightweight Directory Access Protocol (LDAP),
the SID to UID mapping is kept in LDAP itself, so this is typically a very straightforward
authentication environment. with few or no issues on ID mapping in LDAP.

3.6 Data repository layer: SONAS file system

In this section, we provide an overview of the internal architecture of the SONAS file system,
which is based upon the IBM General Parallel File System.

In the SONAS Software, the parallel file system, which includes the central policy engine and
the high performance scan engine, are at the heart of SONAS Software functionality
(Figure 3-25).

CIFS NFS FTP HTTPS future

SONAS Cluster Manager

HSM and ILM Monitoring Agents

Parallel
File System
Backup & Restore GUI/CLI mgmt
Policy Engine Interfaces

Snapshots and Scan Engine

Security
Replication

Enterprise Linux

IBM Servers

Figure 3-25 SONAS Software - parallel file system, policy engine, scan engine

76 IBM Scale Out Network Attached Storage Concepts

 We discuss core SONAS file system concepts, including the high-performance file system
itself, the manner in which the policy engine and scan engine provide the foundation for
SONAS Information Lifecycle Management (ILM), which is discussed in detail in “SONAS
data management services” on page 83, and characteristics of the SONAS file system for
configuration, performance, scalability, and storage management.

The SONAS file system is more than just a file system; it is the core foundation for an end to
end NAS file management infrastructure within SONAS. IBM utilizes IBM GPFS technology to
provide a proven high performance parallel grid file system architecture, with clustered high
reliability, high scalability.

In addition to providing file storage capabilities, the SONAS file system also provides storage
management and information life cycle management tools, centralized administration and
facilities that, in conjunction with the SONAS Cluster Manager, allows for shared high
performance access from multiple NAS protocols simultaneously.

IBM SONAS was designed to take advantage of the long history of IBM GPFS as a high
performance parallel file system, supporting many types of applications ranging from
relational databases, to digital media, to high performance analytics, to scalable file serving.

The core GPFS technology is installed today in across many industries including financial,
retail, and government applications. GPFS has been tested in very demanding large
environments for over 15 years, making GPFS a solid foundation for use within the SONAS
as the central parallel file system.

3.6.1 SONAS file system scalability and maximum sizes

The SONAS maximum file system size (in TB) for standard support is 2 PB. Larger PB file
systems are possible by submitting a request to IBM for support. The SONAS file system is
based upon IBM GPFS technology, which today runs on many of the world’s largest
supercomputers. The largest existing GPFS configurations run into the tens of thousands of
nodes. IBM GPFS has been available on IBM AIX since 1998, and on Linux since 2001. IBM
GPFS has been field proven time and again on the world's most powerful supercomputers to
provide efficient use of disk bandwidth, and it is this technology that is being packaged in a
SONAS form factor, manageable with standard NAS administrator skills, that is available in
SONAS.

SONAS utilizes the fact that GPFS was designed from the beginning to support extremely
large, extremely challenging high performance computing environments. Today, SONAS uses
that technology to supports building a single global namespace and a single file system over
the entire 14.4 PB current maximum size of a physical SONAS system.

The theoretical limit to the SONAS file system size is (2 to the 99th power) bytes. The
maximum SONAS file size is (2 to the 64th power) bytes. The current maximum SONAS
number of files per file system is (2 to the 31st power) -1 (approximately 2 billion). You can
have up to 256 file systems in a SONAS system. Architecturally, a SONAS file system or a
single SONAS file can be as large as the entire SONAS system.

3.6.2 Introduction to SONAS file system parallel clustered architecture

The SONAS file system is built upon a collection of disks that contain the file system data and
metadata. A file system can be built from a single disk or contain thousands of disks storing
petabytes of data.

Chapter 3. SONAS architecture 77

 SONAS implements its file system upon a grid parallel architecture, in which every “node”
runs a copy of SONAS Software and thus has a copy of the SONAS parallel file system code
running on it. SONAS implements a two-tier global cluster, with SONAS interface nodes as
the upper tier of network file serving nodes, and with SONAS storage nodes serving as the
lower tier, acting as storage serving nodes. SONAS file system code is present on all nodes.

On the interface nodes, the SONAS file system code serves as “file system storage
requesters.” On the storage nodes, the SONAS file system code serves as “file system
storage servers.” All SONAS nodes (interface, management, and storage) are a peer-to-peer
global cluster.

There is no limit placed upon the number of simultaneously opened files within a single file
system. SONAS utilizes the experience from worldwide current GPFS customers who are
using single file systems from 10 PB to 20 PB in size, and growing. Other GPFS user file
systems contain hundreds of millions of files.

3.6.3 SONAS file system performance and scalability

SONAS file system achieves high performance I/O in the following ways:
򐂰 Stripes data across multiple disks attached to multiple nodes:
– All data in the SONAS file system is read and written in wide parallel stripes.
– The blocksize for the file system determines the size of the block writes.
– The blocksize is specified at the time the file system is defined, is global across the file
system, and cannot be changed after the file system is defined
򐂰 Optimizes for small block writes; a SONAS block is also sub-divided into 32 sub-blocks, so
that multiple small block application writes can be aggregated and stored in a SONAS file
system block, without unnecessarily wasting space in the block
򐂰 Provides a high performance metadata (inode) scan engine, to scan the file system very
rapidly in order to enable fast identification of data that needs to managed or migrated in
the automated tiered storage environment
򐂰 Supports a large block size, configurable by the SONAS administrator, to fit I/O
requirements:
– Typical blocksizes are the default (256 KB) which is good for most workloads,
especially mixed small random and large sequential workloads.
– For large sequential workloads, the SONAS file system can optionally be defined with
blocksizes at 1 MB or 4 MB.
򐂰 Utilizes advanced algorithms that improve read-ahead and write-behind file functions for
caching in the interface node
򐂰 Uses block level locking based on a very sophisticated scalable token management
system to provide data consistency while allowing multiple application nodes concurrent
access to the files.

Let us see how SONAS scalability is achieved using the expandability of the SONAS building
block approach.

Figure 3-26 shows a SONAS interface node, performing a small single read or write to or from
a disk. Because this read or write is small in size, only the resources of one path, one storage
node, and one RAID controller / disk are sufficient to handle the I/O operation.

78 IBM Scale Out Network Attached Storage Concepts

 Interface
Interface Node
Node

InfiniBand network

Storage Pod
Storage
Storage Node
Node Storage
Storage Node
Node
(NSD
(NSD Server)
Server) (NSD
(NSD Server)
Server)

RAID RAID
Controller Controller

Raid
disk

Figure 3-26 A small single read or write in the SONAS file system

The power of the SONAS file system, however, is in its ability to read or write files in parallel
(in blocksize “chunks”), across multiple disks, controllers, and storage nodes (Figure 3-27).

Interface
Interface Node
Node

InfiniBand network

Storage Pod
Storage
Storage Node
Node Storage
Storage Node
Node
(NSD
(NSD Server)
Server) (NSD
(NSD Server)
Server)

RAID RAID RAID RAID

Controller Controller Controller Controller

Raid Raid Raid Raid Raid Raid Raid Raid Raid Raid Raid Raid
(NSD) (NSD) (NSD) (NSD) (NSD) (NSD) (NSD) (NSD) (NSD) (NSD) (NSD) (NSD)

Figure 3-27 A high parallel read or write in the SONAS file system; can be one file

Chapter 3. SONAS architecture 79

 The scalability of the SONAS file system does not stop with a single storage pod, in addition
to the very large parallel write capability of a single storage pod (Figure 3-28).

Interface
Interface Node
Node

InfiniBand network

Storage
Storage Storage
Storage
Node
Node Node
Node

High Density
Storage Array

High Density
Storage Array

Storage Pod

Figure 3-28 SONAS file system parallel read / write capability to one storage pod

If the file is big enough, or if the aggregate workload is big enough, the SONAS file system
easily expands to multiple storage pods in parallel (Figure 3-29).

Interface
Interface Node
Node

InfiniBand network

Storage
Storage Storage
Storage Storage
Storage Storage
Storage Storage
Storage Storage
Storage Storage
Storage Storage
Storage
Node
Node Node
Node Node
Node Node
Node Node
Node Node
Node Node
Node Node
Node

High Density High Density High Density High Density

Storage Array Storage Array Storage Array Storage Array

High Density High Density High Density High Density

Storage Array Storage Array Storage Array Storage Array

Storage Pod Storage Pod Storage Pod Storage Pod

Figure 3-29 SONAS file system parallel read/ write capability to multiple storage pods

80 IBM Scale Out Network Attached Storage Concepts

We can see that the SONAS file system provides the capability for extremely high parallel
performance. This is especially applicable to modern day analytics-intensive data types with
the associated large data objects and modern unstructured data.

The SONAS file system recognizes typical access patterns such as sequential, reverse
sequential, and random, and optimizes I/O access for these patterns.

Distributed metadata and distributed locking

The SONAS file system also implements the sophisticated GPFS-based token (lock)
management, which coordinates access to shared disks, ensuring the consistency of file
system data and metadata when various nodes access the same file.

The SONAS file system has implemented a sophisticated distributed metadata server
function, in which multiple nodes act, share, acquire, and relinquish roles as token managers
for a single file system. This distributed architecture avoids metadata server bottlenecks, and
has been proven to scale to very large file systems.

Along with distributed token management, SONAS file system provides scalable metadata
management by allowing all nodes of the cluster accessing the file system to perform file
metadata operations. This key and unique feature distinguishes SONAS from other cluster
file systems which have a centralized metadata server handling fixed regions of the file
namespace. SONAS file system is specifically designed to avoid a centralized metadata
serve, to avoid problems where there is a performance bottleneck for metadata intensive
operations. This also improves availability, as the distributed metadata server function
provides additional insulation against a metadata server single point of failure.

SONAS implements the GPFS technology that solves this problem by managing metadata at
the node which is using the file or in the case of parallel access to the file, at a dynamically
selected node which is using the file.

SONAS file system administration

The SONAS file system provides an administration model that is easy to use and consistent
with standard Linux file system administration, while providing extensions for the clustering
aspects of SONAS.

These functions support cluster management and other standard file system administration
functions such as user quotas, snapshots and extended access control lists.

The SONAS file system provides functions that simplify cluster-wide tasks. A single SONAS
command or GUI command can perform a file system function across the entire SONAS file
system cluster.

The distributed SONAS file system architecture facilitates a rolling upgrade methodology, to
allow you to upgrade individual SONAS odes in the cluster while the file system remains
online. The SONAS file system also supports a mix of nodes running at current and new
release levels, to enable dynamic SONAS Software upgrades.

The SONAS file system implements quotas to enable the administrator to control and monitor
file system usage by users and groups across the cluster. The SONAS file system also
provides commands to generate quota reports including user, group, and fileset inode and
data block usage.

Chapter 3. SONAS architecture 81

 SONAS file system snapshots
In the current release, up to 256 read-only snapshots of an entire GPFS file system can be
created to preserve the file system's contents at a single point in time. The SONAS file
system implements a space efficient snapshot mechanism that generates a map of the file
system at the time it was taken and a copy of only the file system data that has been changed
since the snapshot was created. This is done using a copy-on-write techniques. The snapshot
function allows a backup program, for example, to run while the file system is in use and still
obtain a consistent copy of the file system as it was when the snapshot was created.

In addition, SONAS snapshots provide an online backup capability that allows files to be
recovered easily from common problems such as accidental deletion of a file. It is a known
requirement to increase the snapshot granularity to include filesets, directories, and individual
files, and IBM intends to address these requirements in a future SONaS release.

High performance scan engine

The SONAS file system implements a high performance scan engine, which can be used to
rapidly identify files that need to be managed within the SONAS file system concept of logical
tiers storage pools. SONAS file system can transparently perform physical movement of data
between pools of logical tiered storage, and can also perform Hierarchical Storage
Management (HSM) to external storage, using an external Tivoli Storage Manager server.

Access control
SONAS provides NFSv4 enhanced access control at the file system level, to allow all SONAS
users, regardless of the NAS protocol by which they access the SONAS, to be able to take
advantage of this robust level of central security and access control to protect directories and
files. SONAS file system implements NFSv4 access control lists (ACLs) in addition to
traditional ACL support.

SONAS ACLs are based on the POSIX model. Access Control Lists (ACLs) extend the base
permissions, or standard file access modes, of read (r), write (w), and execute (x) beyond the
three categories of file owner, file group, and other users, to allow the definition of additional
users and user groups. In addition, SONAS introduces a fourth access mode, control (c),
which can be used to govern who can manage the ACL itself.

Exporting or sharing the SONAS file system

The SONAS file system is exported (NFS), shared (CIFS), or made available by FTP or
HTTPS to SONAS users through the previously mentioned robust clustered capability of the
SONAS Cluster Manager.

The SONAS Cluster Manager function works in conjunction with the SONAS file system to
provide clustered NFS, clustered CIFS, clustered FTP, and clustered HTTPS. With the
SONAS Cluster Manager, SONAS provides a super-scalable, high performance file system
capability with simultaneous access to a common set of data from multiple interface nodes,

The SONAS file system works in conjunction with the rest of the SONAS Software to provide
a comprehensive, integrated set of storage management tools including monitoring of file
services, load balancing and IP address fail over.

82 IBM Scale Out Network Attached Storage Concepts

 File system high availability
The SONAS clustered file system architecture provides high availability, parallel cluster fault
tolerance. The SONAS file system provides for continuous access to data, even if cluster
nodes or storage systems fail. This is accomplished though robust clustering features and
support for internal or external data replication.

The SONAS file system continuously monitors the health of the file system components. If
failures are detected, appropriate recovery action is taken automatically. Extensive logging
and recovery capabilities are provided which maintain metadata consistency when nodes
holding locks or performing services fail.

Internal data replication (SONAS Software RAID-1 mirroring) is optional and can be
implemented to provide further protection over and above the SONAS hardware storage
redundancy and RAID. In addition, the SONAS file system automatically self-replicates and
internally mirrors file system journal logs and metadata to assure hot failover and redundancy
and continuous operation of the rest of the file system, even if all paths to a disk or storage
pod fail.

SONAS file system summary

The SONAS file system is at the heart of the SONAS Software stack. Based on GPFS, a
SONAS file is highly scalable:
򐂰 Symmetric, scalable software architecture
򐂰 Distributed metadata management
򐂰 Allows for incremental scaling of system (nodes, disk space) with ease
򐂰 Based on GPFS technology that today runs tens of thousands of nodes in a single cluster)

The SONAS file system is a high performance file system:

򐂰 Large block size (tunable) support with wide striping (across nodes and disks)
򐂰 Parallel access to files from multiple nodes
򐂰 Supports byte-range locking and distributed token / locking management
򐂰 Efficient deep prefetching: read ahead, write behind
򐂰 Recognizes access patterns (adaptable mechanism)
򐂰 Highly multi threaded

The SONAS file system is highly available and fault tolerant:

򐂰 Data protection mechanisms that include journaling, replication, mirroring
򐂰 Internal peer-to-peer global cluster heartbeat mechanism to recover from multiple disk,
node, and connectivity failures
򐂰 Recovery software mechanisms implemented in all layers

Let us now examine the SONAS data management services in more detail.

3.7 SONAS data management services

In the following sections we describe the operational concepts of the SONAS central policy
engine and the automated tiered storage methodology by which SONAS Software provides
Hierarchical Storage Management (HSM). Figure 3-30 shows the data management services
on the left center of the diagram.

Chapter 3. SONAS architecture 83

 CIFS NFS FTP HTTPS future

SONAS Cluster Manager

HSM and ILM Monitoring Agents

Parallel
File System
Backup & Restore GUI/CLI mgmt
Policy Engine Interfaces

Snapshots and Scan Engine

Security
Replication

Enterprise Linux

IBM Servers

Figure 3-30 SONAS Software data management services components

We also discuss the role and use of external Tivoli Storage Manager servers to provide
accelerated backup and restore, and tor provide HSM to external storage pools. Finally, we
describe local data resiliency using Snapshots, and remote resiliency using asynchronous
replication.

3.7.1 SONAS: Using the central policy engine and automatic tiered storage
SONAS uses policies to control the lifecycle of files that it manages and consequently control
the costs of storing data by automatically aligning data to the appropriate storage tier based
on the policy rules setup in by the SONAS administrator. Figure 3-31 illustrates a tiered
storage environment that contains multiple storage tiers and each tier has specific
performance characteristics and associated costs, for example, the poolfast contains fast and
expensive disk, whereas pooltape contains relatively inexpensive tapes.

Figure 3-31 Policy-based storage tiering

84 IBM Scale Out Network Attached Storage Concepts

Evidently performance comes at a price, and is the main cost differentiator in storage
acquisitions. For this reason setting policies can help control costs by using the appropriate
storage tier for specific sets of data and making room on the more expensive tiers for new
data with higher performance requirements.

The SONAS policy implementation is based on and uses the GPFS policy implementation.
Files reside in SONAS storage pools, and policies are assigned to files and control the
placement and movement of files between storage pools.

A SONAS policy consists of a collection of rule, and the rules control what actions are
executed and against what files the actions are performed. So the smallest entity controlled
by a rule is a file. SONAS has three types of rules:
Initial file placement These rules control the placement of newly created files.
File management These rules control movement of existing files between storage pools
and the deletion of old files. Migration policies are used to transfer data
between the SONAS storage pools and to the external HSM storage
pool.
Restore of file data These rules control what happens when data gets restored to a
SONAS file system.

Each SONAS filesystem is mapped to storage pools. The default pool for a filesystem is the
system pool, also called pool1. A file system can have one or more additional storage pools
after the system pool.

The SONAS administrator assigns physical disk LUNs to a logical SONAS storage pool.

SONAS also manages external storage pools. An external storage pool is a mechanism for
SONAS to store data in an external HSM storage manager such as Tivoli Storage Manager.
Policies control the location of files among storage pools in the same filesystem. Figure 3-32
shows a conceptual representation of a filesystem, pools, and physical disk LUNs.

Figure 3-32 SONAs filesystem and policies

Chapter 3. SONAS architecture 85

 A filesystem is managed by one active policy, policy1 in the example. The initial file placement
policies control the placement of new files. File placement policies are evaluated and applied
at file creation time. If placement policies are not defined all new files are placed in the system
storage pool. Migration and deletion rules, or file management rules, control the movement of
files between SONAS disk storage pools and external storage pools such as Tivoli Storage
Manager HSM and the deletion of old files. Migration and deletion rules can be scheduled
using the cron scheduler. File migration between pools can also be controlled by specifying
thresholds. Figure 3-33 shows a conceptual representation of these rules.

Figure 3-33 File placement and migration rules

SONAS introduces the concept of tiered and peered storage pools:

Tiered Pools The pools that NSDs are assigned to can be tiered in a hierarchy using
GPFS policies. These hierarchies are typically used to transfer data
between a fast pool and a slower pool (Pool1  Pool2) using
migration. When coupled with HSM, data flows in a hierarchy from
Pool1  Pool2  Pool3(HSM).
Peered Pools The pools that NSDs are assigned to can be operated as peers in a
hierarchy using GPFS file placement policies. These policies allow
files to be placed according to rules in either the fast pool Pool1or the
slower pool Pool2. When coupled with HSM data flows to either Pool1
or Pool2 pool based on file placement policies, then from both Pool1
and Pool2 pools the data flows to Pool3 (HSM).

To simplify implementation of HSM and storage pooling, SONAS provides templates for
standard usage cases. Customized cases can be created from the default templates by using
the SONAS CLI. The standard usage cases, also called ILM profiles, are shown in the
diagram in Figure 3-34.

86 IBM Scale Out Network Attached Storage Concepts

 Default pool:
All NSDs in the same pool
new file pool1

Peered pools:
Placement policies only
new file pool1 pool2

Tiered pools:
Files placed in pool1 and
new file pool1 pool2 then moved to pool2

Default pool and HSM:

Files placed in pool1 then
new file pool1 pool3 moved to TSM HSM pool3

Peered pools and HSM:

Placement policies for
new file pool1 pool2 pool3 pool1,2 and migration from
pool1,2 to pool3
Tiered pools and HSM:
Files placed in pool1, then
new file pool1 pool2 pool3 migrated to pool2 and then
to TSM HSM pool3
Figure 3-34 Standard ILM policy profiles

The standard ILM policy profiles are based on the assumption that pool1 is the fastest pool
using the fastest storage devices such as SAS disks, and pool2 is based on less expensive
disks such as SATA. The SONAS GPFS metadata must always reside in the fastest storage
pool, pool1 in our examples, because it is the data that has the highest IO requirements when
SONAS GPFS file system scan operations are performed.

3.7.2 Using and configuring Tivoli Storage Manager HSM with SONAS basics
The use of SONAS HSM provides the following advantages:
򐂰 It frees administrators and users from manual file system pruning tasks, and defers the
need to purchase additional disk storage.
򐂰 It allows the Tivoli Storage Manager HSM to extend the SONAS disk space and
automates the movement of seldom-used files to and from external near line storage.
򐂰 It allows pre-migration, a method that sends a copy of the file to be migrated to the Tivoli
Storage.
򐂰 It manages the server prior to migration, allowing threshold migration to quickly provide
space by simply stubbing the premigrated files.
򐂰 It allows you to pre-define the stub file size, which will eliminate the recall of the entire file
for programs that browse only the first part of a file.

To use the Tivoli Storage Manager HSM client, you must provide a Tivoli Storage Manager
server external to the SONAS system, and the server is accessed through the Ethernet
connections on the interface nodes.

Chapter 3. SONAS architecture 87

 The physical configuration and connection of a SONAS system and Tivoli Storage Manager
server are explained in “SONAS and Tivoli Storage Manager configuration” on page 94.

The current version of SONAS requires that HSM be configured and managed using the CLI
as at the time of writing GUI support is not present for HSM. HSM migration work might cause
additional overhead on the SONAS interface nodes and so take care when planning the
timing and frequency of migration jobs.

When using HSM space management on a filesystem, each file in the filesystem can be in
one of three states:
򐂰 Resident when the file resides on disk in the SONAS appliance
򐂰 Premigrated when the file resides both on the disk in the SONAS and in Tivoli Storage
Manager HSM
򐂰 Migrated when the file resides only in Tivoli Storage Manager

Files are created and modified on the SONAS filesystem and when they are present they are
said to be in the resident state.

Files in an HSM managed filesystem can be migrated to Tivoli Storage Manager HSM storage
for a variety of reasons, such as when a predefined file system utilization threshold is
exceeded. Migrated files are copied to Tivoli Storage Manager and replaced by a stub file that
has a preset size. Using a stub file leaves a specified amount of file data at the front of the file
on the SONAS disk, allowing it to be read without triggering a recall.

In a SONAS GPFS environment, a small file that is less than the 1/32 of the filesystem
blocksize, or one subblock, can become larger after an HSM migration because SONAS
GPFS adds meta information to the file during the migration. Because another block on the
file system is allocated for the meta information, this increases the space allocated for the file.
If a file system is filled to its maximum capacity with many small files, it is possible that the file
system can run out of space during the file migration.

A recall is triggered when the first byte of storage not on the SONAS disk is accessed. When
a migrated file is accessed, it is recalled from the external Tivoli Storage Manager storage into
the SONAS storage. If you have files with headers that will be periodically accessed and do
not want to trigger recalls on those header accesses, use the appropriate stub file size to
ensure that an appropriate amount of file header stays on the SONAS disk.

As data is accessed by CIFS or NFS, when a migrated file is opened and a byte of data that is
not in the SONAS cache is accessed, that access triggers a Data Management API (DMAPI)
event in the SONAS. That event is sent to the primary Tivoli Storage Manager client, that
resides on one of the interface nodes, and it triggers a recall. If the primary Tivoli Storage
Manager client is not overloaded, it issues the recall itself, otherwise it sends the recall to
another Tivoli Storage Manager client node. In practice most recalls will be performed by the
primary Tivoli Storage Manager client interface node.

Because a recall from physical tape requires waiting for cartridge fetching, tape drive loading
and tape movement to the desired file, physical tape recalls can take significant numbers of
seconds to start, so the application needs to plan for this delay.

88 IBM Scale Out Network Attached Storage Concepts

 Tivoli Storage Manager has the following requirements for HSM:
򐂰 You must supply a Tivoli Storage Manager server that can be accessed by the SONAS
interface nodes.
򐂰 You must ensure that sufficient network bandwidth and connectivity exists between the
interface nodes they select to run HSM on to the external storage server they are
providing.
򐂰 The Tivoli Storage Manager server has to be prepared for use by the SONAS system.
򐂰 A Tivoli Storage Manager backup pool using the SONAS backup feature is set up.
򐂰 Server time must be synchronized with the SONAS system; both systems must access
the same NTP server.
򐂰 Tivoli Storage Manager server authentication must be enabled.

HSM can be added to a filesystem at the time of filesystem creation or at a later time.

3.8 SONAS resiliency using snapshots

In this section, we overview how SONAS Software implements space-efficient Snapshots.
Snapshots are a standard, included feature of the SONAS Software and do not require any
additional licensing. SONAS Snapshots enable online backups to be maintained, providing
near instantaneousness access to previous versions of data without requiring complete,
separate copies or resorting to offline backups.

3.8.1 SONAS Snapshots

SONAS Snapshots can be scheduled or performed by authorized users or by the SONAS
administrator, with the capability of up to 256 active Snapshots, per file system, at any one
time.

SONAS Snapshot technology makes efficient use of storage by storing only block-level
changes between each successive Snapshot. Only the changes made to the original file
system consume additional physical storage, thus minimizing physical space requirements
and maximizing recoverability.

At the current release level, SONAS Snapshot is a read-only, point-in-time consistent version
of an entire SONAS file system, frozen at a point in time:
򐂰 Each SONAS file system can maintain up to 256 Snapshots concurrently.
򐂰 Snapshots only consume space when the file system changes.
򐂰 Snapshots use no additional disk space when first taken.
򐂰 Snapshots are enforced to be consistent across the file system to a single point in time.
򐂰 Snapshots can be taken manually or automatically on a schedule.
򐂰 For CIFS users, SONAS Snapshots are readily accessible by Microsoft Volume Shadow
Services (VSS) integration into the Windows Explorer interface.

Chapter 3. SONAS architecture 89

 Snapshots can be made (with proper authority) through the SONAS Management GUI, or
through the SONAS Command Line Interface (CLI). The SnapShot appears as a special
directory, as shown in Figure 3-35.

Filesystem /fs1/file1
/fs1/file2
Before /fs1/subdir1/file3
Read-only copy of
/fs1/subdir1/file4
Snapshot /fs1/subdir2/file5 directory
structure and files

Only changes to the

/fs1/file1 original file consume
/fs1/file2 disk space
/fs1/subdir1/file3
Filesystem /fs1/subdir1/file4
/fs1/subdir2/file5
After
/fs1/.snapshots/snap1/file1
Snapshot /fs1/.snapshots/snap1/file2
/fs1/.snapshots/snap1/subdir1/file3
/fs1/.snapshots/snap1/subdir1/file4
/fs1/.snapshots/snap1/subdir2/file5

Figure 3-35 SONAS Snapshot appears as a special directory in the file system

Snapshots of a SONAS file system are read-only; changes are made only to the active
(that is, normal, non-snapshot) files and directories. Snapshots are only made of active file
systems;, you cannot make a Snapshot of an existing Snapshot.

Individual files, groups of files, or entire directories can be restored or copied back from
Snapshots.

3.8.2 Integration with Windows

SONAS Snapshot supports the Microsoft Volume Shadow Services (VSS) function to allow
display of older file and folder versions, for example, from within the Microsoft Explorer.
Snapshots are exported to Windows CIFS clients by the Volume Shadow copy Service (VSS)
API.

90 IBM Scale Out Network Attached Storage Concepts

 This means that SONAS Snapshot data can be accessed and copied back, through the
Previous Versions dialog in the Microsoft Windows Explorer. Figure 3-36 depicts the Previous
Versions dialog.

Use these
buttons
copy or
restore the
snapshot

Figure 3-36 SONAS Snapshots are accessible for Windows CIFS users by Windows Explorer

SONAS Snapshots are intended as a point in time copy of an entire SONAS file system, and
preserve the contents of the file system at a single point in time. The Snapshot function allows
a backup or mirror program to run concurrently with user updates and still obtain a consistent
copy of the file system as of the time that the snapshot was created. SONAS Snapshots also
provide an online backup capability that allows easy recovery from common problems such
as accidental deletion of a file, and comparison with older versions of a file.

3.9 SONAS resiliency using asynchronous replication

In this section, we overview how SONAS asynchronous replication is designed to provide a
bandwidth-friendly mechanism that is tolerant of telecommunication bandwidth shortages.
This implementation is space efficient, transferring only the changed blocks of a file, not the
whole file again. Resource efficiency and high performance is achieved by using multiple
interface nodes in parallel, to transfer the data.

SONAS asynchronous replication can also be useful for the idea of “backup-less backup”
disaster recovery, in other words, using direct disk to disk incremental change replication to a
disaster recovery remote site. This is particularly important when the raw amount of data for
backup/restore for large amounts of storage, is so large that a tape restore at a disaster
recovery site might be unfeasible from a time-to-restore standpoint.

In this section, we discuss the SONAS asynchronous replication capability that is designed to
address these requirements.

At a high level, SONAS asynchronous replication works as follows.

Chapter 3. SONAS architecture 91

 The first step is to execute a central policy engine scan for async replication. The SONAS
high performance scan engine is used for this scan. As part of the asynchronous replication,
an internal snapshot will be made of both the source file system and the target file system.
The first step is shown in Figure 3-37.

Async replication
begins by
executing a policy
IBM Scale Out NAS

1. Read policies
Global Namespace R em ote S ca le
Out NA S
Policy Engine
Inte rf ac e
node … Inter fa ce
node …
Target file system 1 snapshot

St or age
node
… Stor age
node
…
Target file system 2 snapshot

File system 1 snapshot File system 2 snapshot

Figure 3-37 SONAS async replication step 1 - execute a policy, make snapshot at local and remote

The next step is to make a mathematical hash of the source and target snapshots, and
compare them, as shown in Figure 3-38.

Hash compare:
determines
incremental
changes to send
IBM Scale Out NAS

Global Namespace
R em ote Sc ale
Policy Engine Out NA S
Inte rf ac e
node … Inter fa ce
node … hash

hash hash
Target file system 1 snapshot

2. Scan, hash compare St or age

node
… Stor age
node
… hash

Target file system 2 snapshot

File system 1 snapshot File system 2 snapshot

Figure 3-38 SONAS async replication step 2 - compare mathematical hash of snapshots

The final step is to exploit the parallel data transfer capabilities of SONAS by having multiple
nodes participate in the transfer of the async replication changed blocks to the target remote
file systems, as shown in Figure 3-39.

92 IBM Scale Out Network Attached Storage Concepts

 IBM Scale Out NAS 3. Parallel transmit to remote site(s)

Global Namespace
R em ote Sc ale
Policy Engine Out NA S
Inte rf ac e
node … Inter fa ce
node …
Target file system 1 snapshot
St or age
node
… Stor age
node
…
Target file system 2 snapshot
File system 1 snapshot File system 2 snapshot

Figure 3-39 SONAS async replication step 3 - transfer data using multiple interface nodes

The internal snapshot at the source side assures that data being transmitted is in data
integrity and consistency, and is at a single point in time. The internal snapshot at the target is
there to provide a backout point in time capability, if for any reason the drain of the changes
from source to target fails before it is complete.

Let us review a few more details about the SONAS asynchronous replication.

SONAS asynchronous replication is designed to cope with connections that provide low
bandwidth, high latency and low reliability. The basic philosophy of SONAS asynchronous
replication proceeds as follows:
1. Take a snapshot of both the local and remote file systems. This ensures first that we are
replicating a frozen and consistent state of the source file system.
2. Collect a file path list with corresponding stat information, by comparing the two with a
mathematical hash, in order to identify changed blocks
3. Distribute the changed file list to a specified list of source interface nodes
4. Run a scheduled process that performs rsync operations on the set of interface nodes, for
a given file list, to the destination SONAS. Rsync is a well-understood open source utility,
that will pick-up the changed blocks on the source SONAS file system, and stream those
changes in parallel to the remote, and write them to the target SONAS file system.
5. The snapshot at the remote SONAS system ensures that a safety fallback point is
available if there is a failure in the drain of the new updates.
6. When the drain is complete, then the remote file system is ready for use.
7. Both snapshots are automatically deleted after a successful replication run.

The target SONAS system is an independent SONAS cluster that can be thousands of miles
away.

At the current release level SONAS R1.1.1, asynchronous replication is available for
replicating incremental changes at the file system level to one other site, meaning SONAS
machines are paired on a 1:1 basis.

Multiple file systems can be replicated between this pair of SONAS systems. As long as they
are unique pairs of file systems, a given SONAS can have certain file systems be sources for
async replication, and other file systems can be the targets.

Chapter 3. SONAS architecture 93

 Asynchronous replication is done using an IBM enhanced and IBM supported version of open
source “rsync.” The enhancements include the ability to have multiple SONAS nodes in
parallel work on the rsync transfer of the files.

The replication schedule is configured thru the SONAS GUI or by Command Line Interface.
Depending on the number of files included in the replication; the minimal interval will vary
depending on the amount of data and files to be sent.

3.10 SONAS and Tivoli Storage Manager configuration

In this section, we provide more SONAS Software configuration details on how the SONAS
and Tivoli Storage Manager work together and are configured. You can choose to use the
SONAS-specific integration and exploitation with Tivoli Storage Manager for these two
functions:
򐂰 The SONAS and Tivoli Storage Manager integration is designed to provide SONAS
customers the ability to perform HSM to external storage, or to Accelerated backup/restore
of the file systems as compared with normal backup/restore servers.
򐂰 It is also designed to support accelerated backups on a file-level of entire GPFS
filesystems with Tivoli Storage Manager to an external Tivoli Storage Manager server, and
to provide a file-level based restore capability including patterns.

Figure 3-40 shows a diagram of the SONAS and Tivoli Storage Manager configuration.

TSM client
code is pre- SONAS to TSM
installed is Ethernet only

Mgmt Interface Interface

Node Node Node

Storage
Pod
IBM If using SONAS HSM to
SONAS external storage, an external
TSM server is used

Storage

Figure 3-40 SONAS and Tivoli Storage Manager configuration

94 IBM Scale Out Network Attached Storage Concepts

 As compared to normal, conventional backup software, SONAS and Tivoli Storage Manager
integration is designed to provide significantly accelerated backup elapsed times or high
performance HSM to external storage, by exploiting the following technologies:
򐂰 A very fast SONAS scan engine is used to identify files for Tivoli Storage Manager to
backup or HSM. This is much faster compared to standard Tivoli Storage Manager
backups, or other conventional backup software, that needs to traverse potentially large)
filesystems and checking each file against the Tivoli Storage Manager server.
򐂰 Multiple SONAS interface nodes work in parallel under the covers to stream data to the
Tivoli Storage Manager server at a greatly accelerated rate, over multiple parallel interface
nodes in parallel
򐂰 The SONAS Software will distribute parts of the filelist as backup jobs across a given set
of interface nodes. Each interface node then operates on its own filelists. Each Tivoli
Storage Manager process can establish a few sessions to the Tivoli Storage Manager
server.

Existing Tivoli Storage Manager customers can make use of existing Tivoli Storage Manager
servers to backup SONAS. SONAS is designed to perform Tivoli Storage Manager functions
on SONAS with only a few commands that need to be issued on the Tivoli Storage Manager
server and the SONAS system to make use of Tivoli Storage Manager with SONAS, and to
perform scheduled backups or HSM data movements.

In SONAS, Tivoli Storage Manager backup is performed over LAN through the interface
nodes to the Tivoli Storage Manager servers. It is not possible to do LAN-free backup at this
time from SONAS to the Tivoli Storage Manager server.

3.10.1 Tivoli Storage Manager terminology and operational overview

IBM Tivoli Storage Manager, working together with IBM SONAS, provides an end-to-end
comprehensive solution for backup/restore, archival, and hierarchical storage management
(HSM).

Tivoli Storage Manager overview

In order to best understand how SONAS works together with IBM Tivoli Storage Manager, it is
useful here to review and compare the specific Tivoli Storage Manager terminology and
processes involved with the following activities:
򐂰 Backing up and restoring files
򐂰 Archiving and retrieving them
򐂰 Migrating and recalling them (Hierarchical Storage Management)

Tivoli Storage Manager terminology

If you use Tivoli Storage Manager to back up files (which will invoke the Tivoli Storage
Manager backup-archive client code on the interface nodes), copies of the files are created
on the Tivoli Storage Manager server external storage, and the original files remain in your
local file system. To obtain a backed file from Tivoli Storage Manager storage, for example,
in case the file is accidentally deleted from the local file system, you restore the file.

If you use Tivoli Storage Manager to archive files to Tivoli Storage Manager storage, those
files are removed from your local file system, and if needed later, you retrieve them from Tivoli
Storage Manager storage.

Chapter 3. SONAS architecture 95

 If you use Tivoli Storage Manager to migrate SONAS files to external storage (which will
invoke the Tivoli Storage Manager HSM client code on the interface nodes), you move the
files to external storage attached to the Tivoli Storage Manager server, and Tivoli Storage
Manager will replace the file with a stub file in the SONAS file system. You can accept the
default stub file size, or if you want, specify the size of your Tivoli Storage Manager HSM stub
files to accommodate needs or applications that want to read headers or read initial portions
of the file.

To users, the files appear to be online in the file system. If the migrated file is accessed, Tivoli
Storage Manager HSM will automatically initiate a recall of the full files from their migration
location in external Tivoli Storage Manager-attached storage. The effect to the user will
simply be elongated response time while the file is being recalled and reloaded into internal
SONAS storage. You can also initiate recalls proactively if desired.

3.10.2 General Tivoli Storage Manager and SONAS guidelines

Follow these general guidelines:
򐂰 Customers must use a Tivoli Storage Manager disk-pool as the primary pool to store data
and its size must be larger than the normal amount of backup data that gets backed up per
backup-run, so that all data first gets copied to disk, and no tape mount is required during
a backup
򐂰 Depending on the amount of data in SONAS, it might be necessary to have one dedicated
Tivoli Storage Manager server per filesystem considering that one SONAS filesystem can
contain 2 billion files.

Note that the Tivoli Storage Manager backup is not using the classical Tivoli Storage Manager
backup process to traverse the filesystem, query the server and identify the changes.

Instead, the SONAS Software is called to use the high performance scan engine and the
policy engine to identify changes in the filesystem, and to generate a list of files that need to
be expired, and a list of files that need to be backed up.

Scripts are provided with the SONAS Software to define the interface nodes involved in the
backup, the relationship of which filesystem needs to be backed up to which Tivoli Storage
Manager server, and to start/stop backups or restores.

Do not consider the use of SONAS HSM with Tivoli Storage Manager as a replacement for
backup/restore. Rather, consider HSM as an external storage extension of local SONAS disk
storage.

When a file is migrated using the Tivoli Storage Manager HSM server to the external Tivoli
Storage Manager HSM storage, there is still only one copy of the file available, because the
original is deleted on the SONAS file system itself, and replaced by the Tivoli Storage
Manager/HSM stub file. Also, HSM with Tivoli Storage Manager maintains only the last copy
of the file, giving no opportunity to store multiple versions. In comparison, Tivoli Storage
Manager backup/archive (or typically any backup/archive software) gives you the full ability to
storage multiple backup versions of a file, and to track and manage these backup copies in an
automated way.

According to Tivoli Storage Manager best practices, you must back up / archive a file before
the file has been migrated by Tivoli Storage Manager HSM to external storage. With proper
configuration, you can specify in Tivoli Storage Manager management classes that a file is
not eligible for HSM migration unless a backup has been made first with the Tivoli Storage
Manager backup-archive capability. This is an extremely useful function.

96 IBM Scale Out Network Attached Storage Concepts

 It is good practice to back up and migrate your SONAS files to the same IBM® Tivoli Storage
Manager server. If you are doing both back up and HSM migration of files to the same Tivoli
Storage Manager server, the Tivoli Storage Manager HSM code on the interface node can
verify and enforce that a current Tivoli Storage Manager backup version of your files exists,
before you migrate the files.

Another good reason for using the same Tivoli Storage Manager server for both backup
restore and HSM, is that if the file is migrated and the same Tivoli Storage Manager server
destination is used, the Tivoli Storage Manager server, outboard of the SONAS and without
any involvement from the SONAS, can copy the file from the Tivoli Storage Manager HSM
migration storage pool to the backup pool without recalling the file, which is another very
useful capability.

3.10.3 Tivoli Storage Manager software licensing

The Tivoli Storage Manager client software code is always pre-installed within SONAS
Software, and is resident in the code from the factory.

As of the writing of this book, the Tivoli Storage Manager client version 6.1.3 is used internally
in SONAS Software. There is no need to order it or install it separately on the interface nodes.

If you are not using Tivoli Storage Manager functions, there is no charge for the fact that this
Tivoli Storage Manager client code is present in the SONAS Software.

Note that an IBM Tivoli Storage Manager server talks to two unique types of Tivoli Storage
Manager clients. One client is backup/archive client for backup, restore, archive and retrieve
operations, and the other is a separate Tivoli Storage Manager HSM client for migrating and
recalling files. Both of these Tivoli Storage Manager client software are pre-installed on the
SONAS Software at the factory and are present on every SONAS system; you do not need to
order them or install them yourself. If you are not using Tivoli Storage Manager software
functions, there is of course no charge. If you are using Tivoli Storage Manager software
functions, then there is the normal Tivoli Storage Manager licensing charge for the
appropriate Tivoli Storage Manager client software, however, this is just on the interface
nodes that you are attaching to the Tivoli Storage Manager server.

If you use Tivoli Storage Manager functions, then you are required to pay a license charge for
the appropriate Tivoli Storage Manager client code. You only have to pay the license charge
for the interface nodes that are attached to Tivoli Storage Manager servers. The licensing is
for the processor value units of the SONAS interface node for a Linux server. Note that this
Tivoli Storage Manager client licensing is not for the terabytes of storage that might be on the
SONAS.

As for the Tivoli Storage Manager server, this Tivoli Storage Manager Server licensing (and if
necessary, the associated servers that the Tivoli Storage Manager Server will run on) needs
to be licensed and setup independently of SONAS.

If you are an existing Tivoli Storage Manager server user, then you can, of course, use that
existing Tivoli Storage Manager infrastructure and licenses with IBM SONAS.

Various Redbooks publications and white papers provide further information about Tivoli
Storage Manager and its sizing guidelines, performance optimizations, and tuning knobs.

Chapter 3. SONAS architecture 97

3.11 SONAS system management services
SONAS provides a complete centralized set of facilities for globally managing and centrally
deploying NAS storage. In this section we provide an overview of the Management GUI, the
Command Line Interface and the Health Center.

3.11.1 Management GUI

SONAS provides a centralized web-based graphical user interface and Health Center for
configuration and monitoring tasks. Users access the GUI / Health Center by a standard web
browser. There is a command line interface (CLI) as well.

SONAS Management GUI runs on the SONAS Management Node, and is web-based (https).
It provides role-based authorization for users, and enables the administrator to maintain the
SONAS cluster. These roles are used to segregate GUI administrator users according to their
working scope within the Management GUI. These roles are as follows:
򐂰 Administrator: This role has access to all features and functions provided by the GUI, and
is the only role that can manage GUI users and roles.
򐂰 Operator: The operator can do the following tasks:
– Check the health of the cluster.
– View the cluster configuration.
– Verify the system and file system utilization.
– Manage to set thresholds and notifications settings.
򐂰 Export administrator: The export administrator is allowed to create and manage shares,
plus perform the tasks the operator can execute.
򐂰 Storage administrator: The storage administrator is allowed to manage disks and storage
pools, plus perform the tasks the operator can execute.
򐂰 System administrator: The system administrator is allowed to manage nodes and tasks,
plus perform the tasks the operator can execute.

SONAS has a central database that stores configuration information and events. This
information is partly pulled by the management node and partly pushed to the management
node by the other nodes in the cluster.

98 IBM Scale Out Network Attached Storage Concepts

The SONAS Management GUI and Health Center provides the following panels:
򐂰 Storage management
򐂰 File system management
򐂰 Pool management
򐂰 Fileset management
򐂰 Policy management
򐂰 Access control list (ACL) management
򐂰 Synchronous replication management
򐂰 Hierarchical Storage management
򐂰 Tivoli Storage Manager backup management
򐂰 Async replication management
򐂰 Snapshot management
򐂰 Quota management
򐂰 Cluster management
򐂰 Protocol management (CIFS, NFS, HTTPS, FTP)
򐂰 Export management
򐂰 Event log
򐂰 Node availability
򐂰 Node utilization (CPU, memory, I/O)
򐂰 Performance management (CPU, memory, I/O)
򐂰 File system utilization (capacity)
򐂰 Pool / disk utilization (capacity)
򐂰 Notifications / call-home
򐂰 Hardware monitoring
򐂰 File access services such as NFS, HTTPS, FTP, and CIFS
򐂰 File system services
򐂰 Nodes including CPU’s, memory DIMMs, VRM, disk drives, power supplies, fans, and
onboard network interface ports
򐂰 I/O adapters including storage and network access
򐂰 Storage utilization

Panels are available for most of the major functions, as shown in Figure 3-41.

Figure 3-41 SONAS Management GUI has panels for most aspects of SONAS

Chapter 3. SONAS architecture 99

 SONAS has a complete topology viewer, that shows in graphical format, the internal
components of the SONAS, reports on their activity, and provides a central place to monitor
and if necessary, click on an icon and drill down into the details of the particular component.
In Figure 3-42, we see an example of the SONAS Management GUI Topology Viewer:

External network
send/receive
average MB/sec

At a glance look Exports /

at all interface shares
node status status

File systems
status
Internal data
network
performance

Storage node
status and
performance

Figure 3-42 SONAS Management GUI - Topology Viewer

Each of the icons is clickable, and will expand to show status of an individual components.

The SONAS Management GUI is the focal point for extended monitoring facilities and the
SONAS Health Center.

3.11.2 Health Center

The SONAS Health Center provides a central place to view the overall SONAS health,
including examining the System Log, Alert Log, and System Utilization Reports and graphs.
Through the SONAS Management GUI, repeating tasks can be set up, utilization thresholds
set, notification settings refined and notification recipients defined.

SONAS tracks historical performance and utilization information, and provides the ability to
graphically display the current and historical trends.

100 IBM Scale Out Network Attached Storage Concepts

 The SONAS Health Center is shown in Figure 3-43.

Interface Interface node

Example historical reports memory
node CPU
utilization for a SONAS interface node utilization

Interface node Storage node

network utilization disk utilization

Figure 3-43 SONAS Health Center historical system utilization graphical reports

The length of time that can be reported is determined by the amount of log space set aside to
capture data.

3.11.3 Command Line Interface

The SONAS Command Line Interface (CLI) runs on the SONAS Management Node. The CLI
provides the ability to perform SONAS administrative tasks, and implements about 110 CLI
commands. The focus is on enabling scripting of administrative tasks. CLI primarily for
installation and setup commands, with additional configuration functionality.

The CLI includes the following commands:

򐂰 Cluster configuration
򐂰 Authentication
򐂰 Network
򐂰 Files
򐂰 File Systems
򐂰 Exports
򐂰 File Sets
򐂰 Quotas
򐂰 Snapshots
򐂰 Replication
򐂰 ILM automatic tiered storage
򐂰 Hierarchical storage management
򐂰 Physical management of disk storage

Chapter 3. SONAS architecture 101

 򐂰 Performance and Reports
򐂰 System Utilization
򐂰 SONAS Console Settings
򐂰 Scheduled Tasks

The SONAS CLI is designed to be familiar to the standard UNIX, Windows, and NAS
administrator.

3.12 Summary: SONAS hardware and software

As we conclude this chapter, we have seen that SONAS Software provides a comprehensive,
integrated software functionality stack that includes in one software license, all capabilities
required to manage a SONAS system from the very small to the very large.

SONAS Software provides this functionality in one license, which includes all of the
components shown in Figure 3-44.

CIFS NFS FTP HTTPS future

SONAS Cluster Manager

HSM and ILM

Parallel Monitoring Agents
File System
Backup & Restore GUI/CLI mgmt
Policy Engine Interfaces
Snapshots and Scan Engine
Security
Replication

Enterprise Linux

IBM Servers

Figure 3-44 Summary - SONAS Software functional components

The SONAS Software provides the ability for central management of storage, providing the
functionality for a highly automated, extremely flexible, and highly scalable self-managing
system. You can start with a small SONAS system with less than 100 TB, and continue to
seamlessly grow and linearly scale and increase performance, using the SONAS Software to
manage scalability at petabytes.

102 IBM Scale Out Network Attached Storage Concepts

SONAS Software supports the full capability of the current SONAS to scale that is up to 30
interface nodes and 60 storage nodes. The current largest SONAS storage subsystem is
capable of supporting up to 14.4 petabytes of raw storage. One copy of SONAS Software
runs on each node of a SONAS system. A current maximum SONAS configuration is shown
in Figure 3-45.

Example of a fully configured IBM SONAS

/ho m
/ho
/home/a
me/app
ppl/data
l/d ata/we
/web/importa
b/important _big_ spr
nt_big_ spreads
e adsheet.x ls
heet .xls
e

/a ppl /hom
/ho e/ap
me/a pppl/data
l/d ata/w
/weeb /big_arch
b/big_ architectur e_ drawing.ppt
itect ur e_drawing.ppt
/d ata
/ho
/homme/a
e/appppl/data /weeb
l/d ata/w b/unstr
/uns tructur ed_ big_
uc tured_b ig_vide o. mpg
video.m pg
/web Logical

IBM Scale Out NAS

Physical
Global Namespace
Policy Engine In e
t r f ac I t er fa c
n I nt er f ac In e
t rf ac
In t er fa c In t erf ac In e
t rf ac I nt er f ac I nt er f ac In t er fac
… > scale
… > out
e n od es e no des e no de s e n od es e no des en od es e n od es e no des e n od es en o des

I nt er f ac I t erfac
n I nt er f ac In e
t rf ac I t er fac
n I nt er f ac
In t er fa c I nt er f ac In e
t rf ac I nt er f ac

…>
e no des e n od es e n od es e no des e n od es en o des e no des e n od es en o des e no d es

In t er fac I nt er f ac In e
t rf ac I nt er fa c I nt er f ac I n t erf ac
I nt er f ac In t er fac I nt erf ac In e
t rf ac
e n od es en o des e no des e n od es en o des e no d es e n od es e no des e no d es e n od es

… … … … … …
… … …

Stor ag Stor a
g Stora
g S tor a
g St o
r ag Stor ag St o
r ag Stor ag St o
r ag Stor a
g Stor ag Stor ag St orag St o
r ag St roag St roag St orag Stor ag St roag Stor a
g
… > scale
… > out
e e e e e e e e e e e e e e e e e e e e
nodes nodes nodes nodes nodes nodes odes
n nodes nodes nodes nodes nodes oe
n ds odes
n nodes nodes oe
n ds nodes nodes nodes

St roag Stor ag St o
r ag Stor ag S tor a
g S toa
rg St orag St o
r ag S tor g
a St o
r ag St orag Stor ag Stor ag Stor a
g Stor a
g S tor g
a S toa
rg St orag Stor a
g St o
r ag

…>
e e e e e e e e e e e e e e e e e e e e
odes
n nodes nodes nodes nodes node s oe
n ds odes
n nodes nodes oe
n ds nodes nodes nodes nodes nodes node s oe
n ds nodes nodes

Stor a
g St o
r ag S tor a
g St o
r ag Stor ag Stor ag Stor ag Stor a
g Stor a
g S tor a
g Stor ag St orag St o
r ag St roag St roag Stor ag Stor ag Stor ag Stor a
g Stor a
g
e e e e e e e e e e e e e e e e e e e e
nodes odes
n nodes nodes nodes nodes nodes nodes nodes nodes nodes oe
n ds odes
n nodes nodes nodes nodes nodes nodes nodes

…
… >
> scale
…
… >> out
etc….. etc….. etc…..

Storage Pool 1 Storage Pool 2 Storage Pool 3….. etc.

Figure 3-45 SONAS Software manages all aspects of a maximum size SONAS

As storage needs continue to grow over time, the SONAS Software is designed to continue to
scale out and support even larger configurations, while still maintaining all the storage
management and high performance storage characteristics that we have discussed in this
chapter.

SONAS Software is designed to provide the following features:

򐂰 A single software license that provides all capabilities required to manage a simple,
expandable storage appliance, including ease of ordering, deployment, and management
򐂰 Centralized Management of all files, single namespace, which provides reduced
Administrative costs and faster response time to end users
򐂰 File Placement policies including automation, which provides optimized storage costs and
reduced administrative costs
򐂰 No individual chargeable add-on software, which provides reduced TCO and simpler
procurement process
򐂰 Automated policy based hierarchical storage management: HSM, ILM, which provides
reduced administrative costs, and optimized storage costs
򐂰 Independent scalability of storage and nodes, which provides simple but flexible
configurations tailored to your specific workload characteristics, yet remains flexible and
reconfigurable for the future

Chapter 3. SONAS architecture 103

 򐂰 Concurrent access to files from all nodes, distributed token management, automatic
self-tuning and workload balancing, and high availability by the Cluster Manager,
combining to provide very high performance and reduced administrative costs related to
migrating hot spot files
򐂰 Storage Pool striping, which provides very high performance with fast access to data
򐂰 High performance metadata scanning across all available resources/nodes, integrated
Tivoli Storage Manager clients, which provides ability to perform HSM and automatic
tiered storage at high scalability, and well as accelerating faster backups of files
򐂰 Snapshots and Asynchronous Replication, which provide robust data protection and
disaster Recovery.

3.13 SONAS goals

In summary, SONAS Software provides the software foundation to meet the goals of IBM
SONAS as stated in Figure 3-46.

Goals of IBM Scale Out NAS (SONAS)

 Unified management of petabytes of storage
– Automated tiered storage, centrally managed and deployed
 Global access to data, from anywhere
– Single global namespace, across petabytes of data
 Based on standard, open architectures
– Not proprietary
– Avoids lock-ins
– Leverage worldwide O pen Source innovative technology
 Provides and exceeds today’s needed requirements
for:
– Scale-out capacity, performance, global virtual file server
– Extreme scalability with modular expansion
 High ROI
– Significant cost savings due to auto-tune, auto-balance, automatic
tiered storage
 Position to exploit the next generation of storage
technology
– Superb foundation for cloud storage

Figure 3-46 Goals of IBM Scale Out NAS

In the remaining chapters of this book, we review typical SONAS usage cases and ISV
software integration.

A much more detailed discussion of SONAS hardware and software architecture can be
found in the companion Redbooks publication, SONAS Architecture, Planning, and Basic
Implementation, SG24-7875.

104 IBM Scale Out Network Attached Storage Concepts

 4

Chapter 4. SONAS usage cases

In this chapter we consider the reasons why you might choose to implement a SONAS
system, as well as good possible candidates for this. We discuss various usage scenarios,
general cases, and specific industry usages.

We also provide insight into the role of SONAS in Cloud Storage.

© Copyright IBM Corp. 2010. All rights reserved. 105

4.1 Types of network attached storage
Network attached storage is today a broad category of storage, with many implementations
and sizings, ranging from entry to mid-range to high end. In order to differentiate which type of
network attached storage is most appropriate, following is a generalized description of three
major types of network attached storage, as shown in Figure 4-1.

Scale Out Network Attached Storage –

Definition

General purpose file storage. One or two node clusters. Usually

Classical NAS allow capacity expansion only behind the clusters. Cost
effective, lower price points effectively addressing the small to
midrange NAS market space

Single Namespace to view all files. May provide limited node

Scalable NAS clustering for performance and capacity expansion. Not
necessarily linear or independent scaling.

Single Namespace. Ability to scale large numbers of clusters.

Scale Out NAS Aggregation of storage for high utilization. Independent scale of
GBs/sec of performance vs. PB’s of capacity. N+1 Availability

Figure 4-1 Definitions: three types of Network Attached Storage

We can make the following general comments about these types of NAS.
򐂰 Classical NAS storage:
This class of storage is general purpose file storage, and usually has one or two
controllers in an active-active or active standby configuration. The large growth of
classical NAS storage has been due to it’s ability to provide cost-effective, easy-to-use
storage for the small to mid-range NAS market space
򐂰 Scalable NAS:
This class of NAS storage has arisen with features that are designed to resolve
considerations and restrictions in classical NAS storage, in terms of scalability, user
manageability, and expandability. The focus is on resolving the usability and management
from a user perspective.
򐂰 Scale out NAS:
This final class of storage has arisen to address limitations and restrictions in scalable
NAS architectures. In this case, a parallel clustered file system is implemented, with global
distribution of workload management, workload balancing, and centralized management
and deployment capabilities.

Let us examine each of these considerations in more detail.

106 IBM Scale Out Network Attached Storage Concepts

4.1.1 Classical NAS storage
In classical NAS storage, the controller is a single or dual head controller with active-active or
active-passive modes. Capacity is added to each controller pair until the capacity of the
controller is reached. This comprises the typical NAS filer.

Each NAS filer presents a namespace to the network as shown in Figure 4-2.

Example of classic NAS storage architecture

Cluster Cluster Cluster
#1 #2 #3
Primary Secondary Primary Secondar y Primary Secondary
Filer Filer Filer Filer Filer Filer
Filer File r Filer Filer File r Filer

D i sk Enc los ure D isk En closu re D i sk Enc los ure

D i sk Enc los ure D isk En closu re D i sk Enc los ure

D i sk Enc los ure D isk En closu re D i sk Enc los ure

D i sk Enc los ure D isk En closu re D i sk Enc los ure

Scale Name
D i sk Enc los ure
Scale Name
D isk En closu re
Scale Name
D i sk Enc los ure

Up Space
D isk En clo su re Up Space
Di sk Enc los ur e Up Space
D i sk Enc los ure

#1
D isk En clo su re

D isk En clo su re
#2
Di sk Enc los ur e

Di sk Enc los ur e
#3
D i sk Enc los ure

D i sk Enc los ure

D isk En clo su re Di sk Enc los ur e D i sk Enc los ure

D isk En clo su re Di sk Enc los ur e D i sk Enc los ure

Figure 4-2 Example of a classic NAS architecture

With today’s highly powerful microprocessors and large commodity storage, classical NAS
filers have provided and continue to provide a cost-effective solution for users.

However, as the storage continues to grow in size, considerations appear when considering
large scale and using classical NAS storage servers:
򐂰 Each NAS filer is an independent namespace. There is no sharing of resources with
another NAS filer or another namespace.
򐂰 Failover occurs between primary and secondary filer controllers within the filer, however,
there is no ability to failover to any of the other NAS filers.
򐂰 Ability to scale is limited to the controller capacity. When the controller is at capacity, then
another NAS filer must be implemented; each of these NAS filers must be managed
independently.
򐂰 Balancing of workload between NAS filers must be done manually, and when data is
moved between filers, the corresponding changes need to be made in the domain name
servers, authentication servers, and the user paths and directory definitions. As the
number of NAS filers increases, this can increasingly become a management problem.

4.1.2 Scalable NAS storage

In order to address the issues of management and scalability, the next general type of NAS
storage was developed, which we call Scalable NAS, as shown in Figure 4-3.

Chapter 4. SONAS usage cases 107

 Example of scalable NAS storage architecture

Virtual Global Namespace (software)

#1
Actu al
Actual cl uster Actual
#1 Cluster #2 Clu ster #3

Primary Secondary Primary Sec ondary Primary Sec ondary

Controlle r Controlle r Controller C ontr oller Controller Controlle r

Di sk Enclo sure Di sk Enclo sure Di sk Enclo sure

Di sk Enclo sure Di sk Enclo sure Di sk Enclo sure
Di sk Enclo sure Di sk Enclo sure Di sk Enclo sure
Di sk Enclo sure Di sk Enclo sure Di sk Enclo sure
Di skActual
Enclo sure Di skActual
Enclo sure Di skActual
Enclo sure
Scale Scale Scale
Up Disk Enclosure
Name Up Disk Enclosure
Na me Up Disk Enclosure
Na me
Space Space Space
Disk Enclosure
#1 Disk Enclosure
#2 Disk Enclosure
#3
Disk Enclosure Disk Enclosure Disk Enclosure
Disk Enclosure Disk Enclosure Disk Enclosure
Disk Enclosure Disk Enclosure Disk Enclosure

Figure 4-3 Example of a scalable NAS architecture

The characteristics and considerations when considering implementing large scale NAS
storage using Scalable NAS architectures are:
򐂰 Scalable NAS typically provides a software re-direction layer. providing the appears to all
users of a single (virtual) global namespace. This software re-direction layer can be
implemented in multiple ways; it can reside in any of the following layers:
– NAS controllers
– Network switches and routers
– Servers which front-end the network
򐂰 Regardless of where the software re-direction layer is, the goal is to provide the
appearance of a single virtual global namespace to the users, thus minimizing or
eliminating changes to the user environment, The software layer redirects the virtual
global namespace among the actual physical namespaces that reside on separate
physical NAS storage controllers.
򐂰 However, at the physical level, each NAS file and controller set continues to be
independent. Data might have limitations in how widely it can be spread across multiple
controllers (if it can be spread at all), and failover might be limited to a certain subset of the
controllers.
򐂰 From a physical implementation standpoint, the same physical controllers that were in the
classical NAS type of storage are used. At a physical level, there might not be any change
to the ability to failover to other controller pairs, and physical tuning might continue to
required manual monitoring and adjustment.
򐂰 If data is physically moved, then coordination is required between the virtual software
redirection layer, and the physical controllers and their associated physical namespaces.

Certainly, scalable NAS implementations have value and can address a few of the many user
namespace management concerns. However, these implementations have not addressed

108 IBM Scale Out Network Attached Storage Concepts

 the physical data management concerns, and have not implement an architectural solution to
constraints in scalability, scale out performance, and manageability.

4.1.3 Scale out NAS storage

To address these concerns, scale out NAS storage architectures have emerged, as shown in
Figure 4-4.

Scale Out NAS storage architecture - used in IBM SONAS

Data
Global Scale striped
across all
cluster Out disks in
pool
IF Node IF Node IF Node IF Node
Multiple IF Node IF Node Aggregate
storage Infiniband parallel
nodes S. N ode S . N ode
read/write
S. N ode S . N ode S . N ode S. N ode
working
in
parallel Di sk En clo su re Di sk Enc los ure Di sk Enc los ur e

D isk En clo su re D isk En clo su re Di sk Enc los ur e

D isk En clo su re
Logical Storage
Di sk Enc los ure
Tier Gold Di sk Enc los ur e

Scale D isk En clo su re Di sk Enc los ure Di sk Enc los ur e IL M

Up D isk En clo su re Di sk Enc los ure Di sk Enc los ur e

Di sk Enc lo sur e Logical Storage

D isk Encl osure Tier Silver Di sk Enc los ur e

Di sk Enc lo sur e D isk Encl osure Di sk Enc los ur e

Di sk Enc lo sur e
Logical Storage Tier Copper
D isk Encl osure Di sk Enc los ur e

Figure 4-4 Example of scale out NAS storage architecture as implemented in IBM SONAS

We define a true scale out NAS architecture as one where the physical implementation of the
filer has been re-architected, to provide the following capabilities:
򐂰 Provide a true parallel global clustered environment for network serving, data serving,
any-to-any failover, global workload balancing, global centralized policy management,
global logical storage pools, integrated tiered storage
򐂰 Provision of a true physical global namespace that can scale to petabyte levels, that can
remain constant and unchanging regardless of scale, for all users to see/access.
򐂰 Utilize commercial off the shelf (COTS) enterprise servers, in order to take advantage of
today’s fast pace of technology advancement. Utilize these servers in a grid, where all
nodes share workload equally, and all nodes of a particular role can dynamically failover to
any other node with the same role
򐂰 Provide integrated, advanced features such central policy-managed automated tiered
storage, cross-platform (UNIX, Linux, Windows) support of concurrent reads and writes to
the same file from within the global namespace, wide parallel data stripings, and high
performance scan engines

IBM SONAS is designed as a true scale out NAS architecture. SONAS is designed for NAS
environments where large scale, high performance and capacity requirements for file serving
are present, that cannot be adequately served by classic NAS or scalable NAS architectures.
Each of the features above are implemented within SONAS.

Chapter 4. SONAS usage cases 109

 In the next section, we examine the business and application requirements that SONAS can
be used for, that are increasingly requesting this kind of high scale, high performance storage
requirements.

4.2 Usage cases for scale out NAS

We begin by observing that today’s smarter planet, driven by an amazing level of human
innovation on a worldwide scale, is placing tremendous demands on IT. New image-intensive,
data-intensive, compute-intensive applications including pervasive instrumentation, is
creating vast amounts of data, and driving new types of applications that require real time
data analysis and prediction. Examples of these applications, requirements, and the
industries they are creating, are shown in Figure 4-5.

Medical
Advanced Financial Imaging
Search Analytics

3D On-Line
Infotainment

Video
Surveillance

Analytics with Online Network Security

Transaction Processing & Threat Management

Figure 4-5 New demands on IT for large scale, real-time data analysis and prediction

Let us examine the requirements that these applications are placing on the IT infrastructure.
These requirements are the driving factors behind the IBM approach to SONAS, and are the
reason for the IBM SONAS design and implementation.

4.2.1 General NAS consolidation and streamlining of operations

NAS filers have been popular because of their ease of use, and ability to provide relatively
inexpensive means to provide storage, across a widely dispersed locations, across a wide
diversity of applications.

However, each classical NAS filer has to be individually managed, and as the scale of today’s
IT systems have dramatically grown, the following issues and requirements have occurred:
򐂰 Tremendous growth of “unstructured data” in recent years
򐂰 That must be managed with the same (or less) administrative staff
򐂰 Limitations as described earlier in scalability in both performance and capacity
򐂰 No parallel access to data, yet today’s data rate requirements are continually climbing
򐂰 Multiple filers have to be managed individually

110 IBM Scale Out Network Attached Storage Concepts

򐂰 Backup windows are too small as the among of storage grows to hundreds of terabytes or
more
򐂰 No easy way to apply storage policies across multiple, independent filers
򐂰 No policies to automatically migrate data
򐂰 Difficult to globally implement centrally managed, centrally deployed Automatic Tiered
Storage (for example, Information Lifecycle Management) - essential as terabytes multiply
into petabytes
򐂰 Operations costs can grow linearly with each additional NAS filer

These issues, and the value that IBM SONAS is designed to provide, can be summarized as
shown in Figure 4-6.

Centralized Management and Administration

Classic Filers IBM Scale Out NAS

Figure 4-6 SONAS Usage case - large scale NAS management and administration

IBM Scale Out NAS is designed to provide a solution to these large scale requirements, by
providing a super-scalable, scale out NAS cluster that is centrally managed, centrally
deployed, and provides integrated, automated tiered storage, with centralized management of
logical storage tiers (instead of individual disks and filers).

Chapter 4. SONAS usage cases 111

 SONAS is also designed to provide high storage utilization and automated central policy
management of tiered storage, as shown in Figure 4-7.

Storage Utilization, Automated Tiered Storage

Classic Filers IBM Scale Out NAS

$ $ $ $
$

Figure 4-7 SONAS Usage case - large scale NAS storage utilization, tiered storage

On the left, we see a typical case in the classical NAS environment, where:
򐂰 Capacity must be managed individually by each NAS filer.
򐂰 Average storage utilization is not as high as we prefer; it many environments, it can easily
be less than 50%.
򐂰 All files are online in the file system, but large amounts of these files (perhaps as many as
80% of the files) have not been accessed during the last 6 months, legitimately can be
migrated to lower classes of storage, or to external tape storage.

SONAS can provide a powerful solution to address these requirements, by providing:

򐂰 Large, centralized, sharable storage capacity that is centrally managed and centrally
deployed
򐂰 With policy driven file movement between automatically managed tiers of storage
򐂰 With a central policy engine and data movement that is actively monitoring and tuning, to
drive storage utilization of a storage pool to greater than 80%
򐂰 Ability to destage inactive files automatically to external tape, virtual tape library, or
de-duplication devices, by using the integrated automated tiered storage capabilities. In
this manner, store petabytes of storage in the online file system, but only require terabytes
of disk
򐂰 Automated tools to centrally managed, centrally schedule, and centrally execute migration
of data; this migration can be as granular as an individual file basis

112 IBM Scale Out Network Attached Storage Concepts

For the general use case of large scale NAS storage consolidation, SONAS provides a
means to transform the management of the NAS storage environment (Example 4-8).

Storage environment …. Before:

High-End
Workstations

Database

End Users

Application
File Servers
Servers

Copies of Data Underutilized Segmented Storage

Figure 4-8 Example of NAS storage environment - before

SONAS can transform the foregoing environment into a globally virtualized file server
environment that can be seen in Figure 4-9.

IBM SONAS End users High end

workstations Application
Centralized Storage: servers

Global
Database
virtual
file
SONAS
server Global Global namespace
Virtual
File High performance
Server Protocols Management petabyte
CIFS Central Availability
Clustered NFS Administration Data Migration scale
Auto-failover HTTP Monitoring Replication
FTP File Mgmt Backup

Tiered storage
Auto-Tiered De-dup VT L
Storage Or tape

High perf.
Scan engine Automated movement between
tiered storage pools

Figure 4-9 Example of NAS environment - after - by using IBM SONAS

Chapter 4. SONAS usage cases 113

 SONAS is designed to provide these values to the general NAS consolidation and operations
streamlining usage case:
򐂰 Provides a super-scalar, expandable NAS storage appliance: Providing ease of ordering,
deployment, and management
򐂰 Provides centralized management of all NAS files, single namespace: Providing reduced
administrative costs and faster provisioning, along with high performance response time to
end users
򐂰 File Placement policies including automation: Providing optimized storage costs, reduced
administrative costs
򐂰 Provides all functions in one IBM SONAS Software license: Thus removing the need to
manage individual chargeable add-on software, leading to reduced Total Cost of
Ownership and simpler procurement process
򐂰 Automated policy based hierarchical storage management: Providing global Hierarchical
Storage Management, integrated tiered storage, leading to reduced administrative costs
and optimized storage asset utilization
򐂰 Independent scalability of storage and nodes: Providing simple but flexible configurations
that can be tailored to any customer specific workloads
򐂰 Truly parallel file system with concurrent access to files from all nodes, from multiple
platforms, while maintaining high performance: Providing an optimum storage platform for
very high performance yet providing ability to reduce administrative and management
costs even as the file system grows to petabytes
򐂰 Automatic load balancing and high availability, with integrated wide data striping: Providing
very high performance and fast access to data, in a 24 x 7 x 365 environment
򐂰 High performance scan engine: Providing integrated ability to perform metadata scanning
at a very high rate of speed, utilizing in parallel all available resources/nodes,
򐂰 Integrated high performance backup / restore / HSM- integration of the scan engine and
parallel data serving with IBM Tivoli Storage Manager, thus providing the ability for high
performance backups / restores / recalls of data that had previously been migrated to
external storage
򐂰 Snapshots and Asynchronous Replication: For data protection and disaster recovery

These values have a broad applicability, across a multitude of industries and applications, and
are valuable to any industry and any environment needing easy-to-manage, high
performance, scalable NAS storage.

114 IBM Scale Out Network Attached Storage Concepts

4.2.2 Industry usage cases
Examples of many of these industry usage cases are shown in Figure 4-10.

IBM SONAS – what kinds of usage cases?

Many environments needing easy-to-manage high performance, scalable storage.

Collaboration General purpose file storage environments Retail Banking &

Financial Markets
Data & General where clients are challenged with the
File Storage manageability of current NAS systems
High performance, simplified management
Digital Media for widely varying use cases in digital
medi a environments.
Hyper-scalable storage for large Web 2.0
Web Content stores and for other vendors looking to
Store
build their own Cloud/SaaS applications

High Performance Business applications such as financi al

Analytics services interested in cloud deployments
with single namespace

Energy exploration and geo-sciences Chemical & Petroleum

Energy & Geo-
Sciences require huge addressable namespaces and
very high performance.
Auto / Aero / Electronics design processes
CAE experiencing rapid file-centric storage
growth as simul ation expands. Healthcare

Figure 4-10 Overview of SONAS usage cases

In the rest of this chapter, we examine a few of these usage cases in more detail:
򐂰 Digital media and entertainment
򐂰 Analytics for competitive advantage
򐂰 Deep computing, high performance computing
򐂰 Healthcare
򐂰 Cloud Storage

4.2.3 Digital media

One of the popular usage cases for SONAS is the digital media environment.

What is digital media

In this section, we define digital media is any type of unstructured (non text, non numeric) rich
media object, such as a video or audio clips, graphics, images, or email, or a combination of
these.

Today in a media environment we are facing the convergence of two key trends:
򐂰 The growing demand for rich media content
򐂰 The exploding availability of affordable bandwidth.

This is causing enterprises and organizations of all types to evaluate how they can more
efficiently and strategically create, manage and distribute digital media.

Chapter 4. SONAS usage cases 115

 The digital media covers the following aspects:
򐂰 Content creation (harvesting, pre-processing)
򐂰 Content management (post-processing, classification, archiving)
򐂰 Content distribution (Internet, digital radio and TV, movies, pervasive devices)

The growing demand for compelling content, affordable bandwidth, and advancing network
technologies are prompting organizations to evaluate how they can efficiently manage the
digital media content throughout its entire lifecycle, as shown in the following figure. Efficient
digital media management can be used for the following purposes:
򐂰 Increase the media impact on audiences
򐂰 Enhance user experiences
򐂰 Improve productivity
򐂰 Tighten security
򐂰 Reduce costs
򐂰 Contents reutilization
򐂰 Generate new revenue

The Digital Media environment basically can be understood as a three stage data lifecycle,
broken down into the Create, Manage, and Distribute sections.

The Digital Media environment is shown in Figure 4-11.

Digital Media lifecycle

Figure 4-11 Digital media data and content lifecycle

In today’s digital media industry, this lifecycle is serviced by a large number of integrated
technologies provided by multiple technology vendors, including many technologies from IBM
and IBM partners. Consulting and integration skills in digital media are used to take
advantage of existing and evolving digital media business solutions.

116 IBM Scale Out Network Attached Storage Concepts

Clearly, there are many detailed 'ecosystem' technology subcomponents, links, and
workflows between the stages of the lifecycle, all related to the many various business
systems that handle digital media.

This ecosystem can be diagrammed in detail as a digital media workflow (Figure 4-12):

Digital media workflow

Create Manage Distribute & Transact
Set-Top
Box
Editing
Retail
Rendering LAN Display

Ingest &
Animation Indexing PC
Interactive Engine
Media Streaming
Media Server Wireless
Digital Rights Device
Video System
Content Edge
Infrastructure Content
Management Cache Wireless
Text Services Aggregation Gateway
System
Business
Image Rights System Kiosk
Distribution
E-sourcing Storage Scheduling WAN
Multi- Media
Audio repository
Gateway
Search
Middleware Managed Transaction
Business Server Playout
Hosting Server
Other Return
Data Network

Application Integration Middleware

Business Support Systems, Operational Support Systems
Rich Media Creation Warehousing Protection Infrastructure Delivery Devices

Figure 4-12 Digital media workflow

The foregoing detailed digital media open “ecosystem” is commonly implemented today using
widely available common specifications for formats, protocols, and APIs - which facilitate
interoperable solutions based on multiple vendors, including IBM vendors and IBM business
partner components. This ecosystem framework provides the assurance of a solid
architectural and organization foundation for viable, repeatable, efficient workflows when
dealing with heterogeneous platforms and applications (which is almost always the case in
digital media).

From a competitive advantage standpoint, digital media production is all about managing the
workflow, making it efficient, and speeding the time to completion.

To bring value to this workflow, IBM SONAS is designed to provide the time-saving,
labor-saving efficiency of large central high performance shared data storage, providing fast
ingest, secure yet sharable data storage, and high performance delivery capability to this
ecosystem. SONAS as a centrally managed, centrally deployed, shared storage server allows
the digital media organization to implement a robust set of media-based accelerated
workflows that can quickly deliver projects, and move media through the workflow at ever
accelerating time to delivery. At the same time, SONAS provides standard NAS storage
interfaces to allow taking advantage of already installed application and technology
infrastructure.

The value of SONAS to today’s dynamic digital media workflows by providing centrally
managed, cost-effective, high performance storage, can be summarized as shown in
Figure 4-13.

Chapter 4. SONAS usage cases 117

 Optimize digital media workflow using centralized, shared storage

Customers
(Playout,
Information & online, etc.)
Archives
Message &
Workgroups
Ca Control Services r
talo live
g De
Ac
c es
s
External
Central Deliver Customers
Storage
shes
& Ru
i ves
Arch Acc
Ingest ister ess
Workgroups Re g ro ug h
Acc / fi
ne c
og Pub ess u ts
dL
n l i sh
ss

Reg is
ea ts fini s
ni z ce Cu hed
Or
g a
Ac /Fine pr o
gr a Post

ter C
h m
ug Production
h Ro craft edit

onten
s
bli finishing
Production Pu
Workgroups

t
External
Production
Companies

Figure 4-13 Digital media - optimizing workflow using centralized, shared storage

We consider how this common theme of exploiting a centrally managed, high performance,
highly sharable storage subsystem is common across many digital media applications.

We first examine how this applies to a broadcast environment.

Broadcast environment
In a typical broadcasting environment, there are three distinct areas working with the high
quality digital streams (video and audio):
򐂰 Ingest stations (incoming stream)
򐂰 Non-linear editing (NLE) stations (handling stream)
򐂰 Play out server (play the stream out)

In addition, for any high resolution stream, there will be a low resolution version created
(for catalog and archiving purposes). You will also find common IT tasks such as
backup/restore and database operations.

Analyzing an NLE system only, you typically have from two to five parallel data clips
referenced at a time, and most of the action is “reading,” which includes shuttling through the
material. When you decide to render the material, the read/write ratio is almost 50/50,
because an NLE system reads exactly one stream as input and writes one stream as an
output (with caching in between).

Looking in addition at an entire production day, you might consider that a lot of time is used by
seeking for appropriate material and this is read-only activity. In a typical NLE environment,
there are multiple files open at a time, and the result is one file being written when rendered.
Also, most of the edit work involves to shuttle through the content, which is heavy read
activity.

118 IBM Scale Out Network Attached Storage Concepts

SONAS can provide a significant value this workflow, by centralizing the management and
optimization of this workflow, by servicing all the users simultaneously with a centrally
managed, high performance storage, as shown in Figure 4-14.

Centralized SONAS storage: usage case for Broadcast

Editing
Direct & Near-line

Broadcast

Ingest File System

Satellite, upload
servers
SONAS

Media Asset
Mgmt
Archive/HSM, DR
Copy
Compositing
Dubbing, graphics

Figure 4-14 Digital media - SONAS usage case for broadcast environment

Let us next see how SONAS might apply to a post production environment.

Post-production
Compared to broadcasting, the post-production (mastering and distribution) environments
require much higher bandwidth, because the resolution of the content is higher. Also, the
access behavior in post-production environments is more sequential, because this is
predefined work, which involves a “shuttle-in,” then working with the editing or color correction
stations. Hence, the material will be opened for read, updated on the workstation, and
written/rendered back to the storage.

The conclusion we can draw is that a centralized storage is the key to make the workflow
more efficient and to save costs. The storage in such an environment must meet the following
demands:
򐂰 Storage must be scalable: Because islands are difficult to manage and/or are slowly
accessible (possibly needing their own content management), islands must be avoided.
򐂰 Storage must perform: It must provide the speed and bandwidth for multiple read and write
streams that must be handled without interruption (concurrent access).
򐂰 Storage must be highly available and must provide for data protection: Production must be
online at all times.

It is fair to recognize that storage is not the only component involved. However, the storage is
foundational key to analyze and improving the workflow capability of all infrastructure
components. Every component can have a big influence (impact) in the infrastructure’s
performance, scalability, or high availability.

Chapter 4. SONAS usage cases 119

 We have to look at these components as part of the overall design:
򐂰 Storage subsystem (controller plus attached disks)
򐂰 File system (nodes, operating system and the file system design)
򐂰 Local area network (LAN) and bandwidth
򐂰 Attached clients (operating system, access method, specific requirements)

In all cases, it comes back to having centrally manageable, high performance storage that
can be utilized across the entire workflow to operate in an optimum manner. This is the value
that centralized SONAS storage is intended to offer in the usage case for post-production, as
shown in Figure 4-15.

Centralized SONAS storage: usage case for post-production

Editing
Color correction, grain, clean, etc

Film
Recorder
Ingest
Scanner, DataCine, File System (s)
Camera
SONAS

Media Asset
Mgmt
Archive/HSM, DR
Copy

Effects
Autodesk, DaVinci, CG, etc.

Figure 4-15 Digital media - SONAS usage case for post-production environment

Content archiving and distribution

By now, we can see the common themes that occur when applying centralized storage to a
high performance digital media workflow. In the same manner as was described for
broadcasting and post-production, we can apply the values of SONAS centralized high
performance storage in the usage case for content archiving and distribution, as shown in
Figure 4-16.

120 IBM Scale Out Network Attached Storage Concepts

 Centralized SONAS storage: usage case for
content archiving and distribution
Transcoding

Distribution
Upload
to Content
Ingest File System (s) Providers, CDNs
Upload Server, etc
SONAS

Media Asset
Mgmt
Archive/HSM, DR
Copy

Encoding

Figure 4-16 Digital media - SONAS usage case for content archiving and distribution

In summary, SONAS is designed to provide these values to the general NAS consolidation
and operations streamlining digital media usage case:
򐂰 Provides a super-scalar, expandable NAS storage appliance: Providing ease of ordering,
deployment, and management
򐂰 Provides centralized management of all NAS files, single namespace: Providing reduced
administrative costs and faster provisioning, along with high performance response time to
end users
򐂰 File Placement policies including automation: Providing optimized storage costs, reduced
administrative costs
򐂰 Provides all functions in one IBM SONAS Software license: Thus removing the need to
manage individual chargeable add-on software, leading to reduced Total Cost of
Ownership and simpler procurement process
򐂰 Automated policy based hierarchical storage management: Providing global Hierarchical
Storage Management, integrated tiered storage, leading to reduced administrative costs
and optimized storage asset utilization
򐂰 Independent scalability of storage and nodes: Providing simple but flexible configurations
that can be tailored to any customer specific workloads
򐂰 Truly parallel file system with concurrent access to files from all nodes, from multiple
platforms, while maintaining high performance: Providing an optimum storage platform for
very high performance yet providing ability to reduce administrative and management
costs even as the file system grows to petabytes.

SONAS provides all of these values and more to the digital media environment.

Let us now turn our attention to other modern IT application and environments that require
this kind of storage capability.

Chapter 4. SONAS usage cases 121

4.2.4 Modern analytics applications for competitive advantage
The emergence of today's global economy is forcing today's businesses to seek to become
more nimble with their operations and more innovative with their decisions.

In the face of exploding data volumes and shrinking batch-time windows, these businesses
are struggling to make real-time or near-real-time decisions, and thereby gain competitive
advantage.

Valuable information comes in many forms, from structured to unstructured, operational to

transactional, real-time to historical; it is scattered throughout every enterprise. This
information can reside in databases and data warehouses, emails, and transaction logs,
customer call logs, shopper behavior or repair orders, or it can be XML data locked up inside
transactional systems that cannot be used or analyzed in databases.

If this data can be unleashed and utilized properly, then modern day businesses can make
better decisions to drive sales, improve processes and services, boost team productivity,
reduce the risks inherent in doing business, and streamline relationships with customers,
trading partners, and suppliers. These are the goals of today's modern day analytics
applications.

To achieve these goals, we fundamentally must supply a way to centralize and share data
with high performance on an enterprise level, efficiently and cost-effectively.

High performance, high concurrency, highly scalable, high capacity, easily manageable
central storage (such as IBM SONAS) provides a way to do exactly that, as shown in
Figure 4-17.

Modern enterprise analytics requires

centralized, high performance, shareable storage

Data Mining Dashboards

Applications

Spreadsheets
Data Model

central
storage

Extract Load and Transform

Cubing Services

Figure 4-17 SONAS provides ability for centralized, shareable data for enterprise-wide analytics

To understand and fully take advantage of the power of this usage case, we need to
appreciate the advanced characteristics of today’s modern analytics applications, and thus
understand the specific kinds of high performance and concurrent data access that they
demand. In the remainder of this chapter we see how the high performance central storage
(such as IBM SONAS) provides significant advantage in the usage case of supporting today’s
competitive advantage analytics applications.

122 IBM Scale Out Network Attached Storage Concepts

Modern day analytics application stack
Let us begin by examining a modern Analytics application stack, and in the process, we shall
see how IBM SONAS provides a superb foundation for the unique requirements of today’s
modern competitive advantage IT applications. SONAS is positioned in the analytics
application stack as shown in Figure 4-18.

Location of shared, scalable storage in Analytics application stack

Users

User Interface Layer

Location of Dashboards, Mashups, Search,
customer Ad hoc reporting, Spreadsheets
competitive

authorization
advantage

Security
Analytic Process Layer
applications Real-time computing and anal ysis, stream computing,
entity analytics, data mining, content management, text
analytics, etc.

Infrastructure layer
Virtualization, central end to end management, control,
data proximity, deployment on global virtual file server
with high performance scale out NAS storage

Cloud Storage
IBM SONAS IBM SONAS scale out based on IBM
technology capacity / performance SONAS

Figure 4-18 Analytics - location of shared, scalable storage in analytics application stack

In this diagram, you can see that SONAS provides a foundational of storage. But what kinds
of data access patterns are generated by today’s modern analytics applications? The answer
to that question will illustrate the reason that SONAS can be a significant differentiating
solution to the greatly expanded high performance, high scale types of data access required
by modern image-intensive, real-time compute intensive, stream computing applications.

Dashboards, mashups, and search

Let us begin by discussing the topmost layer - the modern user interface.

This layer is in itself a very image-intensive, compute-intensive consumer of data and

compute cycles. As an example, notice the breadth and wide scale of data that can be
requested and integrated into today’s graphics-intensive, media-rich dashboards, mashups,
and search, as shown in Figure 4-19.

Chapter 4. SONAS usage cases 123

 Dashboards, mashups, search: collecting ever larger amounts of data
into one user interface
Enterprise Information
MQSeries
<WSDL>
JDBC DB MQ
We b services
Platforms
Web
Info Server IMS
Po rtal/por tlets D omino

WAS
We b

Google Gadge ts

Enterprise Applications

Quickr
Connections
Commerce

ERP
ECM
CRM
Legacy
Dashboards,
Mashups and Search

Figure 4-19 Analytics dashboards, mashups, and search are more data intensive than ever before

These kinds of application requirements are further expanded by the competitive need to
provide real-time updates on continuous basis, as depicted in Figure 4-20.

Advanced image-rich, real-time-update interactive tools

Figure 4-20 Analytics-based results displayed in real-time, with continuous updates

124 IBM Scale Out Network Attached Storage Concepts

These data-intensive, image-rich continuously updated modern dashboard and mashups, are
the direct result of a new type of computing that has emerged as a result of today’s readily
available amounts of affordable, huge compute power. In particular, we see a continuing and
rapid expansion of real-time stream computing.

Real time stream computing

Stream computing is a new paradigm. In “traditional” processing, one can think of running
queries against relatively large amounts of static data: for instance, “List the sum of all traffic
and energy usage data, for all sensors within 50 kilometers of the city center.” This (rather
large) query will result in a single result set.

With stream computing, one can execute a process similar to a “continuous query,” however,
it is a query that continuously identifies and tracks the constantly changing data, identifying
peaks and trends of “analyze all traffic and energy usage data within 50 kilometers of city
center” - creating continuously generated and up-to-date real-time results. These results
might be broadcast to a broad variety of locations, in addition to mashup/dashboard/search
users. For example, these results can be sent as “smart city” controls and energy
reallocations to a widespread network of large and small and mobile devices such as GPSs
and mobile handsets - and be continually refreshed over time.

In the older traditional case of data queries, static questions are asked of static data; in the
stream computing case, continuously changing data is continuously evaluated by
continuously evolving questions. Modern day stream computing analytics in this way creates
an imaginative new extension of what is possible. A simple view of this new paradigm is
shown in Figure 4-21.

Stream Computing Static data versus streaming

data: conceptual overview

Older style static data queries

Queries Database Results

Modern day stream computing analytics:

Data Queries Results

Contin uously query streaming data in real time

Continuously modify queries based on results

Figure 4-21 Analytics - stream computing

One can easily imagine how that this kind of computing starts to rapidly create major new
competitive advantage capabilities. As an example, consider how modern day analytics can
be applied and integrated into financial market workflow for trading decisions, as shown in
Figure 4-22.

Chapter 4. SONAS usage cases 125

 Trading example of real-time stream computing

Data Queries Results

Figure 4-22 Analytics - trading system example of real-time stream computing

A key aspect of this real-time stream computing technology trend is that extracting data from
a traditional OLTP database, and then loading it to a data warehouse, is becoming viewed as
strategically non-competitive. Traditional analytics had previously required data to be
extracted out of the transaction databases and loaded to the data warehouse, and this also
resulted in the creation of multiple silos of information. In addition to that problem, what is
real-time about a time-consuming extract of data from one source, and loading it to another.

Online analytical processing (OLAP)

These kinds of issues have given rise to a new evolution of traditional online transaction
processing (OLTP), and this newer evolution is named Online Analytical Processing (OLAP).
OLAP is one of various competitive advantage “real-time” imperatives that are driving the
notion of a single converged data source for all data and all analytics.

The key requirement to enable real-time stream computing analytical processing has
therefore rapidly moved in the direction of converging the transaction data and the data
warehouse, and imbedding the analytics, onto a single data source.

This has been an elusive goal for businesses for a while now; with SONAS as a foundation,
we are closer to that goal.

IBM SONAS helps achieve this goal by providing the required parallel data access, very high
scalability, linear increase in performance, manageability, and high performance
characteristics required for such environments. SONAS can provide a foundation for both a
single data source for data and also real-time analytics - directly producing the ability to
implement real-time competitive advantage applications as a result of the reduced latency in
getting access to the data in a globally shareable manner.

With these capabilities now available through SONAS, now OLAP analytics can be performed
without moving the data. This no-copy analytics capability thus sets the stage for more even
more innovation in IT competitive advantage.

126 IBM Scale Out Network Attached Storage Concepts

Data mining
Traditional data mining and search has evolved in a similar manner to stream computing,
utilizing ever larger amounts of readily available storage high performance, high scale, and
concurrent access.

Data mining has always used advanced statistical techniques and mathematical algorithms to
analyze very large volumes of historical data. The objectives of data mining are to discover
and model unknown or poorly understood patterns and behaviors inherent in the data, thus
creating descriptive and/or predictive models to gain valuable insights and predict outcomes
with high business value.

One modern example is to use data mining to improve the human condition by highly tailored
health care. By studying vast amounts of health and disease characteristics spread across
entire populations, consider the possibilities of identifying medication formulas that have been
proven to be specifically effective with other patients that have a similar genetic and DNA
makeup as your own, as shown in Figure 4-23.

Data mining

Data mining
Process

Identify
‘clustering’

Figure 4-23 Analytics - data mining in real time on the Online Analytic Processing database

However, modern real-time analytics-based data mining, takes this to a new level. One of the
great advantages of real-time, in-database mining, compared to mining in a separate
analytical environment, is direct access to the data in its primary store rather than having to
move data back and forth between the database and the analytical environment. This
approach enables the mining functions to operate with lower latency, supporting real-time or
near-real-time mining as data arrives in the data warehouse, particularly with automated data
mining processes.

Modern data mining thus includes new descriptive and predictive techniques. Descriptive
mining methods (sometimes called “unsupervised learning”) include clustering
(segmentation), associations (link analysis), and sequences (temporal links), and much more.

Summary
When requirements include the need to build a modern analytic-centric IT competitive
advantage application environment, IBM SONAS can provide strategic benefits in delivering
the storage scalability, capacity, high performance, concurrent access, and central
manageability.

Chapter 4. SONAS usage cases 127

 If this data can be unleashed and utilized properly, modern day businesses can make better
decisions to drive sales, improve processes and services, boost team productivity, reduce the
risks inherent in doing business, and streamline relationships with customers, trading
partners, and suppliers. These are the goals of today's modern day analytics applications.

To achieve these goals, SONAS can provide a fundamental storage solution to centralize and
share data with high performance, for enterprise-wide analytics storage, with efficiency and
cost-effectiveness, as shown in Figure 4-24.

SONAS provides centralized, high performance,

shareable storage suitable for modern analytics

Data Mining Dashboards

Applications

Spreadsheets
Data Model

IBM
SONAS
Storage

Extract Load and

Transform

Cubing Services

Figure 4-24 Analytics - SONAS as a central high performance, shareable storage

Many more examples of innovative thinking in competitive advantage IT applications exist,

and an easy to read, complete discussion of all of these factors and possibilities in today’s
new Analytics-based applications can be found in the IBM booklet “New Intelligence for a
Smarter Planet™”, published in October 2009, as shown in Figure 4-25.

Figure 4-25 Analytics - New intelligence for a Smarter Planet booklet

Link: A copy of this booklet can be downloaded from the Internet at the following link:
ftp://ftp.software.ibm.com/common/ssi/pm/bk/n/imm14055usen/IMM14055USEN.PDF

128 IBM Scale Out Network Attached Storage Concepts

4.2.5 Deep computing and high performance computing
“Deep Computing” is an IBM terminology that captures the notion that scientists and
researchers seek insight, that comes from exploring far below the surface of a problem. While
“high performance computing” emphasizes computer performance, the IBM term “Deep
Computing” is meant to recognize that real innovation and competitive business advantage,
requires much more than fast computing processors. Deep computing combines techniques
such as these:
򐂰 Advanced mathematics
򐂰 Domain-specific knowledge
򐂰 Specialized software
򐂰 Powerful hardware

From a usage case perspective, the goal of Deep Computing is to create solutions that help
clients develop deep insights that lead to innovation and competitive advantage. Obviously, a
large amount of storage is usually needed when building solutions that mean to exploit
today’s modern Deep Computing capabilities.

Today's deep computing can span many industries and many applications, as shown in
Figure 4-26:

Deep Computing industries and applications

Electronic
Life Sciences Design
Research, drug discovery, diagnostics,
information-based medicine Automation
Chip design and
Digital Media fabrication
Advanced digital content
creation, management and
distribution, online gaming,
surveillance
High Performance
Supercomputing driving leading Financial Services
edge applications Optimizing IT infrastructure,
Petroleum risk management and
Oil and gas exploration compliance, analytics
and production

Automotive/Aerospace
Engineering
Government & Scientific research,
Automotive, Aerospace
and Defense
Higher Educationweather/environmental
classified/defense,

Electronics & Engineering sciences

Figure 4-26 Deep Computing industries and applications

Here are typical examples of today's Deep Computing possibilities:

򐂰 Biological and Life Sciences
򐂰 Pharmaceutical discovery
򐂰 Materials and manufacturing
򐂰 Automotive and aerospace design
򐂰 Environment
򐂰 Energy

Chapter 4. SONAS usage cases 129

 򐂰 Finance
– Inventory optimization
– Value at risk
– Portfolio and trading optimization
򐂰 Security
– Homeland security
– Military simulations

These powerful technologies hold the possibility of not only of providing innovation and
competitive advantage, but also of enabling a smarter planet that improves the human
condition by stopping epidemics, inventing new medicines, understanding nature, and
protecting the environment. Modern deep computing capabilities can acquire, compute, and
create massive amounts of data, in pursuit of truly magnificent results (Figure 4-27).

Figure 4-27 Deep computing - examples of modern deep computing applications

To perform Deep Computing, a series of interlocking technologies are used, including

servers, blades, clusters, special-purpose systems and accelerators, powered by high
performance interconnects and bandwidth, driven by sophisticated software.

All of these technologies require storage. Not surprisingly, the storage requirements for Deep
Computing environments faces the same challenges in high performance, high scalability,
manageability, concurrent parallel access, high availability, and cost-effectiveness as was
previously described in 4.2.1, “General NAS consolidation and streamlining of operations” on
page 110.

IBM SONAS in Deep Computing

IBM SONAS can be an excellent fit to provide the storage for Deep Computing environments,
especially in the following circumstances:
򐂰 The workload operates well over CIFS (Windows) and NFS (UNIX/Linux).
򐂰 Concurrent access to a file from both CIFS (Windows) and NFS (UNIX/Linux) users is
required.
򐂰 A packaged, fully managed, scale out NAS storage appliance solution is desired.
򐂰 The performance requirements of an individual user can be handled by a single SONAS
interface node.

130 IBM Scale Out Network Attached Storage Concepts

SONAS is an additional option for high performance parallel file system Deep Computing
storage - SONAS is not a replacement for Deep Computing capabilities such as the IBM
General Parallel File System (GPFS). By providing a powerful subset of the IBM GPFS
capabilities, in an easy to use, easy to manage appliance format, SONAS provides a means
to gain access to existing supercomputer parallel file system, deep computing storage
capabilities, while at the same time significantly reducing the skill and implementation time
requirements that might have previously been needed.

SONAS can provide high performance streaming storage capability for Deep Computing
environments where the throughput requirements for any specific application or specific user
can be handled by the high throughput capability of a single SONAS interface node (with an
appropriate amount of attached SONAS storage).

For Deep Computing, SONAS provides all of these capabilities of IBM GPFS, in an easy to
use, appliance form factor:
򐂰 Provides a super-scalar, expandable high performance NAS storage appliance: Providing
high performance with ease of storage management, central deployment, and high
scalability
򐂰 Provides centralized management of all NAS files, single namespace: Providing reduced
administrative costs and faster provisioning, along with high performance response time to
end users
򐂰 File Placement policies including automation: Providing optimized storage costs, reduced
administrative costs for managing a Deep Computing environment
򐂰 Automated policy based hierarchical storage management: Providing global Hierarchical
Storage Management, integrated tiered storage. This is especially useful when large
amounts of Deep Computing research data needs to be staged onto and off of the actual
disk storage, allowing storing of petabytes of data on terabytes of disk. Reduced
administrative costs and optimized storage asset utilization also can result
򐂰 Independent scalability of storage and nodes: Providing very scalable, flexible
configurations that can be tailored to many types of Deep Computing workloads
򐂰 Truly parallel file system with concurrent access to files from all nodes, from multiple
platforms, while maintaining high performance: Providing an optimum storage platform for
very high performance Deep Computing, supporting multiple processes running against
the same Deep Computing data, even as the file system grows to petabytes
򐂰 Automatic load balancing and high availability, with integrated wide data striping:
Providing very high performance and fast access to data, in a 24 x 7 x 365 environment
򐂰 High performance scan engine: Providing integrated ability to perform metadata scanning
at a very high rate of speed, thus allowing ability to locate, stage, and destage data in a
large data farm that can be petabytes in size
򐂰 Integrated high performance backup / restore / HSM: Providing integration of the scan
engine and parallel data serving with IBM Tivoli Storage Manager, thus providing the
ability for high performance backups / restores / recalls of data that had previously been
migrated to external storage
򐂰 Snapshots and asynchronous Replication: For data protection and disaster recovery

SONAS, as an appliance, is designed to provide accessibility to Deep Computing parallel file

system capabilities that previously were only available in customized large scale parallel file
system environments such as IBM General Parallel File System (GPFS).

Chapter 4. SONAS usage cases 131

 IBM General Parallel File System in Deep Computing
In addition, there are many Deep Computing environments where the IBM General Parallel
File System (GPFS) might be preferred, as the requirements are a superset of IBM SONAS
capability. In particular, IBM GPFS is the solution in those environments where the client has
Deep Computing requirements to support the following needs:
򐂰 Applications that are specifically written for, and require direct parallel access to a native
parallel file system. These can include requirements for:
– Data rates to a single user that exceed the capability of a single SONAS interface node
– Application requirement to run the GPFS client code on the user’s system
򐂰 Clients that require direct Fibre Channel SAN access to the storage for specific
performance requirements
򐂰 Cross-cluster mount to existing IBM GPFS parallel file systems
򐂰 Scalability required is beyond SONAS number of interface and storage nodes, and
beyond SONAS storage scalability

For more information on IBM General Parallel File System, see the URLs shown in
Figure 4-28.

http://www.ibm.com/systems/gpfs

http://www-03.ibm.com/systems/software/gpfs/resources.html
Figure 4-28 Deep computing - URLs for IBM General Parallel File System

Summary for SONAS in Deep Computing

We are living in a time of significant challenge and extraordinary potential and change. The
emerging Deep Computing and High Performance computing landscape is extremely
complex, but this technology is also clearly allowing us to enable new applications that
address problems that were viewed only yesterday, as unsolvable.

Innovation is spanning multiple disciplines, and the creative solutions of the future are the
result of ever increasing collaboration and sharing of information between universities,
research institutes, and industry. In the end, it is not about the technology, it is what we can
do with it all that counts. Certainly, centrally manageable, securely sharable, high
performance storage clearly plays a key role in this collaboration and innovation.

With the availability of IBM SONAS, IBM now has an additional storage option to add to the
existing portfolio of IBM Deep Computing capabilities.

SONAS storage provides an additional capability, not a replacement for IBM General Parallel
File System. SONAS provides a means to gain access to powerful IBM GPFS parallel file
system deep computing storage capabilities, while simultaneously reducing the skill and
implementation time requirements that might have previously been needed to gain access to
these storage functions and performance.

Deep Computing and high performance computing installations are strongly influenced by
application needs. These applications can have a wide array of requirements, and the
specifics of the environments will determine when SONAS can provide the needed
capabilities, and when GPFS might be the preferred choice in others.

132 IBM Scale Out Network Attached Storage Concepts

4.2.6 Healthcare
Modern health care is an extremely sophisticated ecosystem, made up of a large network of
patients, providers, medical expertise, security, and financials, as shown in Figure 4-29.

Healthcare
Ecosystem
PHARMACIES 


RADIOLOGY LABS PATIENTS

Terminology
Mapping

Claims Message
Translation Normalization GOVERNMENT
PHYSICIANS’OFFICES

Clinical Data Healthcare

Health Regulatory
CDR y
Repositor Inform ation Reporting
Exchange
PROVIDERS EMPLOYERS
Security (AAA)
Access/Security Unified MPI
EMPI

Audit Logging Patient

Data Location

LABS LIFE SCIENCES

ACCESS CONTROL

PAYERS

THERAPEUTICS

1
Figure 4-29 Healthcare - ecosystem

There are many aspects to modern healthcare. We discuss one of the most important and
data intensive of these applications: modern medical imaging. Medical images are at the
center of today’s healthcare diagnostic procedures. Images have provided not only a
noninvasive means to view anatomical cross-sections of internal organs, tissues, bones, and
other human body features, but also as a means for physicians to evaluate the patient’s
diagnosis, monitor effects of treatment, and for scientific investigators to conduct research.

Medical images come from a broad spectrum of imaging sources such as computed axial
tomography (CT), magnetic resonance imaging (MRI), digital mammography and positron
emission tomography (PET), and they generate a staggering amount of image data and
important medical information. Modern medical healthcare therefore has large scale, high
performance, long term medical image data repository as a central, critical need. This image
data repository can contain comprehensive patient records, including information such as
demographics, medical history, clinical data and related diagnostic images, and
post-processed images.

With a large volume and complexity of the data, as well as diversified user access
requirements, the implementation of modern medical Picture Archiving and Communication
System (PACS) are sophisticated tasks which of course, require large amounts of centrally
manageable, cost-effective, highly available storage. Specific requirements for long term
secure archival and retrieval systems, that are scalable and upgradable with time, are also
part of the scope of these technologies. IT storage industry studies have indicated that by
2010, as much as 30% of the world’s storage can be medical images1 (Figure 4-30).

1 IBM Marketplace study Building a Smarter Planet, Healthcare, 2009-2010

Chapter 4. SONAS usage cases 133

 180MB/3D
image

10X

18MB/2D
image

2004 2007

Estimation: by 2010, medical images may take

up as much as 30% of the world’s storage

Figure 4-30 Healthcare - medical images

Regardless of the actual number, it is clear that storage requirements for Healthcare
environments supporting modern medical imaging and diagnostics, faces the same
challenges in high performance, high scalability, manageability, concurrent parallel access,
high availability, and cost-effectiveness as was previously described in 4.2.1, “General NAS
consolidation and streamlining of operations” on page 110.

We can envision IBM SONAS as providing capabilities that can help address these vital
needs for the future. In particular, previously described SONAS capabilities can be useful for
these requirements of a medical imaging system:
򐂰 Medical image data is stored and managed based on integrated ILM rules:
– Rules dictate: number of replicas, location of replicas and tier of storage
– Data can be automatically destaged and restaged either on demand or according to
central policy engine
– Aligns value of data to storage costs. Can easily change criteria of data over time or
according to other needs
򐂰 Medical image data is cached and indexed
򐂰 Medical image data stored and retrieved using standard interface (such as CIFS/NFS):
– All other operations are transparent to users / applications
򐂰 Optimized performance:
– Parallel loading of entire study / data set
– Intelligent caching speeds subsequent requests
򐂰 Stream based transport:
– Low latency access

134 IBM Scale Out Network Attached Storage Concepts

An example of such an application, which uses the IBM SONAS toolset for automated tiered
storage and ILM central policy engine, is shown in Figure 4-31.

Healthcare Medical Image: Data Storage

Using ILM toolset provided by IBM SONAS

ILM Policy

# Copies

Storage tier
Radiology
Radiology Application
Applic ation
Storage location

CIFS Share Retention period

SONAS SONAS

1001010 1001010
0011011 0011011
1000101 1000101
0101010 0101010
Storage
Storage

Primary site Secondary site

Figure 4-31 Healthcare - medical image storage example using SONAS ILM

Upon retrieval, the SONAS provides the required capabilities to serve the image, from
wherever it happens to be in the tiered storage hierarchy, including restaging from external
storage, as shown in Figure 4-32.

Healthcare Image: Data Retrieval

Using ILM toolset provided by IBM SONAS

Retrieval request

Radiology
Radiology Application
Application

CIFS Share

SONAS SONAS

1011 1011
1001010
0101 0101
0011011
1000101
1011 1011
0101010
SATA
0101 0101
Storage
St orage
1TB
SATA
Drives

Primary Site Secondary Site

Figure 4-32 Healthcare - medical image retrieval example using SONAS parallel reads

Chapter 4. SONAS usage cases 135

 IBM SONAS can provide a powerful set of tools that can be utilized for the specific
requirements of high resolution medical image storage, archival and retrieval. Of course,
SONAS, as a central shared storage infrastructure, can also be simultaneously applied to
many other aspects of the healthcare ecosystem.

In the final analysis, SONAS provides the following capabilities:

򐂰 A super-scalar, expandable high performance NAS storage appliance: Providing high
performance with ease of storage management, central deployment, and high scalability
򐂰 Centralized management of all NAS files, single namespace: Providing reduced
administrative costs and faster provisioning, along with high performance response time to
end users
򐂰 File Placement policies including automation: Providing optimized storage costs, reduced
administrative costs for managing a healthcare environment
򐂰 Automated policy based hierarchical storage management: Providing global Hierarchical
Storage Management, integrated tiered storage; all especially useful when large amounts
of medical or medical research data needs to be recalled or destaged on and off of the
actual disk storage. This supports storing of petabytes of data on terabytes of disk.
Reduced administrative costs and optimized storage asset utilization also can result
򐂰 Independent scalability of storage and nodes: Providing very scalable, flexible
configurations that can be tailored to many various types of healthcare workloads
򐂰 Truly parallel file system with concurrent access to files from all nodes, from multiple
platforms, while maintaining high performance: Providing an optimum storage platform for
very high performance healthcare image serving and medical research, supporting
multiple processes running against the same healthcare data, even as the file system
grows to petabytes
򐂰 Automatic load balancing and high availability, with integrated wide data striping:
Providing very high performance and fast access to data, in a 24 x 7 x 365 environment
򐂰 High performance scan engine: Providing integrated ability to perform metadata scanning
at a very high rate of speed, thus allowing ability to locate, stage, and destage data in a
large data farm that can be petabytes in size
򐂰 Integrated high performance backup / restore / HSM: Providing the ability for high
performance backups / restores / recalls of data that had previously been migrated to
external storage

In the end, the goal of modern healthcare is to provide affordable, optimum patient care, as
well as healthcare organizational performance. SONAS can be a powerful component to
provide the storage foundation for healthcare systems for a smarter planet.

4.3 Usage cases for scale out NAS in cloud storage

In this section we introduce cloud computing, explain the reasons why cloud is gaining
increased attention and then we discuss IBM cloud storage and the reasons for SONAS in a
storage cloud.

136 IBM Scale Out Network Attached Storage Concepts

4.3.1 The transition to cloud service delivery models
Over time, many businesses have streamlined and industrialized operations to become
smarter and more efficient, as the example shown in Figure 4-33.

Telcos Banks Manufacturers

Figure 4-33 Industrialization across multiple industries

򐂰 Telcos used to have switching centers staffed by many persons to connect calls and this
process has been replaced by switches that automate traffic to assure service and lower
cost.
򐂰 Manufacturers used to rely heavily on human labor, but now manufacturers use robotics to
improve quality and lower cost.
򐂰 Banks used to handle transactions manually and employ large numbers of persons to
perform calculations, now they use automated teller machines to improve service and
lower cost.

Breakthroughs like these are enabled by service management systems that enable whole
industries to become smarter and more efficient. IT needs to become smarter as well, and
cloud computing can bee seen as the natural evolution of IT to its next industrialized stage.

4.3.2 What is cloud computing

Cloud Computing is an emerging style of computing in which applications, data, and IT
resources are provided as services to users. Cloud computing is a service delivery model that
promises advantages such as those shown in Figure 4-34. These advantages include agility
to react to change and new requirements, cost attribution and control, and efficiency in the
utilization of underlying resources.

Chapter 4. SONAS usage cases 137

 Capability From To
Server/Storage
10-20% Cloud is a synergistic 70-90%
Utilization
fusion which accelerates
Self service None business value across Unlimited
a wide variety
Provisioning Weeks
of domains. Minutes
Change
Months Days/Hours
Management
Release
Weeks Minutes
Management Legacy
Fixed cost environments
Metering/Billing Granular
model Cloud-enabled
Payback period enterprise
Years Months
for new services

Figure 4-34 Benefits introduced by cloud computing

The drivers for cloud services can be summarized in the following points:
򐂰 First and foremost, service consumers demand lower cost options, cost control is
paramount
򐂰 As a consequence service consumers want to only pay for what they use
򐂰 Service consumers want speedy service, they want to get it fast, when they need it.
򐂰 Last but not least, to enable speed and cost control, they want it to be really easy to
manage.

4.3.3 Delivery models

Cloud computing can be implemented using multiple delivery models, as shown in
Figure 4-35, depending on the division of roles and responsibilities between the service
provider and the service consumer or end user, the customer. The cases vary depending on
who owns the infrastructure assets, where the infrastructure is located and who manages the
infrastructure.

Figure 4-35 Cloud services delivery models

138 IBM Scale Out Network Attached Storage Concepts

 The multiple delivery models illustrated above are as follows:
1. Dedicated infrastructure owned, hosted and managed by the service consumer, accessed
locally by the service consumer
2. Dedicated infrastructure owned and hosted by the service consumer and managed by the
service provider, accessed locally by the service consumer
3. Infrastructure hosted by the service consumer, owned and managed by the service
provider, accessed locally by the service consumer
4. Shared infrastructure owned, hosted and managed by the service provider and accessed
remotely by the service consumer
5. Shared infrastructure owned, hosted and managed by the service provider and accessed
remotely by the service consumer as end user

Cases 4 and 5 differ in the management of access authorizations to the service, case 4 is in a
certain sense wholesale as the service consumer manages authorization and authentication
where case 5 is a retail case where individual users access the shared infrastructure and
authentication and authorization is managed by the service provider.

4.3.4 SONAS as foundation for cloud storage

SONAS is an excellent foundation for cloud storage services. Most cloud storage services are
offered and consumed using file-level protocols such as CIFS and NFS and SONAS offers
access through these protocols and other file-level protocols such as HTTP and FTP.

SONAS is a truly scalable solution, scalability is at the core of the SONAS architecture.
Physical storage capacity can be scaled to the tens of terabytes in a single file system and
front-end nodes can scale out horizontally to offer bandwidths up to tens of gigabytes per
second.

SONAS offers availability and resiliency functions such as snapshots, integrated backups and
replication that enable uninterruptible service delivery with minimum downtime for
housekeeping operations.

4.3.5 IBM cloud storage

IBM has been offering cloud-like storage solutions for years, since the mid-1990s, so let us
look at what is new about cloud storage from what IBM has delivered in the past.

First, utilization of our storage cloud systems is greatly increased because all of the storage
capacity is managed as one unit. The data is spread evenly between all devices making
capacity management extremely simple. This makes it very easy for clients to order and then
consume what they really need, even in a private cloud system.

Our storage cloud offering also offers a simplified web user interface which IBM has
enhanced to be fully integrated into our public cloud offering as well for self-service
consumption of the service.

System management tasks such as provisioning, change and release management can all
executed by IBM greatly easing client management load.

Additionally, the storage cloud can also easily feed your chargeback and billing systems with
granular tracking of how assets are being used in the system, allowing you to build your own
internal shared storage clouds.

Chapter 4. SONAS usage cases 139

 The IBM Smart Business Storage Cloud allows you to abstract and remove the complexity of
these environments away from users and get storage resources as they are needed in a more
agile way. The market has increasing demand for file services. The growth of so-called
unstructured data outpaces that of databases by far.

Current NAS solutions do not scale. File systems are often limited in practice to a few
terabytes and a few million objects, computing power in a single system is miniscule in an
Enterprise context, so you have to add box by box and manage them individually.

Difficult way to apply policies across this independent data islands. Workflow, regulatory
compliance hampered. “Find the file” becomes “find the server with the file, then find the file”.
Massive scalability in multiple dimensions, ability to scale Interface and Storage nodes
separately to suit workload, and greatly simplified administration/management.)

Deep integration of Information Lifecycle Management (ILM) functions into the file system
becomes more important as the amount of TB's explode – scalability and speed critical to
actual execution of an ILM or Tiered Storage strategy. Global namespace, deeply integrated
policy engine, integrated backup and replication of single logical device. Backup windows are
a big issue and get worse while the amount of data continuously increase. Internal distribution
of data often causes “hotspots” and very low total utilization (as low as ~15%)

We see commercial requirements on data access rates and response times to individual files
that have been unique to HPC environments in the past. Logical storage pools of many
physical devices increases utilization, eliminates hotspots and increases performance by
wide striping of data across all devices. Making file servers clustered is easy in theory but
hard to build and maintain, there is much complexity.

Migration, integration or removal of storage for file services is typically difficult. Very proven,
mature IBM core technology coupled to simplified interfaces allowing non-disruptive online
building block addition or removal of individual components, easing growth or technology
upgrades.

IBM has announced the client-premise hosted service called Smart Business Storage Cloud
that can be implemented behind client firewall in managed or un-managed configurations and
as an IBM hosted offering. IBM has also announced the Smart Business Storage on the IBM
Cloud service that offers access by subscription to IBM hosted storage cloud services on IBM
infrastructure based on SONAS. Figure 4-36 illustrates these two offerings.

Figure 4-36 IBM Smart Business Storage Cloud services

140 IBM Scale Out Network Attached Storage Concepts

 The Smart Business Storage Cloud service is focused on the needs of enterprise clients,
particularly those with a focus on the highest degrees of security, privacy and control and for
those that have a business need for a higher level of customization that is typically available in
a public cloud model. This offering can be implemented behind your firewall or as part of our
strategic outsourcing capability.

We have also launched a true public cloud for storage as well, called Smart Business Storage
on the IBM Cloud. Delivered in a public cloud model with a full pay-per-usage pricing model
and all the high degrees of automation, this cloud will also be focused on the needs of our
enterprise clients. We are raising the bar for storage cloud offerings by providing higher
performance, more reliability, and an all-inclusive pricing model that the competition cannot
match.

It is important to understand these services in context of one another as IBM sees the future
of cloud technology for most installations to be in the use of what we call ‘hybrid’ cloud
implementation models. In this model the clients workloads can be even further expanded
through the use of both cloud types in tandem. Inherent technology in the cloud software will
allow us to enable seamless, policy-based movement of data to and from the private and
public cloud spaces.

IBM Smart Business Storage Cloud offers design and implementation services as well as
ongoing basic, premium and fully managed support services, providing you with a single point
of contact for asset, configuration, performance and problem call management. Through
around-the-clock monitoring and reporting of the production cycle, we help you optimize your
data storage environment. Additionally, you also have the option of allowing us to manage
your storage cloud, so you can reduce the need for high skills in network file system and
storage technologies.

4.3.6 Sharing the storage cloud: Tenancy

When discussing storage cloud solutions we often start discussing the concept of tenancy
and multi-tenancy. An individual storage infrastructure can have one or more service
consumers or tenants that use it, a tenant is a service consumer or customer using an
infrastructure. When a single customer uses a physical infrastructure we have single -tenancy
and when multiple customers share a physical infrastructure we have multi-tenancy.

Authentication and access to file services is controlled by an authentication mechanism in the

case of SONAS, a Microsoft Active Directory (AD) server or an LDAP server.

A Service consumer customer accessing file services might have their own LDAP or AD
server that contains the profiles of the service consumer customer’s end users. Alternatively,
the service consumer can rely on an AD/LDAP service provided by the service provider.

Currently a SONAS cluster can be connected to a single AD or LDAP server, and so we have
two possible use cases:
򐂰 In the first case the service consumer or customer uses a central AD/LDAP service
provided by the service provider, the service provider manages and assigns users for
each customer. In this case multiple service consumers can access the same SONAS
cluster.
򐂰 In the second case the service consumer or user uses and maintains his own AD/LDAP
server, in this case the SONAS cluster can be used by a single service consumer or
customer.

Chapter 4. SONAS usage cases 141

4.4 Summary
In this chapter we have defined three general types of NAS storage:
򐂰 Traditional NAS
򐂰 Scalable NAS
򐂰 Scale out NAS

We then saw that large scale, fast growing, modern competitive advantage IT applications,
scale out NAS can often be a highly desirable approach to solving requirements for:
򐂰 Tremendous growth of “unstructured data” with the same (or less) administrative staff
򐂰 Need for NAS solutions tot scale in both performance and capacity
򐂰 Provide parallel access to data to support modern analytics and compute intensive
competitive advantage application
򐂰 Provide ability to centrally manage hundreds of terabytes to petabytes of data, while
driving high utilization of storage with high performance
򐂰 Resolve situation where backup windows are too small for 100’s of terabytes
򐂰 Provide an easy way to centrally apply and enforce policies across all the NAS storage
򐂰 Provide method to transparently and automatically migrate data as needed
򐂰 Provide an automated method to globally implement centrally managed, centrally
deployed Automatic Tiered Storage (i.e. Information Lifecycle Management)

IBM SONAS is designed as a scale out NAS appliance, to provide a solution to all of these
issues.

We then proceeded to examine SONAS applicability to a variety of usage cases, including

generalized NAS consolidation, high performance analytics and deep computing, and also
examined specific applications for the digital media and healthcare industries.

Finally, we saw the applicability of IBM SONAS to cloud storage.

IBM SONAS is designed to provide a cost-effective, highly scalable, centrally managed NAS
storage solution for all of these usage cases. In the remainder of this book, we review the
high-level concepts of IBM Scale Out NAS (SONAS).

142 IBM Scale Out Network Attached Storage Concepts

 5

Chapter 5. Next steps

Now that we have seen the IBM Scale Out NAS (SONAS) concepts, architecture, and usage
cases, let us consider what are the next steps.

After reading this chapter, you will know where to go next to investigate and evaluate IBM
SONAS further for possible use in your IT organization.

This chapter directs you to information about SONAS in other Redbooks publications, to
websites and white papers, and provides information about IBM Services related to SONAS.

© Copyright IBM Corp. 2010. All rights reserved. 143

5.1 Where to go from here
In the following sections, we list where you can go to obtain more information about SONAS.

5.1.1 Other SONAS Redbooks publications

If you want to read more technical information about IBM SONAS, download and read the
companion book SONAS Architecture, Planning and Implementation Basics, SG24-7875.

This book is available online at the following URL:

http://www.redbooks.ibm.com/abstracts/sg247875.html?Open

This book goes into more detail on IBM SONAS, and discusses topics in detail, such as:
򐂰 SONAS Hardware architecture
򐂰 SONAS Software architecture
򐂰 SONAS networking considerations
򐂰 SONAS backup and restore, availability, and resiliency
򐂰 SONAS configuration and sizing
򐂰 SONAS installation and configuration
򐂰 SONAS administration, including:
– Security
– Information Lifecycle Management and automated tiered storage
– Asynchronous replication
– Tivoli Storage Manager integration
򐂰 SONAS migration overview
򐂰 Getting started with SONAS

You can download this Redbooks publication for more detailed SONAS information.

5.1.2 Websites about IBM SONAS

IBM maintains websites where you can find more information about IBM SONAS, such as
these.
򐂰 SONAS Manuals can be read online at the SONAS information Center:
http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/index.jsp
򐂰 IBM general information pages about SONAS:
http://www-03.ibm.com/systems/storage/network/sonas/
򐂰 IBM technical support pages about SONAS:
http://www-947.ibm.com/systems/support/supportsite.wss/selectproduct?brandind=5
000029&familyind=5388917&oldbrand=5000029&oldfamily=5388917&oldtype=0&taskind=1
&psid=bm&continue.x=15&continue.y=20

5.1.3 ISV software supported

When designing SONAS, IBM planned enablement of infrastructure ISV applications from the
beginning. This means that a wide variety of ISV applications can be utilized quickly with the
knowledge that IBM has confirmed their interoperability.

IBM has created a whole SONAS Eco-System in order to validate the SONAS solution into
existing and relevant ISV environments. SONAS provides File Services and Resource

144 IBM Scale Out Network Attached Storage Concepts

 provisioning and relies on ISVs for Storage Management applications. The SONAS
Eco-System consists of validated and qualified ISV applications that utilize SONAS storage to
provide you risk-minimized as well as scale out solutions.

You might already be committed to an ISV solution in wide domains such as Storage
Management, Data Protection, Virtualization or Data Base environment. And obviously it is a
hard requirement to have business continuity when your SONAS solution is integrated into
your existing infrastructure. You then have to ensure that the SONAS Storage solution is
compatible with your ISV solutions and it will scale to meet your expectations.

The validation of the SONAS solution to be interoperable with middleware such as databases
or security protections, as well as with your business applications, is a key criteria, but
SONAS has also to provide you acceptable performance and throughputs.

This validation process done by IBM with ISVs is aimed to help you to reduce the deployment
effort and risks by delivering you already tested applications. For additional information about
the SONAS Storage solution and compatibility with your ISV solutions, see the website:
https://www-03.ibm.com/systems/storage/solutions/isv/index.html#symantec

5.1.4 More information about IBM SONAS components

IBM SONAS is a scale out NAS storage appliance. IBM SONAS Software is an integrated
packaging of many existing and powerful functionalities, as shown in Figure 5-1.

CIFS NFS FTP HTTPS future

Cluster manager
SONAS
HSM and ILM Monitoring Agents
Parallel Software
File System
Backup & Restore GUI/CLI mgmt functions
And Interfaces

Snapshots and Policy Engine

Security
Replication

Enterprise Linux

IBM Servers

Figure 5-1 SONAS Software functionality and components

There is a broad variety of existing information available on the Internet about each of these
functions and components, and we list a few of those here.

Chapter 5. Next steps 145

 SONAS parallel file system and policy engine (GPFS)
The SONAS parallel file system and integrated storage policy engine is based on the IBM
General Parallel File System (GPFS).

To learn more about IBM GPFS concepts and capabilities, watch this 21-minute self-playing
website video:
http://www-03.ibm.com/systems/data/flash/clusters/software/gpfs/training/intro/she
ll_popup.html

General information about IBM GPFS can be found at:

http://www.ibm.com/systems/gpfs

Excellent detailed GPFS technical information can be found on IBM DeveloperWorks:

http://www.ibm.com/developerworks/wikis/display/hpccentral/General+Parallel+File+S
ystem+%28GPFS%29

Here is another excellent page of GPFS resources:

http://www-03.ibm.com/systems/software/gpfs/resources.html

There are a number of Redbooks publications that have been written on IBM GPFS. You can
search for these books on the Redbooks publications website:
http://www.redbooks.ibm.com/cgi-bin/searchsite.cgi?query=GPFS

GPFS is also available on Wikipedia:

http://en.wikipedia.org/wiki/GPFS

Tivoli Storage Manager

IBM Tivoli Storage Manager performs two major functions with SONAS:
򐂰 Provides an accelerated backup and restore capability with high performance, using the
Tivoli Storage Manager server connected to SONAS by either NFS or CIFS
򐂰 Provides Hierarchical Storage Management (HSM) of SONAS disk data to external tape,
tape libraries, virtual tape libraries, or de-duplication devices.

An existing Tivoli Storage Manager customer can use their existing Tivoli Storage Manager
Server to perform backups and restores on SONAS, as well as to perform HSM. The SONAS
Software (5639-SN1) includes imbedded Tivoli Storage Manager client code inside the
SONAS (you do not need a separate license for the internal Tivoli Storage Manager client
code inside the SONAS). The following links provide additional information about Tivoli
Storage Manager:
򐂰 General information about Tivoli Storage Manager:
http://www-01.ibm.com/software/tivoli/products/storage-mgr/
򐂰 Tivoli Storage Manager manuals online:
http://publib.boulder.ibm.com/infocenter/tsminfo/v6/index.jsp
򐂰 Tivoli Storage Manager publications on the Redbooks website:
http://www.redbooks.ibm.com/cgi-bin/searchsite.cgi?query=Tivoli+AND+storage+AND
+manager

146 IBM Scale Out Network Attached Storage Concepts

 Red Hat Enterprise Linux
The internal operating system of SONAS is based upon Red Hat Enterprise Linux (RHEL).
More information about RHEL can be found at:
http://www.redhat.com/rhel/

IBM System x servers

SONAS is based upon standard commercial enterprise IBM System x servers. You can find
more information about these servers at:
http://www-03.ibm.com/systems/x/

5.1.5 Education
For general educational information about clustered file systems, a set of web articles on this
topic has been written by the independent consultant “Clustermonkey”. You can view these
articles at:
http://www.clustermonkey.net/

5.1.6 IBM Services

IBM Global Services provides installation, planning, and education about IBM SONAS.
IBM Business Partners might also offer similar services for installing IBM SONAS. For
information about this topic, contact your IBM Business Partner.

To learn more about IBM Implementation Services for Network Attached Storage systems –
IBM Scale Out Network Attached Storage, contact your IBM representative or IBM Business
Partner, or visit the following website:
http://www-03.ibm.com/systems/storage/network/sonas/

5.1.7 White papers on SONAS ISV software certification

IBM has established an external website, where technical white papers on certification of
various software on SONAS are hosted and available for download.

5.1.8 TCOnow! for SONAS

IBM specialists and IBM business partners have access a Total Cost of Ownership (TCO)
analysis tool called TCOnow! for SONAS, provided by the third-party software company
CIOview. TCOnow! for SONAS allows them to generate a business case for replacing an
existing storage infrastructure with a SONAS solution. The tool is based on a built-in online
questionnaire that performs an assessment of the existing storage infrastructure and
workloads. The tool does provide many inputs with dynamic defaults based on industry best
practices, so this makes it easier to collect the data for a specific installation.

This assessment covers information about the existing storage hardware infrastructure that
can either being direct attached storage (DAS), SAN and NAS storage from major vendors.

The tool allows you to asses Information about installed storage and fabric components,
planned and unplanned outages, personnel costs and maintenance costs.

Chapter 5. Next steps 147

 The assessment also collects information about the current system metrics such as
performance requirements, percentage of reads/writes and random/sequential usage,
throughput any cache hit ratios.

All the above information, together with the expected growth rates for storage, number of
users and servers is used to generate a SONAS design proposal. This proposal together with
the existing infrastructure data is then used to calculate and compare the cost breakdown in
the following categories: hardware, software, networking, services, facilities, ongoing
personnel, and downtime.

The results, presented both in tables or graphs, together with all input data can be used to
generate an extensive report about the current storage and proposed SONAS environments.

5.2 Contact your IBM or IBM Business Partner representative

For more information about IBM Scale Out NAS, or to arrange a consultation about SONAS,
contact your IBM or IBM Business Partner representative.

148 IBM Scale Out Network Attached Storage Concepts

Related publications

The publications listed in this section are considered particularly suitable for a more detailed
discussion of the topics covered in this book.

IBM Redbooks publications

For information about ordering these publications, see “How to get Redbooks publications” on
page 149. Note that various documents referenced here might be available in softcopy only.
򐂰 IBM Scale Out Network Attached Storage Architecture, Planning and Implementation
Basics, SG24-7875
򐂰 IBM eServer xSeries and BladeCenter Server Management, SG24-6495
򐂰 Configuration and Tuning GPFS for Digital Media Environments, SG24-6700
򐂰 IBM Tivoli Storage Manager Implementation Guide, SG24-5614

Online resources
These websites are also relevant as further information sources:
򐂰 SONAS Support Site:
http://www.ibm.com/storage/support/
From this website, select:
Product family: Network Attached Storage (NAS)
Product: Scale Out Network Attached Storage
Click Go.
򐂰 Support for IBM System Storage, TotalStorage and Tivoli Storage products:
http://www.ibm.com/storage/support/
򐂰 Additional GPFS documentation sources:
http://www.ibm.com/systems/gpfs
http://www-03.ibm.com/systems/software/gpfs/resources.html
򐂰 NFS V4 ACL information:
http://www.nfsv4.org/

How to get Redbooks publications

You can search for, view, or download Redbooks publications, Redpapers publications,
Technotes, draft publications, and Additional materials, as well as order hardcopy Redbooks
publications, at this website:
ibm.com/redbooks

© Copyright IBM Corp. 2010. All rights reserved. 149

Help from IBM
IBM Support and downloads:
ibm.com/support

IBM Global Services:

ibm.com/services

150 IBM Scale Out Network Attached Storage Concepts

Index
data access layer 60
Numerics data growth contributors 3
96-port switch 47 data lifecycle management 26
data management services 83
A data mining 127
access control 82 data repository 76
Access Control List 82 database application
ACL 82 file access 4
Active Directory 75, 141 Deep Computing
analytics applications 122 examples 129
Apache daemon 63 GPFS 132
architecture 8, 41 deep computing 129
asynchronous replication 91, 93 descriptive mining methods 127
authentication 141 Digital media
active directory 75 broadcasting environment 118
ID mapping 75 Digital Media environment. 115
authentication methods 74 direct attached storage 6
disk expansion unit 49
Domain Name Server 16
B drive types 56
backplane bandwidth 48 drives intermix 56
backup restore 35
base rack
Feature Code 9003 53 E
Feature Code 9004 53 education 147
Feature Code 9005 53 Ethernet switch
base racks 52 private internal 48
Block I/O 4 Ethernet switches
external 48
expansion rack 55
C interface 54
cabling 48 expansion unit 50
CIFS 60 external network 48
CIFS file serving 64 external ports 49
classical NAS external storage
workload balance 107 data movement 35
classical NAS environment 112 external storage pools 28, 85
cloud computing
definition 137
delivery models 138 F
ILM functions 140 File I/O 4
Cloud Storage 105 File management policies
cloud storage 136 file attributes 29
Cluster Manager 63 file management policies 29
failover failback 70 threshold option 30
function 70 file placement policies 28
functions 64 file sharing 72
clustered grid architecture 16 file-serving capabilities 43
Command Line Interface 101 FTP support 62
configurations 52
connection types 50 G
content archiving and distribution 120 General Parallel File System 8
global cluster 64
D global namespace 10
dashboard and mashups 125 GPFS 131

© Copyright IBM Corp. 2010. All rights reserved. 151

GPFS file system maximum file system 77
snapshot 82 multi-tenancy 141
GPFS technology 77
GPFS-based token 81
grid parallel architecture 78 N
NAS
access 6
H limitations 7
hardware structure 42 overview 6
Health Center 99–100 storage consolidation 113
healthcare 133 NAS consolidation 110
Hierarchical Storage Management 28 NAS storage
high availability 83 classical 107
high availability design 12 scalable 106–107
high concurrency 24 Scale Out NAS 109
HTTPS protocol 62 Scale out NAS 106
NFS clients 62
NFSv4 82
I
IBM Global Services 147
ID mapping 75 O
infiniband fabric 43 Online Analytical Processing 126
InfiniBand switches 46 overview 1
interface expansion rack 54
interface node 23
automatic balancing 18 P
cache memory 10 parallel data stripe 22
interface node overview 43 parallel global clustered environment 109
interface nodes 10, 15, 42 parallel grid architecture 36
failover 68 peered pools 86
locking 66 performance 22
recovery 66 physical disk 56
internal connections 51 physical disk drive 56
internal data replication 83 Physical disk drives 16
internal data transfer 51 policies 84
internal Infiniband switch 47 policy
internal storage pools 28 collection of rules 85
IP address balancing 67 policy engine 16, 83
ISV software 147 policy engine rules 28
policy rule syntax 30
post-production environment 119
L predictive techniques 127
LDAP 74
lifecycle
summary 39 R
lifecycle of a file 13 rack types 52
lifecycle of files 84 RAID controller 50
locking capability 72 read performance 24
logging and trace 47 real time stream computing 125
logical storage pools 16, 26 Redbooks publications website 149
Contact us xi
replication
M asynchronous 91
manage storage 26
management GUI 98
Management node 46 S
component connection 51 SAS configuration 55
management node 42 SAS HDDs 44
system health center 46 SATA drive considerations 55
management services 98 Scalable NAS 107
marketplace requirements 2 architectures 108
applications 3 scalable NAS 106

152 IBM Scale Out Network Attached Storage Concepts

scale out capability 10 storage consolidation 113
Scale Out NAS 106 storage management 11
Scale out NAS architecture 109 storage node 45
scan engine 30, 82 storage pod 49
functions 33 switches 47
summary 34 System Software product 46
security 141 Tivoli Storage Manager 37, 95
server nodes 42 Tivoli Storage Manager guidelines 96
Services for UNIX 76 use cases 105
Share Modes 61 SONAS Software 16
Snapshots 89 Storage attachment 51
snapshots 82 Storage capacity
software 11 adding 49
software functionality 14 storage node 45
software updates 43 storage nodes 15, 43
SONAS 98 storage pod 45, 49
architecture 8, 41 streamlining NAS 114
authentication methods 74 stripe data 67
backup 36 switches 47
backup restore 35 Symantec security 12
CIFS 60 System Health Center 46
cloud storage 136, 139
Cluster Manager 63
clustered node architecture 25 T
component connections 50 tenancy 141
configurations 52 tiered pools 86
create file 16 Tivoli Storage Manager 35
data access layer 60 client software 97
data repository 76 configuration 94
Deep Computing 130 HSM requirements 89
deep computing 129 licensing 97
Digital Media environment 116 Tivoli Storage Manager HSM 87, 95
external storage pools 28 Tivoli Storage Manager server 28
file access 44 topology viewer 100
FTP support 62 Total Cost of Ownership 147
GPFS 131
hardware structure 42 U
healthcare 133 usage cases
high concurrency 24 analytics applications 122
HTTP protocol 62 use cases 105
internal storage pools 28
lifecycle summary 39
logical storage pools 26 V
manage storage 26 virtualized file server 113
management services 98 Volume Shadow Services 90
maximum file system 77
NFS clients 62
overview 1, 8
X
XCOPY utility. 69
parallel file system 71
parallel global clustered 109
performance 22
policy engine 83
raw storage 9
requirements 110
scale out capability 10
scale to petabyte level 109
scan engine 30
Snapshot 61
software 11
solution 112

Index 153
154 IBM Scale Out Network Attached Storage Concepts
IBM Scale Out Network Attached
Storage Concepts
IBM Scale Out Network Attached
Storage Concepts
IBM Scale Out Network Attached Storage Concepts
IBM Scale Out Network Attached Storage Concepts
(0.2”spine)
0.17”<->0.473”
90<->249 pages
IBM Scale Out Network Attached
Storage Concepts
IBM Scale Out Network Attached
Storage Concepts
 Back cover ®

IBM Scale Out

Network Attached
Storage Concepts ®

Introduces IBM Scale IBM Scale Out Network Attached Storage (IBM SONAS) is a Scale Out
NAS offering designed to manage vast repositories of information in INTERNATIONAL
Out NAS concepts
enterprise environments requiring very large capacities, high levels of TECHNICAL
performance, and high availability. SUPPORT
Details hardware and
ORGANIZATION
software architecture SONAS provides a range of reliable, scalable storage solutions for a
variety of storage requirements. These capabilities are achieved by
Provides using network access protocols such as NFS, CIFS, HTTP, and FTP.
configuration
considerations Utilizing built-in RAID technologies, all data is well protected with BUILDING TECHNICAL
options to add additional protection through mirroring, replication, INFORMATION BASED ON
snapshots, and backup. These storage systems are also characterized PRACTICAL EXPERIENCE
by simple management interfaces that make installation,
administration, and troubleshooting uncomplicated and IBM Redbooks are developed
straightforward. by the IBM International
Technical Support
In this IBM Redbooks publication, we discuss the marketplace Organization. Experts from
requirements that led to the SONAS stand-alone storage system. We IBM, Customers and Partners
introduce storage architectures and file systems. We then introduce from around the world create
the reader to SONAS and describe the hardware and software that timely technical information
make up the SONAS appliance.
based on realistic scenarios.
Specific recommendations
are provided to help you
implement IT solutions more
effectively in your
environment.

For more information:

ibm.com/redbooks

SG24-7874-00 ISBN 0738434957