IBM Systems Storage

IBM Storwize V7000 Unified

Planning & Implementation

Matthew Robinson – IBM Australia ([email protected])

© 2012 IBM Corporation
 Special Programs:

Comprehensive education, training and service
IBM Systems ‘Guaranteed to Run’ Classes --
offerings Make your education plans for classes
with confidence!

Expert instructors and consultants,
world-class content and skills

Instructor-led online (ILO) training

Multiple delivery options for training The classroom comes to you.
and services

Conferences explore emerging trends
Customized, private training
and product strategies

Lab-based services assisting in high tech
solutions
www.ibm.com/training
© 2012 IBM Corporation
Agenda

Overview

Hardware & Components

Software & Features

Cabling

System Initialization

Online code upgrade

Shares, Exports, and Disk structure

Security & Authentication

Asynchronous Remote Replication

File clones

Snapshots

3
© 2012 IBM Corporation
One Year Later… Technology Integration Continues
Storwize V7000 Unified Storage

Leveraging the best of IBM storage technologies in a midrange disk system

Snapshots Innovative
GUI
SAN
Easy Tier

Virtualization
RAID

IBM Active
Clustered File
Cloud
Engine™ Systems Support
4
© 2012 IBM Corporation
What’s different?

Active/Active Unified Namespace across all file modules

IBM Active Cloud Engine™ technology delivers automated storage efficiency
capabilities
– True policy-based management of files
– With user-defined policies!

Based on highly successful Storwize V7000 block storage system

Fully integrated user interface for block and file data simplifies management
– Not a “unified” launcher for two different interfaces

Simple USB key based initial setup
– Extends successful Storwize V7000 approach

Built-in IBM Tivoli Storage Manager client simplifies
– backup to TSM server
– and Hierarchical Storage Management to tape or virtual tape library with dedu-plication
– System also supports NDMP for third-party backup clients

5
© 2012 IBM Corporation

OVERVIEW

6
© 2012 IBM Corporation
Overview

Network File Serving
– NFS v2/v3, CIFS, HTTP, FTP & SCP HTTPS

– Clustered parallel file system CIFS NFS

– Quotas Management NIS/ AD FCP

● User, group and fileset level quotas
● Soft limits, hard limits, grace periods IP layer SAN layer
DFS

Block Access
iSCSI
– FCP & iSCSI
CIFS NFS HTTPS FTP SCP NDMP
– Disk Storage Virtualisation Engine Active Cloud Engine

Scalability and Performance Clustered parallel file system
iSCS
FCP
– 2 file modules I
Storage Virtualisation Engine
– Up to 240 hard disk drives + external virtualisation IBM Storwize V7000U
– Directly attached to Storwize V7000

Information Lifecycle Management
Storage Pools
– Policy driven placement, movement, migration &
deletion of files
– Storage tiering (Disk-to-Disk-to-Tape)
– Integrated HSM agent to migrate inactive files to
Shared Integrated Disk
tape or Deduplication VTL
7
© 2012 IBM Corporation
Overview (cont.)

Data Protection & Security
– File Access Protocols
● Up to 256 snapshots per file system and/or filesets
● Asynchronous IP Remote replication for file access
● Integrated TSM 6.2 Backup/Archive client or NDMP
● “Scan on access” & “scan on demand” Antivirus
– Block Access Protocols
● Point-In-Time Copy (FlashCopy/snapshot)
● Virtual Disk Mirroring
● Sync. & Async. Remote Replication for block access
● FlashCopy Manager for Windows, VMware, Linux, AIX, Solaris & HP-UX
● VMware Site Recovery Manager

Centralized Management & Administration
– GUI & CLI
– Centralized alert & event log
– Event notifications via email or SNMP
– Performance Monitoring

RAS
– Centralized monitoring of entire system via System Health Center
– Call home and remote service features
– Fully redundant capability in all components for High Availability
8
 © 2012 IBM Corporation
Software & Hardware Components

Backup & • Symantec Netbackup

Migration • Legato Networker
File Access Protocols Protocol • CommVaultSampana

CIFS NFS HTTPS FTP SCP NDMP

Cluster Manager
Clustered Trivial Database (CTDB)
IBM Support Call home Parallel File System Monitoring agents
& • Symantec
Storwize GUI (web) & CLI Anti-Virus Scan AV server
• McAfee
V7000U Admins Management Interfaces Active Cloud Engine Security AD/LDAP/NIS server
WAN caching IBM GPFS
Remote Storwize Backup & Recovery TSM server
V7000U Panache ILM TSM client
Remote Storwize Disaster Recovery Rapid Data Recovery HSM TSM server
V7000U Remote Replication Snapshots HSM agent
Base Operating System
RedHat Enterprise Linux 6 (64bit)
Hardware
Servers: IBM System x 3650M3 (x2)
Storage: Storwize V7000

9
© 2012 IBM Corporation
Enterprise Workload Landscape…
What Fits Where?
LEAD
Application Workload Class Comments
WITH
DBMS B (F) For larger (>20TB) instances, lead with XIV
Oracle
eBusiness Suite B (F)
OLTP B OLTP/BASIS
SAP
BWH F (B) Analytics
Content Mgt. Filenet, Documentum, etc. F Metadata layer may be block/RDBMS
Media Streaming VOD, AOD, IPTV F Very performance/latency sensitive; also potential for tape/LTFS
2010 B Strong ESRP results for both V7K & XIV (see here)
MS Exchange
2003/2007 B
Lotus Notes B (F) Back end is DB2 database; predominantly block

Virt. Infrastructure B (F) Block: mature (also XIV); File: emerging (MS HyperV – block only)
VMware
VDI B/F
IBM DB2 RDBMS B
SAS Analytics B/F Block: mid-range, File: grid; XIV certified & strong; V7KU TBD
Imaging F Ex.: Cachet Database
PACS/EMR
Prod. Workflow B/F “Front Office” OLTP: block; Patient archives: file

B: Block
10
F: File
© 2012 IBM Corporation

HARDWARE & COMPONENTS

11
© 2012 IBM Corporation
Base Configuration

6 up to 24 RU solution

File modules are provided in a pair (cluster)

Same code as SONAS
– Storwize V7000U is like a SONAS appliance’s storage pod with Storwize V7000 control
enclosure as a back-end storage subsystem, and the interface/management nodes
embedded in the storage nodes

Each file module has all functions
– Management node  a node is active while the other is standby
– Interface node  both nodes are active
– Storage node  both nodes are active and directly attached to Storwize V7000

Direct redundant LAN connectivity between both file modules

12
© 2012 IBM Corporation
Storwize V7000 Unified Architecture

External Local Area Network

(1GbE & 10GbE)

Global Name Space

Each File Module has the Interface, Storage & Management role functions Installed & Activated
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Management Interface role 1 2

Sys t emx3 650 M 3 Interface role Management
1 2
Sys t emx3 650 M 3

role Active*
5 G0 B
0
N L -S A S
5 0 0
NL
G B
- S AS

Storage role
3 4

5 G
0 0
NL S
B
- AS
5 0 0
NL

Storage role
G B
- S AS role Standby*
3 4

No Single Point
File Module File Module
Of Failure
Single Management
Interface
To manage both file & block access Active Tiering

Storage Module

13 (*) only the management role function is active on 1 File Module and Standby on the other File Module
All Interface & Storage role functions are active at the same time on both File Modules
© 2012 IBM Corporation
Storwize V7000 Unified File Module

IBM System x 3650M3 (2U)

Quad Core Intel® Xeon® E5620 (Westmere)
– 8 threads, 2.40 GHz, 12MB cache, 80W

72GB DDR3 memory Custom bezel to mask
the unused drive bays

2 x 500GB NL-SAS
mirrored drives
DVD drive

Internal SAN Free/Unused PCIe slots

2 x 8Gbps FC ports
QLogic HBA directly
connected to the
Storage Module

Management
Ethernet (IMM)
Unused

External IP Network Internal IP Network External IP Network Redundant PSU

2 x GbE ports 2 x GbE ports 2 x 10GbE ports
14 Directly connected to Emulex CNA
the partner File Module
© 2012 IBM Corporation
Storwize V7000 Unified Storage Module
Storwize V7000 Enclosure

Storwize V7000 enclosure (2U)
– 2 Storage Bridge Bay (SBB) canisters for the electronics
– Redundant power supplies
● 764W per PSU in the control enclosures
● 500W per PSU in the enclosure enclosures
– Internal drive bays
● Enclosure for 12 x 3.5” drives (NL-SAS)

● Enclosure for 24 Small Form-Factor 2.5” drives (SSD, SAS & NL-SAS)
Note: Both types may be mixed freely in the same subsystem.

15
© 2012 IBM Corporation
Storwize V7000 Unified Storage Module
Storwize V7000 control canister

1st enclosure is configured with 2 controller canisters

– Intel Jasper Forest processor (SKU 4, EC3539, 4 cores, 8MB cache, 2.13 GHz, 65W)
– 8 GB DDR3 memory
– Internal 32GB MLC SSD for boot and fire-hose dump
– Both controller canisters communicate over an internal 8xPCIe Gen2 link in the mid-plane

Two 6Gbps SAS ports

Two USB2 ports (connections to expansions)
Four 2/4/8Gbps Two 1Gbps
Ethernet ports Optional Two 10Gbps Top node canister
Fibre Channel
Ethernet ports
ports

Bottom node canister

Note: identical to the top node
Redundant Power Supply Unit (PSU) canister, but installed upside down
Note: also Power Cooling Module (PCM)
& Battery

16
© 2012 IBM Corporation
Storwize V7000 Unified Storage Module

Up to 9 expansion enclosures may be attached

to the control enclosure
– An expansion enclosure is configured with two
SBOD canisters (Switched Bunch Of Drives)
– Each SBOD canister contains the following
components:
● 36-port SAS expander to attach the internal drives
and provide an enclosure management processor
● Two 4x6Gbps mini-SAS connectors for chaining
enclosures
● Both SBOD canisters communicate via internal I2C
busses in the enclosure mid-plane

Up to 120 3.5” drives, 240 SFF drives or a
mixture of both in a Storwize V7000 Storage
Module

17
© 2012 IBM Corporation

SOFTWARE & FEATURES OVERVIEW

18
© 2012 IBM Corporation
Storwize V7000 Unified Features

Centralized Management & Administration
Block Access Protocols
– GUI & CLI – FCP & iSCSI
– Performance Monitoring – Disk Storage Virtualisation Engine
– Centralized alert & event log ● Expand or shrink Volumes on-line
Volume

– Event notifications via email, ● Thin-provisioned Volumes Storwize V7000

Syslog or SNMP
– Call home and remote service features ● On-line Volume Migration MDisk MDisk
Source Target

– Authentication service for Single Sign-On & LDAP ● EasyTier: Automatic relocation of hot and cold extents
SSDs HDDs SSDs HDDs

Network File Serving Automatic
Relocation
– NFS v2/v3, CIFS, HTTP, FTP & SCP
– Clustered parallel file system
Hot-spots Optimized performance and
– Active Cloud Engine throughput
– Data protection & Security
● Policy driven placement, movement, migration & deletion of files
● Point-In-Time Copy (FlashCopy/snapshot)
● Parallel Scan Engine
> Full , Partial, Space Efficient, Incremental,
● Storage tiering (Disk-to-Disk-to-Tape) Cascaded, Consistency Groups & Reverse
Up to 256
Volume

Tier
Tier33 Storwize V7000
● Integrated HSM agent to migrate inactive files to tape or Dedup VTL
● Virtual Disk Mirroring
● WAN caching (SoD) Volume Volume
copy 1 copy 2
– Active Directory, or LDAP or NIS support
● Synchronous & Asynchronous Remote Replication
– Quotas Management
● FlashCopy Manager for Windows, VMware, Linux, AIX,
– Data protection & Security Solaris & HP-UX
● Up to 256 snapshots per file system

VMware
● Asynchronous IP Remote replication for file access
● Integrated TSM 6.2 Backup/Archive client or NDMP – vCenter Server management plug-in & VAAI
● “Scan on access” & “scan on demand” Antivirus support
19
– Storage Replication Adaptor for SRM
© 2012 IBM Corporation
Virtualisation Concepts & Limits

Hosts
Single Multipath driver •Up to 256/1024 Hosts per IO group/cluster
Volumes •Up to 512/2048 FC ports per IO group/cluster
•Up to 2,048/8,192 Volumes per IO group/cluster (SDD, MPIO)
•Up to 256/1024 iSCSI ports per IO group/cluster
•Max. Volume size of 256TiB
iSCSI, FCP or FCoE
• Image Mode = Native Mode
Host Mapping
• Managed Mode = Virtualized Mode LAN SAN

Storwize
V7000
SSD SAS NL-SAS

Storage Pools
•Up to 128 MDisks per
Pool
Vol11
Vol2 Pool
Vol32 Pool 3
Vol7
Vol4 Pool 4 Pool
Vol65
Vol5 Storage Pool
•Up to 128 storage pools
per cluster

Internal Storage External Storage

Internal Storage Up to 4,096 MDisks Max. External MDisk External Storage

1 to 8/16 drives (HDD) to form an Array Size of 2TiB Multiple Storage Subsystems
1 Array  Managed Disk (MD) Or greater(1) 1 LUN  Managed Disk (MDisk)
Round-robin port selection for back-end storage subsystems
20 (1) Check Storwize V7000 interoperability website to obtain the list of Disk Subsystems supported with greater than 2TB MDisks
http://www-03.ibm.com/systems/storage/disk/storwize_v7000/interop.html
© 2012 IBM Corporation

CABLING

21
 Internal Cabling of Storwize V7000 Unified
© 2012 IBM Corporation

Between File Modules & with the Storage Module

A pair of direct Ethernet connections (1Gbps) between each File Modules
– Clustering connectivity

Each File module has a direct FC (8Gbps) connection to a port per node canister
in the Storage Module
– 32Gbps of bandwidth between File and Storage Modules
File Module 1 File Module 2

Storage Module

2 FC ports (8Gbps) per node canister in the Storage Module can be used for
external FC connection
– Host/Server connectivity for native block access using FC protocol (target SCSI port)
– Back-End virtualised external storage (initiator port)
– Remote Replication of Volumes (LUNs) with another Storwize V7000 Storage Module
22
© 2012 IBM Corporation
Internal SAS Cabling of the Storwize V7000
Storage Module

A Storage Module (Storwize V7000 Control Enclosure) can connect to up to 9

Expansion Enclosures

96Gbps of Back-End bandwidth
– Redundant SAS connectivity
– 24Gbps bandwidth per SAN cable
● 6Gbps per physical link
● 4 physical links per SAS cable
– Each node canister in the Control Enclosure is connected to up to 2 daisy chains
23
© 2012 IBM Corporation
External Cabling of Storwize V7000 Unified

Up to 120Gbps of Front-End bandwidth

– 44 Gbps for File Access protocols, NDMP, management & IP remote replication
– 76Gbps for Block Access protocols & management

External SAN

File Module 1 File Module 2 Storage Module

24
© 2012 IBM Corporation
Minimum Cabling of Storwize V7000 Unified

Provide connectivity for the cluster management as well as the service of each File Module 
ethX0 MUST be connected
– ethX0 is made of the GbE ports 3 & 4 on the motherboard (forms by default an adaptive load balancing bond)

Direct point-to-point Ethernet Connection between each File Modules  mgmt0 MUST be
connected
– mgmt0 is made of the GbE ports 1 and 2 on the motherboard (forms by default an Active-Passive bond)

Provide connectivity for the cluster management as well as the service of each Storage Module 
Ethernet port 1 of each node canister MUST be connected

Direct point-to-point Fiber Connection between the File Modules and the Storage Module
– File Module 1 port 1 (PCI card 2, right port)  Storage Module node 1 (top canister) port 1 (top left port)
– File Module 1 port 2 (PCI card 2, left port)  Storage Module node 2 (bottom canister) port 1 (top left port)
– File Module 2 port 1 (PCI card 2, right port)  Storage Module node 1 (top canister) port 1 (top left port)
– File Module 1 port 2 (PCI card 2, left port)  Storage Module node 2 (bottom canister) port 1 (top left port)

Port 1 of each node canister of the

Storage Module must be cabled
ethX0
mgmt0

Direct FCP connectivity

25
© 2012 IBM Corporation

SYSTEM INITIALIZATION

26
© 2012 IBM Corporation
Documentation & Links

To start… 3 documents to download and read
– IBM Storwize V7000 Unified Preinstall Guide
● http://www-01.ibm.com/support/docview.wss?uid=ssg1S1004016
– Storwize V7000 Unified Installation: Error Recovery Hints and Tips
● http://www-01.ibm.com/support/docview.wss?uid=ssg1S1004017
– IBM Storwize V7000 Unified authentication: The how-to guide
● https://www-304.ibm.com/partnerworld/wps/servlet/ContentHandler/stg_ast_sto_wp_ibm_storwize_unified_authentication

Then access all ... most.... Storwize V7000 Unified documentation
– Information Centre
● http://publib.boulder.ibm.com/infocenter/storwize/unified_ic/index.jsp
● http://publib.boulder.ibm.com/infocenter/storwize/ic/index.jsp
– SAN Volume Controller, Storwize V7000 & Storwize V7000 Unified PDF documentation & MIB (for SNMP traps)
PREINSTALL
● ftp://ftp.software.ibm.com/storage/san/sanvc/V6.3.0/
GUIDE_V1_1.doc

Upgrade, maintenance, plan, limits...
– Interoperability, configuration limits and restrictions
● http://www-03.ibm.com/systems/storage/disk/storwize_v7000/interop.html
– Concurrent Compatibility and Code Cross Reference for Storwize V7000
● http://www-01.ibm.com/support/docview.wss?&uid=ssg1S1003705
– Concurrent Compatibility and Code Cross Reference for Storwize V7000 Unified
● http://www-01.ibm.com/support/docview.wss?uid=ssg1S1003918
– Supported Drive Types and Firmware Levels for IBM Storwize V7000 and IBM Storwize V7000 Unified
● https://www-304.ibm.com/support/docview.wss?uid=ssg1S1003842
– IBM Tivoli Storage Manager Server/Client Compatibility and Upgrade Considerations
● https://www-304.ibm.com/support/docview.wss?rs=663&context=SSGSG7&q1=1053218&uid=swg21053218
– Storwize V7000 Unified environmental requirements
● http://publib.boulder.ibm.com/infocenter/storwize/unified_ic/topic/com.ibm.storwize.v7000.unified.doc/easy_preparephysicalenv.html?resultof=%
22%65%6e%76%69%72%6f%6e%6d%65%6e%74%61%6c%22%20%22%65%6e%76%69%72%6f%6e%6d%65%6e%74%22%20
– Support Matrix for Subsystem Device Driver, Subsystem Device Driver Path Control Module, and Subsystem Device Driver
Device Specific Module
27 ● http://www-01.ibm.com/support/docview.wss?rs=540&context=ST52G7&uid=ssg1S7001350&loc=en_US&cs=utf-8&lang=en
© 2012 IBM Corporation
Sample Statement Of Work (SOW)

Planning workshop (1/2 day)
– Review implementation roles and responsibilities
– Verify that all hardware, software and site prerequisites are met and reviewed based on the End User’s current environment
– Discuss specific implementation requirements, including: IP addresses, Domain Name, authentication server address,
security accounts, NTP server address, TSM server address and account, etc.

Hardware implementation (1/2 day)
– Racking & cabling
– Check Ethernet and Fiber Channel cabling

Software implementation (3 days)
– Initialisation of the storage and file modules using the USB key
– Preparing the Security Domain
– Preparing the DNS
– Ezsetup wizard
– Creating file systems (up to 3)
– Creating file sets (up to 6)
– Creating shares/exports (up to 6) & test access to them for each share type: CIFS, NFS, HTTPS, FTP & SCP
– Setting up and scheduling snapshots (up to 6) & test restore from a Windows client and a Unix client
– Setting up TSM (optional HSM set up) or NDMP  An additional day to setup and test (would require more days if TSM
server must be setup 3 to 5 days)
● Scheduling the backup of each file system
– Setup & test placement, migration and deletion policies  Advance policies with lots of granularity could require more days
– Setup antivirus  Would require more days if you go beyond the setup
– Setup remote replication & test Failover/failback  Would require 1 additional day at least to go beyond the setup and test
Failover/Failback

Handover and Documentation (1 day)
– Conduct a walkthrough of configured systems with an assigned technical staff member
– Prepare As-Built documentation
28
© 2012 IBM Corporation
Understanding Block Access vs. File Access
from an implementation point of view

File access protocols and block access protocols have very different requirements
– Block Access protocols are using your Storage Area network to access storage devices
● LUN masking and SAN Networking are used provide access to storage
● Networking is usually straight forward with a single or dual core-edge topology
● Fabric redundancy is more than recommended
– File Access protocols are using your Local Area Network to access shared resources
● Routing & gateways must be carefully understood and set up
> ethX0 correspond to the internal GbE ports which is bond and used primarily for management and service
♦ May be used for client access to shared resources
> ethX1 correspond to the 10GbE CNA used primarily for front-end client access via CIFS, NFS, etc.
● Authentication is the level of security to filter access to shared resources
> It is using either of the following method: Active Directory, LDAP & NIS (Usually AD is being used)
> Networking is critical
> DNS reverse lookup must be defined
> Time on the File Modules must be synchronised with the time on the directory services controller  Usually a
Network Time Protocol server should be defined and commonly used by Storwize V7000 Unified and the
controllers such as AD controllers

29
© 2012 IBM Corporation
Prerequisites

Authentication
– Set up to handle UID and GID identifiers

Domain Name Services

Network connectivity
– Switches configured to handle LAGs
– Connectivity between file modules and V7000 enclosure
– No VLAN tagging during initial setup

Can be a good idea to bring your own switch for set up

– Not depending on the networking guys

30
© 2012 IBM Corporation
USB key 1st initialisation wizard

1. Put the USB key onto your workstation and enter IP addresses for Storage
Module (Storwize V7000) and File Modules
2. Plug the USB key into the Storage Module (Storwize V7000)
– Wait for both green LEDs at the rear of the node canisters to stop blinking and the
orange LED to disappear
3. Remove USB key then plug USB key into one of File Modules
– Wait for blue LEDs at the rear and in front of the File Modules to go off
● The LEDs will go from a blinking state to a steady state then will either
> go off  configuration was successful
> blink again  configuration was unsuccessful
Note: the LED on the File Module in which the USB key was inserted could be in steady state, while it would be
blinking again after a steady state on the remote File Module, when the code of the Storage Module is
upgraded

4. Remove USB key then return it into your workstation to confirm results

Notes:
– USB key configures network addressing between Storage and File Modules
● Allows Storage and File Modules to refer to and address each other
– USB key also carries credential (ssh key) from the Storage Module to the File Modules
● Enables secure communications between the both layers
31 ● Utilizes the NAS “User role”
© 2012 IBM Corporation
Storwize V7000 Unified Initial Setup

32
© 2012 IBM Corporation
Sample IP addresses assignment
Description Link Speed Bonding IP address Status 1st time setup Following changes

10.2.20.200
Storage Module - Cluster Management 1GbE No Mandatory USB 1st Initialisation wizard2 CLI or GUI
Virtual1
Storage Module - Service Node 1 Port 1 1GbE No 10.2.20.201 Recommended USB key or CLI or GUI USB key or CLI or GUI
Storage Module - Service Node 2 Port 1 1GbE No 10.2.20.202 Recommended USB key or CLI or GUI USB key or CLI or GUI
Storage Module – iSCSI Node 1 Port 1 1GbE No 10.2.20.203 If required CLI or GUI CLI or GUI
Storage Module – iSCSI Node 1 Port 2 1GbE No 10.2.20.204 If required CLI or GUI CLI or GUI
Storage Module – iSCSI Node 2 Port 1 1GbE No 10.2.20.205 If required CLI or GUI CLI or GUI
Storage Module – iSCSI Node 2 Port 2 1GbE No 10.2.20.206 If required CLI or GUI CLI or GUI
Storage Module – iSCSI Node 1 Port 3 10GbE No 10.2.20.207 If required CLI or GUI CLI or GUI
Storage Module – iSCSI Node 1 Port 4 10GbE No 10.2.20.208 If required CLI or GUI CLI or GUI
Storage Module – iSCSI Node 2 Port 3 10GbE No 10.2.20.209 If required CLI or GUI CLI or GUI
Storage Module – iSCSI Node 2 Port 4 10GbE No 10.2.20.210 If required CLI or GUI CLI or GUI
File Modules - Cluster Management 10.2.20.211
1GbE Yes Mandatory USB 1st Initialisation wizard3 CLI or GUI
Ports 3 & 4 of active management node Virtual1
File Modules - Service Node 1 Ports 3 & 4 1GbE Yes 10.2.20.212 Mandatory USB 1st Initialisation wizard3 CLI or GUI
File Modules - Service Node 2 Ports 3 & 4 1GbE Yes 10.2.20.213 Mandatory USB 1st Initialisation wizard3 CLI or GUI
File Modules - Client Access 10.2.20.214
1GbE Yes If required EZsetup wizard or CLI or GUI CLI or GUI
ethX0 on Node 1 Ports 5 & 6 Virtual4
File Modules - Client Access 10.2.20.215
1GbE Yes If required EZsetup wizard or CLI or GUI CLI or GUI
ethX0 on Node 2 Ports 5 & 6 Virtual4
File Modules - Client Access 10.2.20.216
10GbE Yes If required EZsetup wizard or CLI or GUI CLI or GUI
ethX1 on Node 1 Ports 7 & 8 Virtual4
File Modules - Client Access 10.2.20.217
10GbE Yes If required EZsetup wizard or CLI or GUI CLI or GUI
ethX1 on Node 2 Ports 7 & 8 Virtual4
(1) Always active on the management/configuration node; could failover to the other node (Port 1 of Storage node canisters, Ports 3 & 4 of File modules)
(2) In the second screen of the USB initial setup preparation
33 (3) In the third screen of the USB initial setup preparation
(4) Multiple virtual IP addresses with their own VLAN tagging ID and gateway could be assigned; They can also failover within the same port group to the partner node
© 2012 IBM Corporation
V7000 Storage Module

Will flash orange when USB stick is placed in canister

Solid green if node is part of a cluster, Solid green if power is on

otherwise it will flash green

Solid green if node is part of a cluster,

Solid green if power is on otherwise it will flash green

Will flash orange when USB stick is placed in canister

34
© 2012 IBM Corporation
V7000 File Headers

Solid green when switched on

Flashing blue when ready to set up in a cluster

If setup was successful the blue light will switch off

35
© 2012 IBM Corporation
USB Setup Fails

Check the V7000 Unified Information Centre

– http://publib.boulder.ibm.com/infocenter/storwize/unified_ic/index.jsp

36
© 2012 IBM Corporation
EZSetup wizard – Step 3

Use the management IP address defined for the File Modules during the USB key
initialisation wizard
– ie. https://172.17.1.203
● Default ID = admin
● Default Password = admin

Note: With the storage module, the default ID is “superuser” and its password “passw0rd”

37
© 2012 IBM Corporation
EZSetup wizard – Step 1

License agreement

38
© 2012 IBM Corporation
EZSetup wizard – Step 2

The “System Name” is applied to the Storwize V7000 Storage Module & File
Modules
– The GPFS cluster name & Storwize V7000 Unified UID is based on the serial number of
the first management node
● The SONAS/GPFS/Storwize V7000U cluster’s name is created using the serial number of that first
management node: ie. 1300006.ibm or 1300037.ibm, etc.

The “NetBIOS” will be used for authentication with the Active Directory and
cannot be changed once it is defined with this wizard

Even if Network Time Protocol servers are defined during this step in the wizard,
they will only be really setup later at the end of the setup
39
© 2012 IBM Corporation
EZSetup wizard – Step 3

This correspond the Storwize V7000 Storage Module licenses

Can easily be setup later from the Web Interface

40
© 2012 IBM Corporation
EZSetup wizard – Step 4

Can easily be setup later from the Web Interface

41
© 2012 IBM Corporation
EZSetup wizard – Step 5

It is very important to define here DNS servers which can be accessed using the
management and service IP addresses defined during the USB key initialisation
process

Some of the following steps rely on this set up

Can easily be setup later from the Web Interface but should be defined now

42
© 2012 IBM Corporation
EZSetup wizard – Step 6

Authentication with AD or LDAP could be defined at this stage

It can also easily be defined later from the Web Interface

It may also easily fail since Network Time Protocol servers have not been defined
yet and time synchronisation is a fundamental requirement

43
© 2012 IBM Corporation
EZSetup wizard – Step 6

44
© 2012 IBM Corporation
EZsetup wizard – Step 7

Organise your enclosures to match your current rack configuration

Can easily be setup later from the Web Interface

45
© 2012 IBM Corporation
EZsetup wizard – Step 8

This correspond to Storwize V7000 Storage Module RAID Array configuration

Can easily be setup later from the Web Interface

46
© 2012 IBM Corporation
EZsetup wizard – Step 9

This correspond to the network connections used for the front-end client access
to shares and exports

Can easily be setup later from the Web Interface

47
© 2012 IBM Corporation

48
© 2012 IBM Corporation

ONLINE CODE UPGRADE

49
© 2012 IBM Corporation
Online & Automated Code Upgrade
Upload the file

Once you have

– Checked for new upgrades
– Downloaded the upgrade code from IBM’s support site

You can upload the file on Storwize V7000 Unified cluster

50
© 2012 IBM Corporation
Online & Automated Code Upgrade
Install/Upgrade the Code

51
© 2012 IBM Corporation

SHARES, EXPORTS, & DISK

STRUCTURE

52
© 2012 IBM Corporation
Terminology

A File is made up of metadata and data

A Fileset is a logical sub-tree of a file system that provides more

granularity, flexibility and manageability for features such as Quota
management, Snapshots, Policies, etc.

A File System is created using Network Shared Disk (a LUN)

from 1 or more storage pools

A Storage pool is a group of NSDs within a file system

A Network Shared Disk (NSD) a Logical Unit Number (LUN) assigned to

a number of File Modules

53
© 2012 IBM Corporation
Network Shared Disks

A NSD is a V7000 block volume that has been assigned to the file headers to be
used as file storage
– Invisible in the volumes menu of the GUI

Can be placed on internal or external drive
– Currently CIFS is not supported on external storage

NSDs belong to only one storage pool and only one file system

NSDs

54
© 2012 IBM Corporation
Array’s Considerations
with Storwize V7000 Unified

Try to use 8+P (RAID5), or 8+P+Q (RAID6) with Storwize V7000 Unified
– The GPFS file system could use a block size of 256KiB (default), 1MiB or 4MiB when
accessing its file volumes (NSD) on Storwize V7000 file modules
– An Array on the internal drives (Hard Disk Drives) of Storwize V7000 storage module
would have a default stripe unit size of 256KiB at an array level, or an optional stripe unit
size of 128KiB (when using CLI)

RAID RAID Stripe Stripe Stripe

Topology Type Unit Width Size
256KiB 8 2MiB
8+P 5
128KiB 8 1MiB
256KiB 8 2MiB
8+P+Q 6
128KiB 8 1MiB

55
© 2012 IBM Corporation
Storage Pools

Logical unit inside a file system

Relates to one physical (block) storage pool
– A file storage pool belongs to one physical storage pool
– A physical storage pool can have file storage pools in many file systems

File storage pools are used for automatic tiering inside a file system
– Files are moved between pools based on policies

Storage Pools

56
© 2012 IBM Corporation
Automatic Tiered Storage

Information Lifecycle Management means Tiering between
internal Disk Pools

Hierarchical Storage Management means Tiering with external
storage pool (usually tape) with file pointer left in the File System

At multiple levels
– Storage Pool level (A group of LUNs)
– Fileset level (Define sub-trees of a File-System)

Using Policies for rule based management of files
– Placement
– Migration
– Deletion
– Exclusion

Delivering
– Scalability  billions of files
– One global File-System name space across independent storage pools
– Hierarchical storage based on files
● Files in the same directory can be in different pools
Tier
Tier 33 ● Files placed in storage pools at create time using policies
● Files moved between pools via automated policy-driven tiered storage
– Hierarchical to both disk & tape
57 – Allows classification of data according to Service Level Agreements
© 2012 IBM Corporation
Scan Engine

/home

Scan Engine reads internal file system metadata

Does not need to read the file or directory tree
/appl

All Modules can participate in scan of file system
/data Metadata scan  Scan > 10 million files per minute per File Modules
/web
All File Modules participate in parallel scan engine

1. Start scan

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Management Interface role 1 2

Syst emx3 650 M 3 Interface role Management 1 2
Sys t emx3 650 M 3

role Active*
5 G0 B
0
N L -S A S NL

Storage role
5 0 0
G B
- S AS
3 4

5 G
0 0
NL S
B
- AS
5 0 0
NL

Storage role
G B
- S AS role Standby* 3 4

File Module File Module

Storwize 2. Read policies
V7000 ACE
Unified 3. Parallel Scan
4. Return results of Scan Storage Module

58
(*) only the management role is active on 1 File Module and Standby on the other Module (All Interface & Storage roles are active at the same time on both File Modules
© 2012 IBM Corporation
Data Movement
Information Lifecycle Management
/home
/home/appl/data/web/
/appl
important_big_spreadsheet.xls
No change to the
big_architecture_drawing.ppt
/data logical directory
unstructured_big_video.mpg
/web

All File Modules participate in

parallel data movement
5. Perform results of scan
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Management Interface role 1 2

Syst emx3 650 M 3 Interface role Management 1 2
Sys t emx3 650 M 3

role Active*
5 G0 B
0
N L -S A S
5 0 0
NL
G B
- S AS

Storage role
3 4

5 G
0 0
NL S
B
- AS
5 0 0
NL

Storage role
G B
- S AS role Standby* 3 4

File Module File Module

Storwize
V7000 ACE
Unified
Storage Module

59
(*) only the management role is active on 1 File Module and Standby on the other Module (All Interface & Storage roles are active at the same time on both File Modules
© 2012 IBM Corporation
Data Movement
Hierarchical Storage Management
/home
/home/appl/data/web/
/appl
important_big_spreadsheet.xls External TSM server

/data
big_architecture_drawing.ppt
unstructured_big_video.mpg
/web

Tier
Tier 33

All File Modules participate in

5. Perform results of scan parallel data movement
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Management Interface role 1 2

Syst emx3 650 M 3 Interface role Management 1 2
Sys t emx3 650 M 3

role Active*
5 G0 B
0
N L -S A S
5 0 0
NL
G B
- S AS

Storage role
3 4

5 G
0 0
NL S
B
- AS
5 0 0
NL

Storage role
G B
- S AS role Standby* 3 4

File Module File Module

Storwize
V7000 ACE
Unified
Storage Module
Stub files left on disk  Auto-recall if accessed
...

60
(*) only the management role is active on 1 File Module and Standby on the other Module (All Interface & Storage roles are active at the same time on both File Modules
© 2012 IBM Corporation
Data Placement & Lifecycle Policies

ACE ensures efficiencies by identifying

– Which files need to be moved
– And where they should be stored over their lifetime

Example 1: “Placement policies, evaluated at file creation”

→ rule important_files set pool gold for fileset key_application_fileset
→ rule other_files set pool silver

Example 2: “Migration policies, evaluated periodically”

→ rule ilm_gold migrate from pool gold threshold (90,70) to pool silver
→ rule ilm_silver when day_of_week() = monday migrate from pool silver to pool
bronze where access_age > 30 days

Example 3: “Deletion policies, evaluated periodically”

→ rule purge_bronze when day_of_month() = 1 delete from pool bronze where access_age
> 365 days

61
© 2012 IBM Corporation
Define Active Cloud Engine ILM Policies

Modeled on Storwize
V7000 volume
creation dialog

62
© 2012 IBM Corporation
Define Active Cloud Engine ILM Policies
Display and manually edit policy rules

Advanced users can

display rules created
through the GUI

63
© 2012 IBM Corporation
File System

A file system is a separate instance of a GPFS file system

A file system is made up of one or more storage pools

Upon creation of a file system:
– A storage pool named “system” is created
● You can define additional file system at creation time as well as later
– Five NSDs equally sized (or as close to equal) NSDs are created for storage pool

File Systems

64
© 2012 IBM Corporation
Logical Structure of a File System

File System

Up to 256 File Systems per Storwize V7000
Unified cluster

Max. 8PB per file system
File Sets

Up to 3,000* filesets per File System

Up to 65,536 (216) sub-directories per

directory

Up to 263 files per file system (4,294,967,296

= 232 currently tested)

File attributes include folder name

→ File data can be moved to different storage pools
within the file system

Folders
(*) Up to 3,000 dependent and 1,000 independent filesets:
65 •An independent file set has a separate inode space, but shares physical storage with the remainder of the file system
•A dependent file set shares the inode space, quota limits and snapshot capability of the containing independent file set
© 2012 IBM Corporation

66
© 2012 IBM Corporation

67
© 2012 IBM Corporation
File Sets

File sets are a logical subset of a file system

Two types of file sets:
– Independent
● Has its own inode table, can be snapshotted and have quotas set against it
● Up to 1000 in a file system
– Dependant
● Inherits inode table space and quotas
● Cannot be snapshotted
● Up to 3000 in a file system

Provide the ability to partition a file system to provide more granular
administrative options

68
© 2012 IBM Corporation

69
© 2012 IBM Corporation

70
© 2012 IBM Corporation

71
© 2012 IBM Corporation

72
© 2012 IBM Corporation
Shares/Exports

Shares and/or exports are used to make sections of your file system accessible

Shares can be accessed via CIFS, NFS, HTTP, FTP, or SCP

Ensure that share ACLs if implemented match up with directory and file ACLs

73
© 2012 IBM Corporation

74
© 2012 IBM Corporation

75
© 2012 IBM Corporation

76
© 2012 IBM Corporation

77
© 2012 IBM Corporation

78
© 2012 IBM Corporation

SECURITY & AUTHENTICATION

79
© 2012 IBM Corporation
Authentication Overview

SONAS & Storwize V7000 Unified supports the following authentication methods
– Active Directory (Active Directory itself provides Kerberos Infrastructure)
– Active Directory with Services for Unix (SFU), also called Integrated Management for Unix
(IMU) and/or RFC2307 schema
– LDAP
– LDAP with Kerberos
– SAMBA PDC / NT4 mode

Prerequisites
– Authentication server is external to Storwize V7000 Unified and must have a proper
network connectivity
● Authentication Server needs to configured separately
– Administrative information for authentication server such as admin account, password,
SSL certificate, Kerberos keytab file, … are obtained in advance, as required
– Network MUST be properly set up before configuring the authentication method
– All the Storwize V7000 Unified nodes have synchronized time to be same as
authentication server such as Active Directory Server and/or Kerberos KDC server
● Active Directory domain controller can be used as a NTP time source

80
© 2012 IBM Corporation
Authentication Limitations

Only one of the authentication methods can be configured at a time

The authentication method is chosen during installation of the NAS system and it
is not recommended to change it later

The number of groups per user that are evaluated for Authorization on files and
directories is limited to ~1000

The authentication server is a essential component in the data access flow  if
the authentication server is unavailable then the data access is disturbed

At least one authentication command must be run to set up the NAS protocols
– To enable the use of CIFS, HTTP, FTP/SCP, the authentication command must configure
an external authentication server such as Active Directory or LDAP*

If future use of Async-Replication or Caching is considered, external ID mapping is
mandatory from the start

81
(*) a configuration allowing NFS only access is possible without external authentication server
© 2012 IBM Corporation
User ID mapping Concepts

NFS CIFS

UID / GID SID (User or Group Windows Active

Directory

Storwize V7000 Unified

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Management Interface role 1 2

Sys t emx3 650 M 3 Interface role Management 1 2
Sys t emx3 650 M 3

role Active*
5 G0 B
0
N L -S A S
5 0 0
NL
G B
- S AS

Storage role
3 4

5 G
0 0
NL S
B
- AS
5 0 0
NL

Storage role
G B
- S AS role Standby* 3 4

File Module File Module

GPFS uses Linux

UID / GID

User mapping between Shared

LD MAP DB
Windows SID and UNIX CTDB
UID/GID

82
© 2012 IBM Corporation

ASYNCHRONOUS REMOTE
REPLICATION

83
© 2012 IBM Corporation
Asynchronous Remote Replication - Overview

Connects 2 Storwize V7000U over a LAN or WAN

Asynchronous protection of 1 or more File Systems through file based replication
– Each source File System is a separate relationship
– Each relationship is invoked on a periodic basis based on RPO requirements

A high speed scan of the source File System is performed to determine the
created/modified/deleted files and directories

Replication performed by “rsync” tool
– Only moves changed portions of files to destination
– Replication over SSH provides encrypted movement of data between
Storwize V7000U

A “default” snapshot of the source file system is created at the beginning of the
async process
– Snapshot is deleted after successful async

A “default” snapshot of the target file system is created after successful async
process
– Provides image of target file system after async completes

Next completed async will replace contents target snapshot
84
© 2012 IBM Corporation
Asynchronous Remote Replication - Architecture

The primary site holds the production File System

The secondary site receives async replicas of the primary site’s File System

Changes are tracked on the primary site  replica File System should not be modified
– Exports to the destination file system should be defined as Read-Only to prevent accidental
delete or modify
– Future async cycles will update/replace destination File System contents to match source
– Destination file content will be replaced by async during the process

CIFS
NFS AD or
HTTPS
FTP
IP layer LDAP
or NIS

Snapshot Snapshot

Read-Only

Source File-tree Replica File-tree

85
© 2012 IBM Corporation
Schedule Remote Replication with the GUI

New Replication

Source File System:

Education

Destination File System:

Education Backup

Frequency of Replication:
Once a day

Days of Week: Hour: Minute:

Every day 02 00

Encryption Method:
Strong

Create Snapshot on Source System

Create Snapshot on Target System

Compress Files

OK Cancel

86
© 2012 IBM Corporation
Asynchronous Remote Replication - Failover

To continue Read/Write production at site 2 through a site 1 outage, exports to
replica at destination should be changed to R/W
– Modification to site 2’s file system can now be made
– Assumption is that concurrent changes to site 1’s file system are not taking place

Remote Storwize V7000U should be accessed for their production workload

DNS should be changed to connect to remote Storwize V7000U

Modify their application shares to point to remote Storwize V7000U

CIFS
NFS AD or
HTTPS
FTP
IP layer LDAP
or NIS

Snapshot Snapshot

Read-Only

Source File-tree Replica File-tree

87
© 2012 IBM Corporation
Asynchronous Remote Replication - Failback

Recovering source Storwize V7000U is achieved by defining a reverse async
relationship from site 2 back to site 1

Replication times back to source depends on scope and duration of outage

A “fullsync” option should be used for initial sync back from site 2 to site 1
– Will attempt to sync all data from site 2 to site 1, will discover existing files at site 1, rsync
will determine which files needs to be synced back

A staged recovery back to site 1 may be required depending on restore time

CIFS
NFS AD or
HTTPS
FTP
IP layer LDAP
or NIS

Snapshot Snapshot

Read-Only

Source File-tree Replica File-tree

88
© 2012 IBM Corporation
Multiple Uni-directional Relationships

Each Storwize V7000U location has production file system(s) which are replicated
between each other
– Can be more file systems from Storwize V7000U 1 to Storwize V7000U 2
– Can be some going from Storwize V7000U 2 to Storwize V7000U 1
Snapshot Snapshot

Read-Only

Source File-tree Replica File-tree

A A’

Snapshot Snapshot

Read-Only

Replica File-tree Source File-tree

B’ B
89
© 2012 IBM Corporation

FILE CLONES

90
© 2012 IBM Corporation
File Clones

A File Clone is a writeable Snapshot of a file

Enable file owner to clone files
– Only files may be cloned

Clones are writable space efficient copies of individual files

Potential Usage
– Provision virtual machines
– Clone common base image
– Create a file for each machine
– Database cloning

No file space is used by clones
– Clones ‘point’ back to the parent file
– Only blocks from the modified clone are stored

Multiple clones of the same file incur no additional space overhead
– Clones of clones possible

91
© 2012 IBM Corporation
File Clones Implementation

A read only copy is made of the source

– Called a parent file
– An attempt to modify a parent file results in an access denied return code Blocks
– Two ways to create a parent on the Disk
● The source file becomes the parent Inode i1
> The source file becomes immutable Before
A A
> The clone is modifiable
B B
● A parent file is created from the source
> The created parent file becomes immutable C C
> The clone and the source file are both modifiable Inode i1 B1

A B2
Inode i0
B1 parent C2

C A
Inode i2 B

A C After
A parent is created
B2
B1
from the source
C2
92
© 2012 IBM Corporation

SNAPSHOTS

93
© 2012 IBM Corporation
Snapshot

Redirect On Write point-in-time copy feature

Snapshots available at a File System and File Set level
– Space occupied by a snapshot is charged against the file set quota
– 32 snapshots are reserved for services such as NDMP
– 224 snapshots per file system
– 224 snapshots per file set (not charged against the file system)

Rules are defined to
– Set schedules for all File Systems or File Sets
– Define retention policies to control retention of snapshots
● Automatically deleting the oldest snapshots as new ones are created
● Maintaining a set number of snapshots for specified periods
– The Snapshot manager runs once a minute to create or delete snapshots based on
defined rules  minimum frequency every minute

Snapshots may be named (default to the Windows standard of @GMT-date-time)

94
© 2012 IBM Corporation
Schedule your snapshots with the GUI

95
© 2012 IBM Corporation
Schedule your snapshots with the GUI

96
© 2012 IBM Corporation
Snapshot Function

Blocks In Active Blocks

Take snapshot 1
File System or File Set on the Disk
– Copy pointers only
• Snapshot lives within the same File

AA A
System or File Set as the active data
BB

B – No data movement
CC

C – Near instantaneous Snapshot
creation
– Consistent point-in-time copy
– Ready to use (read-only)
• Original data is never re-written
– Consumes no space!
• Finest granularity for a “snapshot”
is 1 minute
• Rolls over after 224 max/volume is
Snap 1 reached
Blocks are “frozen” on disk

97
© 2012 IBM Corporation
Snapshot Function

Take snapshot 1
Blocks in Active Blocks
File System or File Set on the Disk

Continue writing data

AA A

Take snapshot 2

B1
B1
B B
– Copy pointers only

CC C
• Snapshot lives within the same File System
B1 or File Set as the active data
– No data movement
– Near instantaneous Snapshot creation
– Consistent point-in-time copy
A


A
– Ready to use (read-only)

B
B
• Original data is never re-written

C
C
– Consumes no* space!
Snap 1 Snap 2
1 • Finest granularity for a snapshot is 1
minute
• Rolls over after 224 max/volume is
reached
98
© 2012 IBM Corporation
Snapshot Function

Take snapshot 1
Blocks in Active Blocks
File System or File Set on the Disk

Continue writing data
A A

Take snapshot 2
B1
B B
C2
C C
Continue writing data
B1
C2
Take snapshot 3
+ Excellent disk utilization

A
A
A + Excellent performance for the 1st
incoming write requests

B B1


B1
– Potential performance on the

C
C
C2
production volume when backup is
accessing the target volume on the
Snap 1 Snap 2 Snap 3
same spindles
– No protection against the loss of the
production disks
99
© 2012 IBM Corporation
Restore Snapshot

Restore from a Snapshot

– Windows file restore via Windows
Volume Shadow Copy Service (VSS)
● In Windows Explorer, right clic on a
directory and select “Properties”
● Then select the “previous Versions”
● You can then view the content of the
directory, restore it or copy it
Note: if you restore a file that you do not own,
you will not be able to restore it in the same
location

– Unix or Linux file restore via mounting

the .snapshots directory from the root
of the root independent fileset
Snapshots Integrated into Windows Explorer using the
Volume Shadow Copy Services (VSS)

100
101