Oracle Solaris Cluster Upgrade Guide

Part No: 821125610

September 2010, Revision 10
Copyright 2000, 2010, Oracle and/or its afliates. All rights reserved.
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual
property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license,
transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software,
unless required by lawfor interoperability, is prohibited.
The information contained herein is subject to change without notice and is not warranted to be error-free. If you fnd any errors, please report themto us in writing.
If this is software or related software documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following
notice is applicable:
U.S. GOVERNMENTRIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are
commercial computer software or commercial technical data pursuant to the applicable Federal Acquisition Regulation and agency-specifc supplemental
regulations. As such, the use, duplication, disclosure, modifcation, and adaptation shall be subject to the restrictions and license terms set forth in the applicable
Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR52.227-19, Commercial
Computer Software License (December 2007). Oracle America, Inc., 500 Oracle Parkway, Redwood City, CA94065.
This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently
dangerous applications, including applications which may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you
shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its afliates disclaimany
liability for any damages caused by use of this software or hardware in dangerous applications.
Oracle and Java are registered trademarks of Oracle and/or its afliates. Other names may be trademarks of their respective owners.
AMD, Opteron, the AMDlogo, and the AMDOpteron logo are trademarks or registered trademarks of Advanced Micro Devices. Intel and Intel Xeon are
trademarks or registered trademarks of Intel Corporation. All SPARCtrademarks are used under license and are trademarks or registered trademarks of SPARC
International, Inc. UNIXis a registered trademark licensed through X/Open Company, Ltd.
This software or hardware and documentation may provide access to or information on content, products, and services fromthird parties. Oracle Corporation and
its afliates are not responsible for and expressly disclaimall warranties of any kind with respect to third-party content, products, and services. Oracle Corporation
and its afliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.
110714@25097
Contents
Preface .....................................................................................................................................................5
1 PreparingtoUpgrade Oracle Solaris Cluster Software ................................................................ 11
Upgrade Requirements and Software Support Guidelines ............................................................ 11
Choosing an Oracle Solaris Cluster Upgrade Method .................................................................... 13
Standard Upgrade ........................................................................................................................ 14
Dual-Partition Upgrade .............................................................................................................. 14
Live Upgrade ................................................................................................................................ 15
Rolling Upgrade ........................................................................................................................... 16
2 Performinga StandardUpgrade toOracle Solaris Cluster 3.3Software ....................................17
Performing a Standard Upgrade of a Cluster ................................................................................... 17
Howto Upgrade QuorumServer Software ............................................................................... 18
Howto Prepare the Cluster for Upgrade (Standard) ............................................................... 20
Howto Upgrade the Solaris OS and Volume Manager Software (Standard) ....................... 27
Howto Upgrade Oracle Solaris Cluster 3.3 Software (Standard) .......................................... 32
3 Performinga Dual-PartitionUpgrade toOracle Solaris Cluster 3.3Software ..........................39
Performing a Dual-Partition Upgrade of a Cluster ......................................................................... 39
Howto Upgrade QuorumServer Software ............................................................................... 40
Howto Prepare the Cluster for Upgrade (Dual-Partition) ..................................................... 42
Howto Upgrade the Solaris OS and Volume Manager Software (Dual-Partition) ............. 50
Howto Upgrade Oracle Solaris Cluster 3.3 Software (Dual-Partition) ................................. 56
4 Performinga Live Upgrade toOracle Solaris Cluster 3.3Software .............................................67
Performing a Live Upgrade of a Cluster ........................................................................................... 67
Howto Upgrade QuorumServer Software ............................................................................... 68
3
Howto Prepare the Cluster for Upgrade (Live Upgrade) ....................................................... 70
Howto Upgrade the Solaris OS and Oracle Solaris Cluster 3.3 Software (Live Upgrade) .. 71
5 Performinga RollingUpgrade .......................................................................................................... 85
Performing a Rolling Upgrade of a Cluster ...................................................................................... 86
Howto Upgrade QuorumServer Software ............................................................................... 86
Howto Prepare a Cluster Node for a Rolling Upgrade ............................................................ 88
Howto Performa Rolling Upgrade of a Solaris Maintenance Update .................................. 92
Howto Performa Rolling Upgrade of Oracle Solaris Cluster 3.3 Software .......................... 94
6 Completingthe Upgrade .................................................................................................................... 99
Completing a Cluster Upgrade .......................................................................................................... 99
Howto Commit the Upgraded Cluster to Oracle Solaris Cluster 3.3 Software .................... 99
Howto Verify Upgrade of Oracle Solaris Cluster 3.3 Software ............................................ 100
Howto Finish Upgrade to Oracle Solaris Cluster 3.3 Software ............................................ 101
7 RecoveringFromanIncomplete Upgrade .................................................................................... 111
Cluster Recovery After an Incomplete Upgrade ............................................................................ 111
Howto Recover froma Failed Dual-Partition Upgrade ........................................................ 111
SPARC: Howto Recover Froma Partially Completed Dual-Partition Upgrade ................ 114
x86: Howto Recover Froma Partially Completed Dual-Partition Upgrade ...................... 115
Recovering FromStorage Confguration Changes During Upgrade .......................................... 117
Howto Handle Storage Reconfguration During an Upgrade ............................................. 117
Howto Resolve Mistaken Storage Changes During an Upgrade ......................................... 118
8 SPARC: UpgradingSunManagement Center Software .............................................................. 121
Upgrading the Oracle Solaris Cluster Module for Sun Management Center ............................ 121
SPARC: Howto Upgrade Oracle Solaris Cluster Module Software for Sun Management
Center .......................................................................................................................................... 121
Upgrading Sun Management Center Software .............................................................................. 123
SPARC: Howto Upgrade Sun Management Center Software .............................................. 123
Index ................................................................................................................................................... 127
Contents
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 4
Preface
The Oracle Solaris Cluster Upgrade Guide for Solaris OS contains guidelines and procedures for
upgrading the Oracle Solaris Cluster software on both SPARCbased systems and x86 based
systems.
Note This Oracle Solaris Cluster release supports systems that use the SPARCand x86 families
of processor architectures: UltraSPARC, SPARC64, AMD64, and Intel 64. In this document, x86
refers to the larger family of 64-bit x86 compatible products. Information in this document
pertains to all platforms unless otherwise specifed.
This document is intended for experienced systemadministrators with extensive knowledge of
Oracle software and hardware. Do not use this document as a presales guide. You should have
already determined your systemrequirements and purchased the appropriate equipment and
software before reading this document.
The instructions in this book assume knowledge of the Oracle Solaris Operating Systemand
expertise with the volume-manager software that is used with Oracle Solaris Cluster software.
Note Oracle Solaris Cluster software runs on two platforms, SPARCand x86. The information
in this document pertains to both platforms unless otherwise specifed in a special chapter,
section, note, bulleted item, fgure, table, or example.
UsingUNIXCommands
This document contains information about commands that are used to install, confgure, or
upgrade an Oracle Solaris Cluster confguration. This document might not contain complete
information about basic UNIXcommands and procedures such as shutting down the system,
booting the system, and confguring devices.
See one or more of the following sources for this information.

Online documentation for the Oracle Solaris OS

Other software documentation that you received with your system

Oracle Solaris OS man pages

5
Typographic Conventions
The following table describes the typographic conventions that are used in this book.
TABLE P1 Typographic Conventions
Typeface Meaning Example
AaBbCc123 The names of commands, fles, and directories,
and onscreen computer output
Edit your .login fle.
Use ls -a to list all fles.
machine_name% you have mail.
AaBbCc123 What you type, contrasted with onscreen
computer output
machine_name% su
Password:
aabbcc123 Placeholder: replace with a real name or value The command to remove a fle is rm
flename.
AaBbCc123 Book titles, newterms, and terms to be
emphasized
Read Chapter 6 in the User's Guide.
Acache is a copy that is stored
locally.
Do not save the fle.
Note: Some emphasized items
appear bold online.
Shell Prompts inCommandExamples
The following table shows the default UNIXsystemprompt and superuser prompt for shells
that are included in the Oracle Solaris OS. Note that the default systemprompt that is displayed
in command examples varies, depending on the Oracle Solaris release.
TABLE P2 Shell Prompts
Shell Prompt
Bash shell, Korn shell, and Bourne shell $
Bash shell, Korn shell, and Bourne shell for superuser #
Cshell machine_name%
Cshell for superuser machine_name#
Preface
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 6
RelatedDocumentation
Information about related Oracle Solaris Cluster topics is available in the documentation that is
listed in the following table. All Oracle Solaris Cluster documentation is available at
http://docs.sun.com.
Topic Documentation
Overview Oracle Solaris Cluster Overview
Oracle Solaris Cluster 3.3 Documentation Center
Concepts Oracle Solaris Cluster Concepts Guide
Hardware installation and
administration
Oracle Solaris Cluster 3.3 Hardware Administration Manual
Individual hardware administration guides
Software installation Oracle Solaris Cluster Software Installation Guide
Data service installation and
administration
Oracle Solaris Cluster Data Services Planning and Administration Guide
Individual data service guides
Data service development Oracle Solaris Cluster Data Services Developers Guide
Systemadministration Oracle Solaris Cluster SystemAdministration Guide
Oracle Solaris Cluster Quick Reference
Software upgrade Oracle Solaris Cluster Upgrade Guide
Error messages Oracle Solaris Cluster Error Messages Guide
Command and function references Oracle Solaris Cluster Reference Manual
Oracle Solaris Cluster Data Services Reference Manual
Oracle Solaris Cluster QuorumServer Reference Manual
For a complete list of Oracle Solaris Cluster documentation, see the release notes for your
release of Oracle Solaris Cluster software at http://wikis.sun.com/display/SunCluster/Home/.
Documentation, Support, andTraining
See the following web sites for additional resources:

Documentation (http://docs.sun.com)

Support (http://www.oracle.com/us/support/systems/index.html)

Training (http://education.oracle.com) Click the Sun link in the left navigation bar.
Preface
7
OracleWelcomesYour Comments
Oracle welcomes your comments and suggestions on the quality and usefulness of its
documentation. If you fnd any errors or have any other suggestions for improvement, go to
http://docs.sun.com and click Feedback. Indicate the title and part number of the
documentation along with the chapter, section, and page number, if available. Please let us
knowif you want a reply.
Oracle Technology Network (http://www.oracle.com/technetwork/index.html) ofers a
range of resources related to Oracle software:

Discuss technical problems and solutions on the Discussion Forums

(http://forums.oracle.com).

Get hands-on step-by-step tutorials with Oracle By Example (http://www.oracle.com/

technology/obe/start/index.html).

Download Sample Code (http://www.oracle.com/technology/sample_code/

index.html).
GettingHelp
If you have problems installing or using Oracle Solaris Cluster software, contact your service
provider and supply the following information.

Your name and email address (if available)

Your company name, address, and phone number

The model number and serial number of your systems

The release number of the Oracle Solaris OS (for example, Oracle Solaris 10)

The release number of Oracle Solaris Cluster (for example, Oracle Solaris Cluster 3.3)
Use the following commands to gather information about your systemfor your service
provider.
Command Function
prtconf -v Displays the size of the systemmemory and reports
information about peripheral devices
psrinfo -v Displays information about processors
showrev -p Reports which patches are installed
SPARC: prtdiag -v Displays systemdiagnostic information
/usr/cluster/bin/clnode show-rev Displays Oracle Solaris Cluster release and package
version information
Preface
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 8
Also have available the contents of the /var/adm/messages fle.
Preface
9
10
Preparing to Upgrade Oracle Solaris Cluster
Software
This chapter provides the following information and procedures to prepare to upgrade to
Oracle Solaris Cluster 3.3 software:

Upgrade Requirements and Software Support Guidelines on page 11

Choosing an Oracle Solaris Cluster Upgrade Method on page 13

Upgrade Requirements andSoftware Support Guidelines
Observe the following requirements and software-support guidelines when you upgrade to
Oracle Solaris Cluster 3.3 software:

Upgrade of x86 based systems - On x86 based systems, you cannot upgrade fromthe
Solaris 9 OS to the Oracle Solaris 10 OS. You must reinstall the cluster with a fresh
installation of the Oracle Solaris 10 OS and Oracle Solaris Cluster 3.3 software for x86 based
systems. Followprocedures in Chapter 2, Installing Software on Global-Cluster Nodes, in
Oracle Solaris Cluster Software Installation Guide.

MinimumOracle Solaris Cluster software version - Oracle Solaris Cluster 3.3 software
supports the following direct upgrade paths:

SPARC: Fromversion 3.1 8/05 through version 3.2 11/09 - Use the standard,
dual-partition, or live upgrade method.

Fromversion 3.2 including update releases through version 3.2 11/09 - Use the
standard, dual-partition, or live upgrade method.

On version 3.3 to an Oracle Solaris Cluster 3.3 update release with no Oracle Solaris
upgrade except to an Oracle Solaris update release, or to upgrade only Oracle Solaris to an
update release You can also use the rolling upgrade method.
See Choosing an Oracle Solaris Cluster Upgrade Method on page 13 for additional
requirements and restrictions for each upgrade method.
1
C H A P T E R 1
11

MinimumSolaris OS - The cluster must run on or be upgraded to at least Solaris 10 10/09

software, including the most current required patches, before you upgrade the Oracle
Solaris Cluster software.

Supported hardware - The cluster hardware must be a supported confguration for Oracle
Solaris Cluster 3.3 software. Contact your Sun representative for information about current
supported Oracle Solaris Cluster confgurations.

Architecture changes during upgrade - Oracle Solaris Cluster 3.3 software does not
support upgrade between architectures.

Software migration - Do not migrate fromone type of software product to another product
during Oracle Solaris Cluster upgrade. For example, migration fromSolaris Volume
Manager disk sets to VxVMdisk groups or fromUFS fle systems to VxFS fle systems is not
supported during Oracle Solaris Cluster upgrade. Performonly software confguration
changes that are specifed by upgrade procedures of an installed software product.

Global-devices partition size - If a node uses a dedicated partition for the

/global/.devices/node@nodeid fle systemand the partition is less than 512 Mbytes but it
provides sufcient space for existing device nodes, you do not need to change the fle-system
size. The 512-Mbyte minimumapplies to newinstallations of Oracle Solaris Cluster
software.
However, you must still ensure that the global-devices fle systemhas ample space and
ample inode capacity for existing devices and for any newdevices that you intend to
confgure. Certain confguration changes, such as adding disks, disk volumes, or
metadevices, might require increasing the partition size to provide sufcient additional
inodes.

Data services - You must upgrade data-service software to the Oracle Solaris Cluster 3.3
version.

Upgrading to compatible versions - You must upgrade all software on the cluster nodes to
a version that is supported by Oracle Solaris Cluster 3.3 software. For example, if a version of
an application is supported on Sun Cluster 3.2 software but is not supported on Oracle
Solaris Cluster 3.3 software, you must upgrade that application to the version that is
supported on Oracle Solaris Cluster 3.3 software, if such a version exists. See Supported
Products in Oracle Solaris Cluster 3.3 Release Notes for information about supported
products.

Downgrade - Oracle Solaris Cluster 3.3 software does not support any downgrade of Oracle
Solaris Cluster software.

Limitation of scinstall for data-service upgrades - The scinstall upgrade utility only
upgrades those data services that are provided with Oracle Solaris Cluster 3.3 software. You
must manually upgrade any customor third-party data services.
Upgrade Requirements and Software Support Guidelines
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 12
ChoosinganOracle Solaris Cluster Upgrade Method
The following matrixes summarize the supported upgrade methods for each Oracle Solaris OS
version and platform, provided that all other requirements for any supported method are met.
Check the documentation for other products in the cluster, such as volume management
software and other applications, for any additional upgrade requirements or restrictions.
Note If your cluster uses a ZFS root fle system, you can upgrade the Oracle Solaris OS only by
using the live upgrade method. See Oracle Solaris upgrade documentation for more
information.
This limitation does not apply if you are not upgrading the Oracle Solaris OS.
TABLE 11 Upgrade FromOracle Solaris Cluster 3.1 8/05 Through 3.2 11/09 Software, Including Oracle
Solaris OS Upgrade
Method
Oracle Solaris 10
SPARC x86
Standard upgrade X X
Dual-partition upgrade X X
Live upgrade X X
Rolling upgrade - -
TABLE 12 Upgrade on Oracle Solaris Cluster 3.3 Software of Oracle Solaris OS Update Only
Method
Oracle Solaris 10
SPARC x86
Standard upgrade X X
Dual-partition upgrade X X
Live upgrade X X
Rolling upgrade X X
Choose fromthe following methods to upgrade your cluster to Oracle Solaris Cluster 3.3
software:

Standard Upgrade on page 14

Dual-Partition Upgrade on page 14

Live Upgrade on page 15

Rolling Upgrade on page 16

Choosing an Oracle Solaris Cluster Upgrade Method
Chapter 1 Preparing to Upgrade Oracle Solaris Cluster Software 13
For overviewinformation about planning your Oracle Solaris Cluster 3.3 confguration, see
Chapter 1, Planning the Oracle Solaris Cluster Confguration, in Oracle Solaris Cluster
Software Installation Guide.
StandardUpgrade
In a standard upgrade, you shut down the cluster before you upgrade the cluster nodes. You
return the cluster to production after all nodes are fully upgraded.

ZFS root fle systems - If your cluster uses a ZFS root fle system, you cannot use standard
upgrade to upgrade the Solaris OS. You must use only the live upgrade method to upgrade
the Solaris OS. But you can use standard upgrade to separately upgrade Oracle Solaris
Cluster and other software.
Dual-PartitionUpgrade
In a dual-partition upgrade, you divide the cluster into two groups of nodes. You bring down
one group of nodes and upgrade those nodes. The other group of nodes continues to provide
services. After you complete upgrade of the frst group of nodes, you switch services to those
upgraded nodes. You then upgrade the remaining nodes and boot themback into the rest of the
cluster.
The cluster outage time is limited to the amount of time that is needed for the cluster to switch
over services to the upgraded partition, with one exception. If you upgrade fromthe Sun Cluster
3.1 8/05 release and you intend to confgure zone clusters, you must temporarily take the
upgraded frst partition out of cluster mode to set newprivate-network settings that were
introduced in the Sun Cluster 3.2 release.
Observe the following additional restrictions and requirements for the dualpartition upgrade
method:

ZFS root fle systems - If your cluster uses a ZFS root fle system, you cannot use
dual-partition upgrade to upgrade the Solaris OS. You must use only the live upgrade
method to upgrade the Solaris OS. But you can use dual-partition upgrade to separately
upgrade Oracle Solaris Cluster and other software.

HAfor Sun Java SystemApplication Server EE (HADB) - If you are running the HAfor
Sun Java SystemApplication Server EE (HADB) data service with Sun Java System
Application Server EE (HADB) software as of version 4.4, you must shut down the database
before you begin the dual-partition upgrade. The HADB database does not tolerate the loss
of membership that would occur when a partition of nodes is shut down for upgrade. This
requirement does not apply to versions before version 4.4.
Choosing an Oracle Solaris Cluster Upgrade Method
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 14

Data format changes - Do not use the dual-partition upgrade method if you intend to
upgrade an application that requires that you change its data format during the application
upgrade. The dual-partition upgrade method is not compatible with the extended downtime
that is needed to performdata transformation.

Location of application software - Applications must be installed on nonshared storage.

Shared storage is not accessible to a partition that is in noncluster mode. Therefore, it is not
possible to upgrade application software that is located on shared storage.

Division of storage - Each shared storage device must be connected to a node in each
group.

Single-node clusters - Dual-partition upgrade is not available to upgrade a single-node

cluster. Use the standard upgrade or live upgrade method instead.

Confguration changes - Do not make cluster confguration changes that are not
documented in the upgrade procedures. Such changes might not be propagated to the fnal
cluster confguration. Also, validation attempts of such changes would fail because not all
nodes are reachable during a dual-partition upgrade.
Live Upgrade
Alive upgrade maintains your previous cluster confguration until you have upgraded all nodes
and you commit to the upgrade. If the upgraded confguration causes a problem, you can revert
to your previous cluster confguration until you can rectify the problem.
The cluster outage is limited to the amount of time that is needed to reboot the cluster nodes
into the upgraded boot environment.
Observe the following additional restrictions and requirements for the live upgrade method:

ZFS root fle systems - If your cluster confguration uses a ZFS root fle system, you must
use only live upgrade to upgrade the Solaris OS. See Solaris documentation for more
information.

Dual-partition upgrade - The live upgrade method cannot be used in conjunction with a
dual-partition upgrade.

Non-global zones - Unless the cluster is already running on at least Solaris 10 11/06, the live
upgrade method does not support the upgrade of clusters that have non-global zones that
are confgured on any of the cluster nodes. Instead, use the standard upgrade or
dual-partition upgrade method.

Disk space - To use the live upgrade method, you must have enough spare disk space
available to make a copy of each node's boot environment. You reclaimthis disk space after
the upgrade is complete and you have verifed and committed the upgrade. For information
about space requirements for an inactive boot environment, refer to or Allocating Disk and
Swap Space in Solaris 10 10/09 Installation Guide: Planning for Installation and Upgrade.
Choosing an Oracle Solaris Cluster Upgrade Method
Chapter 1 Preparing to Upgrade Oracle Solaris Cluster Software 15
RollingUpgrade
In a rolling upgrade, you upgrade software to an update release on one node at a time. Services
continue on the other nodes except for the time it takes to switch services froma node to be
upgraded to a node that will remain in service.
Observe the following additional restrictions and requirements for the rolling upgrade method:

MinimumOracle Solaris Cluster version - The cluster must be running an Oracle Solaris
Cluster 3.3 release.

Solaris upgrade paths - You can upgrade the Solaris OS only to an update version of the
same release. For example, you can performa rolling upgrade fromSolaris 10 5/08 to Solaris
10 10/09. But you cannot performa rolling upgrade froma version of Solaris 9 to a version
of Oracle Solaris 10.

ZFS root fle systems - If your cluster confguration uses a ZFS root fle system, you cannot
use rolling upgrade to upgrade the Solaris OS. You must use only live upgrade to upgrade
the Solaris OS. See Solaris documentation for more information.

Hardware confguration changes - Do not change the cluster confguration during a rolling
upgrade. For example, do not add to or change the cluster interconnect or quorumdevices.
If you need to make such a change, do so before you start the rolling upgrade procedure or
wait until after all nodes are upgraded and the cluster is committed to the newsoftware
version.

Duration of the upgrade - Limit the amount of time that you take to complete a rolling
upgrade of all cluster nodes. After a node is upgraded, begin the upgrade of the next cluster
node as soon as possible. You can experience performance penalties and other penalties
when you run a mixed-version cluster for an extended period of time.

Software confguration changes - Avoid installing newdata services or issuing any

administrative confguration commands during the upgrade.

New-feature availability - Until all nodes of the cluster are successfully upgraded and the
upgrade is committed, newfeatures that are introduced by the newrelease might not be
available.
Choosing an Oracle Solaris Cluster Upgrade Method
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 16
Performing a Standard Upgrade to Oracle
Solaris Cluster 3.3 Software
This chapter provides the following information to upgrade to Oracle Solaris Cluster 3.3
software by using the standard nonrolling upgrade method:

Howto Prepare the Cluster for Upgrade (Standard) on page 20

Howto Upgrade the Solaris OS and Volume Manager Software (Standard) on page 27

Howto Upgrade Oracle Solaris Cluster 3.3 Software (Standard) on page 32

Note If your cluster uses a ZFS root fle systemand is confgured with zone clusters, you cannot
use standard upgrade to upgrade the Solaris OS. Use only the live upgrade method to upgrade
the Solaris OS. After Solaris is upgraded, you can use standard upgrade to upgrade the Oracle
Solaris Cluster software.
Performinga StandardUpgrade of a Cluster
The following table lists the tasks to performto upgrade to Oracle Solaris Cluster 3.3 software.
You also performthese tasks to upgrade only the Solaris OS.
Note If you upgrade the Solaris OS to a newmarketing release, such as fromSolaris 9 to Oracle
Solaris 10 software, you must also upgrade the Oracle Solaris Cluster software and dependency
software to the version that is compatible with the newOS version.
TABLE 21 Task Map: Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software
Task Instructions
1. Read the upgrade requirements and restrictions. Determine the
proper upgrade method for your confguration and needs.
Upgrade Requirements and Software Support Guidelines on
page 11
Choosing an Oracle Solaris Cluster Upgrade Method on
page 13
2
C H A P T E R 2
17
TABLE 21 Task Map: Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software (Continued)
Task Instructions
2. If a quorumserver is used, upgrade the QuorumServer
software.
Howto Upgrade QuorumServer Software on page 18
3. Remove the cluster fromproduction and back up shared data.
If Oracle Solaris Cluster Geographic Edition software is installed,
uninstall it.
Howto Prepare the Cluster for Upgrade (Standard) on page 20
4. Upgrade the Solaris software, if necessary, to a supported
Solaris update. If the cluster uses dual-string mediators for Solaris
Volume Manager software, unconfgure the mediators. As
needed, upgrade Veritas Volume Manager (VxVM) and Veritas
File System(VxFS). Solaris Volume Manager software is
automatically upgraded with the Solaris OS.
Howto Upgrade the Solaris OS and Volume Manager Software
(Standard) on page 27
5. Upgrade to Oracle Solaris Cluster 3.3 framework and
data-service software. If necessary, upgrade applications. If the
cluster uses dual-string mediators and you upgraded the Solaris
OS, reconfgure the mediators. If you upgraded VxVM, upgrade
disk groups.
Howto Upgrade Oracle Solaris Cluster 3.3 Software (Standard)
on page 32
6. Use the scversions command to commit the cluster to the
upgrade.
Howto Commit the Upgraded Cluster to Oracle Solaris Cluster
3.3 Software on page 99
7. Verify successful completion of upgrade to Oracle Solaris
Cluster 3.3 software.
Howto Verify Upgrade of Oracle Solaris Cluster 3.3 Software
on page 100
8. Enable resources and bring resource groups online. Migrate
existing resources to newresource types. Upgrade to Oracle
Solaris Cluster Geographic Edition 3.3 software, if used.
Howto Finish Upgrade to Oracle Solaris Cluster 3.3 Software
on page 101
9. (Optional) SPARC: Upgrade the Oracle Solaris Cluster module
for Sun Management Center, if needed.
SPARC: Howto Upgrade Oracle Solaris Cluster Module
Software for Sun Management Center on page 121

HowtoUpgrade QuorumServer Software

If the cluster uses a quorumserver, upgrade the QuorumServer software on the quorumserver
before you upgrade the cluster.
Note If more than one cluster uses the quorumserver, performon each cluster the steps to
remove the quorumserver and later the steps to add back the quorumserver.
Performall steps as superuser on the cluster and on the quorumserver.
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 18
If the cluster has twonodes andthe quorumserver is the cluster's only quorumdevice,
temporarily adda secondquorumdevice.
See Adding a QuorumDevice in Oracle Solaris Cluster SystemAdministration Guide.
If you add another quorumserver as a temporary quorumdevice, the quorumserver can run
the same software version as the quorumserver that you are upgrading, or it can run the 3.3
version of QuorumServer software.
Unconfgure the quorumserver fromeachcluster that uses the quorumserver.
phys-schost# clquorum remove quorumserver
Fromthe quorumserver toupgrade, verify that the quorumserver nolonger serves any cluster.
quorumserver# clquorumserver show +
If the output shows any cluster is still served by the quorumserver, unconfgure the quorum
server fromthat cluster. Then repeat this step to confrmthat the quorumserver is no longer
confgured with any cluster.
Note If you have unconfgured the quorumserver froma cluster but the clquorumserver show
command still reports that the quorumserver is serving that cluster, the command might be
reporting stale confguration information. See Cleaning Up Stale QuorumServer Cluster
Information in Oracle Solaris Cluster SystemAdministration Guide.
Fromthe quorumserver toupgrade, halt all quorumserver instances.
quorumserver# clquorumserver stop +
Uninstall the QuorumServer software fromthe quorumserver toupgrade.
a. Navigate tothe directory where the uninstaller is located.
quorumserver# cd /var/sadm/prod/SUNWentsysver
ver The version that is installed on your system.
b. Start the uninstallationwizard.
quorumserver# ./uninstall
c. Followinstructions onthe screentouninstall the QuorumServer software fromthe
quorum-server host computer.
After removal is fnished, you can viewany available log. See Chapter 8, Uninstalling, in
Sun Java Enterprise System5 Update 1 Installation Guide for UNIXfor additional
information about using the uninstall program.
d. (Optional) Cleanupor remove the quorumserver directories.
By default, this directory is /var/scqsd.
1
2
3
4
5
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 19
Install the Oracle Solaris Cluster 3.3 QuorumServer software, reconfgure the quorumserver,
andstart the quorumserver daemon.
Followthe steps in Howto Install and Confgure QuorumServer Software in Oracle Solaris
Cluster Software Installation Guide for installing the QuorumServer software.
Froma cluster node, confgure the upgradedquorumserver as a quorumdevice.
Followthe steps in Howto Confgure QuorumDevices in Oracle Solaris Cluster Software
Installation Guide.
If youconfgureda temporary quorumdevice, unconfgure it.
phys-schost# clquorum remove tempquorum

HowtoPrepare the Cluster for Upgrade (Standard)

Performthis procedure to remove the cluster fromproduction before you performa standard
upgrade. Performall steps fromthe global zone only.
Performthe following tasks:

Ensure that the confguration meets the requirements for upgrade. See Upgrade
Requirements and Software Support Guidelines on page 11.

Have available the installation media, documentation, and patches for all software products
that you are upgrading, including the following software:

Solaris OS

Oracle Solaris Cluster 3.3 framework

Oracle Solaris Cluster 3.3 patches

Oracle Solaris Cluster 3.3 data services (agents)

Applications that are managed by Oracle Solaris Cluster 3.3 data services

Veritas Volume Manager, if applicable

See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes
for the location of patches and installation instructions.

If you use role-based access control (RBAC) instead of superuser to access the cluster nodes,
ensure that you can assume an RBACrole that provides authorization for all Oracle Solaris
Cluster commands. This series of upgrade procedures requires the following Oracle Solaris
Cluster RBACauthorizations if the user is not superuser:

solaris.cluster.modify

solaris.cluster.admin

solaris.cluster.read
6
7
8
BeforeYouBegin
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 20
See Role-Based Access Control (Overview) in SystemAdministration Guide: Security
Services for more information about using RBACroles. See the Oracle Solaris Cluster man
pages for the RBACauthorization that each Oracle Solaris Cluster subcommand requires.
Ensure that the cluster is functioningnormally.
a. Viewthe current status of the cluster by runningthe followingcommandfromany node.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost% scstat

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
phys-schost% cluster status
See the scstat(1M) or cluster(1CL) man page for more information.
b. Searchthe /var/adm/messages logonthe same node for unresolvederror messages or
warningmessages.
c. Check the volume-manager status.
Notify users that cluster services will be unavailable duringthe upgrade.
If Geographic Editionsoftware is installed, uninstall it.
For uninstallation procedures, see the documentation for your version of Geographic Edition
software.
Become superuser ona node of the cluster.
Take eachresource groupofine anddisable all resources.
Take ofine all resource groups in the cluster, including those that are in non-global zones.
Then disable all resources, to prevent the cluster frombringing the resources online
automatically if a node is mistakenly rebooted into cluster mode.

If youwant touse the scsetup or clsetup utility, performthe followingsteps:

a. Start the utility.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scsetup

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following
command:
phys-schost# clsetup
1
2
3
4
5
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 21
The Main Menu is displayed.
b. Choose the menuitem, Resource Groups.
The Resource Group Menu is displayed.
c. Choose the menuitem, Online/Ofine or Switchover a Resource Group.
d. Followthe prompts totake ofine all resource groups andtoput theminthe unmanaged
state.
e. Whenall resource groups are ofine, type q toreturntothe Resource GroupMenu.
f. Exit the scsetup utility.
Type q to back out of each submenu or press Ctrl-C.

Touse the commandline, performthe followingsteps:

a. Take eachresource ofine.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scswitch -F -g resource-group
-F Switches a resource group ofine.
-g resource-group Specifes the name of the resource group to take ofine.

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following
command:
phys-schost# clresource offline resource-group
b. Fromany node, list all enabledresources inthe cluster.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scrgadm -pv | grep "Res enabled"
(resource-group:resource) Res enabled: True

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following
command:
phys-schost# clresource show -p Enabled
=== Resources ===
Resource: resource
Enabled{nodename1}: True
Enabled{nodename2}: True
...
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 22
c. Identify those resources that dependonother resources.
phys-schost# clresource show -p resource_dependencies
=== Resources ===
Resource: node
Resource_dependencies: node
...
You must disable dependent resources frst before you disable the resources that they
depend on.
d. Disable eachenabledresource inthe cluster.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scswitch -n -j resource
-n Disables.
-j resource Specifes the resource.

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following
command:
phys-schost# clresource disable resource
See the scswitch(1M) or clresource(1CL) man page for more information.
e. Verify that all resources are disabled.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scrgadm -pv | grep "Res enabled"
(resource-group:resource) Res enabled: False

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following
command:
phys-schost# clresource show -p Enabled
=== Resources ===
Resource: resource
Enabled{nodename1}: False
Enabled{nodename2}: False
...
f. Move eachresource grouptothe unmanagedstate.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scswitch -u -g resource-group
-u Moves the specifed resource group to the unmanaged state.
-g resource-group Specifes the name of the resource group to move into the
unmanaged state.
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 23

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following
command:
phys-schost# clresourcegroup unmanage resource-group
Verify that all resources onall nodes are Offline andthat all resource groups are inthe
Unmanaged state.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scstat -g

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
phys-schost# cluster status -t resource,resourcegroup
Stopall applications that are runningoneachnode of the cluster.
Ensure that all shareddata is backedup.
If youwill upgrade the Solaris OS andyour cluster uses dual-stringmediators for Solaris Volume
Manager software, unconfgure your mediators.
See Confguring Dual-String Mediators in Oracle Solaris Cluster Software Installation Guide
for more information about mediators.
a. Runthe followingcommandtoverify that nomediator data problems exist.
phys-schost# medstat -s setname
-s setname Specifes the disk set name.
If the value in the Status feld is Bad, repair the afected mediator host. Followthe procedure
Howto Fix Bad Mediator Data in Oracle Solaris Cluster Software Installation Guide.
b. List all mediators.
Save this information for when you restore the mediators during the procedure Howto
Finish Upgrade to Oracle Solaris Cluster 3.3 Software on page 101.
c. For a disk set that uses mediators, take ownershipof the disk set if nonode already has
ownership.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scswitch -z -D setname -h node
-z Changes mastery.
-D devicegroup Specifes the name of the disk set.
-h node Specifes the name of the node to become primary of the disk set.

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
6
7
8
9
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 24
phys-schost# cldevicegroup switch -n node devicegroup
d. Unconfgure all mediators for the disk set.
phys-schost# metaset -s setname -d -m mediator-host-list
-s setname Specifes the disk set name.
-d Deletes fromthe disk set.
-m mediator-host-list Specifes the name of the node to remove as a mediator host for the
disk set.
See the mediator(7D) man page for further information about mediator-specifc options to
the metaset command.
e. Repeat Stepc throughStepdfor eachremainingdisk set that uses mediators.
Fromone node, shut downthe cluster.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scshutdown -g0 -y

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
phys-schost# cluster shutdown -g0 -y
See the scshutdown(1M)man page for more information.
Boot eachnode intononcluster mode.

OnSPARCbasedsystems, performthe followingcommand:

ok boot -x

Onx86 basedsystems, performthe followingcommands:

a. Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry andtype e
toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based Systemby
Using GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
10
11
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 25
b. Inthe boot parameters screen, use the arrowkeys toselect the kernel entry andtype e
toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
|+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
c. Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
d. Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
e. Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over the
systemboot. The next time you reboot the node, it will boot into cluster mode. To boot
into noncluster mode instead, performthese steps to again add the -x option to the
kernel boot parameter command.
Ensure that eachsystemdisk is backedup.
Upgrade software on each node.

To upgrade Solaris software before you performOracle Solaris Cluster software upgrade, go
to Howto Upgrade the Solaris OS and Volume Manager Software (Standard) on page 27.
12
Next Steps
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 26

You must upgrade the Solaris software to a supported release if Oracle Solaris Cluster 3.3
software does not support the release of the Solaris OS that your cluster currently runs .
See Supported Products in Oracle Solaris Cluster 3.3 Release Notes for more
information.

If Oracle Solaris Cluster 3.3 software supports the release of the Solaris OS that you
currently run on your cluster, further Solaris software upgrade is optional.

Otherwise, upgrade to Oracle Solaris Cluster 3.3 software. Go to Howto Upgrade Oracle
Solaris Cluster 3.3 Software (Standard) on page 32.

HowtoUpgrade the Solaris OS andVolume Manager

Software (Standard)
Performthis procedure on each node in the cluster to upgrade the Solaris OS and optionally
also VxVM, if used. Performall steps fromthe global zone only. If the cluster already runs on a
version of the Solaris OS that supports Oracle Solaris Cluster 3.3 software, further upgrade of
the Solaris OS is optional.
If you do not intend to upgrade the Solaris OS or volume management software, proceed to
Howto Upgrade Oracle Solaris Cluster 3.3 Software (Standard) on page 32.
Note The cluster must already run on, or be upgraded to, at least the minimumrequired level
of the Oracle Solaris OS to support upgrade to Oracle Solaris Cluster 3.3 software. See
Supported Products in Oracle Solaris Cluster 3.3 Release Notes for more information.
Ensure that all steps in Howto Prepare the Cluster for Upgrade (Standard) on page 20 are
completed.
Become superuser onthe cluster node toupgrade.
If you are performing a dual-partition upgrade, the node must be a member of the partition that
is in noncluster mode.
Determine whether the followingApache run-control scripts exist andare enabledor disabled:
/etc/rc0.d/K16apache
/etc/rc1.d/K16apache
/etc/rc2.d/K16apache
/etc/rc3.d/S50apache
/etc/rcS.d/K16apache
Some applications, such as Oracle Solaris Cluster HAfor Apache, require that Apache run
control scripts be disabled.
BeforeYouBegin
1
2
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 27

If these scripts exist and contain an uppercase K or S in the fle name, the scripts are enabled.
No further action is necessary for these scripts.

If these scripts do not exist, in Step 7 you must ensure that any Apache run control scripts
that are installed during the Solaris OS upgrade are disabled.

If these scripts exist but the fle names contain a lowercase k or s, the scripts are disabled. In
Step 7 you must ensure that any Apache run control scripts that are installed during the
Solaris OS upgrade are disabled.
Comment out all entries for globally mountedfle systems inthe node's /etc/vfstab fle.
a. For later reference, make a recordof all entries that are already commentedout.
b. Temporarily comment out all entries for globally mountedfle systems inthe /etc/vfstab
fle.
Entries for globally mounted fle systems contain the global mount option. Comment out
these entries to prevent the Solaris upgrade fromattempting to mount the global devices.
Determine whichprocedure tofollowtoupgrade the Solaris OS.

To use Live Upgrade, go instead to Chapter 4, Performing a Live Upgrade to Oracle Solaris
Cluster 3.3 Software.

To upgrade a cluster that uses Solaris Volume Manager by a method other than Live
Upgrade, followupgrade procedures in Solaris installation documentation.

To upgrade a cluster that uses Veritas Volume Manager by a method other than Live
Upgrade, followupgrade procedures in Veritas Storage Foundation installation
documentation.
Note If your cluster has VxVMinstalled and you are upgrading the Solaris OS, you must
reinstall or upgrade to VxVMsoftware that is compatible with the version of Oracle Solaris 10
that you upgrade to.
Upgrade the Solaris software, followingthe procedure that youselectedinStep4.
Note Do not performthe fnal reboot instruction in the Solaris software upgrade. Instead, do
the following:
a. Return to this procedure to performStep 6 and Step 7.
b. Reboot into noncluster mode in Step 8 to complete Solaris software upgrade.

When prompted, choose the manual reboot option.

3
4
5
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 28

When you are instructed to reboot a node during the upgrade process, always reboot into
noncluster mode. For the boot and reboot commands, add the -x option to the command.
The -x option ensures that the node reboots into noncluster mode. For example, either of
the following two commands boot a node into single-user noncluster mode:

OnSPARCbasedsystems, performeither of the followingcommands:

phys-schost# reboot -- -xs
or
ok boot -xs
If the instruction says to run the init S command, use the reboot -- -xs command
instead.

Onx86 basedsystems, performthe followingcommand:

phys-schost# shutdown -g -y -i0
Press any key to continue
a. Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry andtype e
toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+-------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based Systemby
Using GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
b. Inthe boot parameters screen, use the arrowkeys toselect the kernel entry andtype e
toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 29
c. Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
d. Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
e. Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over the
systemboot. The next time you reboot the node, it will boot into cluster mode. To boot
into noncluster mode instead, performthese steps to again to add the -x option to the
kernel boot parameter command.
If the instruction says to run the init S command, shut down the systemthen change the
GRUB kernel boot command to /platform/i86pc/multiboot -sx instead.
Inthe /a/etc/vfstab fle, uncomment those entries for globally mountedfle systems that you
commentedout inStep3.
If Apache runcontrol scripts were disabledor didnot exist before youupgradedthe Solaris OS,
ensure that any scripts that were installedduringSolaris upgrade are disabled.
To disable Apache run control scripts, use the following commands to rename the fles with a
lowercase k or s.
phys-schost# mv /a/etc/rc0.d/K16apache /a/etc/rc0.d/k16apache
phys-schost# mv /a/etc/rc1.d/K16apache /a/etc/rc1.d/k16apache
phys-schost# mv /a/etc/rc2.d/K16apache /a/etc/rc2.d/k16apache
phys-schost# mv /a/etc/rc3.d/S50apache /a/etc/rc3.d/s50apache
phys-schost# mv /a/etc/rcS.d/K16apache /a/etc/rcS.d/k16apache
Alternatively, you can rename the scripts to be consistent with your normal administration
practices.
6
7
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 30
Reboot the node intononcluster mode.
Include the double dashes (--) in the following command:
phys-schost# reboot -- -x
If your cluster runs VxVMandyouare upgradingit as well as upgradingthe Solaris OS, perform
the remainingsteps inthe procedure toreinstall or upgradeVxVM.
Make the following changes to the procedure:

After VxVMupgrade is complete but before youreboot, verify the entries inthe /etc/vfstab
fle.
If any of the entries that you uncommented in Step 6 were commented out, make those
entries uncommented again.

If theVxVMprocedures instruct youtoperforma fnal reconfgurationreboot, do not use the

-r optionalone. Instead, reboot intononcluster mode by usingthe -rx options.

OnSPARCbasedsystems, performthe followingcommand:

phys-schost# reboot -- -rx

Onx86 basedsystems, performthe shutdownandboot procedures that are describedin

Step5 except add-rx tothe kernel boot commandinsteadof -sx.
Note If you see a message similar to the following, type the root password to continue upgrade
processing. Do not run the fsck command nor type Ctrl-D.
WARNING - Unable to repair the /global/.devices/node@1 filesystem.
Run fsck manually (fsck -F ufs /dev/vx/rdsk/rootdisk_13vol). Exit the
shell when done to continue the boot process.
Type control-d to proceed with normal startup,
(or give root password for system maintenance): Type the root password
(Optional) SPARC: UpgradeVxFS.
Followprocedures that are provided in your VxFS documentation.
Install any requiredSolaris software patches andhardware-relatedpatches, anddownloadany
neededfrmware that is containedinthe hardware patches.
Note Do not reboot after you add patches. Wait to reboot the node until after you upgrade the
Oracle Solaris Cluster software.
See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes for
the location of patches and installation instructions.
8
9
10
11
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 31
If you are only upgrading the Solaris OS to a Solaris update release and are not upgrading the
Oracle Solaris Cluster software, skip to Chapter 6, Completing the Upgrade.
Otherwise, upgrade to Oracle Solaris Cluster 3.3 software. Go to Howto Upgrade Oracle
Solaris Cluster 3.3 Software (Standard) on page 32.
Note To complete the upgrade to a newmarketing release of the Solaris OS, such as from
Solaris 9 to Oracle Solaris 10 software, you must also upgrade the Oracle Solaris Cluster
software and dependency software to the version that is compatible with the newversion of the
OS.

HowtoUpgrade Oracle Solaris Cluster 3.3Software

(Standard)
Performthis procedure to upgrade each node of the cluster to Oracle Solaris Cluster 3.3
software. You must also performthis procedure after you upgrade to a diferent marketing
release of the Solaris OS, such as fromSolaris 9 to Oracle Solaris 10 software.
Performall steps fromthe global zone only.
Tip You can use the cconsole utility to performthis procedure on multiple nodes
simultaneously. See Howto Install Cluster Control Panel Software on an Administrative
Console in Oracle Solaris Cluster Software Installation Guide for more information.
Performthe following tasks:

Ensure that all steps in Howto Prepare the Cluster for Upgrade (Standard) on page 20 are
completed.

If you upgraded to a newmarketing release of the Solaris OS, such as fromSolaris 9 to

Oracle Solaris 10 software, ensure that all steps in Howto Upgrade the Solaris OS and
Volume Manager Software (Standard) on page 27 are completed.

Ensure that you have installed all required Solaris software patches and hardware-related
patches.
Become superuser ona node of the cluster.
Loadthe installationDVD-ROMintothe DVD-ROMdrive.
If the volume management daemon vold(1M) is running and is confgured to manage
CD-ROMor DVDdevices, the daemon automatically mounts the media on the /cdrom/cdrom0
directory.
Next Steps
BeforeYouBegin
1
2
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 32
Change tothe /Solaris_arch/Product/sun_cluster/Solaris_ver/Tools/ directory, where
arch is sparc or x86 andwhere ver is 10 for Oracle Solaris 10.
phys-schost# cd /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools
Start the scinstall utility.
phys-schost# ./scinstall
Note Do not use the /usr/cluster/bin/scinstall command that is already installed on the
node. You must use the scinstall command that is located on the installation DVD-ROM.
The scinstall Main Menu is displayed.
Choose the menuitem, UpgradeThis Cluster Node.
*** Main Menu ***
Please select from one of the following (*) options:
1) Create a new cluster or add a cluster node
2) Configure a cluster to be JumpStarted from this install server
* 3) Manage a dual-partition upgrade
* 4) Upgrade this cluster node
* 5) Print release information for this cluster node
* ?) Help with menu options
* q) Quit
Option: 4
The Upgrade Menu is displayed.
Choose the menuitem, Upgrade Oracle Solaris Cluster Framework onThis Node.
Followthe menuprompts toupgrade the cluster framework.
During the Oracle Solaris Cluster upgrade, scinstall might make one or more of the following
confguration changes:

Rename the ntp.conf fle to ntp.conf.cluster, if ntp.conf.cluster does not already

exist on the node.

Set the local-mac-address? variable to true, if the variable is not already set to that value.
Upgrade processing is fnished when the systemdisplays the message Completed Oracle
Solaris Cluster framework upgrade and prompts you to press Enter to continue.
Quit the scinstall utility.
Upgrade data service packages.
You must upgrade all data services to the Oracle Solaris Cluster 3.3 version.
3
4
5
6
7
8
9
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 33
Note For HAfor SAP Web Application Server, if you are using a J2EE engine resource or a web
application server component resource or both, you must delete the resource and recreate it
with the newweb application server component resource. Changes in the newweb application
server component resource includes integration of the J2EE functionality. For more
information, see Oracle Solaris Cluster Data Service for SAP Web Application Server Guide.
a. Start the upgradedinteractive scinstall utility.
phys-schost# /usr/cluster/bin/scinstall
Note Do not use the scinstall utility that is on the installation media to upgrade data
service packages.
The scinstall Main Menu is displayed.
b. Choose the menuitem, UpgradeThis Cluster Node.
The Upgrade Menu is displayed.
c. Choose the menuitem, Upgrade Oracle Solaris Cluster Data Service Agents onThis Node.
d. Followthe menuprompts toupgrade Oracle Solaris Cluster data service agents that are
installedonthe node.
You can choose fromthe list of data services that are available to upgrade or choose to
upgrade all installed data services.
e. When the system displays the message Completed upgrade of Oracle Solaris Cluster
data services agents, press Enter.
The Upgrade Menu is displayed.
Quit the scinstall utility.
Unloadthe installationDVD-ROMfromthe DVD-ROMdrive.
a. Toensure that the DVD-ROMis not beingused, change toa directory that does not reside on
the DVD-ROM.
b. Eject the DVD-ROM.
phys-schost# eject cdrom
If youhave HAfor NFS confguredona highly available local fle system, ensure that the
loopback fle system(LOFS) is disabled.
10
11
12
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 34
Note If you have non-global zones confgured, LOFS must remain enabled. For guidelines
about using LOFS and alternatives to disabling it, see Cluster File Systems in Oracle Solaris
Cluster Software Installation Guide.
To disable LOFS, ensure that the /etc/system fle contains the following entry:
exclude:lofs
This change becomes efective at the next systemreboot.
As needed, manually upgrade any customdata services that are not suppliedonthe product
media.
Verify that eachdata-service update is installedsuccessfully.
Viewthe upgrade log fle that is referenced at the end of the upgrade output messages.
Install any Oracle Solaris Cluster 3.3 framework anddata-service software patches.
See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes for
the location of patches and installation instructions.
Upgrade software applications that are installedonthe cluster.
If you want to upgrade VxVMand did not upgrade the Solaris OS, followprocedures in Veritas
Storage Foundation installation documentation to upgrade VxVMwithout upgrading the
operating system.
Note If any upgrade procedure instruct you to performa reboot, you must add the -x option to
the boot command. This option boots the cluster into noncluster mode.
Ensure that application levels are compatible with the current versions of Oracle Solaris Cluster
and Solaris software. See your application documentation for installation instructions.
If youupgradedfromSunCluster 3.1 8/05 software, reconfgure the private-network address
range.
Performthis step if you want to increase or decrease the size of the IP address range that is used
by the private interconnect. The IP address range that you confgure must minimally support
the number of nodes and private networks in the cluster. See Private Network in Oracle
Solaris Cluster Software Installation Guide for more information.
13
14
15
16
17
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 35
If you also expect to confgure zone clusters, you specify that number in Howto Finish
Upgrade to Oracle Solaris Cluster 3.3 Software on page 101, after all nodes are back in cluster
mode.
a. Fromone node, start the clsetup utility.
When run in noncluster mode, the clsetup utility displays the Main Menu for
noncluster-mode operations.
b. Choose the menuitem, Change IPAddress Range.
The clsetup utility displays the current private-network confguration, then asks if you
would like to change this confguration.
c. Tochange either the private-network IPaddress or the IPaddress range, type yes andpress
the Returnkey.
The clsetup utility displays the default private-network IP address, 172.16.0.0, and asks if
it is okay to accept this default.
d. Change or accept the private-network IPaddress.

Toaccept the default private-network IPaddress andproceedtochangingthe IPaddress

range, type yes andpress the Returnkey.
The clsetup utility will ask if it is okay to accept the default netmask. Skip to the next
step to enter your response.

Tochange the default private-network IPaddress, performthe followingsubsteps.

i. Type no inresponse tothe clsetup utility questionabout whether it is okay toaccept
the default address, thenpress the Returnkey.
The clsetup utility will prompt for the newprivate-network IP address.
ii. Type the newIPaddress andpress the Returnkey.
The clsetup utility displays the default netmask and then asks if it is okay to accept
the default netmask.
e. Change or accept the default private-network IPaddress netmask andrange.
The default netmask is 255.255.240.0. This default IP address range supports up to 64
nodes, up to 10 private networks, and up to 12 zone clusters in the cluster. If you choose to
change the netmask, you specify in the following substeps the number of nodes and private
networks that you expect in the cluster.
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 36
If you also expect to confgure zone clusters, you specify that number in Howto Finish
Upgrade to Oracle Solaris Cluster 3.3 Software on page 101, after all nodes are back in
cluster mode.

Toaccept the default IPaddress netmask andrange, type yes andpress the Returnkey.
Then skip to the next step.

Tochange the IPaddress netmask andrange, performthe followingsubsteps.

i. Type no inresponse tothe clsetup utility's questionabout whether it is okay to
accept the default address range, thenpress the Returnkey.
When you decline the default netmask, the clsetup utility prompts you for the
number of nodes and private networks that you expect to confgure in the cluster.
ii. Enter the number of nodes andprivate networks that youexpect toconfgure inthe
cluster.
Fromthese numbers, the clsetup utility calculates two proposed netmasks:

The frst netmask is the minimumnetmask to support the number of nodes and
private networks that you specifed.

The second netmask supports twice the number of nodes and private networks
that you specifed, to accommodate possible future growth.
iii. Specify either of the calculatednetmasks, or specify a diferent netmask that
supports the expectednumber of nodes andprivate networks.
f. Type yes inresponse tothe clsetup utility's questionabout proceedingwiththe update.
g. Whenfnished, exit the clsetup utility.
After all nodes inthe cluster are upgraded, reboot the upgradednodes.
a. Shut downeachnode.
phys-schost# shutdown -g0 -y
b. Boot eachnode intocluster mode.

OnSPARCbasedsystems, dothe following:

ok boot
18
Performing a Standard Upgrade of a Cluster
Chapter 2 Performing a Standard Upgrade to Oracle Solaris Cluster 3.3 Software 37

Onx86 basedsystems, dothe following:

When the GRUB menu is displayed, select the appropriate Solaris entry and press Enter.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+-------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based Systemby
Using GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
Go to Chapter 6, Completing the Upgrade. Next Steps
Performing a Standard Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 38
Performing a Dual-Partition Upgrade to Oracle
Solaris Cluster 3.3 Software
This chapter provides the following information to upgrade a multiple-node cluster to Oracle
Solaris Cluster 3.3 software by using the dual-partition upgrade method:

Howto Prepare the Cluster for Upgrade (Dual-Partition) on page 42

Howto Upgrade the Solaris OS and Volume Manager Software (Dual-Partition) on

page 50

Howto Upgrade Oracle Solaris Cluster 3.3 Software (Dual-Partition) on page 56

Note If your cluster uses a ZFS root fle systemand is confgured with zone clusters, you cannot
use dual-partition upgrade to upgrade the Solaris OS. Use only the live upgrade method to
upgrade the Solaris OS. After Solaris is upgraded, you can use dual-partition upgrade to
upgrade the Oracle Solaris Cluster software.
Performinga Dual-PartitionUpgrade of a Cluster
The following table lists the tasks to performto upgrade to Oracle Solaris Cluster 3.3 software.
You also performthese tasks to upgrade only the Solaris OS.
Note If you upgrade the Solaris OS to a newmarketing release, such as fromSolaris 9 to Oracle
Solaris 10 software, you must also upgrade the Oracle Solaris Cluster software and dependency
software to the version that is compatible with the newOS version.
3
C H A P T E R 3
39
TABLE 31 Task Map: Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software
Task Instructions
1. Read the upgrade requirements and restrictions. Determine the
proper upgrade method for your confguration and needs.
Upgrade Requirements and Software Support Guidelines on
page 11
Choosing an Oracle Solaris Cluster Upgrade Method on
page 13
2. If a quorumserver is used, upgrade the QuorumServer
software.
Howto Upgrade QuorumServer Software on page 40
3. If Oracle Solaris Cluster Geographic Edition software is
installed, uninstall it. Partition the cluster into two groups of
nodes.
Howto Prepare the Cluster for Upgrade (Dual-Partition) on
page 42
4. Upgrade the Solaris software, if necessary, to a supported
Solaris update. If the cluster uses dual-string mediators for Solaris
Volume Manager software, unconfgure the mediators. As
needed, upgrade Veritas Volume Manager (VxVM) and Veritas
File System(VxFS). Solaris Volume Manager software is
automatically upgraded with the Solaris OS.
Howto Upgrade the Solaris OS and Volume Manager Software
(Dual-Partition) on page 50
5. Upgrade to Oracle Solaris Cluster 3.3 framework and
data-service software. If necessary, upgrade applications. If the
cluster uses dual-string mediators and you upgraded the Solaris
OS, reconfgure the mediators. If you upgraded VxVM, upgrade
disk groups.
Howto Upgrade Oracle Solaris Cluster 3.3 Software
(Dual-Partition) on page 56
6. Use the scversions command to commit the cluster to the
upgrade.
Howto Commit the Upgraded Cluster to Oracle Solaris Cluster
3.3 Software on page 99
7. Verify successful completion of upgrade to Oracle Solaris
Cluster 3.3 software.
Howto Verify Upgrade of Oracle Solaris Cluster 3.3 Software
on page 100
8. Enable resources and bring resource groups online. Optionally,
migrate existing resources to newresource types. Upgrade to
Oracle Solaris Cluster Geographic Edition 3.3 software, if used.
Howto Finish Upgrade to Oracle Solaris Cluster 3.3 Software
on page 101
9. (Optional) SPARC: Upgrade the Oracle Solaris Cluster module
for Sun Management Center, if needed.
SPARC: Howto Upgrade Oracle Solaris Cluster Module
Software for Sun Management Center on page 121

HowtoUpgrade QuorumServer Software

If the cluster uses a quorumserver, upgrade the QuorumServer software on the quorumserver
before you upgrade the cluster.
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 40
Note If more than one cluster uses the quorumserver, performthese steps for each of those
clusters.
Performall steps as superuser on the cluster and on the quorumserver.
If the cluster has twonodes andthe quorumserver is the cluster's only quorumdevice,
temporarily adda secondquorumdevice.
See Adding a QuorumDevice in Oracle Solaris Cluster SystemAdministration Guide.
If you add another quorumserver as a temporary quorumdevice, the quorumserver can run
the same software version as the quorumserver that you are upgrading, or it can run the 3.3
version of QuorumServer software.
Unconfgure the quorumserver fromeachcluster that uses the quorumserver.
phys-schost# clquorum remove quorumserver
Fromthe quorumserver toupgrade, verify that the quorumserver nolonger serves any cluster.
quorumserver# clquorumserver show +
If the output shows any cluster is still served by the quorumserver, unconfgure the quorum
server fromthat cluster. Then repeat this step to confrmthat the quorumserver is no longer
confgured with any cluster.
Note If you have unconfgured the quorumserver froma cluster but the clquorumserver show
command still reports that the quorumserver is serving that cluster, the command might be
reporting stale confguration information. See Cleaning Up Stale QuorumServer Cluster
Information in Oracle Solaris Cluster SystemAdministration Guide.
Fromthe quorumserver toupgrade, halt all quorumserver instances.
quorumserver# clquorumserver stop +
Uninstall the QuorumServer software fromthe quorumserver toupgrade.
a. Navigate tothe directory where the uninstaller is located.
quorumserver# cd /var/sadm/prod/SUNWentsysver
ver The version that is installed on your system.
b. Start the uninstallationwizard.
quorumserver# ./uninstall
1
2
3
4
5
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 41
c. Followinstructions onthe screentouninstall the QuorumServer software fromthe
quorum-server host computer.
After removal is fnished, you can viewany available log. See Chapter 8, Uninstalling, in
Sun Java Enterprise System5 Update 1 Installation Guide for UNIXfor additional
information about using the uninstall program.
d. (Optional) Cleanupor remove the quorumserver directories.
By default, this directory is /var/scqsd.
Install the Oracle Solaris Cluster 3.3 QuorumServer software, reconfgure the quorumserver,
andstart the quorumserver daemon.
Followthe steps in Howto Install and Confgure QuorumServer Software in Oracle Solaris
Cluster Software Installation Guide for installing the QuorumServer software.
Froma cluster node, confgure the upgradedquorumserver as a quorumdevice.
Followthe steps in Howto Confgure QuorumDevices in Oracle Solaris Cluster Software
Installation Guide.
If youconfgureda temporary quorumdevice, unconfgure it.
phys-schost# clquorum remove tempquorum

HowtoPrepare the Cluster for Upgrade

(Dual-Partition)
Performthis procedure to prepare a multiple-node cluster for a dual-partition upgrade. These
procedures will refer to the two groups of nodes as the frst partition and the second partition.
The nodes that you assign to the second partition will continue cluster services while you
upgrade the nodes in the frst partition. After all nodes in the frst partition are upgraded, you
switch cluster services to the frst partition and upgrade the second partition. After all nodes in
the second partition are upgraded, you boot the nodes into cluster mode to rejoin the nodes
fromthe frst partition.
Note If you are upgrading a single-node cluster, do not use this upgrade method. Instead, go to
Howto Prepare the Cluster for Upgrade (Standard) on page 20 or Howto Prepare the
Cluster for Upgrade (Live Upgrade) on page 70.
Performall steps fromthe global zone only.
Performthe following tasks:
6
7
8
BeforeYouBegin
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 42

Ensure that the confguration meets the requirements for upgrade. See Upgrade
Requirements and Software Support Guidelines on page 11.

Have available the installation media, documentation, and patches for all software products
that you are upgrading, including the following software:

Solaris OS

Oracle Solaris Cluster 3.3 framework

Oracle Solaris Cluster 3.3 patches

Oracle Solaris Cluster 3.3 data services (agents)

Applications that are managed by Oracle Solaris Cluster 3.3 data services

Veritas Volume Manager, if applicable

See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes
for the location of patches and installation instructions.

If you use role-based access control (RBAC) instead of superuser to access the cluster nodes,
ensure that you can assume an RBACrole that provides authorization for all Oracle Solaris
Cluster commands. This series of upgrade procedures requires the following Oracle Solaris
Cluster RBACauthorizations if the user is not superuser:

solaris.cluster.modify

solaris.cluster.admin

solaris.cluster.read
See Role-Based Access Control (Overview) in SystemAdministration Guide: Security
Services for more information about using RBACroles. See the Oracle Solaris Cluster man
pages for the RBACauthorization that each Oracle Solaris Cluster subcommand requires.
Ensure that the cluster is functioningnormally.
a. Viewthe current status of the cluster by runningthe followingcommandfromany node.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost% scstat

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
phys-schost% cluster status
See the scstat(1M) or cluster(1CL) man page for more information.
b. Searchthe /var/adm/messages logonthe same node for unresolvederror messages or
warningmessages.
c. Check the volume-manager status.
1
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 43
If necessary, notify users that cluster services might be temporarily interruptedduringthe
upgrade.
Service interruption will be approximately the amount of time that your cluster normally takes
to switch services to another node.
Become superuser.
Ensure that the RG_system property of all resource groups inthe cluster is set toFALSE.
Asetting of RG_system=TRUE would restrict certain operations that the dual-partition software
must perform.
a. Oneachnode, determine whether any resource groups are set toRG_system=TRUE.
phys-schost# clresourcegroup show -p RG_system
Make note of which resource groups to change. Save this list to use when you restore the
setting after upgrade is completed.
b. For eachresource groupthat is set toRG_system=TRUE, change the settingtoFALSE.
phys-schost# clresourcegroup set -p RG_system=FALSE resourcegroup
If Geographic Editionsoftware is installed, uninstall it.
For uninstallation procedures, see the documentation for your version of Geographic Edition
software.
If youwill upgrade the Solaris OS andyour cluster uses dual-stringmediators for Solaris Volume
Manager software, unconfgure your mediators.
See Confguring Dual-String Mediators in Oracle Solaris Cluster Software Installation Guide
for more information about mediators.
a. Runthe followingcommandtoverify that nomediator data problems exist.
phys-schost# medstat -s setname
-s setname Specifes the disk set name.
If the value in the Status feld is Bad, repair the afected mediator host. Followthe procedure
Howto Fix Bad Mediator Data in Oracle Solaris Cluster Software Installation Guide.
b. List all mediators.
Save this information for when you restore the mediators during the procedure Howto
Finish Upgrade to Oracle Solaris Cluster 3.3 Software on page 101.
c. For a disk set that uses mediators, take ownershipof the disk set if nonode already has
ownership.
2
3
4
5
6
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 44

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scswitch -z -D setname -h node
-z Changes mastery.
-D devicegroup Specifes the name of the disk set.
-h node Specifes the name of the node to become primary of the disk set.

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
phys-schost# cldevicegroup switch -n node devicegroup
d. Unconfgure all mediators for the disk set.
phys-schost# metaset -s setname -d -m mediator-host-list
-s setname Specifes the disk set name.
-d Deletes fromthe disk set.
-m mediator-host-list Specifes the name of the node to remove as a mediator host for the
disk set.
See the mediator(7D) man page for further information about mediator-specifc options to
the metaset command.
e. Repeat Stepc throughStepdfor eachremainingdisk set that uses mediators.
If youare upgradinga two-node cluster, skiptoStep17.
Otherwise, proceed to Step 8 to determine the partitioning scheme to use. You will determine
which nodes each partition will contain, but interrupt the partitioning process. You will then
compare the node lists of all resource groups against the node members of each partition in the
scheme that you will use. If any resource group does not contain a member of each partition,
you must change the node list.
Loadthe installationDVD-ROMintothe DVD-ROMdrive.
If the volume management daemon vold(1M) is running and is confgured to manage
CD-ROMor DVDdevices, the daemon automatically mounts the media on the /cdrom/cdrom0
directory.
Become superuser ona node of the cluster.
Change tothe /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools/
directory, where arch is sparc or x86andwhere ver is 10 for Oracle Solaris 10 .
phys-schost# cd /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools
7
8
9
10
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 45
Start the scinstall utility ininteractive mode.
phys-schost# ./scinstall
Note Do not use the /usr/cluster/bin/scinstall command that is already installed on the
node. You must use the scinstall command on the installation DVD-ROM.
The scinstall Main Menu is displayed.
Choose the menuitem, Manage a Dual-PartitionUpgrade.
*** Main Menu ***
Please select from one of the following (*) options:
1) Create a new cluster or add a cluster node
2) Configure a cluster to be JumpStarted from this install server
* 3) Manage a dual-partition upgrade
* 4) Upgrade this cluster node
* 5) Print release information for this cluster node
* ?) Help with menu options
* q) Quit
Option: 3
The Manage a Dual-Partition Upgrade Menu is displayed.
Choose the menuitem, Display andSelect Possible PartitioningSchemes.
Followthe prompts toperformthe followingtasks:
a. Display the possible partitioningschemes for your cluster.
b. Choose a partitioningscheme.
c. Choose whichpartitiontoupgrade frst.
Note Stop and do not respond yet when prompted, Do you want to begin the
dual-partition upgrade?, but do not exit the scinstall utility. You will respond to this
prompt in Step 19 of this procedure.
Make note of whichnodes belongtoeachpartitioninthe partitionscheme.
Onanother node of the cluster, become superuser.
Ensure that any critical data services canswitchover betweenpartitions.
For a two-node cluster, each node will be the only node in its partition.
11
12
13
14
15
16
17
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 46
When the nodes of a partition are shut down in preparation for dual-partition upgrade, the
resource groups that are hosted on those nodes switch over to a node in the other partition. If a
resource group does not contain a node fromeach partition in its node list, the resource group
cannot switch over. To ensure successful switchover of all critical data services, verify that the
node list of the related resource groups contains a member of each upgrade partition.
a. Display the node list of eachresource groupthat yourequire toremaininservice duringthe
entire upgrade.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scrgadm -pv -g resourcegroup | grep "Res Group Nodelist"
-p Displays confguration information.
-v Displays in verbose mode.
-g resourcegroup Specifes the name of the resource group.

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
phys-schost# clresourcegroup show -p nodelist
=== Resource Groups and Resources ===
Resource Group: resourcegroup
Nodelist: node1 node2
...
b. If the node list of a resource groupdoes not containat least one member of eachpartition,
redefne the node list toinclude a member of eachpartitionas a potential primary node.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scrgadm -a -g resourcegroup -h nodelist
-a Adds a newconfguration.
-h Specifes a comma-separated list of node names.

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
phys-schost# clresourcegroup add-node -n node resourcegroup
Determine your next step.

If youare upgradinga two-node cluster, returntoStep8 throughStep14 todesignate your

partitioningscheme andupgrade order.
When you reach the prompt Do you want to begin the dual-partition upgrade?, skip
to Step 19.
18
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 47

If youare upgradinga cluster withthree or more nodes, returntothe node that is running
the interactive scinstall utility.
Proceed to Step 19.
At the interactive scinstall prompt Do you want to begin the dual-partition upgrade?,
type Yes.
The command verifes that a remote installation method is available.
Whenprompted, press Enter tocontinue eachstage of preparationfor dual-partitionupgrade.
The command switches resource groups to nodes in the second partition, and then shuts down
each node in the frst partition.
After all nodes inthe frst partitionare shut down, boot eachnode inthat partitioninto
noncluster mode.

OnSPARCbasedsystems, performthe followingcommand:

ok boot -x

Onx86 basedsystems, performthe followingcommands:

a. Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry andtype e
toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+----------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based Systemby
Using GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
b. Inthe boot parameters screen, use the arrowkeys toselect the kernel entry andtype e
toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
19
20
21
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 48
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
c. Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
d. Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
e. Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over the
systemboot. The next time you reboot the node, it will boot into cluster mode. To boot
into noncluster mode instead, performthese steps to again to add the -x option to the
kernel boot parameter command.
Ensure that eachsystemdisk is backedup.
If any applications that are runninginthe secondpartitionare not under control of the Resource
GroupManager (RGM), create scripts tohalt the applications before youbegintoupgrade those
nodes.
During dual-partition upgrade processing, these scripts would be called to stop applications
such as Oracle Real Application Clusters before the nodes in the second partition are halted.
a. Create the scripts that youneedtostopapplications that are not under RGMcontrol.

Create separate scripts for those applications that you want stopped before applications
under RGMcontrol are stopped and for those applications that you want stop
afterwards.
22
23
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 49

To stop applications that are running on more than one node in the partition, write the
scripts accordingly.

Use any name and directory path for your scripts that you prefer.
b. Ensure that eachnode inthe cluster has its owncopy of your scripts.
c. Oneachnode, modify the followingOracle Solaris Cluster scripts tocall the scripts that you
placedonthat node.

/etc/cluster/ql/cluster_pre_halt_apps - Use this fle to call those scripts that you

want to run before applications that are under RGMcontrol are shut down.

/etc/cluster/ql/cluster_post_halt_apps - Use this fle to call those scripts that you

want to run after applications that are under RGMcontrol are shut down.
The Oracle Solaris Cluster scripts are issued fromone arbitrary node in the partition during
post-upgrade processing of the partition. Therefore, ensure that the scripts on any node of
the partition will performthe necessary actions for all nodes in the partition.
Upgrade software on each node in the frst partition.

To upgrade Solaris software before you performOracle Solaris Cluster software upgrade, go
to Howto Upgrade the Solaris OS and Volume Manager Software (Dual-Partition) on
page 50.

If Oracle Solaris Cluster 3.3 software does not support the release of the Solaris OS that
you currently run on your cluster, you must upgrade the Solaris software to a supported
release. See Supported Products in Oracle Solaris Cluster 3.3 Release Notes for more
information.

If Oracle Solaris Cluster 3.3 software supports the release of the Solaris OS that you
currently run on your cluster, further Solaris software upgrade is optional.

Otherwise, upgrade to Oracle Solaris Cluster 3.3 software. Go to Howto Upgrade Oracle
Solaris Cluster 3.3 Software (Dual-Partition) on page 56.

HowtoUpgrade the Solaris OS andVolume Manager

Software (Dual-Partition)
Performthis procedure on each node in the cluster to upgrade the Solaris OS and optionally
VxVM, if used. Performall steps fromthe global zone only.
If the cluster already runs on a version of the Solaris OS that supports Oracle Solaris Cluster 3.3
software, further upgrade of the Solaris OS is optional. If you do not intend to upgrade the
Solaris OS or VxVM, proceed to Howto Upgrade Oracle Solaris Cluster 3.3 Software
(Standard) on page 32.
Next Steps
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 50
Note The cluster must already run on, or be upgraded to, at least the minimumrequired level
of the Solaris OS to support upgrade to Oracle Solaris Cluster 3.3 software. See Supported
Products in Oracle Solaris Cluster 3.3 Release Notes for more information.
Ensure that all steps in Howto Prepare the Cluster for Upgrade (Standard) on page 20 are
completed.
Become superuser onthe cluster node toupgrade.
The node must be a member of the partition that is in noncluster mode.
Determine whether the followingApache run-control scripts exist andare enabledor disabled:
/etc/rc0.d/K16apache
/etc/rc1.d/K16apache
/etc/rc2.d/K16apache
/etc/rc3.d/S50apache
/etc/rcS.d/K16apache
Some applications, such as Oracle Solaris Cluster HAfor Apache, require that Apache run
control scripts be disabled.

If these scripts exist and contain an uppercase K or S in the fle name, the scripts are enabled.
No further action is necessary for these scripts.

If these scripts do not exist, in Step 7 you must ensure that any Apache run control scripts
that are installed during the Solaris OS upgrade are disabled.

If these scripts exist but the fle names contain a lowercase k or s, the scripts are disabled. In
Step 7 you must ensure that any Apache run control scripts that are installed during the
Solaris OS upgrade are disabled.
Comment out all entries for globally mountedfle systems inthe node's /etc/vfstab fle.
a. For later reference, make a recordof all entries that are already commentedout.
b. Temporarily comment out all entries for globally mountedfle systems inthe /etc/vfstab
fle.
Entries for globally mounted fle systems contain the global mount option. Comment out
these entries to prevent the Solaris upgrade fromattempting to mount the global devices.
Determine whichprocedure tofollowtoupgrade the Solaris OS.

To use Live Upgrade, go instead to Chapter 4, Performing a Live Upgrade to Oracle Solaris
Cluster 3.3 Software.

To upgrade a cluster that uses Solaris Volume Manager by a method other than Live
Upgrade, followupgrade procedures in Solaris installation documentation.
BeforeYouBegin
1
2
3
4
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 51

To upgrade a cluster that uses Veritas Volume Manager by a method other than Live
Upgrade, followupgrade procedures in Veritas Storage Foundation installation
documentation.
Note If your cluster has VxVMinstalled and you are upgrading the Solaris OS, you must
reinstall or upgrade to VxVMsoftware that is compatible with the version of Oracle Solaris 10
you upgraded to.
Upgrade the Solaris software, followingthe procedure that youselectedinStep4.
a. Whenprompted, choose the manual reboot option.
b. Whenpromptedtoreboot, always reboot intononcluster mode.
Note Do not performthe fnal reboot instruction in the Solaris software upgrade. Instead,
do the following:
a. Return to this procedure to performStep 6 and Step 7.
b. Reboot into noncluster mode in Step 8 to complete Solaris software upgrade.
Execute the following commands to boot a node into noncluster mode during Solaris
upgrade:

OnSPARCbasedsystems, performeither of the followingcommands:

phys-schost# reboot -- -x
or
ok boot -x
If the instruction says to run the init S command, use the reboot -- -xs command
instead.

Onx86 basedsystems, performthe followingcommand:

phys-schost# shutdown -g -y -i0
Press any key to continue
i. Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry andtype
e toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+-------------------------------------------------------------------------+
5
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 52
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based System
by Using GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
ii. Inthe boot parameters screen, use the arrowkeys toselect the kernel entry andtype
e toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
iii. Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
iv. Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
v. Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over the
systemboot. The next time you reboot the node, it will boot into cluster mode. To
boot into noncluster mode instead, performthese steps to again to add the -x option
to the kernel boot parameter command.
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 53
If the instruction says to run the init S command, shut down the systemthen change
the GRUB kernel boot command to /platform/i86pc/multiboot -sx instead.
Inthe /a/etc/vfstab fle, uncomment those entries for globally mountedfle systems that you
commentedout inStep3.
If Apache runcontrol scripts were disabledor didnot exist before youupgradedthe Solaris OS,
ensure that any scripts that were installedduringSolaris upgrade are disabled.
To disable Apache run control scripts, use the following commands to rename the fles with a
lowercase k or s.
phys-schost# mv /a/etc/rc0.d/K16apache /a/etc/rc0.d/k16apache
phys-schost# mv /a/etc/rc1.d/K16apache /a/etc/rc1.d/k16apache
phys-schost# mv /a/etc/rc2.d/K16apache /a/etc/rc2.d/k16apache
phys-schost# mv /a/etc/rc3.d/S50apache /a/etc/rc3.d/s50apache
phys-schost# mv /a/etc/rcS.d/K16apache /a/etc/rcS.d/k16apache
Alternatively, you can rename the scripts to be consistent with your normal administration
practices.
Reboot the node intononcluster mode.

OnSPARCbasedsystems, performthe followingcommand.

Include the double dashes (--) in the command:
phys-schost# reboot -- -x

Onx86 basedsystems, performthe shutdownandboot procedures that are describedin

Step5 except add-x tothe kernel boot commandinsteadof -sx.
If your cluster runs VxVMandyouare upgradingit as well as upgradingthe Solaris OS, perform
the remainingsteps inthe procedure toreinstall or upgradeVxVM.
Make the following changes to the procedure:

After VxVMupgrade is complete but before youreboot, verify the entries inthe
/etc/vfstab fle.
If any of the entries that you uncommented in Step 6 were commented out, make those
entries uncommented again.

If theVxVMprocedures instruct youtoperforma fnal reconfgurationreboot, do not use the

-r optionalone. Instead, reboot intononcluster mode by usingthe -rx options.

OnSPARCbasedsystems, performthe followingcommand:

phys-schost# reboot -- -rx
6
7
8
9
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 54

Onx86 basedsystems, performthe shutdownandboot procedures that are describedin

Step5 except add-rx tothe kernel boot commandinsteadof -sx.
Note If you see a message similar to the following, type the root password to continue upgrade
processing. Do not run the fsck command nor type Ctrl-D.
WARNING - Unable to repair the /global/.devices/node@1 filesystem.
Run fsck manually (fsck -F ufs /dev/vx/rdsk/rootdisk_13vol). Exit the
shell when done to continue the boot process.
Type control-d to proceed with normal startup,
(or give root password for system maintenance): Type the root password
(Optional) SPARC: UpgradeVxFS.
Followprocedures that are provided in your VxFS documentation.
Install any requiredSolaris software patches andhardware-relatedpatches, anddownloadany
neededfrmware that is containedinthe hardware patches.
Note Do not reboot after you add patches. Wait to reboot the node until after you upgrade the
Oracle Solaris Cluster software.
See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes for
the location of patches and installation instructions.
If you are already running Oracle Solaris Cluster 3.3 software and only upgrading the Oracle
Solaris 10 OS to an Oracle Solaris 10 update release, you do not need to upgrade the Oracle
Solaris Cluster software. Go to Chapter 6, Completing the Upgrade.
Otherwise, upgrade to Oracle Solaris Cluster 3.3 software. Go to Howto Upgrade Oracle
Solaris Cluster 3.3 Software (Dual-Partition) on page 56.
Note To complete the upgrade to a newmarketing release of the Solaris OS, such as from
Solaris 9 to Oracle Solaris 10 software, you must also upgrade the Oracle Solaris Cluster
software and dependency software to the version that is compatible with the newversion of the
Solaris OS.
10
11
Next Steps
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 55

HowtoUpgrade Oracle Solaris Cluster 3.3Software

(Dual-Partition)
Performthis procedure to upgrade each node of the cluster to Oracle Solaris Cluster 3.3
software. You must also performthis procedure after you upgrade to a diferent marketing
release of the Solaris OS, such as fromSolaris 9 to Oracle Solaris 10 software.
Performall steps fromthe global zone only.
Tip You can use the cconsole utility to performthis procedure on multiple nodes
simultaneously. See Howto Install Cluster Control Panel Software on an Administrative
Console in Oracle Solaris Cluster Software Installation Guide for more information.
Performthe following tasks:

Ensure that all steps in Howto Prepare the Cluster for Upgrade (Dual-Partition) on
page 42 are completed.

Ensure that the node you are upgrading belongs to the partition that is not active in the
cluster and that the node is in noncluster mode.

If you upgraded to a newmarketing release of the Solaris OS, such as fromSolaris 9 to

Oracle Solaris 10 software, ensure that all steps in Howto Upgrade the Solaris OS and
Volume Manager Software (Dual-Partition) on page 50 are completed.

Ensure that you have installed all required Solaris software patches and hardware-related
patches.
Become superuser ona node that is a member of the partitionthat is innoncluster mode.
Loadthe installationDVD-ROMintothe DVD-ROMdrive.
If the volume management daemon vold(1M) is running and is confgured to manage
CD-ROMor DVDdevices, the daemon automatically mounts the media on the /cdrom/cdrom0
directory.
Change tothe /Solaris_arch/Product/sun_cluster/Solaris_ver/Tools/ directory, where
arch is sparc or x86 andwhere ver is 10 for Oracle Solaris 10.
phys-schost# cd /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools
Start the scinstall utility.
phys-schost# ./scinstall
BeforeYouBegin
1
2
3
4
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 56
Note Do not use the /usr/cluster/bin/scinstall command that is already installed on the
node. You must use the scinstall command that is located on the installation DVD-ROM.
The scinstall Main Menu is displayed.
Choose the menuitem, UpgradeThis Cluster Node.
*** Main Menu ***
Please select from one of the following (*) options:
1) Create a new cluster or add a cluster node
2) Configure a cluster to be JumpStarted from this install server
* 3) Manage a dual-partition upgrade
* 4) Upgrade this cluster node
* 5) Print release information for this cluster node
* ?) Help with menu options
* q) Quit
Option: 4
The Upgrade Menu is displayed.
Choose the menuitem, Upgrade Oracle Solaris Cluster Framework onThis Node.
Followthe menuprompts toupgrade the cluster framework.
During the Oracle Solaris Cluster upgrade, scinstall might make one or more of the following
confguration changes:

Rename the ntp.conf fle to ntp.conf.cluster, if ntp.conf.cluster does not already

exist on the node.

Set the local-mac-address? variable to true, if the variable is not already set to that value.
Upgrade processing is fnished when the systemdisplays the message Completed Oracle
Solaris Cluster framework upgrade and prompts you to press Enter to continue.
Quit the scinstall utility.
Upgrade data service packages.
You must upgrade all data services to the Oracle Solaris Cluster 3.3 version.
Note For HAfor SAP Web Application Server, if you are using a J2EE engine resource or a web
application server component resource or both, you must delete the resource and recreate it
with the newweb application server component resource. Changes in the newweb application
server component resource includes integration of the J2EE functionality. For more
information, see Oracle Solaris Cluster Data Service for SAP Web Application Server Guide.
5
6
7
8
9
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 57
a. Start the upgradedinteractive scinstall utility.
phys-schost# /usr/cluster/bin/scinstall
Note Do not use the scinstall utility that is on the installation media to upgrade data
service packages.
The scinstall Main Menu is displayed.
b. Choose the menuitem, UpgradeThis Cluster Node.
The Upgrade Menu is displayed.
c. Choose the menuitem, Upgrade Oracle Solaris Cluster Data Service Agents onThis Node.
d. Followthe menuprompts toupgrade Oracle Solaris Cluster data service agents that are
installedonthe node.
You can choose fromthe list of data services that are available to upgrade or choose to
upgrade all installed data services.
e. When the system displays the message Completed upgrade of Oracle Solaris Cluster
data services agents, press Enter.
The Upgrade Menu is displayed.
Quit the scinstall utility.
Unloadthe installationDVD-ROMfromthe DVD-ROMdrive.
a. Toensure that the DVD-ROMis not beingused, change toa directory that does not reside on
the DVD-ROM.
b. Eject the DVD-ROM.
phys-schost# eject cdrom
If youhave Oracle Solaris Cluster HAfor NFS confguredona highly available local fle system,
ensure that the loopback fle system(LOFS) is disabled.
Note If you have non-global zones confgured, LOFS must remain enabled. For guidelines
about using LOFS and alternatives to disabling it, see Cluster File Systems in Oracle Solaris
Cluster Software Installation Guide.
To disable LOFS, ensure that the /etc/system fle contains the following entry:
exclude:lofs
10
11
12
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 58
This change becomes efective at the next systemreboot.
As needed, manually upgrade any customdata services that are not suppliedonthe product
media.
Verify that eachdata-service update is installedsuccessfully.
Viewthe upgrade log fle that is referenced at the end of the upgrade output messages.
Install any Oracle Solaris Cluster 3.3 framework anddata-service software patches.
See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes for
the location of patches and installation instructions.
Upgrade software applications that are installedonthe cluster.
Ensure that application levels are compatible with the current versions of Oracle Solaris Cluster
and Solaris software. See your application documentation for installation instructions.
If you want to upgrade VxVMand did not upgrade the Solaris OS, followprocedures in Veritas
Storage Foundation installation documentation to upgrade VxVMwithout upgrading the
operating system.
Note If any upgrade procedure instruct you to performa reboot, you must add the -x option to
the boot command. This option boots the cluster into noncluster mode.
Repeat all steps inthis procedure uptothis point onall remainingnodes that youneedto
upgrade inthe partition.
After all nodes ina partitionare upgraded, apply the upgrade changes.
a. Fromone node inthe partitionthat youare upgrading, start the interactive scinstall
utility.
phys-schost# /usr/cluster/bin/scinstall
Note Do not use the scinstall command that is located on the installation media. Only
use the scinstall command that is located on the cluster node.
The scinstall Main Menu is displayed.
b. Type optionnumber for Apply Dual-PartitionUpgrade Changes tothe Partition.
c. Followthe prompts tocontinue eachstage of the upgrade processing.
The command performs the following tasks, depending on which partition the command is
run from:
13
14
15
16
17
18
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 59

First partition - The command halts each node in the second partition, one node at a
time. When a node is halted, any services on that node are automatically switched over
to a node in the frst partition, provided that the node list of the related resource group
contains a node in the frst partition. After all nodes in the second partition are halted,
the nodes in the frst partition are booted into cluster mode and take over providing
cluster services.
Caution Do not reboot any node of the frst partition again until after the upgrade is
completed on all nodes. If you again reboot a node of the frst partition before the second
partition is upgraded and rebooted into the cluster, the upgrade might fail in an
unrecoverable state.

Second partition - The command boots the nodes in the second partition into cluster
mode, to join the active cluster that was formed by the frst partition. After all nodes have
rejoined the cluster, the command performs fnal processing and reports on the status of
the upgrade.
d. Exit the scinstall utility, if it is still running.
e. If youare fnishingupgrade fromSunCluster 3.1 8/05 software of the frst partitionandyou
want toconfgure zone clusters, set the expectednumber of nodes andprivate networks in
the cluster.
If you upgraded fromSun Cluster 3.1 8/05 software and do not want to confgure zone
clusters, or if you upgraded fromSun Cluster 3.2 software, this task is optional.
i. Boot all nodes inthe frst partitionintononcluster mode.

OnSPARCbasedsystems, performthe followingcommand:

ok boot -x

Onx86 basedsystems, performthe followingcommands:

Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry and
type e toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 60
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based
Systemby Using GRUB (Task Map) in SystemAdministration Guide: Basic
Administration.
Inthe boot parameters screen, use the arrowkeys toselect the kernel entry and
type e toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
|+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over
the systemboot. The next time you reboot the node, it will boot into cluster
mode. To boot into noncluster mode instead, performthese steps to again add
the -x option to the kernel boot parameter command.
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 61
ii. Fromone node, start the clsetup utility.
When run in noncluster mode, the clsetup utility displays the Main Menu for
noncluster-mode operations.
iii. Choose the menuitem, Change IPAddress Range.
The clsetup utility displays the current private-network confguration, then asks if you
would like to change this confguration.
iv. Tochange either the private-network IPaddress or the IPaddress range, type yes and
press the Returnkey.
The clsetup utility displays the default private-network IP address, 172.16.0.0, and
asks if it is okay to accept this default.
v. Change or accept the private-network IPaddress.

Toaccept the default private-network IPaddress andproceedtochangingthe IP

address range, type yes andpress the Returnkey.
The clsetup utility will ask if it is okay to accept the default netmask. Skip to the next
step to enter your response.

Tochange the default private-network IPaddress, performthe followingsubsteps.

Type no inresponse tothe clsetup utility questionabout whether it is okay to
accept the default address, thenpress the Returnkey.
The clsetup utility will prompt for the newprivate-network IP address.
Type the newIPaddress andpress the Returnkey.
The clsetup utility displays the default netmask and then asks if it is okay to
accept the default netmask.
vi. Change or accept the default private-network IPaddress netmask andrange.
The default netmask is 255.255.240.0. This default IP address range supports up to 64
nodes, up to 10 private networks, and up to 12 zone clusters in the cluster. If you choose
to change the netmask, you specify in the following substeps the number of nodes and
private networks that you expect in the cluster.
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 62
If you also expect to confgure zone clusters, you specify that number in Howto Finish
Upgrade to Oracle Solaris Cluster 3.3 Software on page 101, after all nodes are back in
cluster mode.

Toaccept the default IPaddress netmask andrange, type yes andpress the Return
key.
Then skip to the next step.

Tochange the IPaddress netmask andrange, performthe followingsubsteps.

Type no inresponse tothe clsetup utility's questionabout whether it is okay to
accept the default address range, thenpress the Returnkey.
When you decline the default netmask, the clsetup utility prompts you for the
number of nodes and private networks that you expect to confgure in the cluster.
Enter the number of nodes andprivate networks that youexpect toconfgure in
the cluster.
Fromthese numbers, the clsetup utility calculates two proposed netmasks:

The frst netmask is the minimumnetmask to support the number of nodes

and private networks that you specifed.

The second netmask supports twice the number of nodes and private
networks that you specifed, to accommodate possible future growth.
Specify either of the calculatednetmasks, or specify a diferent netmask that
supports the expectednumber of nodes andprivate networks.
vii. Type yes inresponse tothe clsetup utility's questionabout proceedingwiththe update.
viii.Whenfnished, exit the clsetup utility.
ix. Boot the nodes of the frst partitionintocluster mode.
f. If youare fnishingupgrade of the frst partition, performthe followingsubsteps toprepare
the secondpartitionfor upgrade.
Otherwise, if you are fnishing upgrade of the second partition, proceed to Howto Verify
Upgrade of Oracle Solaris Cluster 3.3 Software on page 100.
i. Boot eachnode inthe secondpartitionintononcluster mode.

OnSPARCbasedsystems, performthe followingcommand:

ok boot -x
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 63

Onx86 basedsystems, performthe followingcommands:

Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry and
type e toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based
Systemby Using GRUB (Task Map) in SystemAdministration Guide: Basic
Administration.
Inthe boot parameters screen, use the arrowkeys toselect the kernel entry and
type e toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
|+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Performing a Dual-Partition Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 64
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over
the systemboot. The next time you reboot the node, it will boot into cluster
mode. To boot into noncluster mode instead, performthese steps to again add
the -x option to the kernel boot parameter command.
ii. Upgrade the nodes inthe secondpartition.
To upgrade Solaris software before you performOracle Solaris Cluster software upgrade,
go to Howto Upgrade the Solaris OS and Volume Manager Software (Dual-Partition)
on page 50.
Otherwise, upgrade Oracle Solaris Cluster software on the second partition. Return to
Step 1.
If youchangedthe RG_system property of any resource groups toFALSE, change the settings
back toTRUE.
phys-schost# clresourcegroup set -p RG_system=TRUE resourcegroup
Go to Chapter 6, Completing the Upgrade.
If you experience an unrecoverable error during dual-partition upgrade, performrecovery
procedures in Howto Recover froma Failed Dual-Partition Upgrade on page 111.
19
Next Steps
Troubleshooting
Performing a Dual-Partition Upgrade of a Cluster
Chapter 3 Performing a Dual-Partition Upgrade to Oracle Solaris Cluster 3.3 Software 65
66
Performing a Live Upgrade to Oracle Solaris
Cluster 3.3 Software
This chapter provides the following information to upgrade to Oracle Solaris Cluster 3.3
software by using the live upgrade method:

Howto Prepare the Cluster for Upgrade (Live Upgrade) on page 70

Howto Upgrade the Solaris OS and Oracle Solaris Cluster 3.3 Software (Live Upgrade) on
page 71
If your cluster confguration uses a ZFS root fle systemand is confgured with zone clusters,
you can use live upgrade only to upgrade the Solaris OS. To upgrade Oracle Solaris Cluster
software, after using live upgrade to upgrade Solaris software, use either standard upgrade or
dual-partition upgrade to upgrade Oracle Solaris Cluster software.
Performinga Live Upgrade of a Cluster
The following table lists the tasks to performto upgrade to Oracle Solaris Cluster 3.3 software.
You also performthese tasks to upgrade only the Solaris OS.
Note If you upgrade the Solaris OS to a newmarketing release, such as fromSolaris 9 to Oracle
Solaris 10 software, you must also upgrade the Oracle Solaris Cluster software and dependency
software to the version that is compatible with the newOS version.
TABLE 41 Task Map: Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software
Task Instructions
1. Read the upgrade requirements and restrictions. Determine the
proper upgrade method for your confguration and needs.
Upgrade Requirements and Software Support Guidelines on
page 11
Choosing an Oracle Solaris Cluster Upgrade Method on
page 13
4
C H A P T E R 4
67
TABLE 41 Task Map: Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software (Continued)
Task Instructions
2. If a quorumserver is used, upgrade the QuorumServer
software.
Howto Upgrade QuorumServer Software on page 68
3. If Oracle Solaris Cluster Geographic Edition software is
installed, uninstall it.
Howto Prepare the Cluster for Upgrade (Live Upgrade) on
page 70
4. If the cluster uses dual-string mediators for Solaris Volume
Manager software, unconfgure the mediators. Upgrade the
Solaris software, if necessary, to a supported Solaris update.
Upgrade to Oracle Solaris Cluster 3.3 framework and data-service
software. If necessary, upgrade applications. If the cluster uses
dual-string mediators, reconfgure the mediators. As needed,
upgrade Veritas Volume Manager (VxVM)software and disk
groups and Veritas File System(VxFS).
Howto Upgrade the Solaris OS and Oracle Solaris Cluster 3.3
Software (Live Upgrade) on page 71
5. Use the scversions command to commit the cluster to the
upgrade.
Howto Commit the Upgraded Cluster to Oracle Solaris Cluster
3.3 Software on page 99
6. Verify successful completion of upgrade to Oracle Solaris
Cluster 3.3 software.
Howto Verify Upgrade of Oracle Solaris Cluster 3.3 Software
on page 100
7. Enable resources and bring resource groups online. Migrate
existing resources to newresource types. Upgrade to Oracle
Solaris Cluster Geographic Edition 3.3 software, if used.
Howto Finish Upgrade to Oracle Solaris Cluster 3.3 Software
on page 101
8. (Optional) SPARC: Upgrade the Oracle Solaris Cluster module
for Sun Management Center, if needed.
SPARC: Howto Upgrade Oracle Solaris Cluster Module
Software for Sun Management Center on page 121

HowtoUpgrade QuorumServer Software

If the cluster uses a quorumserver, upgrade the QuorumServer software on the quorumserver
before you upgrade the cluster.
Note If more than one cluster uses the quorumserver, performthese steps for each of those
clusters.
Performall steps as superuser on the cluster and on the quorumserver.
If the cluster has twonodes andthe quorumserver is the cluster's only quorumdevice,
temporarily adda secondquorumdevice.
See Adding a QuorumDevice in Oracle Solaris Cluster SystemAdministration Guide.
If you add another quorumserver as a temporary quorumdevice, the quorumserver can run
the same software version as the quorumserver that you are upgrading, or it can run the 3.3
version of QuorumServer software.
1
Performing a Live Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 68
Unconfgure the quorumserver fromeachcluster that uses the quorumserver.
phys-schost# clquorum remove quorumserver
Fromthe quorumserver toupgrade, verify that the quorumserver nolonger serves any cluster.
quorumserver# clquorumserver show +
If the output shows any cluster is still served by the quorumserver, unconfgure the quorum
server fromthat cluster. Then repeat this step to confrmthat the quorumserver is no longer
confgured with any cluster.
Note If you have unconfgured the quorumserver froma cluster but the clquorumserver show
command still reports that the quorumserver is serving that cluster, the command might be
reporting stale confguration information. See Cleaning Up Stale QuorumServer Cluster
Information in Oracle Solaris Cluster SystemAdministration Guide.
Fromthe quorumserver toupgrade, halt all quorumserver instances.
quorumserver# clquorumserver stop +
Uninstall the QuorumServer software fromthe quorumserver toupgrade.
a. Navigate tothe directory where the uninstaller is located.
quorumserver# cd /var/sadm/prod/SUNWentsysver
ver The version that is installed on your system.
b. Start the uninstallationwizard.
quorumserver# ./uninstall
c. Followinstructions onthe screentouninstall the QuorumServer software fromthe
quorum-server host computer.
After removal is fnished, you can viewany available log. See Chapter 8, Uninstalling, in
Sun Java Enterprise System5 Update 1 Installation Guide for UNIXfor additional
information about using the uninstall program.
d. (Optional) Cleanupor remove the quorumserver directories.
By default, this directory is /var/scqsd.
Install the Oracle Solaris Cluster 3.3 QuorumServer software, reconfgure the quorumserver,
andstart the quorumserver daemon.
Followthe steps in Howto Install and Confgure QuorumServer Software in Oracle Solaris
Cluster Software Installation Guide for installing the QuorumServer software.
2
3
4
5
6
Performing a Live Upgrade of a Cluster
Chapter 4 Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software 69
Froma cluster node, confgure the upgradedquorumserver as a quorumdevice.
Followthe steps in Howto Confgure QuorumDevices in Oracle Solaris Cluster Software
Installation Guide.
If youconfgureda temporary quorumdevice, unconfgure it.
phys-schost# clquorum remove tempquorum

HowtoPrepare the Cluster for Upgrade (Live

Upgrade)
Performthis procedure to prepare a cluster for live upgrade.
Performthe following tasks:

Ensure that the confguration meets the requirements for upgrade. See Upgrade
Requirements and Software Support Guidelines on page 11.

Have available the installation media, documentation, and patches for all software products
that you are upgrading, including the following software:

Solaris OS

Oracle Solaris Cluster 3.3 framework

Oracle Solaris Cluster 3.3 patches

Oracle Solaris Cluster 3.3 data services (agents)

Applications that are managed by Oracle Solaris Cluster 3.3 data services

Veritas Volume Manager, if applicable

See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes
for the location of patches and installation instructions.

If you use role-based access control (RBAC) instead of superuser to access the cluster nodes,
ensure that you can assume an RBACrole that provides authorization for all Oracle Solaris
Cluster commands. This series of upgrade procedures requires the following Oracle Solaris
Cluster RBACauthorizations if the user is not superuser:

solaris.cluster.modify

solaris.cluster.admin

solaris.cluster.read
See Role-Based Access Control (Overview) in SystemAdministration Guide: Security
Services for more information about using RBACroles. See the Oracle Solaris Cluster man
pages for the RBACauthorization that each Oracle Solaris Cluster subcommand requires.
7
8
BeforeYouBegin
Performing a Live Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 70
Ensure that the cluster is functioningnormally.
a. Viewthe current status of the cluster by runningthe followingcommandfromany node.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost% scstat

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
phys-schost% cluster status
See the scstat(1M) or cluster(1CL) man page for more information.
b. Searchthe /var/adm/messages logonthe same node for unresolvederror messages or
warningmessages.
c. Check the volume-manager status.
If necessary, notify users that cluster services will be temporarily interruptedduringthe
upgrade.
Service interruption will be approximately the amount of time that your cluster normally takes
to switch services to another node.
If Geographic Editionsoftware is installed, uninstall it.
For uninstallation procedures, see the documentation for your version of Geographic Edition
software.
Become superuser ona node of the cluster.
Ensure that all shareddata is backedup.
Ensure that eachsystemdisk is backedup.
Performa live upgrade of the Solaris OS, Oracle Solaris Cluster 3.3 software, and other software.
Go to Howto Upgrade the Solaris OS and Oracle Solaris Cluster 3.3 Software (Live Upgrade)
on page 71.

HowtoUpgrade the Solaris OS andOracle Solaris

Cluster 3.3Software (Live Upgrade)
Performthis procedure to upgrade the Solaris OS, volume-manager software, and Oracle
Solaris Cluster software by using the live upgrade method. The Oracle Solaris Cluster live
1
2
3
4
5
6
Next Steps
Performing a Live Upgrade of a Cluster
Chapter 4 Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software 71
upgrade method uses the Solaris Live Upgrade feature. For information about live upgrade of
the Solaris OS, refer to the following Solaris documentation:

Solaris 10 10/09 Installation Guide: Solaris Live Upgrade and Upgrade Planning

If non-global zones are installed on the cluster, see Chapter 8, Upgrading the Solaris OS on
a SystemWith Non-Global Zones Installed, in Solaris 10 10/09 Installation Guide: Solaris
Live Upgrade and Upgrade Planning.
Note The cluster must already run on, or be upgraded to, at least the minimumrequired level
of the Solaris OS to support upgrade to Oracle Solaris Cluster 3.3 software. See Supported
Products in Oracle Solaris Cluster 3.3 Release Notes for more information.
Performthis procedure on each node in the cluster.
Tip You can use the cconsole utility to performthis procedure on multiple nodes
simultaneously. See Howto Install Cluster Control Panel Software on an Administrative
Console in Oracle Solaris Cluster Software Installation Guide for more information.

Ensure that all steps in Howto Prepare the Cluster for Upgrade (Live Upgrade) on page 70
are completed.
Install a supportedversionof Solaris Live Upgrade software.
Followinstructions in Solaris Live Upgrade SystemRequirements in Solaris 10 10/09
Installation Guide: Solaris Live Upgrade and Upgrade Planning and Installing Solaris Live
Upgrade in Solaris 10 10/09 Installation Guide: Solaris Live Upgrade and Upgrade Planning.
If youwill upgrade the Solaris OS andyour cluster uses dual-stringmediators for Solaris Volume
Manager software, unconfgure your mediators.
See Confguring Dual-String Mediators in Oracle Solaris Cluster Software Installation Guide
for more information about mediators.
a. Runthe followingcommandtoverify that nomediator data problems exist.
phys-schost# medstat -s setname
-s setname Specifes the disk set name.
If the value in the Status feld is Bad, repair the afected mediator host. Followthe procedure
Howto Fix Bad Mediator Data in Oracle Solaris Cluster Software Installation Guide.
b. List all mediators.
Save this information for when you restore the mediators during the procedure Howto
Finish Upgrade to Oracle Solaris Cluster 3.3 Software on page 101.
BeforeYouBegin
1
2
Performing a Live Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 72
c. For a disk set that uses mediators, take ownershipof the disk set if nonode already has
ownership.

On Sun Cluster 3.1 8/05 software, use the following command:

phys-schost# scswitch -z -D setname -h node
-z Changes mastery.
-D devicegroup Specifes the name of the disk set.
-h node Specifes the name of the node to become primary of the disk set.

On Sun Cluster 3.2 or Oracle Solaris Cluster 3.3 software, use the following command:
phys-schost# cldevicegroup switch -n node devicegroup
d. Unconfgure all mediators for the disk set.
phys-schost# metaset -s setname -d -m mediator-host-list
-s setname Specifes the disk set name.
-d Deletes fromthe disk set.
-m mediator-host-list Specifes the name of the node to remove as a mediator host for the
disk set.
See the mediator(7D) man page for further information about mediator-specifc options to
the metaset command.
e. Repeat Stepc throughStepdfor eachremainingdisk set that uses mediators.
Oneachnode that uses a UFS root fle system, temporarily change the name of the global
devices entry inthe /etc/vfstab fle fromthe DIDname tothe physical name.
This name change is necessary for live upgrade software to recognize the global-devices fle
system. You will restore the DIDnames after the live upgrade is completed.
a. Back upthe /etc/vfstab fle.
phys-schost# cp /etc/vfstab /etc/vfstab.old
b. Openthe /etc/vfstab fle for editing.
c. Locate andedit the line that corresponds to/global/.device/node@N.

Change the DIDnames to the physical names by changing /dev/did/{r}dsk/dYsZ to

/dev/{r}dsk/cNtXdYsZ.

Remove global fromthe entry.

3
Performing a Live Upgrade of a Cluster
Chapter 4 Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software 73
The following example shows the names of DIDdevice d3s3, which corresponds to
/global/.devices/node@2, changed to its physical device names and the global entry
removed:
Original:
/dev/did/dsk/d3s3 /dev/did/rdsk/d3s3 /global/.devices/node@2 ufs 2 no global
Changed:
dev/dsk/c0t0d0s3 /dev/rdsk/c0t0d0s3 /global/.devices/node@2 ufs 2 no -
d. Temporarily comment out any entries for highly available local fle systems that are
managedby HAStoragePlus.
Buildaninactive boot environment (BE).
phys-schost# lucreate options-n BE-name
-n BE-name Specifes the name of the boot environment that is to be upgraded.
For information about important options to the lucreate command, see Solaris 10 10/09
Installation Guide: Solaris Live Upgrade and Upgrade Planning and the lucreate(1M) man
page.
If necessary, upgrade the Solaris OS software inyour inactive BE.
If the cluster already runs on a properly patched version of the Solaris OS that supports Oracle
Solaris Cluster 3.3 software, this step is optional.

If youuse Solaris Volume Manager software, runthe followingcommand:

phys-schost# luupgrade -u -n BE-name -s os-image-path
-u Upgrades an operating systemimage on a boot environment.
-s os-image-path Specifes the path name of a directory that contains an operating
systemimage.

If youuseVeritas Volume Manager, followlive upgrade procedures inyour Veritas Storage

Foundationinstallationdocumentationfor upgradingthe operatingsystem.
Mount your inactive BE by usingthe lumount command.
phys-schost# lumount -n BE-name -m BE-mount-point
-m BE-mount-point Specifes the mount point of BE-name.
For more information, see Solaris 10 10/09 Installation Guide: Solaris Live Upgrade and Upgrade
Planning and the lumount(1M) man page.
4
5
6
Performing a Live Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 74
Apply any necessary Solaris patches.
You might need to patch your Solaris software to use Solaris Live Upgrade. For details about the
patches that the Solaris OS requires and where to download them, see Managing Packages and
Patches With Solaris Live Upgrade in Solaris 9 9/04 Installation Guide or Upgrading a System
With Packages or Patches in Solaris 10 10/09 Installation Guide: Solaris Live Upgrade and
Upgrade Planning.
If necessary, upgrade your VxVMsoftware.
Refer to your Veritas Storage Foundation installation documentation for procedures to use the
live upgrade method.
(Optional) SPARC: UpgradeVxFS.
Followprocedures that are provided in your VxFS documentation.
If your cluster hosts software applications that require anupgrade andthat youcanupgrade by
usingSolaris Live Upgrade, upgrade those software applications.
However, if some software applications to upgrade cannot use Solaris Live Upgrade, such as
Sun QFS software, upgrade the applications in Step 26.
Loadthe installationDVD-ROMintothe DVD-ROMdrive.
If the volume management daemon vold(1M) is running and is confgured to manage
CD-ROMor DVDdevices, the daemon automatically mounts the media on the /cdrom/cdrom0
directory.
Change tothe /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools/
directory, where arch is sparc or x86andwhere ver is 10 for Oracle Solaris 10 .
phys-schost# cd /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools
Upgrade Oracle Solaris Cluster software.
phys-schost# ./scinstall -u update -R BE-mount-point
-u update Specifes that you are performing an upgrade of Oracle Solaris Cluster
software.
-R BE-mount-point Specifes the mount point for your alternate boot environment.
For more information, see the scinstall(1M) man page.
Apply Oracle Solaris Cluster patches tothe alternate BE.
Upgrade data services.
phys-schost# BE-mount-point/usr/cluster/bin/scinstall -u update -s all \
-d /cdrom/cdrom0/Solaris_arch/Product/sun_cluster_agents -R BE-mount-point
7
8
9
10
11
12
13
14
15
Performing a Live Upgrade of a Cluster
Chapter 4 Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software 75
Unloadthe installationDVD-ROMfromthe DVD-ROMdrive.
a. Toensure that the DVD-ROMis not beingused, change toa directory that does not reside on
the DVD-ROM.
b. Eject the DVD-ROM.
phys-schost# eject cdrom
Repeat all steps, startingfromStep1, oneachnode inthe cluster.
Note Do not reboot any node until all nodes in the cluster are upgraded on their inactive BE.
Oneachcluster node that uses a UFS root fle system, restore the DIDnames of the
global-devices entry inthe /etc/vfstab fle.
a. Onthe current, unupgradedBE, restore the original /etc/vfstab fle.
phys-schost# cp /etc/vstab.old /etc/vfstab
b. Inthe alternate BE, openthe /etc/vfstab fle for editing.
c. Locate the line that corresponds to/global/.devices/node@Nandreplace the dash(-) at
the endof the entry withthe wordglobal.
/dev/dsk/cNtXdYsZ /dev/rdsk/cNtXdYsZ /global/.devices/node@N ufs 2 no global
When the node is rebooted into the upgraded alternate BE, the DIDnames are substituted
in the /etc/vfstab fle automatically.
d. Uncomment the entries for highly available local fle systems that youcommentedout in
Step3.
Oneachnode, unmount the inactive BE.
phys-schost# luumount -n BE-name
Oneachnode, activate the upgradedinactive BE.
phys-schost# luactivate BE-name
BE-name The name of the alternate BE that you built in Step 4.
Shut downeachnode inthe cluster.
Note Do not use the reboot or halt command. These commands do not activate a newBE.
phys-schost# shutdown -y -g0 -i0
16
17
18
19
20
21
Performing a Live Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 76
Determine your next step.

If one of the followingconditions applies, skiptoStep24 toboot eachnode intononcluster

mode.

You upgraded fromSun Cluster 3.1 8/05 software and you want to confgure zone
clusters.

Your cluster hosts software applications that require upgrade and for which you cannot
use Solaris Live Upgrade.

(Optional) You want to change the private-network IP address range.

If youhave noadditional software toupgrade, gotoStep23 toboot eachnode intocluster

mode.
Tocomplete the upgrade, boot eachnode intocluster mode.
Ensure that all nodes in the cluster are shut down before you boot nodes into cluster mode.

OnSPARCbasedsystems, performthe followingcommand:

ok boot

Onx86 basedsystems, performthe followingcommands:

When the GRUB menu is displayed, select the appropriate Solaris entry and press Enter.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+-------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
The nodes reboot into cluster mode using the new, upgraded BE. The cluster upgrade is
completed.
Toperformadditional upgrade tasks, boot intononcluster mode.
Ensure that all nodes in the cluster are shut down before you boot nodes into noncluster mode.

OnSPARCbasedsystems, performthe followingcommand:

ok boot -x
22
23
24
Performing a Live Upgrade of a Cluster
Chapter 4 Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software 77

Onx86 basedsystems, performthe followingcommands:

a. Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry andtype e
toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+----------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based Systemby
Using GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
b. Inthe boot parameters screen, use the arrowkeys toselect the kernel entry andtype e
toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
c. Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
d. Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
Performing a Live Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 78
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
e. Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over the
systemboot. The next time you reboot the node, it will boot into cluster mode. To boot
into noncluster mode instead, performthese steps to again to add the -x option to the
kernel boot parameter command.
If the instruction says to run the init S command, shut down the systemthen change the
GRUB kernel boot command to /platform/i86pc/multiboot -sx instead.
The upgraded BE nowruns in noncluster mode.
If youupgradedfromSunCluster 3.1 8/05 software, reconfgure the private-network address
range.
Performthis step to increase or decrease the size of the IP address range that is used by the
private interconnect. The IP address range that you confgure must minimally support the
number of nodes and private networks in the cluster. See Private Network in Oracle Solaris
Cluster Software Installation Guide for more information.
If you also expect to confgure zone clusters, you specify that number in Howto Finish
Upgrade to Oracle Solaris Cluster 3.3 Software on page 101, after all nodes are back in cluster
mode.
a. Fromone node, start the clsetup utility.
When run in noncluster mode, the clsetup utility displays the Main Menu for
noncluster-mode operations.
b. Choose the menuitem, Change IPAddress Range.
The clsetup utility displays the current private-network confguration, then asks if you
would like to change this confguration.
c. Tochange either the private-network IPaddress or the IPaddress range, type yes andpress
the Returnkey.
The clsetup utility displays the default private-network IP address, 172.16.0.0, and asks if
it is okay to accept this default.
25
Performing a Live Upgrade of a Cluster
Chapter 4 Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software 79
d. Change or accept the private-network IPaddress.

Toaccept the default private-network IPaddress andproceedtochangingthe IPaddress

range, type yes andpress the Returnkey.
The clsetup utility will ask if it is okay to accept the default netmask. Skip to the next
step to enter your response.

Tochange the default private-network IPaddress, performthe followingsubsteps.

i. Type no inresponse tothe clsetup utility questionabout whether it is okay toaccept
the default address, thenpress the Returnkey.
The clsetup utility will prompt for the newprivate-network IP address.
ii. Type the newIPaddress andpress the Returnkey.
The clsetup utility displays the default netmask and then asks if it is okay to accept
the default netmask.
e. Change or accept the default private-network IPaddress netmask andrange.
The default netmask is 255.255.240.0. This default IP address range supports up to 64
nodes, up to 10 private networks, and up to 12 zone clusters in the cluster. If you choose to
change the netmask, you specify in the following substeps the number of nodes and private
networks that you expect in the cluster.
If you also expect to confgure zone clusters, you specify that number in Howto Finish
Upgrade to Oracle Solaris Cluster 3.3 Software on page 101, after all nodes are back in
cluster mode.

Toaccept the default IPaddress netmask andrange, type yes andpress the Returnkey.
Then skip to the next step.

Tochange the IPaddress netmask andrange, performthe followingsubsteps.

i. Type no inresponse tothe clsetup utility's questionabout whether it is okay to
accept the default address range, thenpress the Returnkey.
When you decline the default netmask, the clsetup utility prompts you for the
number of nodes and private networks that you expect to confgure in the cluster.
ii. Enter the number of nodes andprivate networks that youexpect toconfgure inthe
cluster.
Fromthese numbers, the clsetup utility calculates two proposed netmasks:

The frst netmask is the minimumnetmask to support the number of nodes and
private networks that you specifed.
Performing a Live Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 80

The second netmask supports twice the number of nodes and private networks
that you specifed, to accommodate possible future growth.
iii. Specify either of the calculatednetmasks, or specify a diferent netmask that
supports the expectednumber of nodes andprivate networks.
f. Type yes inresponse tothe clsetup utility's questionabout proceedingwiththe update.
g. Whenfnished, exit the clsetup utility.
Upgrade any software applications that require anupgrade andfor whichyoucannot use
Solaris Live Upgrade.
Note If an upgrade process directs you to reboot, always reboot into noncluster mode, as
described in Step 24, until all upgrades are complete.
After all nodes are upgraded, boot the nodes intocluster mode.
a. Shut downeachnode.
phys-schost# shutdown -g0 -y -i0
b. Whenall nodes are shut down, boot eachnode intocluster mode.

OnSPARCbasedsystems, performthe followingcommand:

ok boot

Onx86 basedsystems, performthe followingcommands:

When the GRUB menu is displayed, select the appropriate Solaris entry and press Enter.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+-------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
The cluster upgrade is completed.
Live Upgrade to Oracle Solaris Cluster 3.3 Software
This example shows a live upgrade of a cluster node. The example upgrades the SPARCbased
node to the Oracle Solaris 10 OS, Oracle Solaris Cluster 3.3 framework, and all Oracle Solaris
26
27
Example 41
Performing a Live Upgrade of a Cluster
Chapter 4 Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software 81
Cluster data services that support the live upgrade method. In this example, sc31u4 is the
original boot environment (BE). The newBE that is upgraded is named sc33 and uses the
mount point /sc33. The directory /net/installmachine/export/solaris10/OS_image/
contains an image of the Oracle Solaris 10 OS. The installer state fle is named sc33state.
The following commands typically produce copious output. This output is shown only where
necessary for clarity.
phys-schost# lucreate sc31u4 -m /:/dev/dsk/c0t4d0s0:ufs -n sc33
...
lucreate: Creation of Boot Environment sc33 successful.
phys-schost# luupgrade -u -n sc33 -s /net/installmachine/export/solaris10/OS_image/
The Solaris upgrade of the boot environment sc33 is complete.
Apply patches
phys-schost# lumount sc33 /sc33
Insert the installation DVD-ROM.
phys-schost# cd /cdrom/cdrom0/Solaris_sparc
phys-schost# ./installer -no -saveState sc33state
phys-schost# ./installer -nodisplay -noconsole -state sc33state -altroot /sc33
phys-schost# cd /cdrom/cdrom0/Solaris_sparc/sun_cluster/Sol_9/Tools
phys-schost# ./scinstall -u update -R /sc33
phys-schost# /sc33/usr/cluster/bin/scinstall -u update -s all \
-d /cdrom/cdrom0 -R /sc33
phys-schost# cd /
phys-schost# eject cdrom
phys-schost# luumount sc33
phys-schost# luactivate sc33
Activation of boot environment sc33 successful.
Upgrade all other nodes
Shut down all nodes
phys-schost# shutdown -y -g0 -i0
When all nodes are shut down, boot each node into cluster mode
ok boot
At this point, you might upgrade data-service applications that cannot use the live upgrade
method, before you reboot into cluster mode.
Performing a Live Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 82
DIDdevice name errors - During the creation of the inactive BE, if you receive an error that a
fle systemthat you specifed with its DIDdevice name, /dev/dsk/did/dNsX, does not exist,
but the device name does exist, you must specify the device by its physical device name. Then
change the vfstab entry on the alternate BE to use the DIDdevice name instead. Performthe
following steps:
1) For all unrecognized DIDdevices, specify the corresponding physical device names as
arguments to the -m or -M option in the lucreate command. For example, if
/global/.devices/node@nodeid is mounted on a DID device, use lucreate -m
/global/.devices/node@nodeid:/dev/dsk/cNtXdYsZ:ufs [-m] -n BE-name to create
the BE.
2) Mount the inactive BE by using the lumount -n BE-name -m BE-mount-point
command.
3) Edit the /BE-name/etc/vfstab fle to convert the physical device name,
/dev/dsk/cNtXdYsZ, to its DIDdevice name, /dev/dsk/did/dNsX.
Mount point errors - During creation of the inactive boot environment, if you receive an error
that the mount point that you supplied is not mounted, mount the mount point and rerun the
lucreate command.
NewBE boot errors - If you experience problems when you boot the newly upgraded
environment, you can revert to your original BE. For specifc information, see Failure
Recovery: Falling Back to the Original Boot Environment (Command-Line Interface) in
Solaris 9 9/04 Installation Guide or Chapter 6, Failure Recovery: Falling Back to the Original
Boot Environment (Tasks), in Solaris 10 10/09 Installation Guide: Solaris Live Upgrade and
Upgrade Planning.
Global-devices fle-systemerrors - After you upgrade a cluster on which the root disk is
encapsulated, you might see one of the following error messages on the cluster console during
the frst reboot of the upgraded BE:
mount: /dev/vx/dsk/bootdg/node@1 is already mounted or
/global/.devices/node@1 is busy
Trying to remount /global/.devices/node@1
mount: /dev/vx/dsk/bootdg/node@1 is already mounted or
/global/.devices/node@1 is busy
WARNING - Unable to mount one or more of the following filesystem(s):
/global/.devices/node@1
If this is not repaired, global devices will be unavailable.
Run mount manually (mount filesystem...).
After the problems are corrected, please clear the
maintenance flag on globaldevices by running the
following command:
/usr/sbin/svcadm clear svc:/system/cluster/globaldevices:default
Troubleshooting
Performing a Live Upgrade of a Cluster
Chapter 4 Performing a Live Upgrade to Oracle Solaris Cluster 3.3 Software 83
Dec 6 12:17:23 svc.startd[8]:
svc:/system/cluster/globaldevices:default: Method
"/usr/cluster/lib/svc/method/globaldevices start" failed with exit
status 96.
[ system/cluster/globaldevices:default misconfigured (see svcs -x for
details) ]
Dec 6 12:17:25 Cluster.CCR: /usr/cluster/bin/scgdevs: Filesystem
/global/.devices/node@1 is not available in /etc/mnttab.
Dec 6 12:17:25 Cluster.CCR: /usr/cluster/bin/scgdevs: Filesystem
/global/.devices/node@1 is not available in /etc/mnttab.
These messages indicate that the vxio minor number is the same on each cluster node. Reminor
the root disk group on each node so that each number is unique in the cluster. See Howto
Assign a NewMinor Number to a Device Group in Oracle Solaris Cluster Software Installation
Guide.
Go to Chapter 6, Completing the Upgrade.
You can choose to keep your original, and nowinactive, boot environment for as long as you
need to. When you are satisfed that your upgrade is acceptable, you can then choose to remove
the old environment or to keep and maintain it.

If you used an unmirrored volume for your inactive BE, delete the old BE fles. For specifc
information, see the appropriate procedure for your original Solaris OS version.

Deleting an Inactive Boot Environment in Solaris 9 9/04 Installation Guide.

Deleting an Inactive Boot Environment in Solaris 10 10/09 Installation Guide: Solaris

Live Upgrade and Upgrade Planning.

If you detached a plex to use as the inactive BE, reattach the plex and synchronize the
mirrors. For more information about working with a plex, see the appropriate procedure for
your original Solaris OS versions.

Example of Detaching and Upgrading One Side of a RAID1 Volume (Mirror)

(Command-Line Interface) in Solaris 9 9/04 Installation Guide.

Example of Detaching and Upgrading One Side of a RAID-1 Volume (Mirror) in

Solaris 10 10/09 Installation Guide: Solaris Live Upgrade and Upgrade Planning.
You can also maintain the inactive BE. For information about howto maintain the
environment, see the appropriate procedure for your original Solaris OS versions.

Chapter 37, Maintaining Solaris Live Upgrade Boot Environments (Tasks), in

Solaris 9 9/04 Installation Guide.

Chapter 7, Maintaining Solaris Live Upgrade Boot Environments (Tasks), in

Solaris 10 10/09 Installation Guide: Solaris Live Upgrade and Upgrade Planning.
Next Steps
See Also
Performing a Live Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 84
Performing a Rolling Upgrade
This chapter provides procedures to performa rolling upgrade of an Oracle Solaris Cluster 3.3
release to an Oracle Solaris Cluster 3.3 update release, or to upgrade Oracle Solaris 10 to an
Oracle Solaris 10 update release. In a rolling upgrade, you upgrade one cluster node at a time,
while the other cluster nodes remain in production. After all nodes are upgraded and have
rejoined the cluster, you must commit the cluster to the newsoftware version before you can
use any newfeatures.
Note Oracle Solaris Cluster software does not support rolling upgrade to Oracle Solaris 10
software froma previous marketing release such as Solaris 9 software, or to Oracle Solaris
Cluster software froma previous marketing release such as Sun Cluster 3.2 software. You can
only performand Oracle Solaris Cluster rolling upgrade of Oracle Solaris Cluster or Solaris
software to an update of the same marketing release.
To upgrade an Oracle Solaris Cluster confguration froman earlier marketing release of Solaris
software, use another upgrade method. See Choosing an Oracle Solaris Cluster Upgrade
Method on page 13 to determine the best upgrade method for your confguration.
This chapter provides the following information to upgrade an Oracle Solaris Cluster 3.3
confguration to an update release of the Oracle Solaris Cluster 3.3 software or to an update
release of the Solaris OS by using the rolling upgrade method:

Performing a Rolling Upgrade of a Cluster on page 86

5
C H A P T E R 5
85
Performinga RollingUpgrade of a Cluster
TABLE 51 Task Map: Performing a Rolling Upgrade to Oracle Solaris Cluster 3.3 Software
Task Instructions
1. Read the upgrade requirements and restrictions. Upgrade Requirements and Software Support Guidelines on
page 11
2. If a quorumserver is used, upgrade the QuorumServer
software.
Howto Upgrade QuorumServer Software on page 86
3. On one node of the cluster, move resource groups and device
groups to another cluster node, and ensure that shared data and
systemdisks are backed up. If Oracle Solaris Cluster Geographic
Edition software is installed, uninstall it. If the cluster uses
dual-string mediators for Solaris Volume Manager software,
unconfgure the mediators. Then reboot the node into noncluster
mode.
Howto Prepare a Cluster Node for a Rolling Upgrade on
page 88
4. Upgrade the Solaris OS on the cluster node, if necessary, to a
supported Solaris update release.
Howto Performa Rolling Upgrade of a Solaris Maintenance
Update on page 92
5. Upgrade the cluster node to Oracle Solaris Cluster 3.3
framework software. Optionally, upgrade data-service software. If
necessary, upgrade applications.
Howto Performa Rolling Upgrade of Oracle Solaris Cluster 3.3
Software on page 94
6. Repeat Tasks 3 through 4 on each remaining node to upgrade.
7. Use the scversions command to commit the cluster to the
upgrade.
Howto Commit the Upgraded Cluster to Oracle Solaris Cluster
3.3 Software on page 99
8. Verify successful completion of upgrade to Oracle Solaris
Cluster 3.3 software.
Howto Verify Upgrade of Oracle Solaris Cluster 3.3 Software
on page 100
9. Enable resources and bring resource groups online. Migrate
existing resources to newresource types. Upgrade to the Oracle
Solaris Cluster Geographic Edition 3.3 software, if used.
Howto Finish Upgrade to Oracle Solaris Cluster 3.3 Software
on page 101
10. (Optional) SPARC: Upgrade the Oracle Solaris Cluster
module to Sun Management Center.
SPARC: Howto Upgrade Oracle Solaris Cluster Module
Software for Sun Management Center on page 121

HowtoUpgrade QuorumServer Software

If the cluster uses a quorumserver, upgrade the QuorumServer software on the quorumserver
before you upgrade the cluster.
Performing a Rolling Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 86
Note If more than one cluster uses the quorumserver, performthese steps for each of those
clusters.
Performall steps as superuser on the cluster and on the quorumserver.
If the cluster has twonodes andthe quorumserver is the cluster's only quorumdevice,
temporarily adda secondquorumdevice.
See Adding a QuorumDevice in Oracle Solaris Cluster SystemAdministration Guide.
If you add another quorumserver as a temporary quorumdevice, the quorumserver can run
the same software version as the quorumserver that you are upgrading, or it can run the 3.3
version of QuorumServer software.
Unconfgure the quorumserver fromeachcluster that uses the quorumserver.
phys-schost# clquorum remove quorumserver
Fromthe quorumserver toupgrade, verify that the quorumserver nolonger serves any cluster.
quorumserver# clquorumserver show +
If the output shows any cluster is still served by the quorumserver, unconfgure the quorum
server fromthat cluster. Then repeat this step to confrmthat the quorumserver is no longer
confgured with any cluster.
Note If you have unconfgured the quorumserver froma cluster but the clquorumserver show
command still reports that the quorumserver is serving that cluster, the command might be
reporting stale confguration information. See Cleaning Up Stale QuorumServer Cluster
Information in Oracle Solaris Cluster SystemAdministration Guide.
Fromthe quorumserver toupgrade, halt all quorumserver instances.
quorumserver# clquorumserver stop +
Uninstall the QuorumServer software fromthe quorumserver toupgrade.
a. Navigate tothe directory where the uninstaller is located.
quorumserver# cd /var/sadm/prod/SUNWentsysver
ver The version that is installed on your system.
b. Start the uninstallationwizard.
quorumserver# ./uninstall
1
2
3
4
5
Performing a Rolling Upgrade of a Cluster
Chapter 5 Performing a Rolling Upgrade 87
c. Followinstructions onthe screentouninstall the QuorumServer software fromthe
quorum-server host computer.
After removal is fnished, you can viewany available log. See Chapter 8, Uninstalling, in
Sun Java Enterprise System5 Update 1 Installation Guide for UNIXfor additional
information about using the uninstall program.
d. (Optional) Cleanupor remove the quorumserver directories.
By default, this directory is /var/scqsd.
Install the Oracle Solaris Cluster 3.3 QuorumServer software, reconfgure the quorumserver,
andstart the quorumserver daemon.
Followthe steps in Howto Install and Confgure QuorumServer Software in Oracle Solaris
Cluster Software Installation Guide for installing the QuorumServer software.
Froma cluster node, confgure the upgradedquorumserver as a quorumdevice.
Followthe steps in Howto Confgure QuorumDevices in Oracle Solaris Cluster Software
Installation Guide.
If youconfgureda temporary quorumdevice, unconfgure it.
phys-schost# clquorum remove tempquorum

HowtoPrepare a Cluster Node for a RollingUpgrade

Performthis procedure on one node at a time. You will take the upgraded node out of the
cluster while the remaining nodes continue to function as active cluster members.
Performthe following tasks:

Ensure that the confguration meets requirements for upgrade. See Upgrade Requirements
and Software Support Guidelines on page 11.

Have available the installation media, documentation, and patches for all the software
products that you are upgrading, including the following software:

Solaris OS

Oracle Solaris Cluster 3.3 framework

Oracle Solaris Cluster 3.3 required patches

Oracle Solaris Cluster 3.3 data services (agents)

Applications that are managed by Oracle Solaris Cluster 3.3 data service agents
See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes
for the location of patches and installation instructions.
6
7
8
BeforeYouBegin
Performing a Rolling Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 88
Ensure that the cluster is functioningnormally.
a. Viewthe current status of the cluster by runningthe followingcommandfromany node.
phys-schost% cluster status
See the cluster(1CL) man page for more information.
b. Searchthe /var/adm/messages logonthe same node for unresolvederror messages or
warningmessages.
c. Check the volume-manager status.
If necessary, notify users that cluster services might be temporarily interruptedduringthe
upgrade.
Service interruption will be approximately the amount of time that your cluster normally takes
to switch services to another node.
If youare upgradingOracle Solaris Cluster 3.3 software andOracle Solaris Cluster Geographic
Editionsoftware is installed, uninstall it.
For uninstallation procedures, see the documentation for your version of Oracle Solaris Cluster
Geographic Edition software.
Become superuser ona node of the cluster.
Move all resource groups anddevice groups that are runningonthe node toupgrade.
phys-schost# clnode evacuate node-to-evacuate
See the clnode(1CL) man page for more information.
Verify that the move was completedsuccessfully.
phys-schost# cluster status -t devicegroup,resourcegroup
Ensure that the systemdisk, applications, andall data are backedup.
If youwill upgrade the Solaris OS andyour cluster uses dual-stringmediators for Solaris Volume
Manager software, unconfgure your mediators.
See Confguring Dual-String Mediators in Oracle Solaris Cluster Software Installation Guide
for more information.
a. Runthe followingcommandtoverify that nomediator data problems exist.
phys-schost# medstat -s setname
-s setname Specifes the disk set name
1
2
3
4
5
6
7
8
Performing a Rolling Upgrade of a Cluster
Chapter 5 Performing a Rolling Upgrade 89
If the value in the Status feld is Bad, repair the afected mediator host. Followthe procedure
Howto Fix Bad Mediator Data in Oracle Solaris Cluster Software Installation Guide.
b. List all mediators.
Save this information for when you restore the mediators during the procedure Howto
Commit the Upgraded Cluster to Oracle Solaris Cluster 3.3 Software on page 99.
c. For a disk set that uses mediators, take ownershipof the disk set if nonode already has
ownership.
phys-schost# cldevicegroup switch -n node devicegr
d. Unconfgure all mediators for the disk set.
phys-schost# metaset -s setname -d -m mediator-host-list
-s setname Specifes the disk-set name
-d Deletes fromthe disk set
-m mediator-host-list Specifes the name of the node to remove as a mediator host
for the disk set
See the mediator(7D) man page for further information about mediator-specifc options to
the metaset command.
e. Repeat these steps for eachremainingdisk set that uses mediators.
Shut downthe node that youwant toupgrade andboot it intononcluster mode.

OnSPARCbasedsystems, performthe followingcommands:

phys-schost# shutdown -y -g0
ok boot -x

Onx86 basedsystems, performthe followingcommands:

a. Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry andtype e
toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+-------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
9
Performing a Rolling Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 90
For more information about GRUB based booting, see Booting an x86 Based Systemby
Using GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
b. Inthe boot parameters screen, use the arrowkeys toselect the kernel entry andtype e
toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
c. Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
d. Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
e. Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over the
systemboot. The next time you reboot the node, it will boot into cluster mode. To boot
into noncluster mode instead, performthese steps to again to add the -x option to the
kernel boot parameter command.
The other nodes of the cluster continue to function as active cluster members.
Performing a Rolling Upgrade of a Cluster
Chapter 5 Performing a Rolling Upgrade 91
To upgrade the Solaris software to a Maintenance Update release, go to Howto Performa
Rolling Upgrade of a Solaris Maintenance Update on page 92.
Note The cluster must already run on, or be upgraded to, at least the minimumrequired level
of the Solaris OS to support Oracle Solaris Cluster 3.3 software. See the Oracle Solaris
Cluster 3.3 Release Notes for information about supported releases of the Solaris OS.
Otherwise, if you do not intend to upgrade the Solaris OS, go to Howto Performa Rolling
Upgrade of Oracle Solaris Cluster 3.3 Software on page 94.

HowtoPerforma RollingUpgrade of a Solaris

Maintenance Update
Performthis procedure to upgrade the Solaris OS to a supported Maintenance Update release.
Note You cannot performa rolling upgrade to upgrade a cluster fromSolaris 9 to Oracle
Solaris 10 software. Go to Choosing an Oracle Solaris Cluster Upgrade Method on page 13 to
identify the appropriate upgrade method to use.
Ensure that all steps in Howto Prepare a Cluster Node for a Rolling Upgrade on page 88 are
completed.
Temporarily comment out all entries for globally mountedfle systems inthe node's
/etc/vfstab fle.
Performthis step to prevent the Solaris upgrade fromattempting to mount the global devices.
Followthe instructions inthe Solaris maintenance update installationguide toinstall the
Maintenance Update release.
Note Do not reboot the node when prompted to reboot at the end of installation processing.
Uncomment all entries inthe /a/etc/vfstab fle for globally mountedfle systems that you
commentedout inStep1.
Install any requiredSolaris software patches andhardware-relatedpatches, anddownloadany
neededfrmware that is containedinthe hardware patches.
Next Steps
BeforeYouBegin
1
2
3
4
Performing a Rolling Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 92
Note Do not reboot the node until Step 5.
Reboot the node intononcluster mode.

OnSPARCbasedsystems, performthe followingcommands:

phys-schost# shutdown -y -g0
ok boot -x

Onx86 basedsystems, performthe followingcommands:

a. Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry andtype e
toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+-------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based Systemby
Using GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
b. Inthe boot parameters screen, use the arrowkeys toselect the kernel entry andtype e
toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
c. Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
5
Performing a Rolling Upgrade of a Cluster
Chapter 5 Performing a Rolling Upgrade 93
d. Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
e. Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over the
systemboot. The next time you reboot the node, it will boot into cluster mode. To boot
into noncluster mode instead, performthese steps to again to add the -x option to the
kernel boot parameter command.
Go to Howto Performa Rolling Upgrade of Oracle Solaris Cluster 3.3 Software on page 94.

HowtoPerforma RollingUpgrade of Oracle Solaris

Cluster 3.3Software
Performthis procedure to upgrade a node that runs Oracle Solaris Cluster 3.3 software while
the remaining cluster nodes are in cluster mode.
Note Until all nodes of the cluster are upgraded and the upgrade is committed, newfeatures
that are introduced by the newrelease might not be available.
Become superuser onthe node of the cluster.
If youupgradedthe Solaris OS but donot needtoupgrade toanOracle Solaris Cluster update
release, skiptoStep13.
Loadthe installationDVD-ROMintothe DVD-ROMdrive.
If the volume management daemon vold(1M) is running and is confgured to manage
CD-ROMor DVDdevices, the daemon automatically mounts the media on the /cdrom/cdrom0
directory.
Next Steps
1
2
3
Performing a Rolling Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 94
Change tothe /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools/
directory, where arch is sparc or x86andwhere ver is 10 for Oracle Solaris 10 .
phys-schost# cd /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools
Start the scinstall utility.
phys-schost# ./scinstall
Note Do not use the /usr/cluster/bin/scinstall command that is already installed on the
node. You must use the scinstall command that is located on the installation DVD-ROM.
The scinstall Main Menu is displayed.
Choose the menuitem, UpgradeThis Cluster Node.
*** Main Menu ***
Please select from one of the following (*) options:
1) Create a new cluster or add a cluster node
2) Configure a cluster to be JumpStarted from this install server
* 3) Manage a dual-partition upgrade
* 4) Upgrade this cluster node
* 5) Print release information for this cluster node
* ?) Help with menu options
* q) Quit
Option: 4
The Upgrade Menu is displayed.
Choose the menuitem, Upgrade Oracle Solaris Cluster Framework onThis Node.
Followthe menuprompts toupgrade the cluster framework.
Upgrade processing is fnished when the systemdisplays the message Completed Oracle
Solaris Cluster framework upgrade and prompts you to press Enter to continue.
Quit the scinstall utility.
(Optional) Upgrade data service packages.
Note For HAfor SAP Web Application Server, if you are using a J2EE engine resource or a web
application server component resource or both, you must delete the resource and recreate it
with the newweb application server component resource. Changes in the newweb application
server component resource includes integration of the J2EE functionality. For more
information, see Oracle Solaris Cluster Data Service for SAP Web Application Server Guide.
4
5
6
7
8
9
10
Performing a Rolling Upgrade of a Cluster
Chapter 5 Performing a Rolling Upgrade 95
a. Start the upgradedinteractive scinstall utility.
phys-schost# /usr/cluster/bin/scinstall
Note Do not use the scinstall utility that is on the installation media to upgrade data
service packages.
The scinstall Main Menu is displayed.
b. Choose the menuitem, UpgradeThis Cluster Node.
The Upgrade Menu is displayed.
c. Choose the menuitem, Upgrade Oracle Solaris Cluster Data Service Agents onThis Node.
d. Followthe menuprompts toupgrade Oracle Solaris Cluster data service agents that are
installedonthe node.
You can choose fromthe list of data services that are available to upgrade or choose to
upgrade all installed data services.
e. When the system displays the message Completed upgrade of Oracle Solaris Cluster
data services agents, press Return.
The Upgrade Menu is displayed.
Quit the scinstall utility.
Unloadthe installationDVD-ROMfromthe DVD-ROMdrive.
a. Toensure that the DVD-ROMis not beingused, change toa directory that does not reside on
the DVD-ROM.
b. Eject the DVD-ROM.
phys-schost# eject cdrom
If youhave HAfor NFS confguredona highly available local fle system, ensure that the
loopback fle system(LOFS) is disabled.
Note If you have non-global zones confgured, LOFS must remain enabled. For guidelines
about using LOFS and alternatives to disabling it, see Cluster File Systems in Oracle Solaris
Cluster Software Installation Guide.
11
12
13
Performing a Rolling Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 96
As of the Sun Cluster 3.2 release, LOFS is no longer disabled by default during Oracle Solaris
Cluster software installation or upgrade. To disable LOFS, ensure that the /etc/system fle
contains the following entry:
exclude:lofs
This change becomes efective at the next systemreboot.
As needed, manually upgrade any customdata services that are not suppliedonthe product
media.
Verify that eachdata-service update is installedsuccessfully.
Viewthe upgrade log fle that is referenced at the end of the upgrade output messages.
Install any Oracle Solaris Cluster 3.3 framework andsoftware patches.
See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes for
the location of patches and installation instructions.
Upgrade software applications that are installedonthe cluster.
If you want to upgrade VxVMand did not upgrade the Solaris OS, followprocedures in Veritas
Storage Foundation installation documentation to upgrade VxVMwithout upgrading the
operating system.
Note If any upgrade procedure instruct you to performa reboot, you must add the -x option to
the boot command. This option boots the cluster into noncluster mode.
Ensure that application levels are compatible with the current versions of Oracle Solaris Cluster
and Solaris software. See your application documentation for installation instructions.
Shut downthe node.
phys-schost# shutdown -g0 -y
Reboot the node intothe cluster.

On SPARCbased systems, performthe following command:

ok boot

On x86 based systems, performthe following commands:

When the GRUB menu is displayed, select the appropriate Solaris entry and press Enter.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
14
15
16
17
18
19
Performing a Rolling Upgrade of a Cluster
Chapter 5 Performing a Rolling Upgrade 97
| |
+-------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
ReturntoHowtoPrepare a Cluster Node for a RollingUpgradeonpage 88 andrepeat all
upgrade procedures onthe next node toupgrade.
Repeat this process until all nodes in the cluster are upgraded.
When all nodes in the cluster are upgraded, go to Chapter 6, Completing the Upgrade.
20
Next Steps
Performing a Rolling Upgrade of a Cluster
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 98
Completing the Upgrade
This chapter provides the following information to complete all Oracle Solaris Cluster 3.3
software upgrade methods:

Howto Commit the Upgraded Cluster to Oracle Solaris Cluster 3.3 Software on page 99

Howto Verify Upgrade of Oracle Solaris Cluster 3.3 Software on page 100

Howto Finish Upgrade to Oracle Solaris Cluster 3.3 Software on page 101
Completinga Cluster Upgrade

HowtoCommit the UpgradedCluster toOracle Solaris

Cluster 3.3Software
Ensure that all upgrade procedures are completed for all cluster nodes that you are upgrading.
Fromone node, check the upgrade status of the cluster.
phys-schost# scversions
Fromthe followingtable, performthe actionthat is listedfor the output message fromStep1.
Output Message Action
Upgrade commit is needed. Proceed to Step 3.
Upgrade commit is NOT needed. All versions match. Go to Howto Verify Upgrade of Oracle Solaris Cluster 3.3
Software on page 100.
Upgrade commit cannot be performed until all cluster
nodes are upgraded. Please run scinstall(1m) on cluster
nodes to identify older versions.
Return to the Oracle Solaris Cluster upgrade procedures that you
used and upgrade the remaining cluster nodes.
6
C H A P T E R 6
BeforeYouBegin
1
2
99
Output Message Action
Check upgrade cannot be performed until all cluster
nodes are upgraded. Please run scinstall(1m) on cluster
nodes to identify older versions.
Return to the Oracle Solaris Cluster upgrade procedures that you
used and upgrade the remaining cluster nodes.
After all nodes have rejoinedthe cluster, fromone node commit the cluster tothe upgrade.
phys-schost# scversions -c
Committing the upgrade enables the cluster to utilize all features in the newer software. New
features are available only after you performthe upgrade commitment.
Fromone node, verify that the cluster upgrade commitment has succeeded.
phys-schost# scversions
Upgrade commit is NOT needed. All versions match.
Go to Howto Verify Upgrade of Oracle Solaris Cluster 3.3 Software on page 100.

HowtoVerify Upgrade of Oracle Solaris Cluster 3.3

Software
Performthis procedure to verify that the cluster is successfully upgraded to Oracle Solaris
Cluster 3.3 software. Performall steps fromthe global zone only.

Ensure that all upgrade procedures are completed for all cluster nodes that you are
upgrading.

Ensure that all steps in Howto Commit the Upgraded Cluster to Oracle Solaris Cluster 3.3
Software on page 99 are completed successfully.
Oneachnode, become superuser.
Oneachupgradednode, viewthe installedlevels of Oracle Solaris Cluster software.
phys-schost# clnode show-rev -v
The frst line of output states which version of Oracle Solaris Cluster software the node is
running. This version should match the version that you just upgraded to.
Fromany node, verify that all upgradedcluster nodes are runningincluster mode (Online).
phys-schost# clnode status
See the clnode(1CL) man page for more information about displaying cluster status.
3
4
Next Steps
BeforeYouBegin
1
2
3
Completing a Cluster Upgrade
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 100
Verifying Upgrade to Oracle Solaris Cluster 3.3 Software
The following example shows the commands used to verify upgrade of a two-node cluster to
Oracle Solaris Cluster 3.3 software. The cluster node names are phys-schost-1 and
phys-schost-2.
phys-schost# clnode show-rev -v
3.3
...
phys-schost# clnode status
=== Cluster Nodes ===
--- Node Status ---
Node Name Status
--------- ------
phys-schost-1 Online
phys-schost-2 Online
Go to Howto Finish Upgrade to Oracle Solaris Cluster 3.3 Software on page 101.

HowtoFinishUpgrade toOracle Solaris Cluster 3.3

Software
Performthis procedure to fnish Oracle Solaris Cluster upgrade. Performall steps fromthe
global zone only.
Ensure that all steps in Howto Verify Upgrade of Oracle Solaris Cluster 3.3 Software on
page 100 are completed.
Copy the security fles for the commonagent container toall cluster nodes.
This step ensures that security fles for the common agent container are identical on all cluster
nodes and that the copied fles retain the correct fle permissions.
a. Oneachnode, stopthe SunJavaWebConsole agent.
phys-schost# /usr/sbin/smcwebserver stop
b. Oneachnode, stopthe security fle agent.
phys-schost# /usr/sbin/cacaoadm stop
c. Onone node, change tothe /etc/cacao/instances/default/ directory.
phys-schost-1# cd /etc/cacao/instances/default/
d. Create a tar fle of the /etc/cacao/instances/default/ directory.
phys-schost-1# tar cf /tmp/SECURITY.tar security
Example 61
Next Steps
BeforeYouBegin
1
Completing a Cluster Upgrade
Chapter 6 Completing the Upgrade 101
e. Copy the /tmp/SECURITY.tar fle toeachof the other cluster nodes.
f. Oneachnode towhichyoucopiedthe /tmp/SECURITY.tar fle, extract the security fles.
Any security fles that already exist in the /etc/cacao/instances/default/ directory are
overwritten.
phys-schost-2# cd /etc/cacao/instances/default/
phys-schost-2# tar xf /tmp/SECURITY.tar
g. Delete the /tmp/SECURITY.tar fle fromeachnode inthe cluster.
You must delete each copy of the tar fle to avoid security risks.
phys-schost-1# rm /tmp/SECURITY.tar
phys-schost-2# rm /tmp/SECURITY.tar
If youupgradedSolaris software, ensure that external access toRPCcommunicationandSun
JavaWebConsole is enabled.
If you did not upgrade the Solaris software, omit this step.
During Solaris OS upgrade, the default restricted network profle is applied, which disables
external access for certain network services. This includes the RPCcommunication service,
which is required for cluster communication, and the Sun Java Web Console service, which is
used by the Oracle Solaris Cluster Manager GUI.
The following steps check whether external access to these services are available and, if
necessary, restore the Solaris functionality.
a. Display the status of external access toRPCcommunicationandOracle JavaWebConsole.

For RPCcommunication, external access is available if the output of the following

command shows that the local_only property is set to false.
phys-schost# svcprop network/rpc/bind:default | grep local_only

For Oracle Java Web Console, the external access is available if the output of the
following command returns an entry for 6789, which is the port number that is used to
connect to Oracle Java Web Console.
phys-schost# netstat -a | grep 6789
If external access to both services is enabled, skip to Step 3. Otherwise, continue to Step b
b. If external access toRPCcommunicationis restricted, performthe followingcommands.
phys-schost# svccfg
svc:> select network/rpc/bind
svc:/network/rpc/bind> setprop config/local_only=false
svc:/network/rpc/bind> quit
phys-schost# svcadm refresh network/rpc/bind:default
2
Completing a Cluster Upgrade
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 102
c. If external access toOracle JavaWebConsole is restricted, performthe followingcommands.
phys-schost# svccfg
svc:> select system/webconsole
svc:/system/webconsole> setprop options/tcp_listen=true
svc:/system/webconsole> quit
phys-schost# /usr/sbin/smcwebserver restart
For more information about what services the restricted network profle restricts to local
connections, see Planning Network Security in Solaris 10 10/09 Installation Guide:
Planning for Installation and Upgrade.
d. Repeat Stepa toconfrmthat external access is restored.
Oneachnode, start the security fle agent andthenstart the SunJavaWebConsole agent.
phys-schost# /usr/sbin/cacaoadm start
phys-schost# /usr/sbin/smcwebserver start
If youupgradedany data services that are not suppliedonthe product media, register the new
resource types for those data services.
Followthe documentation that accompanies the data services.
If youupgradedHAfor SAPliveCache fromthe SunCluster 3.1 8/05 versiontothe Oracle Solaris
Cluster 3.3 version, modify the /opt/SUNWsclc/livecache/bin/lccluster confgurationfle.
a. Become superuser ona node that will host the liveCache resource.
b. Copy the new/opt/SUNWsclc/livecache/bin/lccluster fle tothe
/sapdb/LC_NAME/db/sap/ directory.
Overwrite the lccluster fle that already exists fromthe previous confguration of the data
service.
c. Confgure this /sapdb/LC_NAME/db/sap/lccluster fle as documentedinHowto
Register andConfgure Solaris Cluster HAfor SAP liveCachein Oracle Solaris Cluster Data
Service for SAP liveCache Guide.
If youupgradedthe Solaris OS andyour confgurationuses dual-stringmediators for Solaris
Volume Manager software, restore the mediator confgurations.
a. Determine whichnode has ownershipof a disk set towhichyouwill addthe mediator hosts.
phys-schost# metaset -s setname
-s setname Specifes the disk set name.
b. Onthe node that masters or will master the disk set, become superuser.
3
4
5
6
Completing a Cluster Upgrade
Chapter 6 Completing the Upgrade 103
c. If nonode has ownership, take ownershipof the disk set.
phys-schost# cldevicegroup switch -n node devicegroup
node Specifes the name of the node to become primary of the disk set.
devicegroup Specifes the name of the disk set.
d. Re-create the mediators.
phys-schost# metaset -s setname -a -m mediator-host-list
-a Adds to the disk set.
-m mediator-host-list Specifes the names of the nodes to add as mediator hosts for
the disk set.
e. Repeat these steps for eachdisk set inthe cluster that uses mediators.
If youupgradedVxVM, upgrade all disk groups.
a. Bringonline andtake ownershipof a disk grouptoupgrade.
phys-schost# cldevicegroup switch -n node devicegroup
b. Synchronize the disk group.
This step resolves any changes made to VxVMminor numbers during VxVMupgrade.
phys-schost# cldevicegroup sync devicegroup
c. Runthe followingcommandtoupgrade a disk grouptothe highest versionsupportedby
theVxVMrelease youinstalled.
phys-schost# vxdg upgrade devicegroup
See your VxVMadministration documentation for more information about upgrading disk
groups.
d. Oneachnode that is directly connectedtothe disk group, bringonline andtake ownership
of the upgradeddisk group.
phys-schost# cldevicegroup switch -n node devicegroup
This step is necessary is to update the VxVMdevice fles major number with the latest vxio
number that might have been assigned during the upgrade.
e. Repeat for eachremainingVxVMdisk groupinthe cluster.
Migrate resources tonewresource type versions.
You must migrate all resources to the Oracle Solaris Cluster 3.3 resource-type version to use the
newfeatures and bug fxes that are provided in this release.
7
8
Completing a Cluster Upgrade
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 104
Note For HAfor SAP Web Application Server, if you are using a J2EE engine resource or a web
application server component resource or both, you must delete the resource and recreate it
with the newweb application server component resource. Changes in the newweb application
server component resource includes integration of the J2EE functionality. For more
information, see Oracle Solaris Cluster Data Service for SAP Web Application Server Guide.
See Upgrading a Resource Type in Oracle Solaris Cluster Data Services Planning and
Administration Guide, which contains procedures which use the command line. Alternatively,
you can performthe same tasks by using the Resource Group menu of the clsetup utility. The
process involves performing the following tasks:

Registering the newresource type.

Migrating the eligible resource to the newversion of its resource type.

Modifying the extension properties of the resource type.

Note The Oracle Solaris Cluster 3.3 release might introduce newdefault values for some
extension properties. These changes afect the behavior of any existing resource that uses the
default values of such properties. If you require the previous default value for a resource,
modify the migrated resource to set the property to the previous default value.
If your cluster runs the HAfor SunJava SystemApplicationServer EE (HADB) data service and
youshut downthe HADBdatabase before youbegana dual-partitionupgrade, re-enable the
resource andstart the database.
phys-schost# clresource enable hadb-resource
phys-schost# hadbm start database-name
For more information, see the hadbm(1m) man page.
If youupgradedtothe Oracle Solaris 10 OS andthe Apache httpd.conf fle is locatedona
cluster fle system, ensure that the HTTPD entry inthe Apache control script still points tothat
location.
a. Viewthe HTTPD entry inthe /usr/apache/bin/apchectl fle.
The following example shows the httpd.conf fle located on the /global cluster fle system.
phys-schost# cat /usr/apache/bin/apchectl | grep HTTPD=/usr
HTTPD="/usr/apache/bin/httpd -f /global/web/conf/httpd.conf"
b. If the fle does not showthe correct HTTPD entry, update the fle.
phys-schost# vi /usr/apache/bin/apchectl
#HTTPD=/usr/apache/bin/httpd
HTTPD="/usr/apache/bin/httpd -f /global/web/conf/httpd.conf"
9
10
Completing a Cluster Upgrade
Chapter 6 Completing the Upgrade 105
If youupgradedtothe Oracle Solaris 10 OS andyouintendtoconfgure zone clusters, set the
number of expectedzone clusters.
Specify the number of zone clusters that you expect to confgure in the cluster.
phys-schost# cluster set net-props -p num_zoneclusters=N
The command calculates the number of additional private-network IP addresses that are
needed and automatically modifes the IP address range.
Fromany node, start the clsetup utility.
phys-schost# clsetup
The clsetup Main Menu is displayed.
Re-enable all disabledresources.
a. Choose the menuitem, Resource Groups.
The Resource Group Menu is displayed.
b. Choose the menuitem, Enable/Disable a Resource.
c. Choose a resource toenable andfollowthe prompts.
d. Repeat Stepc for eachdisabledresource.
e. Whenall resources are re-enabled, type q toreturntothe Resource GroupMenu.
Bringeachresource groupback online.
This step includes the bringing online of resource groups in non-global zones.
a. Choose the menuitem, Online/Ofine or Switchover a Resource Group.
b. Followthe prompts toput eachresource groupintothe managedstate andthenbringthe
resource grouponline.
Whenall resource groups are back online, exit the clsetup utility.
Type q to back out of each submenu, or press Ctrl-C.
11
12
13
14
15
Completing a Cluster Upgrade
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 106
If, before upgrade, youenabledautomatic node reboot if all monitoredshared-disk paths fail,
ensure that the feature is still enabled.
Also performthis task if you want to confgure automatic reboot for the frst time.
a. Determine whether the automatic reboot feature is enabledor disabled.
phys-schost# clnode show

If the reboot_on_path_failure property is set toenabled, nofurther actionis

necessary.

If reboot_on_path_failure property is set todisabled, proceedtothe next stepto

re-enable the property.
b. Enable the automatic reboot feature.
phys-schost# clnode set -p reboot_on_path_failure=enabled
-p Specifes the property to set
reboot_on_path_failure=enable Specifes that the node will reboot if all monitored
disk paths fail, provided that at least one of the disks
is accessible froma diferent node in the cluster.
c. Verify that automatic reboot ondisk-pathfailure is enabled.
phys-schost# clnode show
=== Cluster Nodes ===
Node Name: node
...
reboot_on_path_failure: enabled
...
(Optional) Capture the disk partitioninginformationfor future reference.
phys-schost# prtvtoc /dev/rdsk/cNtXdYsZ > flename
Store the fle in a location outside the cluster. If you make any disk confguration changes, run
this command again to capture the changed confguration. If a disk fails and needs replacement,
you can use this information to restore the disk partition confguration. For more information,
see the prtvtoc(1M) man page.
(Optional) Install or complete upgrade of Oracle Solaris Cluster Geographic Edition3.3 software.
See Chapter 4, Upgrading the Geographic Edition Software, in Oracle Solaris Cluster
Geographic Edition Installation Guide.
(Optional) Make a backupof your cluster confguration.
An archived backup of your cluster confguration facilitates easier recovery of the your cluster
confguration,
16
17
18
19
Completing a Cluster Upgrade
Chapter 6 Completing the Upgrade 107
For more information, see Howto Back Up the Cluster Confguration in Oracle Solaris
Cluster SystemAdministration Guide.
Setting the number of zone clusters fails After a cluster is upgraded fromthe Sun Cluster 3.1
8/05 release to the Oracle Solaris Cluster 3.3 release, setting the expected number of zone
clusters might fail if the expected number of nodes and private networks has not yet been set.
These settings were introduced in the Sun Cluster 3.2 release. To correct this problem, do the
following:

1) Set the number of nodes and private networks in the cluster. Followinstructions in How
to Change the Private Network Address or Address Range of an Existing Cluster in Oracle
Solaris Cluster SystemAdministration Guide. This task requires putting all cluster nodes into
noncluster mode.

2) After you set the expected number of nodes and reboot all nodes into cluster mode, rerun
Step 11 to set the expected number of zone clusters.
Resource-type migration failure - Normally, you migrate resources to a newresource type
while the resource is ofine. However, some resources need to be online for a resource-type
migration to succeed. If resource-type migration fails for this reason, error messages similar to
the following are displayed:
phys-schost - Resource depends on a SUNW.HAStoragePlus type resource that is not
online anywhere.
(C189917) VALIDATE on resource nfsrs, resource group rg, exited with
non-zero exit status.
(C720144) Validation of resource nfsrs in resource group rg on node
phys-schost failed.
If resource-type migration fails because the resource is ofine, use the clsetup utility to
re-enable the resource and then bring its related resource group online. Then repeat migration
procedures for the resource.
Java binaries location change - If the location of the Java binaries changed during the upgrade
of Oracle Solaris software, you might see error messages similar to the following when you
attempt to run the cacaoadm start or smcwebserver start commands:
phys-schost# /usr/sbin/cacaoadm start
No suitable Java runtime found. Java 1.5.0_06 or higher is required.
Jan 3 17:10:26 ppups3 cacao: No suitable Java runtime found. Java 1.5.0_06 or
higher is required.
Cannot locate all the dependencies
phys-schost# smcwebserver start
/usr/sbin/smcwebserver: /usr/jdk/jdk1.5.0_06/bin/java: not found
Troubleshooting
Completing a Cluster Upgrade
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 108
These errors are generated because the start commands cannot locate the current location of the
Java binaries. The JAVA_HOME property still points to the directory where the previous version of
Java was located, but that previous version was removed during upgrade.
To correct this problem, change the setting of JAVA_HOME in the following confguration fles to
use the current Java directory:
/etc/webconsole/console/config.properties/etc/opt/SUNWcacao/cacao.properties
If you have a SPARCbased systemand use Sun Management Center to monitor the cluster, go
to SPARC: Howto Upgrade Oracle Solaris Cluster Module Software for Sun Management
Center on page 121.
Otherwise, the cluster upgrade is complete.
Next Steps
Completing a Cluster Upgrade
Chapter 6 Completing the Upgrade 109
110
Recovering Froman Incomplete Upgrade
This chapter provides the following information to recover fromcertain kinds of incomplete
upgrades:

Howto Recover froma Failed Dual-Partition Upgrade on page 111

SPARC: Howto Recover Froma Partially Completed Dual-Partition Upgrade on page 114

x86: Howto Recover Froma Partially Completed Dual-Partition Upgrade on page 115

Recovering FromStorage Confguration Changes During Upgrade on page 117

Cluster Recovery After anIncomplete Upgrade
This section provides information to recover fromincomplete upgrades of an Oracle Solaris
Cluster confguration.

HowtoRecover froma FailedDual-PartitionUpgrade

If you experience an unrecoverable error during upgrade, performthis procedure to back out of
the upgrade.
Note You cannot restart a dual-partition upgrade after the upgrade has experienced an
unrecoverable error.
Become superuser oneachnode of the cluster.
Boot eachnode intononcluster mode.

OnSPARCbasedsystems, performthe followingcommand:

ok boot -x
7
C H A P T E R 7
1
2
111

Onx86 basedsystems, performthe followingcommands:

a. Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry andtype e
toedit its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB based booting, see Booting an x86 Based Systemby
Using GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
b. Inthe boot parameters screen, use the arrowkeys toselect the kernel entry andtype e
toedit the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
|+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
c. Add-x tothe commandtospecify that the systemboot intononcluster mode.
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ESC at any time exits. ]
grub edit> kernel /platform/i86pc/multiboot -x
d. Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.97 (639K lower / 1047488K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
Cluster Recovery After an Incomplete Upgrade
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 112
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
e. Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over the
systemboot. The next time you reboot the node, it will boot into cluster mode. To boot
into noncluster mode instead, performthese steps to again add the -x option to the
kernel boot parameter command.
Oneachnode, runthe upgrade recovery script fromthe installationmedia.
If the node successfully upgraded to Oracle Solaris Cluster 3.3 software froma 3.2 release, you
can alternatively run the scinstall command fromthe /usr/cluster/bin directory.
Note If you upgraded fromthe Sun Cluster 3.1 8/05 release, run the scinstall command only
fromthe installation media. Recovery capability for dual-partition upgrade is not available
fromthe 3.1 version of the scinstall command.
phys-schost# cd /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools
phys-schost# ./scinstall -u recover
-u Specifes upgrade.
recover Restores the /etc/vfstab fle and the Cluster Confguration Repository (CCR)
database to their original state before the start of the dual-partition upgrade.
The recovery process leaves the cluster nodes in noncluster mode. Do not attempt to reboot the
nodes into cluster mode.
For more information, see the scinstall(1M) man page.
Performeither of the followingtasks.

Restore the oldsoftware frombackuptoreturnthe cluster toits original state.

Continue toupgrade software onthe cluster by usingthe standardupgrade method.

This method requires that all cluster nodes remain in noncluster mode during the upgrade.
See the task map for standard upgrade, Table 21. You can resume the upgrade at the last
task or step in the standard upgrade procedures that you successfully completed before the
dual-partition upgrade failed.
3
4
Cluster Recovery After an Incomplete Upgrade
Chapter 7 Recovering Froman Incomplete Upgrade 113

SPARC: HowtoRecover Froma Partially Completed

Dual-PartitionUpgrade
Performthis procedure if a dual-partition upgrade fails and the state of the cluster meets all of
the following criteria:

The nodes of the frst partition are upgraded.

None of the nodes of the second partition are yet upgraded.

None of the nodes of the second partition are in cluster mode.

You can also performthis procedure if the upgrade has succeeded on the frst partition but you
want to back out of the upgrade.
Note Do not performthis procedure after dual-partition upgrade processes have begun on the
second partition. Instead, performHowto Recover froma Failed Dual-Partition Upgrade on
page 111.
Before you begin, ensure that all second-partition nodes are halted. First-partition nodes can be
either halted or running in noncluster mode.
Performall steps as superuser.
Boot eachnode inthe secondpartitionintononcluster mode.
phys-schost# ok boot -x
Fromthe DVDimage, run the scinstall -u recover command on each node in the second
partition.
Change to the /Solaris_arch/Product/sun_cluster/Solaris_ver/Tools/ directory, where
arch is sparc or x86 and where ver is 10 for Oracle Solaris 10.
phys-schost# cd /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Tools/
phys-schost# ./scinstall -u recover
The command restores the original CCRinformation, restores the original /etc/vfstab fle,
and eliminates modifcations for startup.
Boot eachnode of the secondpartitionintocluster mode.
phys-schost# shutdown -g0 -y -i6
When the nodes of the second partition come up, the second partition resumes supporting
cluster data services while running the old software with the original confguration.
Restore the original software andconfgurationdata frombackupmedia tothe nodes inthe
frst partition.
BeforeYouBegin
1
2
3
4
Cluster Recovery After an Incomplete Upgrade
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 114
Boot eachnode inthe frst partitionintocluster mode.
phys-schost# shutdown -g0 -y -i6
The nodes rejoin the cluster.

x86: HowtoRecover Froma Partially Completed

Dual-PartitionUpgrade
Performthis procedure if a dual-partition upgrade fails and the state of the cluster meets all of
the following criteria:

The nodes of the frst partition are upgraded.

None of the nodes of the second partition are yet upgraded.

None of the nodes of the second partition are in cluster mode.

You can also performthis procedures if the upgrade has succeeded on the frst partition but you
want to back out of the upgrade.
Note Do not performthis procedure after dual-partition upgrade processes have begun on the
second partition. Instead, performHowto Recover froma Failed Dual-Partition Upgrade on
page 111.
Before you begin, ensure that all second-partition nodes are halted. First-partition nodes can be
either halted or running in noncluster mode.
Performall steps as superuser.
Boot eachnode inthe secondpartitionintononcluster mode by completingthe followingsteps.
Inthe GRUBmenu, use the arrowkeys toselect the appropriate Solaris entry andtype e toedit
its commands.
The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Solaris 10 /sol_10_x86 |
| Solaris failsafe |
| |
+-------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, e to edit the
commands before booting, or c for a command-line.
For more information about GRUB-based booting, see Booting an x86 Based Systemby Using
GRUB (Task Map) in SystemAdministration Guide: Basic Administration.
5
BeforeYouBegin
1
2
Cluster Recovery After an Incomplete Upgrade
Chapter 7 Recovering Froman Incomplete Upgrade 115
Inthe boot parameters screen, use the arrowkeys toselect the kernel entry andtype e toedit
the entry.
The GRUB boot parameters screen appears similar to the following:
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.
Addthe -x optiontothe commandtospecify that the systemboot intononcluster mode.
Minimal BASH-like line editing is supported.
For the first word, TAB lists possible command completions.
Anywhere else TAB lists the possible completions of a device/filename.
ESC at any time exits.
phys-schost# grub edit> kernel /platform/i86pc/multiboot -x
Press Enter toaccept the change andreturntothe boot parameters screen.
The screen displays the edited command.
GNU GRUB version 0.95 (615K lower / 2095552K upper memory)
+----------------------------------------------------------------------+
| root (hd0,0,a) |
| kernel /platform/i86pc/multiboot -x |
| module /platform/i86pc/boot_archive |
+----------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press b to boot, e to edit the selected command in the
boot sequence, c for a command-line, o to open a new line
after (O for before) the selected line, d to remove the
selected line, or escape to go back to the main menu.-
Type b toboot the node intononcluster mode.
Note This change to the kernel boot parameter command does not persist over the system
boot. The next time you reboot the node, it will boot into cluster mode. To boot into noncluster
mode instead, performthese steps to again to add the -x option to the kernel boot parameter
command.
On each node in the second partition, run the scinstall -u recover command.
phys-schost# /usr/cluster/bin/scinstall -u recover
The command restores the original CCRinformation, restores the original /etc/vfstab fle,
and eliminates modifcations for startup.
3
4
5
6
7
Cluster Recovery After an Incomplete Upgrade
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 116
Boot eachnode of the secondpartitionintocluster mode.
phys-schost# shutdown -g0 -y -i6
When the nodes of the second partition come up, the second partition resumes supporting
cluster data services while running the old software with the original confguration.
Restore the original software andconfgurationdata frombackupmedia tothe nodes inthe
frst partition.
Boot eachnode inthe frst partitionintocluster mode.
phys-schost# shutdown -g0 -y -i6
The nodes rejoin the cluster.
RecoveringFromStorage ConfgurationChanges During
Upgrade
This section provides the following repair procedures to followif changes were inadvertently
made to the storage confguration during upgrade:

Howto Handle Storage Reconfguration During an Upgrade on page 117

Howto Resolve Mistaken Storage Changes During an Upgrade on page 118

HowtoHandle Storage ReconfgurationDuringan

Upgrade
Any changes to the storage topology, including running Oracle Solaris Cluster commands,
should be completed before you upgrade the cluster to Solaris 9 or Oracle Solaris 10 software. If,
however, changes were made to the storage topology during the upgrade, performthe following
procedure. This procedure ensures that the newstorage confguration is correct and that
existing storage that was not reconfgured is not mistakenly altered.
Ensure that the storage topology is correct. Check whether the devices that were fagged as
possibly being replaced map to devices that actually were replaced. If the devices were not
replaced, check for and correct possible accidental confguration changes, such as incorrect
cabling.
Ona node that is attachedtothe unverifeddevice, become superuser.
Manually update the unverifeddevice.
phys-schost# cldevice repair device
8
9
10
BeforeYouBegin
1
2
Recovering FromStorage Confguration Changes During Upgrade
Chapter 7 Recovering Froman Incomplete Upgrade 117
See the cldevice(1CL) man page for more information.
Update the DIDdriver.
phys-schost# scdidadm -ui
phys-schost# scdidadm -r
-u Loads the device-IDconfguration table into the kernel.
-i Initializes the DIDdriver.
-r Reconfgures the database.
Repeat Step2 throughStep3 onall other nodes that are attachedtothe unverifeddevice.
Return to the remaining upgrade tasks. Go to Step 4 in Howto Upgrade Oracle Solaris Cluster
3.3 Software (Standard) on page 32.

HowtoResolve MistakenStorage Changes Duringan

Upgrade
If accidental changes are made to the storage cabling during the upgrade, performthe following
procedure to return the storage confguration to the correct state.
Note This procedure assumes that no physical storage was actually changed. If physical or
logical storage devices were changed or replaced, instead followthe procedures in Howto
Handle Storage Reconfguration During an Upgrade on page 117.
Return the storage topology to its original confguration. Check the confguration of the devices
that were fagged as possibly being replaced, including the cabling.
Oneachnode of the cluster, become superuser.
Update the DIDdriver oneachnode of the cluster.
phys-schost# scdidadm -ui
phys-schost# scdidadm -r
-u Loads the deviceIDconfguration table into the kernel.
-i Initializes the DIDdriver.
-r Reconfgures the database.
See the scdidadm(1M) man page for more information.
3
4
Next Steps
BeforeYouBegin
1
2
Recovering FromStorage Confguration Changes During Upgrade
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 118
If the scdidadm commandreturnedany error messages inStep2, make further modifcations as
neededtocorrect the storage confguration, thenrepeat Step2.
Return to the remaining upgrade tasks. Go to Step 4 in Howto Upgrade Oracle Solaris Cluster
3.3 Software (Standard) on page 32.
3
Next Steps
Recovering FromStorage Confguration Changes During Upgrade
Chapter 7 Recovering Froman Incomplete Upgrade 119
120
SPARC: Upgrading Sun Management Center
Software
This chapter provides the following procedures to upgrade the Oracle Solaris Cluster module
for Sun Management Center:

SPARC: Howto Upgrade Oracle Solaris Cluster Module Software for Sun Management
Center on page 121

SPARC: Howto Upgrade Sun Management Center Software on page 123

Upgradingthe Oracle Solaris Cluster Module for Sun
Management Center
This section provides procedures to upgrade the Oracle Solaris Cluster module for Sun
Management Center software.

SPARC: HowtoUpgrade Oracle Solaris Cluster Module

Software for SunManagement Center
Performthe following steps to upgrade Oracle Solaris Cluster module software on the Sun
Management Center server machine and console machine.
Note To upgrade the Sun Management Center software itself, do not performthis procedure.
Instead, go to SPARC: Howto Upgrade Sun Management Center Software on page 123 to
upgrade the Sun Management Center software and the Oracle Solaris Cluster module.
Have available the installation DVD-ROMfor the SPARCplatformor the path to the
DVD-ROMimage.
8
C H A P T E R 8
BeforeYouBegin
121
As superuser, remove the existingOracle Solaris Cluster module packages fromeachmachine.
Use the pkgrm(1M) command to remove all Oracle Solaris Cluster module packages fromall
locations that are listed in the following table.
Location Module Package toRemove
Sun Management Center console machine SUNWscscn
Sun Management Center server machine SUNWscssv, SUNWscshl
machine# pkgrm module-package
Note Oracle Solaris Cluster module software on the cluster nodes was already upgraded during
the cluster-framework upgrade.
As superuser, reinstall Oracle Solaris Cluster module packages oneachmachine.
a. Insert the installationDVD-ROMfor the SPARCplatformintothe DVD-ROMdrive of the
machine.
b. Change tothe Solaris_sparc/Product/sun_cluster/Solaris_ver/Packages/ directory,
where ver is 10 for Oracle Solaris 10.
machine# cd Solaris_sparc/Product/sun_cluster/Solaris_ver/Packages/
c. Install the SUNWscssv package onthe server machine.
Note that you do not upgrade to a newSUNWscscn package on the console machine or a new
SUNWscshl package on the server machine.
machine# pkgadd -d . SUNWscssv
d. Unloadthe installationDVD-ROMfromthe DVD-ROMdrive.
i. Toensure that the DVD-ROMis not beingused, change toa directory that does not reside
onthe DVD-ROM.
ii. Eject the DVD-ROM.
machine# eject cdrom
1
2
Upgrading the Oracle Solaris Cluster Module for Sun Management Center
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 122
UpgradingSunManagement Center Software
This section provides procedures to upgrade Sun Management Center software.

SPARC: HowtoUpgrade SunManagement Center

Software
Performthe following steps to upgrade fromSun Management Center 3.6 or 3.6.1 software to
Sun Management Center 3.6.1 or 4.0 software.
Have available the following items:

Oracle Solaris Cluster installation DVD-ROMfor the SPARCplatformand, if applicable, for

the x86 platform, or the paths to the DVD-ROMimages. You use the DVD-ROMto reinstall
the Oracle Solaris Cluster 3.3 version of the Oracle Solaris Cluster module packages after
you upgrade Sun Management Center software.
Note The agent packages to install on the cluster nodes are available for both SPARCbased
systems and x86 based systems. The package for the server machine is available for SPARC
based systems only.

Sun Management Center documentation.

Sun Management Center patches and Oracle Solaris Cluster module patches, if any.
See Patches and Required Firmware Levels in the Oracle Solaris Cluster 3.3 Release Notes
for the location of patches and installation instructions.
Stopany SunManagement Center processes.
a. If the SunManagement Center console is running, exit the console.
In the console window, choose FileExit.
b. OneachSunManagement Center agent machine (cluster node), stopthe SunManagement
Center agent process.
phys-schost# /opt/SUNWsymon/sbin/es-stop -a
c. Onthe SunManagement Center server machine, stopthe SunManagement Center server
process.
server# /opt/SUNWsymon/sbin/es-stop -S
BeforeYouBegin
1
Upgrading Sun Management Center Software
Chapter 8 SPARC: Upgrading Sun Management Center Software 123
As superuser, remove Oracle Solaris Clustermodule packages.
Use the pkgrm(1M) command to remove all Oracle Solaris Cluster module packages fromall
locations that are listed in the following table.
Location Module Package toRemove
Each cluster node SUNWscsam, SUNWscsal
Sun Management Center console machine SUNWscscn
Sun Management Center server machine SUNWscssv, SUNWscshl
machine# pkgrm module-package
If you do not remove the listed packages, the Sun Management Center software upgrade might
fail because of package dependency problems. You reinstall these packages in Step 4, after you
upgrade Sun Management Center software.
Upgrade the SunManagement Center software.
Followthe upgrade procedures in your Sun Management Center documentation.
As superuser, reinstall Oracle Solaris Cluster module packages fromthe installationDVD-ROMto
the locations that are listedinthe followingtable.
Location Module Package toInstall
Each cluster node SUNWscsam, SUNWscsal
Sun Management Center server machine SUNWscssv
a. Insert the installationDVD-ROMfor the appropriate platforminthe DVD-ROMdrive of the
machine.
b. Change tothe /Solaris_arch/Product/sun_cluster/Solaris_ver/Packages/ directory,
where arch is sparc or x86, andver is 10 for Oracle Solaris 10.
machine# cd /cdrom/cdrom0/Solaris_arch/Product/sun_cluster/Solaris_ver/Packages/
Note The agent packages to install on the cluster nodes are available for both SPARCbased
systems and x86 based systems. The package for the server machine is available for SPARC
based systems only.
2
3
4
Upgrading Sun Management Center Software
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 124
c. Install the appropriate module package onthe machine.

For cluster nodes, use the followingcommand:

phys-schost# pkgadd -G -d .
module-package
The -G option adds packages to the current zone only. You must add these packages only
to the global zone. Therefore, this option also specifes that the packages are not
propagated to any existing non-global zone or to any non-global zone that is created
later.

SPARC: For the server machine, use the followingcommand:

phys-schost# pkgadd -d . module-package
Apply any SunManagement Center patches andany Oracle Solaris Cluster module patches to
eachnode of the cluster.
Restart SunManagement Center agent, server, andconsole processes.
Followprocedures in Howto Start Sun Management Center in Oracle Solaris Cluster Software
Installation Guide.
Loadthe Oracle Solaris Cluster module.
Followprocedures in Howto Load the Oracle Solaris Cluster Module in Oracle Solaris Cluster
Software Installation Guide.
If the Oracle Solaris Cluster module was previously loaded, unload the module and then reload
it to clear all cached alarmdefnitions on the server. To unload the module, choose Unload
Module fromthe Module menu on the console's Details window.
5
6
7
Upgrading Sun Management Center Software
Chapter 8 SPARC: Upgrading Sun Management Center Software 125
126
Index
A
adapters
local MACaddress changes during upgrade, 33, 57
alternate boot environment, disk space
requirements, 15
Apache
disabling run control scripts, 30
modifying scripts during upgrade, 27, 51
B
boot environment
creating for live upgrade, 74
disk space requirements, 15
booting
into cluster mode, 37
into noncluster mode, 48, 52
into single-user noncluster mode, 29
C
cldevice command, migrating device IDs after
upgrade, 117
clnode command
moving resource groups and device groups, 89
verifying cluster mode, 100
verifying device-IDmigration, 100
clresource command
disabling resources, 23
listing resources, 22
clresource command (Continued)
taking resource groups ofine, 22
clsetup, setting the private-network IP address
range, 35
cluster mode
booting into, 37
verifying, 100
committing the upgrade, 99100
D
data services
upgrading
HAfor SAP liveCache, 103
HAfor SAP Web Application Server, 34, 57, 95,
105
rolling upgrade, 9498
device groups
moving, 89
verifying evacuation, 89
device-IDnames, migrating after upgrade, 117
DIDdriver, updating, 118
disabling
Apache, 30
LOFS after upgrade, 34, 58, 96
resources, 21, 23
dual-partition upgrade, 3965
Oracle Solaris Cluster software, 5665
preparing the cluster, 4250
recovering fromunrecoverable error, 111113
requirements, 1415
127
dual-partition upgrade (Continued)
Solaris software, 50
troubleshooting, 65
dual-string mediators
restoring after upgrade, 103
unconfguring during upgrade, 24, 89
E
error messages, cluster, 9
/etc/inet/ntp.conf fle
changes during upgrade, 33, 57
/etc/system fle
LOFS setting, 34, 58, 96
/etc/vfstab fle
modifying during upgrade
dual-partition upgrade, 51
rolling upgrade, 92
standard upgrade, 28
evacuating, See moving
examples
live upgrade, 8182
live upgrade verifcation, 101
G
Geographic Edition
uninstalling, 21, 44, 71, 89
upgrading, 107
global devices, node@nodeid fle system, 12
H
HAfor SAP liveCache, upgrading, 103
HAfor SAP Web Application Server
upgrading, 34, 57, 95, 105
HAfor Sun Java SystemApplication Server EE (HADB)
disabling before upgrade, 14
re-enabling after dual-partition upgrade, 105
HADB
disabling before upgrade, 14
re-enabling after dual-partition upgrade, 105
help, 89
I
inactive boot environment, 74
L
live upgrade, 6784
Oracle Solaris Cluster software, 7184
preparing the cluster, 7071
requirements, 15
Solaris software, 7184
troubleshooting, 83
upgrading applications that do not support Live
Upgrade, 81
local MACaddress
changes during upgrade, 33, 57
LOFS
disabling after upgrade, 34, 58, 96
loopback fle system(LOFS)
disabling after upgrade, 34, 58, 96
M
messages fles, cluster, 9
moving resource groups and device groups, 89
N
noncluster mode
booting into, 48, 52
booting into single-user, 29
nonrolling upgrade
See standard upgrade
ntp.conf fle
changes during upgrade, 33, 57
Index
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 128
O
Oracle Java Web Console, restoring external
access, 103
Oracle Solaris Cluster Geographic Edition, See
Geographic Edition
Oracle Solaris Cluster module to Sun Management
Center, upgrade, 121122
P
partitions, /global/.devices/node@nodeid fle
system, 12
private-network IP address range
changing to support zone clusters, 106
setting after upgrade, 35
Q
quorumserver, upgrading, 18
R
rebooting
into noncluster mode, 52
into single-user noncluster mode, 29
recovering
See also repairing
See also troubleshooting
dual-partition upgrade unrecoverable
error, 111113
storage confguration changes during
upgrade, 117119
repairing
See also recovering
See also troubleshooting
storage reconfguration during upgrade, 117119
requirements
all upgrade methods, 1112
dual-partition upgrade, 1415
live upgrade, 15
rolling upgrade, 16
standard upgrade, 14
resource groups
making unmanaged, 23
moving, 89
verifying evacuation, 89
resource types, registering after upgrade, 103
resources
disabling, 21, 23
listing, 22
rolling upgrade, 8598
Oracle Solaris Cluster software, 9498
preparing the cluster, 8892
requirements, 16
Solaris software, 9294
unconfguring mediators, 89
RPCservice, restoring external access, 102
S
scinstall command
upgrading Oracle Solaris Cluster software, 33, 56,
95
verifying Oracle Solaris Cluster software, 100
scrgadm command, listing resources, 22
scswitch command
disabling resources, 23
making resource groups unmanaged, 23
moving resource groups and device groups, 89
taking resource groups ofine, 22
scversions command, 99100
security fles, distributing upgraded fles, 101
single-user noncluster mode, booting into, 29
Solaris software
upgrading
dual-partition upgrade, 50
live upgrade, 7184
rolling upgrade, 9294
standard upgrade, 27
verifying device-IDmigration, 100
Solaris zones, upgrading a cluster with non-global
zones, 72
standard upgrade, 1738
Oracle Solaris Cluster software, 3238
preparing the cluster, 2027
requirements, 14
Index
129
standard upgrade (Continued)
Solaris software, 27
volume managers, 27
Sun Cluster Geographic Edition, See Geographic
Edition
Sun Management Center
Oracle Solaris Cluster module
upgrading, 121122
stopping, 123
upgrading, 123125
T
technical support, 89
troubleshooting
See also recovering
See also repairing
dual-partition upgrade, 65
live upgrade, 83
recovering fromunrecoverable error during
dual-partition upgrade, 111113
storage confguration changes during
upgrade, 117119
U
uninstalling
Geographic Edition, 21, 44, 71, 89
upgrading
choosing an upgrade method, 1316
committing the cluster, 99100
dual-partition upgrade, 3965
Oracle Solaris Cluster software, 5665
preparing the cluster, 4250
Solaris software, 50
troubleshooting, 65
Geographic Edition, 107
HAfor SAP liveCache, 103
HAfor SAP Web Application Server, 34, 57, 95, 105
HAfor Sun Java SystemApplication Server EE
(HADB), 105
live upgrade, 6784
Oracle Solaris Cluster software, 7184
upgrading, live upgrade (Continued)
preparing the cluster, 7071
Solaris software, 7184
troubleshooting, 83
upgrading applications that do not support Live
Upgrade, 81
volume managers, 7184
non-global zones, 72
nonrolling upgrade
See standard upgrade
Oracle Solaris Cluster module to Sun Management
Center, 121122
quorumserver software, 18
recovering
storage confguration changes, 117119
unrecoverable error during dual-partition
upgrade, 111113
requirements, 1116
resource types, 103
restoring mediators, 103
rolling
preparing the cluster, 8892
rolling upgrade, 8598
data services, 9498
Oracle Solaris Cluster software, 9498
Solaris software, 9294
unconfguring mediators, 89
standard upgrade, 1738
Oracle Solaris Cluster software, 3238
preparing the cluster, 2027
Solaris software, 27
volume managers, 27
Sun Management Center, 123125
troubleshooting
dual-partition upgrade, 65
live upgrade, 83
recovering fromunrecoverable error during
dual-partition upgrade, 111113
storage confguration changes, 117119
unconfguring mediators, 24
verifying
device-IDconversion, 100
Oracle Solaris Cluster, 100101
Index
Oracle Solaris Cluster Upgrade Guide September 2010, Revision 10 130
upgrading (Continued)
volume manager software
dual-partition upgrade, 50
live upgrade, 7184
standard upgrade, 27
VxVMdisk groups, 104
/usr/cluster/bin/cldevice command, migrating
device IDs after upgrade, 117
/usr/cluster/bin/clnode command
moving resource groups and device groups, 89
verifying cluster mode, 100
verifying device-IDmigration, 100
/usr/cluster/bin/clresource, taking resource
groups ofine, 22
/usr/cluster/bin/clresource command
disabling resources, 23
listing resources, 22
/usr/cluster/bin/scinstall command, verifying
Oracle Solaris Cluster software, 100
/usr/cluster/bin/scrgadm command, listing
resources, 22
/usr/cluster/bin/scswitch command
disabling resources, 23
making resource groups unmanaged, 23
moving resource groups and device groups, 89
taking resource groups ofine, 22
/usr/cluster/bin/scversions command, 99100
V
/var/adm/messages fle, 9
verifying
device group confgurations, 89
device-IDmigration, 100
Oracle Solaris Cluster software version, 100
resource group confgurations, 89
Veritas File System(VxFS)
upgrading
dual-partition upgrade, 50
live upgrade, 7184
standard upgrade, 27
Veritas Volume Manager (VxVM)
upgrading
dual-partition upgrade, 50
Veritas Volume Manager (VxVM), upgrading
(Continued)
live upgrade, 7184
standard upgrade, 27
upgrading disk groups, 104
vfstab fle
modifying during upgrade
dual-partition upgrade, 51
rolling upgrade, 92
standard upgrade, 28
volume managers
upgrading
dual-partition upgrade, 50
live upgrade, 7184
standard upgrade, 27
Z
zone clusters, changing the private-network IP address
range, 106
Index
131
132