Rac10gR2OnWindows

1. *Introduction
1.0.1. Software required for install
Microsoft Windows 2003
For Microsoft Windows 2003 – Service Pack 2 is strongly recommended, do not run with SP1
Base Oracle 10gR2 DVD
The latest patchset (currently 10.2.0.4 available as patch 6810189 for x64)
The latest mini patch (currently PATCH 34 available as patch 9393550 for x64) - These patches are
cumulative - The latest mini patch will have all previous mini patch components with it. Use My
Oracle Support ExtNote:373040.1 : 'How To Identify and Locate the Latest Windows MiniPatch For
Your Patchset Version'.

The latest Oracle Cluster verify Utility (CVU) from here :
http://www.oracle.com/technology/products/database/clustering/cvu/cvu_download_homepage.html

Microsoft Windows 2008
For Microsoft Windows 2008 – Service Pack 2 is strongly recommended, do not run with SP1
Oracle Database 10g Release 2 (10.2.0.4.0) Enterprise/Standard Edition for Microsoft Windows Vista
x64 and Microsoft Windows Server 2008 x64

The latest mini patch (currently PATCH 34 available as patch 9393550 for x64) - These patches are
cumulative - The latest mini patch will have all previous mini patch components with it. Use My
Oracle Support ExtNote:373040.1 : 'How To Identify and Locate the Latest Windows MiniPatch For
Your Patchset Version'.

The latest Oracle Cluster verify Utility (CVU) from here :
http://www.oracle.com/technology/products/database/clustering/cvu/cvu_download_homepage.html

Do not mix Microsoft Operating Systems between nodes in a cluster. e.g Windows 2003 X64 on one node
and
Windows 2003 64bit Itanium IA64 on another
1.0.2. Processor Model
This paper covers both 32bit and 64bit install. Please note that The OS Version of Oracle you install must
match the OS Chip Version.
Microsoft Windows 2003
Windows 2003 32bit - you must install the 32bit versions of the Oracle software.
Windows 2003 64bit X64 - you must install the 64bit X64 versions of the Oracle software.
Windows 2003 64bit Itanium IA64 - you must install the 64bit Itanium IA64 versions of the
Oracle software.


Microsoft Windows 2008
Windows 2008 32bit - you must install the 32bit versions of the Oracle software.

Rac10gR2OnWindows 1
Windows 2008 64bit X64 - you must install the 64bit X64 versions of the Oracle software.
1.1. *Installation steps
To summarise the install steps are:
Preparation
Prerequisites to make sure the cluster is setup OK.
Stage all the software on one node, typically Node1

Establish Oracle Clusterware
Install the Oracle Clusterware (using the push mechanism to install on the other nodes in the
cluster)

Patch the Clusterware to the latest patchset

Establish ASM
Install an Oracle Software Home for ASM
Patch the ASM Home to the latest patchset
Create the Listeners
Create the ASM Instances and initial Disk Group

Establish RAC Database
Install an Oracle Software Home for RAC Database
Patch the RAC Database Home to the latest patchset
Create the RAC Database Instances

1.2. *Schematic
The following is a schematic of the software & hardware layout of a 2-node RAC cluster. As explained in
this
document the actual number of LUN’s required will vary depending on your mirroring requirements.
1.2.1. Hardware/software configuration BEFORE Oracle software install
1.0.2. Processor Model 2
1.2.2. Hardware/software configuration AFTER Oracle software install
Notice that there are a number of LUNs presented to both machines from the shared storage.
5 of 270MB LUNs
2 are used for the OCR devices
3 are used for the vote devices

9 of 1GB LUNs
6 are used for the +DATA ASM diskgroup
3 are used for the +FLASH ASM diskgroup

Also notice there is a dedicated switch on the cluster interconnect.
1.3. *Installation Method
This document details one method of installing a 2-node Oracle 10gR2 RAC cluster on Windows:
NTFS_RAW_ASM: The Oracle Cluster, ASM & Oracle Database Homes are established on the
local disk of each of the RAC nodes. The files required by Oracle Clusterware are on RAW partitions.
The database data-files are on ASM. The local Homes for Oracle Clusterware, ASM and RAC take up
a total of 6.5GB on each nodes' local disk filesystem.

It should be noted that there are 2 other popular installation configurations. This document does not cover
these methods:
NTFS_RAW_RAW: The Oracle Cluster, ASM & Oracle Database Homes are established on the
local disk of each of the RAC nodes. The files required by Oracle Clusterware are on RAW partitions.
The database data-files are also on RAW.

NTFS_OCFS_OCFS: The Oracle Cluster, ASM & Oracle Database Homes are established on the
local disk of each of the RAC nodes. The files required by Oracle Clusterware are on OCFS. The
database data-files are also on OCFS.

There are other possible combinations although Oracle recommends one of the above methods.
1.2.2. Hardware/software configuration AFTER Oracle software install 3
2. Prepare the cluster nodes for Oracle RAC
This page includes hidden sections, use the and image for each section to show/hide the section
or you can 'Show' or 'Hide' all by selecting here : Show all Hide all
This installation routine presumes that you have a 2-node Windows cluster. There are a number of items
that
require checking before the install commences. Getting this bit right will enhance your install experience.
1. User Accounts
2. Installing using Remote desktop client
3. Environment Variables
4. Networking
5. Remote Registry Connect
6. Stopping Services that might impact the installation process
7. Time sync
8. Stage the Oracle Software
9. Run CVU
It is essential that these items are checked and are configured correctly before the install commences.
2.1. User Accounts
Run the installation as the local Administrator account. The local admin username and password must be
the
same on both nodes. If you must use a domain account, then you must ensure that the domain user is
explicitly declared as a member of the local Administrators (not granted membership to this group
through
another group) on each node in the cluster.
Not using an administrator account that is explicitly declared by username in the Local administrators
group
will cause the install to fail
Show the User Account setup information Hide the User Account setup information
2. Prepare the cluster nodes for Oracle RAC 4
For Microsoft Windows 2008, you will need to change the elevation prompt behavior for administrators
to
"Elevate without prompting" to allow for user equivalence to function properly:
Open a command prompt and type â–.secpol.msc▝ to launch the Security Policy Console
management utility.
1.
2. From the Local Security Settings console tree, click Local Policies, and then Security Options
Scroll down to and double-click User Account Control: Behavior of the elevation prompt for
administrators.
3.
From the drop-down menu, select: Elevate without prompting (tasks requesting elevation will
automatically run as elevated without prompting the administrator)
4.
5. Click OK to confirm the changes.
6. Repeat steps 1-5 on ALL cluster nodes.
You also must ensure that the Administrators group is listed under â–.Manage auditing and security
log▝:
Open a command prompt and type â–.secpol.msc▝ to launch the Security Policy Console
management utility.
1.
2. Click on â–.Local Policies▝
3. Click on â–.User Rights Assignment▝
Locate and double click the â–.Manage auditing and security logâ–  in the listing of User Rights
Assignments.
4.
5. If the Administrators group is NOT listed in the â–.Local Security Settingsâ–  tab, add the group now.
6. Click OK to save the changes (if changes were made).
7. Repeat steps 1-6 on ALL cluster nodes.
2.1. User Accounts 5
2.2. Installing Using Windows Remote Desktop Client
For Microsoft Windows 2003, if you must use the MS Terminal Services Client it is essential that you
invoke
the mstsc.exe application with the /console parameter which connects to the console session of the
specified
Windows Server. From the Start-> run menu you must type mstsc /v:nodename /console For
more
information go here :
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/ts_cmd_mstsc.mspx?
mfr=true
For Microsoft Windows 2008, the /admin parameter should be used instead of the /console parameter
with
MS Terminal Services Client. From the Start-> run menu you must type mstsc /v:nodename
/admin
For more information go here: http://support.microsoft.com/kb/947723
2.3. Environment Variables
Make sure that both the TEMP and TMP environment variables for both the install user and the
LocalSystem
account are set to point at directories that do not contain spaces or other special characters. e.g.
TEMP=C:\WINDOWS\TEMP
TMP=C:\WINDOWS\TEMP
and that the directory actually exists. You should check these values are the same on all nodes in the
cluster.
The installer assumes that the local user and system user TMP & TEMP environment variables are the
same.
It copies temporary binaries to remote nodes at that path. If they are not the same on all nodes the install
will
fail.
Show the Enviroment Variable setup information Hide the Enviroment Variable setup information
2.2. Installing Using Windows Remote Desktop Client 6
2.4. Networking
You will need a minimum of 2 network adapters per machine, 1 adapter is the public interface. The other
adapter, the private interface, is used for Oracle RAC node membership and inter-instance traffic. The
private
interface adapters should be connected to each other via a network switch which is used exclusively for
the
interconnect traffic. In 2 node environments do not be tempted to use a 'Crossover Cable' as these are not
supported. Recommended best practice is to use more than 2 adapters. This is known as network teaming
and
is configured before installation of the Oracle software. In basic outline, you take 2 of the adapters on the
private interface and team them together to create what looks like a single, highly available adapter for
the
interconnect traffic. You can then take the remaining 3rd adapter and use it for the public interface. You
can
also team the public adapters as well. In that case you would need to start with 4 adapters, 2 public & 2
private. Ensure you team the adapters before starting installation. Please be aware that even with a 2-node
cluster you must use a dedicated network switch on the interconnect.
You will need a total of 3 IP addresses per node:
The public IP address, which should be recorded in hosts file on each node and, if available, DNS.
This IP Address should be bound to the public adapter before starting the install. It should be a static,
not DHCP, address

The private IP address, which should be from a different subnet than the public IP address. This
address does not require registering in DNS but you should place an entry in the hosts file on each
node. This IP Address should be bound to the private adapter before starting the install. It should be a
static, not DHCP, address

A VIP address, which should be from the same subnet as the public IP address and should be
recorded in DNS and the hosts file on each node. This IP Address should NOT be bound to the public
adapter before starting the install. Oracle Clusterware is responsible for binding this address. It should
be a static, not DHCP, address

If you do not have a DNS server then make sure both the public and the VIP addresses are entered into all
the
relevant hosts files that are normally found in the %SystemRoot
%\system32\drivers\etc\hosts
file.
Show Networking setup information Hide Networking setup information
You should use ipconfig on each node to confirm the correct IP addresses are bound to the correct
adapters
C:\Documents and Settings\Administrator>ipconfig
Windows IP Configuration
Ethernet adapter PublicLAN:
Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 10.10.10.11
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
Ethernet adapter PrivateLAN:
Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 192.168.10.11
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
A sample hosts file
127.0.0.1 localhost
10.10.10.11 winnode1 # Also in DNS if available
10.10.10.21 winnode2 # Also in DNS if available
10.10.10.51 winnode1-vip # Also in DNS if available
2.4. Networking 7
10.10.10.61 winnode2-vip # Also in DNS if available
192.168.10.11 winnode1-priv
192.168.10.21 winnode2-priv
It is important to confirm that networking is configured correctly before commencing install.
2.4.1. Network Ping Tests
There are a series of 'ping' tests that should be completed, and then the network adapter binding order
should
be checked. You should ensure that the public IP addresses resolve correctly and that the private
addresses are
of the form 'nodename-priv' and resolve on both nodes via the hosts file.
Public Ping test
Pinging Node1 from Node1 should return Node1's public IP address Pinging Node2 from Node1 should
return
Node2's public IP address Pinging Node1 from Node2 should return Node1's public IP address Pinging
Node2
from Node2 should return Node2's public IP address
Private Ping test
Pinging Node1 private from Node1 should return Node1's private IP address Pinging Node2 private from
Node1 should return Node2's private IP address Pinging Node1 private from Node2 should return
Node1's
private IP address Pinging Node2 private from Node2 should return Node2's private IP address
VIP Ping test
Pinging the VIP address at this point should fail. VIPs will be activated at the end of the Oracle
Clusterware
install.
If any of the above tests fail you should fix name/address resolution by updating the DNS or local hosts
files
on each node before continuing with the installation.
2.4.2. Network Adapter configuration
If your network adapters allow configuration you should make sure that they are configured for Full
Duplex
and at the 'fastest' speed consistent among nodes. They should not be left to 'auto-negotiate'.
2.4.3. Network Adapter naming and binding order
You must check the network adapter binding order. The public adapter must be listed first. Windows will
normally name the adapters "Local Area Connection" and "Local Area Connection 2". Take the
opportunity to
rename the adapters on each node at this point. It is important the names used for each class of adapter be
consistent across all nodes.
For Microsoft Windows 2003: In the Control Panel double click on the Network (or Network & Dialup)
Connections icon.
For Microsoft Windows 2008: From the Start-> run menu you must type ncpa.cpl, and then click OK
2.4.1. Network Ping Tests 8
As an example, rename the public adapter to PublicLAN and the private adapter to PrivateLAN. It is
important that you use the same names on all nodes. If there are other adapters take the opportunity to
rename
them as well taking care to avoid overly long adapter names and use of any special characters (spaces,
periods, dashes etc) in the adapter names.
2.4.3. Network Adapter naming and binding order 9
You must confirm the network adapter binding order on all the nodes in the cluster. Select the menu item
'Advanced->Advanced settings...'
2.4.3. Network Adapter naming and binding order 10
On the Adapters and Bindings TAB you must ensure that the public adapter is listed first and the private
adapter is listed second. If they are not in this order: select the public adapter and use the up arrow button
to
the right of the Connections list box to move the public adapter to the top of the list. It is very important
to
confirm this on all nodes in the cluster
2.4.4. Net Use Test
Software gets distributed between the nodes via copying to the remote nodes from the local node. You
should
ensure that this procedure will be successful by testing the net use command
On node1 issue the following command:
C:\>net use \\winnode2\c$
The command completed successfully.
This should return success
On node2 issue the following command:
C:\>net use \\winnode1\c$
The command completed successfully.
This should return success
2.4.5. Disable DHCP Media Sense
You should disable media sense. Media Sense allows Windows to uncouple an IP address from a card
when
the link to the local switch is lost. You should disable this activity using the registry editor regedit.
2.4.4. Net Use Test 11
Navigate to the Key HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
and
right click to create a new key of type DWORD.
2.4.5. Disable DHCP Media Sense 12
make sure that the Key is called DisableDHCPMediaSense, is of type DWORD and has a value of 1.
2.5. Remote Registry Connect
There can sometimes be an issue during the install when Oracle attempts to update the registry on a
remote
node. You should check that this is possible. Whilst the registry editor is still open select the File-
>Connect
Network Registry... menu item
Show Remote Registry information Hide Remote Registry information
2.5. Remote Registry Connect 13
In the 'Enter the object name...' edit box enter the name of the other node in the cluster (e.g. winnode2)
and
click OK
2.5. Remote Registry Connect 14
Wait for the node to appear in the registry tree. If this works then remote registry connect is working and
you
can continue.
2.5. Remote Registry Connect 15
If this fails then you probably have a policy set to disallow this. You need to change this policy.
When done you can right click on the node in the regedit tree and select disconnect.
2.5. Remote Registry Connect 16
2.6. Stopping Services
There can be issues with some of the services, which may already be running on the cluster nodes.
Typically a
Microsoft Service: Distributed Transaction Coordinator (MSDTC) can interact with Oracle software
during
install. It is recommended that this service is stopped and set to ‘manual’ start using services.msc on both
nodes.
Show Services information Hide Services information
Select the Action->Properties Menu item.
2.6. Stopping Services 17
Stop the service and set the Startup type to Manual, repeat on all nodes in the cluster. If, after completing
the
install, the MS DTC service is required it can be restarted and set to auto start.
2.7. Time Sync
There is a general requirement for Oracle RAC that the time on all the nodes be the same. Typically this
can
be achieved using Network Time Server protocol. At the very least you should ensure that the system
clocks
on all nodes are as close as possible.
2.8. Stage the Oracle Software
It is recommended that you stage the required software onto a local drive on Node 1 of your cluster.
Important: Ensure that you use only 32 bit versions of the Oracle Software on a 32bit OS and 64 bit
versions
of the Oracle Software on a 64bit OS
A suggested stage for Microsoft Windows 2003 is as follows
Oracle 10g Release 2 EE (10.2.0.1) (1 DVD ) C:\stage\10gR2
The latest patch set 10.2.0.4
The latest mini patch : currently patch 34
A suggested stage for Microsoft Windows 2008 is as follows
Oracle Database 10g Release 2 (10.2.0.4.0) Enterprise/Standard Edition for Microsoft Windows Vista
x64 and Microsoft Windows Server 2008 x64

The latest mini patch : currently patch 34
If you download the software from OTN you will not get the integrated installer. You will have separate
downloads for :
2.7. Time Sync 18
Oracle Clusterware and
ASM/Database
2.9. CVU stage check
Now you can run the CVU to check the state of the cluster prior to the install of the Oracle Software.
Remember to check if there is a newer version of CVU available on otn compared to the one that ships on
the
installation media
Show CVU output Hide CVU output
Z:\Disk1>runcluvfy stage -post hwos -n winnode1,winnode2 -verbose
1 file(s) copied.
C:\WINDOWS\temp\cvtmphome\oui\lib\win32\msvcp71.dll
C:\WINDOWS\temp\cvtmphome\oui\lib\win32\msvcr71.dll
2 file(s) copied.
Performing post-checks for hardware and operating system setup
Checking node reachability...
Check: Node reachability from node "winnode1"
Destination Node Reachable?
------------------------------------ ------------------------
winnode1 yes
winnode2 yes
Result: Node reachability check passed from node "winnode1".
Checking user equivalence...
Check: User equivalence for user "Administrator"
Node Name Comment
------------------------------------ ------------------------
winnode2 passed
winnode1 passed
Result: User equivalence check passed for user "Administrator".
Checking node connectivity...
Interface information for node "winnode2"
Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address
---------------- ------------ ------------ ------------ ------------ ------------
PublicLAN 10.10.10.21 10.10.10.0 10.10.10.21 192.168.1.254 00:0C:29:E3:82:6C
PrivateLAN 192.168.10.21 192.168.10.0 192.168.10.21 192.168.1.254 00:0C:29:E3:82:76
Interface information for node "winnode1"
Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address
---------------- ------------ ------------ ------------ ------------ ------------
PublicLAN 10.10.10.11 10.10.10.0 10.10.10.11 192.168.1.254 00:0C:29:E3:81:6C
PrivateLAN 192.168.10.11 192.168.10.0 192.168.10.11 192.168.1.254 00:0C:29:E3:81:76
Check: Node connectivity of subnet "10.10.10.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
winnode2:PublicLAN winnode1:PublicLAN yes
Result: Node connectivity check passed for subnet "10.10.10.0" with node(s) winnode2,winnode1.
Check: Node connectivity of subnet "192.168.10.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
winnode2:PrivateLAN winnode1:PrivateLAN yes
Result: Node connectivity check passed for subnet "192.168.10.0" with node(s) winnode2,winnode1.
WARNING:
Could not find a suitable set of interfaces for VIPs.
WARNING:
Could not find a suitable set of interfaces for the private interconnect.
Result: Node connectivity check passed.
2.8. Stage the Oracle Software 19
Checking shared storage accessibility...
Shared storage check failed on nodes "winnode2,winnode1".
Post-check for hardware and operating system setup was unsuccessful on all the nodes.
Notice the Shared Storage check failed: This is because this test cluster is using iSCSI and CVU cannot
check
the ‘shared-ness’ of iSCSI disks. Also it could not automatically determine the distinct public and private
network adapters.
Show all Hide all
3. Prepare the shared storage for Oracle RAC
This section describes how to prepare the shared storage for Oracle RAC
1. Shared Disk Layout
2. Enable Automounting of disks
3. Clean the Shared Disks - Optional Step
4. Create Logical partitions inside Extended partitions
5. Drive Letters
6. View Disks
3.1. Shared Disk Layout
It is assumed that the two nodes have local disk primarily for the operating system and the local Oracle
Homes. Labelled C: The Oracle Clusterware software also resides on the local disks on each node. The 2
nodes must also share some central disks.
At least 2 empty partitions are required for the Oracle Clusterware install using external
mirroring/redundancy
One for the Oracle Cluster Registry (OCR)
One for the vote device
If the disk system used to provide the shared physical disk does not provide some form of RAID
protection
then Oracle recommends that you use the built-in mirroring/redundancy capabilities provided by Oracle
Clusterware for the required Clusterware RAW partitions
Two for the Oracle Cluster Registry (OCR)
Three for the vote devices
All of these partitions should be a minimum of 256MB each. Because of space used by extended and
logical
partitions it is recommended that these LUNs be a minimum of 270MB. If you choose to use Oracle
mirroring
you should ensure that the partitions created for the mirrors are on separate physical devices.
2.9. CVU stage check 20
The first disk (Disk 0) is the local C: drive used for the software install. The Storage Administrator has
provisioned 14 LUNs from the shared disk array (Disk1 – Disk14)
Disk1 – Disk5 are used for Oracle Clusterware
Disk6 – Disk11 are used by ASM for the +DATA diskgroup
Disk12– Disk14 are used by ASM for the +FLASH diskgroup
In this document we will use the diskpart command line tool to manage these LUNs. You must create
logical
drives inside of extended partitions for the disks to be used by Oracle Clusterware and Oracle ASM.
There
must be no drive letters assigned to any of the Disks1 – Disk14 on any node. For MIcrosoft Windows
2003 it
is possible to use diskmgmt.msc instead of diskpart (as used in the following sections) to create these
partitions. For Microsoft Windows 2008, diskmgmt.msc cannot be used instead of diskpart to create these
partitions.
3.2. Enable Automount
You must enable automounting of disks for them to be visible to Oracle Clusterware. On each node log in
as a
user with explicit Administrator privileges then Click START->RUN and type diskpart
C:\>diskpart
Microsoft DiskPart version 5.2.3790.3959
Copyright (C) 1999-2001 Microsoft Corporation.
On computer: WINNODE1
DISKPART>AUTOMOUNT ENABLE
Repeat the above command on all nodes in the cluster
3.1. Shared Disk Layout 21
3.3. Clean the Shared Disks - Optional Step
You may want to clean your shared disks before starting the install. Cleaning will remove data from any
previous failed install. Please see a later Appendix for coping with failed installs. On Node1 from within
diskpart you should clean each of the disks. WARNING: this will destroy all of the data on the disk.
Do
not select the disk containing the operating system or you will have to reinstall the OS.
Cleaning the disk ‘scrubs’ every block on the disk. This may take some time to complete.
DISKPART> list disk
Disk ### Status Size Free Dyn Gpt
-------- ---------- ------- ------- --- ---
Disk 0 Online 8189 MB 8033 KB
Disk 1 Online 298 MB 298 MB
Disk 2 Online 298 MB 298 MB
Disk 3 Online 298 MB 298 MB
Disk 4 Online 298 MB 298 MB
Disk 5 Online 298 MB 298 MB
Disk 6 Online 1279 MB 1279 MB
Disk 7 Online 1279 MB 1279 MB
Disk 8 Online 1279 MB 1279 MB
Disk 9 Online 1279 MB 1279 MB
Disk 10 Online 1279 MB 1279 MB
Disk 11 Online 1279 MB 1279 MB
Disk 12 Online 1279 MB 1279 MB
Disk 13 Online 1279 MB 1279 MB
Disk 14 Online 1279 MB 1279 MB
Now you should clean disks 1 – 14 (Not disk0 as this is the local C: drive)
DISKPART>select disk 1
Disk 1 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 2
Disk 2 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 3
Disk 3 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 4
Disk 4 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 5
Disk 5 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 6
Disk 6 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 7
Disk 7 is now the selected disk.
3.3. Clean the Shared Disks - Optional Step 22
DISKPART> clean all
DISKPART> select disk 8
Disk 8 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 9
Disk 9 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 10
Disk 10 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 11
Disk 11 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 12
Disk 12 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 13
Disk 13 is now the selected disk.
DISKPART> clean all
DISKPART> select disk 14
Disk 14 is now the selected disk.
DISKPART> clean all
3.4. Create Logical partitions inside Extended partitions
Assuming the disks you are going to use are completely empty you must create an extended partition and
then
a logical partition inside that partition. In the following example, for Oracle Clusterware, we have
dedicated
LUNS for each device. This is not strictly necessary, you can combine an OCR & vote device onto one
physical LUN, creating 2 partitions inside the extended partition although it would be foolish for example
to
place the OCR and OCRMIRROR devices on the same LUN.
DISKPART> select disk 1
Disk 1 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 2
Disk 2 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 3
Disk 3 is now the selected disk.
3.4. Create Logical partitions inside Extended partitions 23
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition..
DISKPART> select disk 4
Disk 4 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 5
Disk 5 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 6
Disk 6 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 7
Disk 7 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 8
Disk 8 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 9
Disk 9 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 10
Disk 10 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 11
3.4. Create Logical partitions inside Extended partitions 24
Disk 11 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 12
Disk 12 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 13
Disk 13 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
DISKPART> select disk 14
Disk 14 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.
3.4.1. View Created partitions
DISKPART> list disk
Disk ### Status Size Free Dyn Gpt
-------- ---------- ------- ------- --- ---
Disk 0 Online 8189 MB 8033 KB
Disk 1 Online 290 MB 0 B
Disk 2 Online 290 MB 0 B
Disk 3 Online 290 MB 0 B
Disk 4 Online 290 MB 0 B
Disk 5 Online 290 MB 0 B
Disk 6 Online 1271 MB 0 B
Disk 7 Online 1271 MB 0 B
Disk 8 Online 1271 MB 0 B
Disk 9 Online 1271 MB 0 B
Disk 10 Online 1271 MB 0 B
Disk 11 Online 1271 MB 0 B
Disk 12 Online 1271 MB 0 B
Disk 13 Online 1271 MB 0 B
* Disk 14 Online 1271 MB 0 B
3.5. List Drive Letters
Diskpart should not add drive letters to the partitions on the local node. The partitions on the other node
may
have drive letters assigned. You must remove them. On earlier versions of Windows 2003 a reboot of the
‘other’ node will be required for the new partitions to become visible. Windows 2003 SP2 and Windows
2008
do not suffer from this issue.
3.4.1. View Created partitions 25
Using diskpart on Node2
DISKPART> list volume
Volume ### Ltr Label Fs Type Size Status Info
---------- --- ----------- ----- ---------- ------- --------- --------
Volume 0 H RAW Partition 290 MB Healthy
Volume 1 J RAW Partition 1271 MB Healthy
Volume 2 O RAW Partition 1271 MB Healthy
Volume 3 M RAW Partition 1271 MB Healthy
Volume 4 D DVD-ROM 0 B Healthy
Volume 5 L RAW Partition 1271 MB Healthy
Volume 6 K RAW Partition 1271 MB Healthy
Volume 7 C NTFS Partition 8182 MB Healthy System
Volume 8 E RAW Partition 290 MB Healthy
Volume 9 N RAW Partition 1271 MB Healthy
Volume 10 G RAW Partition 290 MB Healthy
Volume 11 F RAW Partition 290 MB Healthy
Volume 12 I RAW Partition 290 MB Healthy
Volume 13 P RAW Partition 1271 MB Healthy
Volume 14 Q RAW Partition 1271 MB Healthy
Volume 15 R RAW Partition 1271 MB Healthy
Notice that the volumes are listed in a completely different order compared to the disk list.
3.5.1. Remove Drive Letters
You need to remove the drive letters H J O M L K E N G F I P Q R These relate to volumes 0 1 2 3 5 6 8
9 10
11 12 13 14 15 Do NOT remove drive letters C & D which, in this case, are local disks (and CD-
ROM
drive) These relate to volumes 7 & 4 (In this example)
DISKPART> select volume 0
Volume 0 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 1
Volume 1 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 2
Volume 2 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 3
Volume 3 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 5
Volume 5 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 6
Volume 6 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
3.5. List Drive Letters 26
DISKPART> select volume 8
Volume 8 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 9
Volume 9 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 10
Volume 10 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 11
Volume 11 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 12
Volume 12 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 13
Volume 13 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 14
Volume 14 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
DISKPART> select volume 15
Volume 15 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.
3.5.2. List volumes on Second node
Check that none of the RAW partitions have drive letters assigned
DISKPART> list vol
Volume ### Ltr Label Fs Type Size Status Info
---------- --- ----------- ----- ---------- ------- --------- --------
Volume 0 RAW Partition 290 MB Healthy
Volume 1 RAW Partition 1271 MB Healthy
Volume 2 RAW Partition 1271 MB Healthy
Volume 3 RAW Partition 1271 MB Healthy
Volume 4 D DVD-ROM 0 B Healthy
Volume 5 RAW Partition 1271 MB Healthy
Volume 6 RAW Partition 1271 MB Healthy
Volume 7 C NTFS Partition 8182 MB Healthy System
Volume 8 RAW Partition 290 MB Healthy
Volume 9 RAW Partition 1271 MB Healthy
Volume 10 RAW Partition 290 MB Healthy
3.5.1. Remove Drive Letters 27
Volume 11 RAW Partition 290 MB Healthy
Volume 12 RAW Partition 290 MB Healthy
Volume 13 RAW Partition 1271 MB Healthy
Volume 14 RAW Partition 1271 MB Healthy
* Volume 15 RAW Partition 1271 MB Healthy
You can now exit diskpart on all nodes
3.6. View Disks using diskmgmt.msc
The disks should look like the above on all nodes in the cluster.
4. Oracle Clusterware Installation and Configuration
This page includes hidden sections, use the and image for each section to show/hide the section
or you can 'Show' or 'Hide' all by selecting here : Show all Hide all
Next we will install the Oracle Clusterware layer. Oracle Clusterware is an essential component of the
Oracle
RAC database infrastructure. Oracle Clusterware does not require any other clustering software. There is
no
need to install Microsoft Cluster Services (MSCS) software.
To install you will need the following information:
The local drive that will host the Oracle Clusterware software
The shared devices that will be used by Oracle Clusterware
The public node names and IP addresses for the nodes to be used by the cluster
The private node names and IP addresses for the nodes to be used by the cluster
3.5.2. List volumes on Second node 28
The Virtual Public IP names and IP addresses for the nodes to be used by the cluster
4.1. CVU Pre Oracle Clusterware install check
Before starting the Oracle Clusterware install it is prudent to complete a Cluster Verification Utility
(CVU)
check for this stage.
Z:\Disk1>runcluvfy.bat stage -pre crsinst -n winnode1,winnode2 -r 10gR2 -verbose
Show CVU output Hide CVU output
1 file(s) copied.
C:\WINDOWS\temp\cvtmphome\oui\lib\win32\msvcp71.dll
C:\WINDOWS\temp\cvtmphome\oui\lib\win32\msvcr71.dll
2 file(s) copied.
Performing pre-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "winnode1"
Destination Node Reachable?
------------------------------------ ------------------------
winnode1 yes
winnode2 yes
Result: Node reachability check passed from node "winnode1".
Checking user equivalence...
Check: User equivalence for user "Administrator"
Node Name Comment
------------------------------------ ------------------------
winnode2 passed
winnode1 passed
Result: User equivalence check passed for user "Administrator".
Checking administrative privileges...
Administrative privileges check passed.
Checking node connectivity...
Interface information for node "winnode2"
Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address
---------------- ------------ ------------ ------------ ------------ ------------
PublicLAN 10.10.10.21 10.10.10.0 10.10.10.21 192.168.1.254 00:0C:29:E3:82:6C
PrivateLAN 192.168.10.21 192.168.10.0 192.168.10.21 192.168.1.254 00:0C:29:E3:82:76
Interface information for node "winnode1"
Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address
---------------- ------------ ------------ ------------ ------------ ------------
PublicLAN 10.10.10.11 10.10.10.0 10.10.10.11 192.168.1.254 00:0C:29:E3:81:6C
PrivateLAN 192.168.10.11 192.168.10.0 192.168.10.11 192.168.1.254 00:0C:29:E3:81:76
Check: Node connectivity of subnet "10.10.10.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
winnode2:PublicLAN winnode2:PublicLAN yes
winnode2:PublicLAN winnode1:PublicLAN yes
winnode2:PublicLAN winnode1:PublicLAN yes
winnode2:PublicLAN winnode1:PublicLAN yes
winnode2:PublicLAN winnode1:PublicLAN yes
winnode1:PublicLAN winnode1:PublicLAN yes
Result: Node connectivity check passed for subnet "10.10.10.0" with node(s) winnode2,winnode1.
Check: Node connectivity of subnet "192.168.10.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
winnode2:PrivateLAN winnode1:PrivateLAN yes
Result: Node connectivity check passed for subnet "192.168.10.0" with node(s) winnode2,winnode1.
4. Oracle Clusterware Installation and Configuration 29
WARNING:
Could not find a suitable set of interfaces for VIPs.
WARNING:
Could not find a suitable set of interfaces for the private interconnect.
Result: Node connectivity check passed.
Checking system requirements for 'crs'...
Check: Operating system version
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
winnode2 Windows Server 2003 Windows Server 2003 passed
winnode1 Windows Server 2003 Windows Server 2003 passed
Result: Operating system version check passed.
Check: Total memory
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
winnode2 1.63GB (1707464KB) 512MB (524288KB) passed
winnode1 1.63GB (1707464KB) 512MB (524288KB) passed
Result: Total memory check passed.
Check: Swap space
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
winnode2 2.99GB (3131544KB) 1GB (1048576KB) passed
winnode1 2.99GB (3131544KB) 1GB (1048576KB) passed
Result: Swap space check passed.
Check: System architecture
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
winnode2 32-bit 32-bit passed
winnode1 32-bit 32-bit passed
Result: System architecture check passed.
Check: Free disk space in "C:\WINDOWS\TEMP" dir
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
winnode2 4.91GB (5150352KB) 400MB (409600KB) passed
winnode1 4.7GB (4924284KB) 400MB (409600KB) passed
Result: Free disk space check passed.
System requirement passed for 'crs'
Pre-check for cluster services setup was successful.