Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1)

This document covers steps in managing Solaris virtualized environments (Zones/Containers and LDOMs) Jeronimo M. Mulato 12/8/2010

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato

Table of Contents
ZFS - An Introduction ..................................................................................................................................... 6 zpool Create a regular raid-z zpool named pool1 with 3 disks ......................................................... 6 zpool Create a mirrored zpool named pool1 with 4 disks.................................................................. 6 zpool Adding mirror to zfs storage pool, pool1 with 2 disks .............................................................. 6 zpool List available storage pools ........................................................................................................ 6 zpool List all pool properties for pool1............................................................................................... 6 zpool Destroy a zfs storage pool ........................................................................................................ 6 zpool Export a zfs storage pool, pool1 ............................................................................................... 6 zpool Import a zfs storage pool, pool1................................................................................................ 6 zpool Upgrading zfs storage pool to current version ......................................................................... 6 zpool Managing/Adding hot spares................................................................................................... 7 zpool Create zfs storage pool with mirrored separate intent logs..................................................... 7 zpool Adding cache devices to zfs storage pool ................................................................................. 7 zpool Remove a mirrored device........................................................................................................ 7 zpool Recovering a Faulted zfs pool ................................................................................................... 7 zpool - reverting a zpool disk back to a regular disk. .......................................................................... 7 zfs - hide from df command .............................................................................................................. 7 zfs mount to a pre-defined mount point (zfs managed) ................................................................... 7 zfs mount to a pre-defined mount point (legacy managed) ............................................................. 7 zfs set limits/quota a zfs filesystem.................................................................................................... 8 zfs destroy a zfs filesystem ................................................................................................................. 8 zfs making snapshot .......................................................................................................................... 8 zfs - rolling back .................................................................................................................................... 8 zfs - removing snapshot....................................................................................................................... 8 ZONES (aka Containers)................................................................................................................................. 8 Easy Steps in creating a Zone .................................................................................................................... 8 Recommendations on Zone Build ............................................................................................................. 9

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato Sample Zone Build ..................................................................................................................................... 9 Steps: ...................................................................................................................................................10 Zone Cloning (Magic)...............................................................................................................................12 Zone Gotchas........................................................................................................................................... 13 Zones Resource (Memory Capping) ........................................................................................................13 Viewing capped swap memory ...........................................................................................................13 Viewing capped locked memory .........................................................................................................13 Viewing zones capped memory usage ................................................................................................13 Change max-swap resource dynamically ............................................................................................13 Change max-locked-memory resource dynamically ...........................................................................13 Change physical-memory capped resource dynamically ....................................................................13 To change capped memory resource permanently ............................................................................13 Zone Resource (Storage Devices|Network) ............................................................................................14 Adding resource ..................................................................................................................................14 Removing resource..............................................................................................................................15 Logical Domains (LDOMs)............................................................................................................................16 Prepping up for Primary LDOM (Control Domain) ..................................................................................16 Installing Firmware patch on the T5240 System Controller................................................................16 Prepping up for Guest LDOMs.................................................................................................................18 Assumptions/Recommendations: .......................................................................................................18 Creating Guest LDOMs ............................................................................................................................18 Steps in Creating Guest LDOMs...........................................................................................................18 Sample LDOM build (after primary or control domain has been created): ............................................21 Steps: ...................................................................................................................................................21 Customizing guest LDOM ........................................................................................................................23 Assigning specific virtual console to Guest LDOM ..................................................................................24 Removing Guest LDOM ...........................................................................................................................24 Updating LDOM Software .......................................................................................................................25 Precautions before LDOM Software Upgrade ....................................................................................25 Steps on Upgrading LDOM software ...................................................................................................26

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato LDOM Gotchas: .......................................................................................................................................27 Housekeeping LDOM guests ...............................................................................................................27 T5240 Additional Patches ....................................................................................................................29 Manually Primary LDOM creation ........................................................................................................29 Manually Guest LDOM creation ...........................................................................................................30 Appendix 1...................................................................................................................................................31 Appendix 2...................................................................................................................................................31

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato This document is intended to provide support information in managing Solaris Virtualized environments or Virtual Machines (VMs) whether it be zones or containers and LDOMs. For intent purpose, zones and containers refer to the same technology and implementation and will just be called zones. See below for a basic comparison of LDOMs and Zones. Description Hardware specific implementation. Full hardware virtualization LDOMs Yes. Currently runs only on TSeries servers. Yes. Hardware resource are assigned specifically to an LDOM and is totally isolated. Yes. An LDOM can run zones Zones No. As long as Solaris 10 runs you can implement zones. No.

Run zones

No.

Solaris LDOMs are virtualized environments assisted by hardware. Currently, LDOMs run only on Sun/Oracle SPARC T-Series servers (servers having the SPARC T-series CPU chip). Whereas zones (or containers) is not restricted to hardware. Prior to delving into the topics of zones and ldoms, a introduction to zfs and its faculties is presented. Much of the examples made use of zfs in both virtualization implementation.

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato

ZFS - An Introduction
ZFS is a combined file system and logical volume manager designed by Sun Microsystems (now Oracle). It features support for high storage capacities, integration of other filesystem and volume management concepts such as snapshots, clones, raid-z, nfs, smbfs, continuous integrity checking and automatic repair. Discussed below are the ways in managing and using zfs. Creating a zpool is the first step in making and using zfs. Prior to making the pool, you have to decide which type of pool to create (raid 0 or concatenated, raid 1 or mirrored, raidz or single crc disk raid, raidz2 or double crc disk raid, and raidz3 or triple crc disk raid). To create a zpool, just enter the command: zpool create <pool_name> <vdev-type> <devices> To illustrate creating zpools, examples will be used below. zpool Create a regular raid-z zpool named pool1 with 3 disks zpool create pool1 raidz c0t0d0 c0t1d0 c0t2d0 If you want to create raid-z2, you would need at least 3 drives. For raid-z3, its only advisable to use with 5 drives or more. zpool Create a mirrored zpool named pool1 with 4 disks zpool create pool1 mirror c0t0d0 c0t1d0 mirror c0t2d0 c0t3d0 zpool Adding mirror to zfs storage pool, pool1 with 2 disks zpool add pool1 mirror c0t3d0 c0t4d0 zpool List available storage pools zpool list zpool List all pool properties for pool1 zpool get all pool1 zpool Destroy a zfs storage pool zpool destroy f pool1 zpool Export a zfs storage pool, pool1 zpool export pool1 zpool Import a zfs storage pool, pool1 zpool import pool1 zpool Upgrading zfs storage pool to current version zpool upgrade a

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato zpool Managing/Adding hot spares zpool create pool1 raidz c0t0d0 c0t1d0 c0t2d0 spare c0t3d0 zpool replace pool1 c0t0d0 c0t3d0 (this replaces the c0t0d0 with c0t3d0 in pool1) zpool remove pool1 c0t2d0 (this removes c0t2d0 from pool1) zpool Create zfs storage pool with mirrored separate intent logs zpool create pool1 mirror c0t0d0 c0t1d0 mirror c0t2d0 c0t3d0 log mirror c0t4d0 c0t5d0 zpool Adding cache devices to zfs storage pool zpool add pool1 cache c1t0d0 c1t1d0 Normally, you add cache devices in which the devices are fast SSD drives. This is to compensate the use of the ARC cache which is in memory to be moved into the fast SSD devices. zpool iostat v pool1 5 zpool Remove a mirrored device zpool remove pool1 mirror-1 zpool Recovering a Faulted zfs pool If the pool faulted but with message as recoverable from the zpool status command, issue the following command: zpool clear F pool1 If the pool configuration was not cached, use the zpool import with the recovery mode flag as follows: zpool import F pool1 zpool - reverting a zpool disk back to a regular disk. format e The -e allows for extended format on the disk. Choose the disk to revert back and issue the command label to label it to SMI format. NOTE: zpool relabels the disk to EFI format. zfs - hide from df command zfs set canmount=off <pool_name> zfs set canmount=on <pool_name>

** Reverse

zfs mount to a pre-defined mount point (zfs managed) zfs get mounted <pool_name>/<zfs_name> zfs set mountpoint=<new_mount_point> <pool_name>/<zfs_name> zfs mount to a pre-defined mount point (legacy managed) zfs create pool1/autonomy mkdir p /apps/autonomy zfs set mountpoint=legacy pool1/autonomy

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato mount -F zfs pool1/autonomy /apps/autonomy NOTE: You have to manually edit /etc/vfstab to enable auto mounting. zfs set limits/quota a zfs filesystem zfs set quota=<n>m|g <pool_name>/<zfs_name> zfs destroy a zfs filesystem zfs destroy <pool_name>/<zfs_name> zfs making snapshot zfs snapshot <pool_name>/<zfs_name>@<current_date|marker> zfs - rolling back zfs list zfs list -H -t snapshot zfs rollback <pool_name>/<zfs_name>@<date_to_revert|marker> zfs - removing snapshot zfs list -H -t snapshot zfs destroy <pool_name>/<zfs_name>@<date_to_revert|marker>

ZONES (aka Containers)

Zones are soft virtual environments which are lighter in weight compared to full VM. With zones, youre not running a different OS but just a copy of the same OS within the contructs of a VM. Resources such as cpu and memory are all shared with the global zone. Resource restrictions may be imposed to the local zones using mechanisms such as fair share scheduling and memory capping. There are two implementations of zones, one is called sparce root and the other is called whole root. With the sparce root implementation, there are four directories in the global zones root filesystem inherited by the local zones. These four directories are as follows: /lib, /platform, /sbin, and /usr. In this model, all packages that are installed on the global zone are made available to the sparce root zone. With the whole root implementation, a copy of the global zones is made for the local zone. Even the package database is made local for the zone. Hence, after the zone creation, you can put in additional packages on the global zones or in the local whole root zone independent of each other. The choice of using whole root versus sparce root depends on the control of resource and administrative trade-off. Whole root allows you to maximize administrative control (independence and isolation) at the cost of more disk space, while sparce root optimizes the efficient sharing of executables and resources using a smaller disk footprint at the cost of administrative independence.

Easy Steps in creating a Zone

1. Create a zone configuration template.

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato global-zone# vi <zone-config-template> 2. Define the zone configuration global-zone# zonecfg z <zone-name> -f <zone-config-template> 3. Install the non-global zone global-zone# zoneadm z <zone-name> install global-zone# zoneadm z <zone-name> ready 4. Create sysidcfg for system and naming service convention (this is optional) global-zone# cd <zone-path>/root/etc global-zone# vi sysidcfg 5. Implement resource management policies (this is optional). Note that you can incorporate this on the configuration template. 6. Boot the non-global zone global-zone# zoneadm z <zone-name> boot 7. Login to the non-global zone global-zone# zlogin C <zone-name> (To exit the console, you need to use ~.) 8. Implement SOE/SSM implementation

Recommendations on Zone Build

1. If you have separate disk space you want to assign to zones implementation, its advisable to create the zones on a ZFS filesystem. 2. Always use the Fair Share Scheduler when configuring zones. This allows you to assign cpushares to a zone and prevent a zone from monopolizing the whole cpu resource. 3. Always use memory capping daemon. This allows capping memory, swap and locked in memory to zones. CAUTION: DO NOT SET memory capping on the global zone as this may impact system availability. 4. Use the given template (see the Appendix A) to use for making the zone.

Sample Zone Build

In this example we will create two zones with the following configuration: The global zone has 4 cpus, 8gb memory and 32gb swap. Drive c1t2d0 will be used for the ZFS. Each non-global zones will have a boot drive of size 30gb Non-global zone1 will have an IP of 169.185.220.15 and will share/use eri0 as its network interface. Non-global zone2 will have an IP of 169.185.220.16 and will share/use eri0 as its network interface. Use Fair Share Scheduler (FSS) to allow assigning cpu shares to each zones. This is to safeguard and prevent a zone from monopolizing the entire cpu resource. Global zone will have 40 cpu shares. Non-global zones will each have 30 cpu shares and will also run with FSS as its scheduler. Memory capping will be implemented for each non-global zones (2gb memory and 8gb swap).

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato Steps: 1. Update the systems default scheduler to run using the Fair Share Scheduler, FSS. Global-zone# dispadmin d FSS 2. Set all processes on the running global zone to run using FSS. Global-zone# priocntl s c FSS i all 3. Start the rcap daemon to enable memory capping. This can be done using the command: Global-zone# svcadm enable rcap 4. Set the global zone cpu-shares as follows: Global-zone# zonecfg z global zonecfg:global> set cpu-shares=40 zonecfg:global> commit zonecfg:global> exit The above will set the global zone cpu-shares. However, to take into effect, the above would require a reboot. To bypass the reboot, issue the following command: Global-zone# prctl n zone.cpu-shares -v 40 -r -i zone global 5. Create a ZFS pool/filesystem for zonepath assignment. zonespool/zone1 for zone1, and zonespool/zone2 for zone2. Give each zfs filesystem a quota of 30gb. Global-zone# zpool create f zonespool c1t2d0 Global-zone# zfs create zonespool/zone1 Global-zone# zfs create zonespool/zone2 Global-zone# zfs set quota=30g zonespool/zone1 Global-zone# zfs set quota=30g zonespool/zone2 6. Create the zone template for both zone1 and zone2. For zone1, the template, /var/tmp/zone1.cfg, will contain the following: create -b set zonepath=/zonespool/zone1 set autoboot=true set scheduling-class=FSS add net set address=169.185.220.15 set defrouter=169.185.220.1 set physical=eri0 end add rctl set name=zone.cpu-shares set value=(priv=privileged,limit=20,action=none) end add capped-memory set physical=2g

10

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato set swap=8g end For zone2, the template, /var/tmp/zone2.cfg will contain the following: create b set zonepath=/zonespool/zone2 set autoboot=true set scheduling-class=FSS add net set address=169.185.220.16 set defrouter=169.185.220.1 set physical=eri0 end add rctl set name=zone.cpu-shares set value=(priv=privileged,limit=20,action=none) end add capped-memory set physical=2g set swap=8g end Configure the zones using the following command:
zonecfg -z zone1 -f /var/tmp/zone1.cfg zonecfg -z zone2 -f /var/tmp/zone2.cfg

To check/verify the zones configuration, use the following command:

zoneadm list cv echo info | zonecfg -z zone1 echo info | zonecfg -z zone2

7. Start the zone install. This will copy the necessary files to the zones location.
zoneadm -z zone1 install zoneadm -z zone2 install

8. Upon confirmation of the zones install, issue the command to make the zones ready for use:
zoneadm -z zone1 ready zoneadm -z zone2 ready 9. Create a sysidcfg template file for each zone (/var/tmp/sysidcfg.zone1 and /var/tmp/sysidcfg.zone2) and copy it to the local zone root location as follows: cp /var/tmp/sysidcfg.zone1 /zonespool/zone1/root/etc cp /var/tmp/sysidcfg.zone2 /zonespool/zone2/root/etc

10. Boot the zones using the following command:

zoneadm -z zone1 boot zoneadm -z zone2 boot

11. Login to the zones and start the your environment customization.

11

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato

Zone Cloning (Magic)

After youve completed one zone install and customization, you really dont need to redo the same steps. If you want to create a copy of the same virtual machine (another local zone), you can use cloning. To clone a zone, you have to follow the steps below: 1. Create the new zone target location: Global-zone# zfs create zonespool/zone3 Global-zone# zfs set quota=30g zonespool/zone3 2. Create the new zone template and configure the new zone. As an example, well create zone3 with the following template -> /var/tmp/zone3.cfg, to contain the following: create -b set zonepath=/zonespool/zone3 set autoboot=true set scheduling-class=FSS add net set address=169.185.220.17 set defrouter=169.185.220.1 set physical=eri0 end add rctl set name=zone.cpu-shares set value=(priv=privileged,limit=20,action=none) end add capped-memory set physical=2g set swap=8g end Configure the new zone using the following command:
zonecfg -z zone3 -f /var/tmp/zone3.cfg

3. Halt the original (source) zone to clone. Global-zone# zlogin zone1 halt 4. Start the cloning as follows: Global-zone# zoneadm -z zone3 clone zone1
5. Create a sysidcfg template file for the new zone (/var/tmp/sysidcfg.zone3) and copy it to the local zone root location as follows: cp /var/tmp/sysidcfg.zone3 /zonespool/zone3/root/etc 6. Start the zones. Global-zone# zoneadm -z zone1 boot Global-zone# zoneadm -z zone3 boot

12

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato

Zone Gotchas
1. If you are to configure multiple zones, its recommended to allocate the cpu shares as follows: Global = 20 Non-global zones = 80/(number of non-global zones) 2. If a decision is made to mirror the ZFS pool on an already made zfs pool with a single drive. In the example given earlier, we already have a ZFS pool named zpool with c1t2d0 as member. To make this a mirror, this can be done simple by issuing the following commands: zpool attach zpool c1t2d0 c1t3d0 3. If IPMP is configured on a physical interface on the global zone and one of the interfaces (the primary active) is used as shared network interface on the local/non-global zone. Then its automatically picked up as IPMP on the non-global zone. Meaning that you do not need to configure IPMP on the non-global zone.

Zones Resource (Memory Capping)

Viewing capped swap memory /bin/prctl -n zone.max-swap `pgrep -z <zone_name> init` Viewing capped locked memory /bin/prctl -n zone.max-locked-memory `pgrep -z <zone_name> init` Viewing zones capped memory usage rcapstat -z 1 1 Change max-swap resource dynamically prctl -z zone.max-swap -r -v 200m `pgrep -z <zone_name> init` Change max-locked-memory resource dynamically prctl -z zone.max-locked-memory -r -v 200m `pgrep -z <zone_name> init` Change physical-memory capped resource dynamically rcapadm -z <zone_name> -m 100m To change capped memory resource permanently zonecfg:zone: > select capped-memory zonecfg:zone:capped-memory> set physical-100m zonecfg:zone:capped-memory> set physical-100m zonecfg:zone:capped-memory> set physical-100m zonecfg:zone:capped-memory> end zonecfg:zone: > commit

13

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato

Zone Resource (Storage Devices|Network)

Adding resource Dataset (ZFS filesystem are treated as datasets): If the primary goal is to delegate the administration of storage to a zone, then ZFS supports adding datasets to a non-global zone. Unlike adding the filesystem, this causes the ZFS filesystem to be visible within the already configured zone. The zone administrator can set file system properties as well as create children. In addition, the zone administrator can take snapshots, create clones and otherwise control the entire file system hierarchy. To use this facility, use the following commands: zonecfg:zone> add dataset zonecfg:zone:dataset> set name=pool/filesys zonecfg:zone:dataset> end zonecfg:zone> verify zonecfg:zone> commit zonecfg:zone> end Zfs filesystem: If the goal is solely to share space with the global zone. zonecfg:zone> add fs zonecfg:zone:fs> set type=zfs zonecfg:zone:fs> set special=<pool_name/zfs_name> zonecfg:zone:fs> set dir=<non-global_zone_local_mount_point> zonecfg:zone:fs> end zonecfg:zone> verify zonecfg:zone> commit zonecfg:zone> end Fs: zonecfg:zone> add fs zonecfg:zone:fs> set dir=<non-global_zone_local_mount_point> zonecfg:zone:fs> set special=/dev/dsk/c#t#d#s# zonecfg:zone:fs> set raw=/dev/rdsk/c#t#d#s# zonecfg:zone:fs> set type=ufs zonecfg:zone:fs> add options logging zonecfg:zone:fs> end zonecfg:zone> verify zonecfg:zone> commit zonecfg:zone> end Inherit-pkg-dir: zonecfg:zone> add fs zonecfg:zone:fs> set dir=<non-global_zone_mount_point> zonecfg:zone:fs> set share=<global_zone_source_dir> zonecfg:zone:fs> set type=lofs zonecfg:zone:fs> end

14

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato zonecfg:zone> verify zonecfg:zone> commit zonecfg:zone> end Export home directory from global zone to local zone: zonecfg:zone> add fs zonecfg:zone:fs> set dir=/export/home zonecfg:zone:fs> set special=/export/home zonecfg:zone:fs> set type=lofs zonecfg:zone:fs> set options=nodevices zonecfg:zone:fs> end zonecfg:zone> verify zonecfg:zone> commit zonecfg:zone> exit net: zonecfg:zone> add net zonecfg:zone:net> set physical=<interface_name> zonecfg:zone:net> set address=<IP_address> zonecfg:zone:fs> end zonecfg:zone> verify zonecfg:zone> commit zonecfg:zone> exit Removing resource Storage: zonecfg:zone> zonecfg:zone> zonecfg:zone> zonecfg:zone> Net: zonecfg:zone> zonecfg:zone> zonecfg:zone> zonecfg:zone> remove net physical=<interface_name> verify commit exit

remove fs dir=<non-global_zone_local_mount_point> verify commit exit

15

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato

Logical Domains (LDOMs)

LDOMs are VMs assisted by hardware in which you can run separate operating system in each domain. Unlike zones, LDOMs operate independently of each other except for the control and resource domains which propagates and controls which resource is given or assigned to the guest domains. Each ldoms are assigned their own resource such as cpu, memory, network and disk space. Note that unlike zones, ldoms are totally isolated from sharing cpu and memory.

Prepping up for Primary LDOM (Control Domain)

1. Make sure that youre on the latest Solaris 10 LDOM patch (138888-02). 2. Install the LDOM software package 3. Make sure you have the correct firmware. This can be done by running showhost from the ILOM/ALOM session. > show /HOSTS Patch
139434-01 139439-02 139444-01 139446-01

Platform
T2000 T5x20 T5240 T5440

Firmware Version
6.7.0 7.2.1.b 7.2.0 7.2.0

LDOM Version
1.1 1.1 1.1 1.1

If you do not have the correct firmware, you need to apply the firmware patch. Installing Firmware patch on the T5240 System Controller a. Copy the patch to the server (control domain) to be patched. Unpack the patch from any directory from the control domain. Then copy the following files to /tmp: sysfwdownload firmware*.pkg NOTE: The firmware package file will no longer be available and different for individual platforms. b. Afterwards, you need to execute the following commands: cd /tmp ./sysfwdownload ./<Firmware.pkg> Shutdown the domain controller using the commands: shutdown y i0 g0 or halt c. Once in the ok prompt, you will need to get to the system controller (ALOM or ILOM) using the key combination #. d. Once on the ILOM, you nee to login as admin to get to the ALOM prompt (sc>). To get to the ALOM prompt you need to login first as root which gives you the standard ILOM prompt (->). Issue the following command to create an admin user which would provide the default ALOM prompt as follows: -> create /SP/users/admin role=Administrator cli_mode=alom Creating user.. Enter new password: *******

16

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato Enter new password again: ******* Created /SP/users/admin -> logout Then login as admin to the ALOM prompt. e. Issue the following commands: sc> poweroff f. Ensure the keyswitch is set to NORMAL. sc> setkeyswitch y normal g. Perform the flash update: sc> flashupdate -s 127.0.0.1 h. After the flash update, be sure to reset the ALOM/ILOM using the command: sc> resetsc i. Log back in to the ALOM prompt and issue the command: sc> poweron j. Logout and log back in to the ILOM prompt and get back to the domain controller prompt as usual: -> start /SP/console Once its up, youre now ready to LDOM! 4. Create the primary domain (also known as the control domain). This domain will be controlling and manipulating all other domains. To do this, issue the command: # Create Virtual Disk Server (vds) ldm add-vds primary-vds0 primary # Create Virtual Console concentrator server (vcc) ldm add-vcc port-range=5000-5100 primary-vcc0 primary # Create Virtual Switch server (vsw) ldm add-vsw net-dev=nxge0 primary-vsw0 primary # Assign 8 virtual CPU ldm set-vcpu 8 primary # Assign 0 MAU (Math Unit) ldm set-mau 0 primary # Assign 4gb memory ldm set-memory 4096m primary # Assign the ldom config to the service processor ldm add-spconfig initial The above commands can be put into a script will assign 8 vCPU (virtual CPU), 0 MAU (math unit), and 4G memory to the primary domain. It will also use the first instance of the network interface (assume nxge0) as the primary network interface for use by itself and the guest domains. NOTE that you can change this later as needed. After the command execution, you would need to reboot the box to boot in LDOM mode. shutdown y i6 g0 5. When the system comes back up, it will be configured with LDOM mode enabled. Confirm using the command:

17

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato # /opt/SUNWldm/bin/ldm list It should provide you the initial ldom named primary indicative of the control domain.

Prepping up for Guest LDOMs

The above command set, basically sets up the initial LDOM (primary LDOM). Its usually setup with the following basic services as: - VCC (virtual console concentrator) - VSW (virtual switch) - and VDS (virtual disk service) After which you would need to customize these services based on the requirements of the guest LDOM(s). Assumptions/Recommendations: - 4 network connections (make sure that physical networks are connected on two different network interfaces and separate cards; say one internal and one external). Make sure that the virtual switches has been setup for each physical network connection. To create a virtual switch issue the following command: /opt/SUNWldm/bin/ldm add-vsw net-dev=<nxgeX|bgeX> primary-vswX primary After the creation of the virtual switches, you would need to remove the original entries for the physical interfaces and instead put in the virtual interfaces. You simply map the file entries and rename the physical network with the virtual switch network. As an example, if nxge0 was used for primary-vsw0, then you would need to rename the file, hostname.nxge0 to hostname.vsw0. Until the virtual switches have been made the active interfaces via the command, ifconfig a, the communication between the control domain and the guest domain would not be feasible.

Creating Guest LDOMs

Before creating the guest LDOMs, make sure to have the following information. - Guest LDOM name - How much vCPU to allocate? (3 cores for production/cob; 1 core for dev/uat. Each core has 8 threads. Hence, for three cores, you would need to specify 24 vCPU and for a single core, 8 vCPU). - How much memory to allocate? (4GB for each core assigned. Hence, for a three core LDOM, you would need to specify 12 GB for the LDOM; for a single core LDOM, you would need to specify 4GB). - How much disk space to allocate(30gb for boot and 25 for apps) ? Make sure that the space is allocated for the guest LDOM. Steps in Creating Guest LDOMs From the control domain (sample is a production/cob guest domain): 1. Setup the proper path for the shell to be used for the administration of LDOM. exec ksh o vi

18

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato export PS1=$LOGNAME@`uname -n` [\$PWD] > export PATH=$PATH:/opt/SUNWldm/bin Create the LDOM guest virtual disk. With the use of zfs and for simplicity, well create a disk/image file which will act as our guest ldom bootdisk. As an example. zfs create ldomroot/ldom<n> cd /ldomroot/ldom<n> mkfile 30g bootdisk.img Create an entry for the bootdisk image to be used for the LDOM guest. ldm add-vdsdev /ldomroot/ldom<n>/bootdisk.img <ldom_name>_boot@primary-vds0 Create the LDOM guest using the command: ldm add-domain gdomain ldm add-vcpu 24 gdomain ldm add-memory 12g gdomain ldm add-vnet vnet1 primary-vsw1 gdomain ldm add-vdsdev /ldomroot/ldom<n>/bootdisk.img gdomain_boot@primary-vds0 ldm add-vdisk vdisk0 gdomain_bootdisk@primary-vds0 ldm bind gdomain ldm set-var auto-boot\?=false gdomain Start the LDOM guest ldm start <ldom_name> Login to the LDOM guest. To login, you need to see which telnet port to login using the virtual console for the newly created LDOM guest. Use the command below to identify the virtual port. ldm list The command above will indicate the <ldom_name> and its console port. As an example, you may see port 5000 for ldom-> gdomain, issue the command: telnet localhost 5000 To exit from the virtual console, use the following sequence -> Ctrl-], then exit. From the ldom guest, issue the command: ok> boot Then login as root and initiate sys-unconfig to rename the LDOM to its new name. Now that youve successfully configured the LDOM guest, you may want to customize. Follow the steps below for additional LDOM customization. Execute the following command: ldm add-spconfig working The above command is to save the current system configuration (both LDOM primary and guests) into the Linux hypervisor running the ldom. **** NOTE **** Always run the command ldm add-spconfig when in doubt to save the current configuration.

2.

3. 4.

5.

6.

19

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato LDOM guest : Adding a virtual network Halt the guest domain. From the control domain: ldm stop <ldom_name> ldm unbind <ldom_name> ldm add-vnet vnetX <virtual_switch_name> <ldom_name> where X is the virtual network number being assigned. ldm bind <ldom_name> ldm start <ldom_name> LDOM guest: Removing a virtual network Halt the guest domain. From the control domain: ldm stop <ldom_name> ldm unbind <ldom_name> ldm remove-vnet vnetX <ldom_name> where X is the virtual network number to be removed. ldm bind <ldom_name> ldm start <ldom_name> LDOM guest : Adding virtual disk using file Halt the guest domain. From the control domain: mkfile <size> <file_image_name> ldm add-vdsdev <file_image_name> <ldom_name>_<image_purpose>@primary-vds0 ldm stop <ldom_name> ldm unbind <ldom_name> ldm add-vdisk vdiskX <ldom_name>_<image_purpose> <ldom_name> where X is the virtual disk number to be assigned. ldm bind <ldom_name> ldm start <ldom_name> LDOM guest : Adding virtual disk using SAN If SAN is made available, it would be best to put the LDOM guest under SAN for performance reasons. You need to register the actual raw (with slice 2) device to the virtual disk server. Halt the guest domain. From the control domain: ldm add-vdsdev /dev/rdsk/c2t1d0s2 gdomain_boot@primary-vds0 ldm stop <ldom_name> ldm unbind <ldom_name> ldm add-vdisk vdiskX <ldom_name>_<image_purpose> <ldom_name> where X is the virtual disk number to be assigned. From the syntax above, ldm add-vdisk vdisk1 gdomain_boot@primary-vds0 gdomain ldm bind <ldom_name>

20

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato ldm start <ldom_name> LDOM guest: Removing a virtual disk Halt the guest domain. From the control domain: ldm stop <ldom_name> ldm unbind <ldom_name> ldm remove-vdisk vdiskX <ldom_name> where X is the virtual disk number to be assigned. ldm bind <ldom_name> ldm start <ldom_name>

Sample LDOM build (after primary or control domain has been created):
Assumptions: Server Network Disks : T5240 : nxge2, nxge3, nxge4, and nxge5 : LDOM guests will be assigned the following disks for both OS and apps (c1t2d0, c1t3d0, c1t4d0, c1t5d0, c1t6d0, and c1t7d0 each 146gb)

Steps: 1. Confirm primary LDOM # exec ksh -o vi # export PATH=$PATH:/opt/SUNWldm/bin # ldm list NAME STATE FLAGS CONS VCPU MEMORY Primary active -n-cv- SP 8 4G # 2. Confirm the primary LDOM services # ldm list-services VCC NAME LDOM PORT-RANGE primary-vcc0 primary 5000-5255

UTIL 0.2%

UPTIME 7h 52m

VSW NAME LDOM MAC NET-DEV DEVICE DEFAULT-VLAN-ID PVID VID MODE primary-vsw0 primary 00:14:4f:f9:d8:76 nxge2 switch@0 1 1 VDS NAME LDOM VOLUME OPTIONS MPGROUP DEVICE # In the above we see that the create_primary initializes the first active network interface as the network device for the primary virtual switch 0. You would need to add the other virtual network interfaces as follows: ldm add-vsw net-dev=nxge3 primary-vsw1 primary ldm add-vsw net-dev=nxge4 primary-vsw2 primary

21

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato ldm add-vsw net-dev=nxge5 primary-vsw3 primary Upon assignment of the virtual switches, you need to make it active and remove the physical network from the active list. This is done by issuing the commands below. This ensures communication between the control domain and the guest domains. for i in 0 1 2 3 do ifconfig vsw${i} plumb done for i in 2 3 4 5 do ifconfig nxge${i} unplumb done ifconfig vsw0 <IP_address_of_nxge2> netmask <netmask> broadcast + up ifconfig vsw1 <IP_address_of_nxge3> netmask <netmask> broadcast + up ifconfig vsw2 <IP_address_of_nxge4> netmask <netmask> broadcast + up ifconfig vsw3 <IP_address_of_nxge5> netmask <netmask> broadcast + up Create the disk device to be used by the guest domains. In our case, ZFS (RAIDZ) is used for flexibility. Note that guests would be assigned a separate filesystem. See below: zpool create -f ldomroot raidz c1t2d0 c1t3d0 c1t4d0 c1t5d0 c1t6d0 c1t7d0 If you make a mistake, you may remove the zpool and recreate the pool as follows: zpool destroy ldomroot zpool create -f ldomroot raidz c1t2d0 c1t3d0 c1t4d0 c1t5d0 c1t6d0 c1t7d0 NOTE : destroy option in zpool should be used with caution and care. Create separate zfs filesystems for each individual LDOM guests. Its also advisable to create a separate zfs filesystem for the apps. zfs create ldomroot/ldom1 zfs create ldomroot/ldom2 zfs create ldomroot/ldom3 zfs create ldomroot/ldom4 zfs create ldomroot/apps By separating the filesystem for the LDOM apps, we can apply the snapshot tool on the guest LDOM OS disk filesystem prior to applying kernel patches. This way, we can revert back to the previous version of the OS if in case we need to (ie. Failed kernel patch that needs to be backed out). Copy the gold build image for each LDOM guest as follows: cp /export/temp/bootdisk.img /ldomroot/ldom1 cp /export/temp/bootdisk.img /ldomroot/ldom2 cp /export/temp/bootdisk.img /ldomroot/ldom3 cp /export/temp/bootdisk.img /ldomroot/ldom4 Assign the virtual disk to the service ldm add-vdsdev /ldomroot/ldom1/bootdisk.img gdom01_boot@primary-vds0 ldm add-vdsdev /ldomroot/ldom2/bootdisk.img gdom02_boot@primary-vds0 ldm add-vdsdev /ldomroot/ldom3/bootdisk.img gdom03_boot@primary-vds0 ldm add-vdsdev /ldomroot/ldom4/bootdisk.img gdom04_boot@primary-vds0 Create each individual LDOM guest using the command:

3.

4.

5.

6.

22

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato /opt/IHSldmcfg/bin/create_guest Provide the initial config as follows: gdom01 24 vCPU 12g Memory gdom01_boot@primary-vds0 as primary drive (vdisk0) primary-vsw0 as the primary network interface (vnet0) gdom02 24 vCPU 12g Memory gdom02_boot@primary-vds0 as primary drive (vdisk0) primary-vsw1 as the primary network interface (vnet0) gdom03 24 vCPU 12g Memory gdom03_boot@primary-vds0 as primary drive (vdisk0) primary-vsw0 as the primary network interface (vnet0) gdom04 24 vCPU 12g Memory gdom04_boot@primary-vds0 as primary drive (vdisk0) primary-vsw0 as the primary network interface (vnet0) Start the guest LDOM ldm start <ldom_name> Login to the LDOM via the virtual console concentrator (VCC) using the port info provided using the command ldm list. Note the port number and login using the command telnet as follows: telnet localhost <port_number> Login as root (password is x!tra123), then issue the command: /usr/sbin/sys-unconfig The above will unconfigure the new LDOM guest, then reboot. From the primary domain, execute the following command: ldm add-spconfig working The above command is to save the current system configuration (both LDOM primary and guests) into the Linux hypervisor running the ldom. **** NOTE **** Always run the command ldm add-spconfig when in doubt to save the current configuration.

7. 8.

9.

10.

Customizing guest LDOM

1. From the control domain, add the virtual network ldm add-vnet vnet1 primary-vsw1 gdom01 Confirm the vnet addition using the command: ldm list-bindings gdom01

23

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato 2. Make sure to unbind and re-bind the new config prior to restart. ldm unbind gdom01 ldm bind gdom01 ldm start gdom01 3. Configure IPMP 4. Continue with the other LDOM using the same steps above. In configuring IPMP, use the following table to effectively utilize the networks: LDOM Guest Vnet0 Vnet1 Primary Active gdom01 primary-vsw0 (nxge2) primary-vsw2 (nxge4) primary-vsw0 (nxge2) gdom02 primary-vsw2 (nxge4) primary-vsw0 (nxge2) primary-vsw2 (nxge4) gdom03 primary-vsw1 (nxge3) primary-vsw3 (nxge5) primary-vsw1 (nxge3) Gdom04 primary-vsw3 (nxge5) primary-vsw1 (nxge3) primary-vsw3 (nxge5) Doing the above assignment, the load will be split on all four physical interface actively. 5. To add the application disks to the guest ldoms. The guest LDOMS needs to be stopped. Then from the control domain, create the application disk assignment on the apps filesystem for all the LDOMS, add and then restart. cd /ldomroot/apps for i in 1 2 3 4 do ldm stop gdom0${i} ldm unbind gdom0${i} mkfile 25g ldom${i}_apps.img ldm add-vdsdev /ldomroot/apps/ldom${i}_apps.img rncardsweb0${i}_apps@primary-vds0 ldm add-vdisk vdisk1 rncardsweb0${i}_apps@primary-vds0 rncardsweb0${i} ldm bind gdom0${i} ldm start gdom0${i} done Connect the guest LDOM and do a device re-configure using the command, reboot -- -r or simply issuing the command devfsadm -C. Also note that the new devices needs to be repartitioned prior to use.

Assigning specific virtual console to Guest LDOM

ldm set-vcons port=<50XX> <ldom_name> Note that you need to specify this command only for unbound guest domains.

Removing Guest LDOM

ldm remove-domain <ldom_name> ldm remove-spconfig <ldom_name>

24

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato

Updating LDOM Software

See the matrix below for updating LDOM package from 1.1 to 1.2. ***** When updating to LDOM 1.2, note that all ldoms (global or guest) should adhere to the minimum memory requirement of 12Mbytes. Patch
139434-03 139439-04 139444-03 139446-03

Platform
T2000 T5x20 T5240 T5440

Firmware Version
6.7.4 7.2.2.e 7.2.2.e 7.2.2.e

LDOM Version
1.2 1.2 1.2 1.2

Precautions before LDOM Software Upgrade Prior to updating the LDOM Software, be sure to take the following precautions: 1. Backup/Save the Autosave Configuration Directories Whenever you upgrade the OS or the LDOM software on the control domain, you must save and restore the Logical Domains autosave configuration data, which is found in the /var/opt/SUNWldm/autosave-autosave-name directories. You can use tar or cpio to save and restore the entire contents of the directories. NOTE: Each autosave directory includes a timestamp for the last SP configuration update for the related configuration. If you restore the autosave files, the timestamp might be out of sync. In this case, the restored autosave configurations are shown in their previous state, either [newer] or up to date. To save : mkdir p /root/ldom cd /root/ldom tar cvf autosave.tar /var/opt/SUNldm/autosave_* To restore, be sure to remove the existing autosave directories to ensure a clean restore operation: cd /root/ldom rm rf /var/opt/SUNWldm/autosave_* tar xvf autosave.tar 2. Backup/Save the Logical Domain Constraints Database File Whenever you upgrade the OS or LDOM package on the control domain, you must save and restore the Logical Domains constraints database file that can be found in /var/opt/SUNWldm/ldom-db.xml. NOTE: Also, save and restore the /var/opt/SUNWldm/ldom-db.xml file when you perform any other operation that is destructive to the control domainss file data, such as disk swap.

25

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato **** When performing a OS Live Upgrade, be sure to preserve the Logical Domains Constraints Database File ***** This can be done by simply adding the following line to /etc/lu/synclist file: /var/opt/SUNWldm/ldom-db.xml OVERWRITE This will cause the database to be copied automatically from the active boot environment to the new boot environment when switching boot environments. 3. Make sure that the current working LDOM configuration is saved in the Service Processor using the command: ldm add-spconfig working Steps on Upgrading LDOM software 1. Stop and unbind the guest LDOMS. 2. Flash update the system firmware. 3. Disable the Logical Domains Manager daemon (ldmd) svcadm disable ldmd 4. Follow the steps below a. Remove the old SUNWldm package. pkgrm SUNWldm b. Add the new SUNWldm package pkgadd d ./SUNWldm c. Use ldm list to verify that the Logical Domain Manager is running. 5. Reboot or issue init 6. Once the system comes up, verify that the other guest domains comes up and do checkout. 6. In some cases, vntsd does not come up, you lose console communication for the guest LDOM. Issue the command: svcadm enable vntsd

WARNING:
In case you lose some of the guest LDOMs configuration resources (ie. vnetX or vdiskX). Dont try to remove and re-create the LDOM. Its easier just to re-add the resource using the commands within the build scripts for the particular LDOM.

26

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato

LDOM Gotchas:
Housekeeping LDOM guests 1. Create an XML backup copy of the LDOM guest as follows: ldm ls-constraints -x <ldom_name> > <ldom_name>.xml This creates a backup XML file in case the guest domain needs to be re-created. To re-create the guest domain using the XML file, issue the following command: ldm add-domain -i <ldom_name>.xml ldm bind <ldom_name> ldm start <ldom_name> Edit the guest XML file and replace the lines with the guest LDOMs reported MAC address and hostid. To get the guest LDOMS MAC and hostid, execute the command: ldm list-domain -l <guest_ldom_name> | more The lines <Content xsi:type="ovf:VirtualSystem_Type" ovf:id="gdom01"> <Section xsi:type="ovf:ResourceAllocationSection_Type"> <Item> <rasd:OtherResourceType>ldom_info</rasd:OtherResourceType> <rasd:Address>auto-allocated</rasd:Address> <gprop:GenericProperty key="hostid">0xffffffff</gprop:GenericProperty> </Item> Replace auto-allocated with the reported MAC address Replace 0xffffffff with the reported hostid of the LDOM guest. 2. From the control domain create an output LDOM resource list and LDOM services using the following commands: ldm list-bindings <ldom_name> > ldm_list_bindings_<ldom_name>.out ldm list-services > ldm_list-services.out 3. Create a link name indicating the respective LDOM filesystem. As an example, ldom1 is rncardsweb01 cd /ldomroot ln -s ldom1 gdom01 4. Make sure to finalize guest LDOM builds with unbind/bind. This is to guarantee that the resource list gets saved on the system controller. NOTE : LDOM resources can be added and deleted dynamically on the fly. To guarantee safe permanent resource assignments on the controller, a unbind/bind has to occur. 5. Make sure that the file permissions of the disk image for the guest LDOMs has the following permission: chmod 1600 <image_file_name> which should provide the following permission as displayed: -rw------T 1 root root <XXXXXX> <Month_date> bootdisk.img

27

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato 6. It would be best to prepare the ldom guest build script containing the information on how the ldom and the primary services was made available. This would ease on re-creating the LDOM guest and the services. The following files are given as examples: =========================================================== =========================================================== =================================== /root/ldom/scripts/ setup_ldm_services.sh =================================== #!/bin/ksh PATH=/sbin:/bin:/usr/sbin:/usr/bin:/opt/SUNWldm/bin # Configure the OS virtual disks ldm add-vdsdev /ldomroot/ldom1/bootdisk.img ldom1_boot@primary-vds0 # Configure the apps virtual disks ldm add-vdsdev /ldomroot/ldom1/apps.img ldom1_apps@primary-vds0 # Configure the Virtual Switches ldm add-vsw net-dev=nxge3 primary-vsw0 primary ldm add-vsw net-dev=nxge4 primary-vsw1 primary ldm add-vsw net-dev=nxge2 primary-vsw2 primary ldm add-vsw net-dev=nxge6 primary-vsw3 primary =========================================================== =========================================================== =================================== /root/ldom/scripts/ build_ldom1.sh =================================== #!/bin/ksh PATH=/sbin:/bin:/usr/sbin:/usr/bin:/opt/SUNWldm/bin # Add the main virtual disks # ldm add-vdisk vdisk0 ldom1_boot@primary-vds0 ldom1 # Add the virtual disks ldm add-vdisk vdisk1 ldom1_apps@primary-vds0 ldom1 ldm add-vdisk vdisk2 ldom1_logs@primary-vds0 ldom1

28

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato # Add the virtual networks ldm add-vnet vnet0 primary-vsw0 ldom1 ldm add-vnet vnet1 primary-vsw1 ldom1 ldm bind ldom1 ldm unbind ldom1 ldm bind ldom1 ldm start ldom1 =========================================================== =========================================================== T5240 Additional Patches 139555-08.zip 140796-01.zip 140899-01.zip 141016-01.zip A script to transfer the files is as follows: cd /export/PATCHES/LDOM dsh -q $i mkdir /var/tmp/LDOM_PATH for i in 139555-08 140796-01.zip 140899-01.zip 141016-01.zip do scp ${i} ${1}:/var/tmp/LDOM_PATCH done

Manually Primary LDOM creation

1. Create the primary virtual disk server ldm add-vds primary-vds0 primary 2. Create the primary virtual console switch ldm add-vcc port-range=5000-5100 primary-vcc0 primary 3. Create the primary virtual switch with the initial network ldm add-vsw net-dev=nxgeX primary-vsw0 primary 4. Set the MAU/vCPU/memory ldm set-mau 1 primary ldm set-vcpu 4 primary ldm set-memory 1024m primary NOTE: DO NOT set MAU on the primary domain as it turns off LDOM dynamic reconfiguration. 5. Create and use the new configuration ldm add-spconfig initial 6. Reboot 7. Enable the virtual network terminal server daemon

29

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato svcadm enable vntsd 8. Execute the following command: ldm add-spconfig working The above command is to save the current system configuration (both LDOM primary and guests) into the Linux hypervisor running the ldom. **** NOTE **** Always run the command ldm add-spconfig when in doubt to save the current configuration.

Manually Guest LDOM creation

1. Add the guest domain ldm add-domain myldom1 2. Set the guest LDOM vCPU, memory, MAU ldm add-vcpu 12 myldom1 ldm add-memory 1g myldom1 ldm add-mau 1 myldom1 3. Add virtual network ldm add-vnet vnet0 primary-vsw0 myldom1 4. Add virtual disk device ldm add-vdsdev /dev/rdsk/c0t1d0s2 vol1@primary-vds0 5. Add the virtual disk ldm add-vdisk vdisk0 vol1@primary-vds0 myldom1 6. Set the guest LDOM OBP ldm set-variable auto-boot\?=false myldom1 ldm set-variable boot-device=/virtual-devices@100/channel-devices@200/disk@0 myldom1 7. Bind the resources to the guest ldom ldm bind myldom1 8. Start the guest ldom ldm start myldom1 9. From the primary ldom, execute the following command: ldm add-spconfig working The above command is to save the current system configuration (both LDOM primary and guests) into the Linux hypervisor running the ldom. **** NOTE **** Always run the command ldm add-spconfig when in doubt to save the current configuration.

30

Solaris Virtualization (Zones/Containers and LDOMs)

Solaris 10 Virtualization (Zones/Containers and LDOMs) - Building and Maintaining (v1.1) Jeronimo M. Mulato

Appendix 1.
Template for the zone configuration: create -b set zonepath=<absolute path for the zone location> set autoboot=true set scheduling-class=FSS add net set address=<zone IP_Address> set defrouter=<zone default IP Router> set physical=<zone physical device to use> end add rctl set name=zone.cpu-shares set value=(priv=privileged,limit=<zone_cpu_share_assigned_value>,action=none) end add capped-memory set physical=<user assigned value> set swap=<user assigned value> set locked=<user assigned value> end

Appendix 2.
Template for sysidcfg for the local zones: system_local=C timezone=US/Eastern terminal=xterm security_policy=NONE root_password=x!tra123 name_service=NONE network_interface=primary {hostname=<zone_name> netmask=<zone_netmask> protocol_ipv6=no default_route=<zone_default_route>} nfs4_domain=dynamic

31