SlideShare a Scribd company logo
CentOS Dojo 2013




KVM Optimizations and Best Practices
    From the Desktop to the Datacenter


    Jaime Melis (opennebula.org)
Summary
●   Tools
●   Drivers
●   Image Backends
●   CPU
●   Memory
●   Networking
●   Desktop Sharing
●   Cloud point-of-view Challenges
Tools
●   Libvirtd !!!!!!
●   virt-manager
●   virt-install
●   qemu-img
●   virt-alignment-scan (libguestfs)
●   lstopo
●   OpenNebula
Drivers
●   Easy... use virtio!!!
●   Huge performance gain
●   No emulation
●   Cooperation with the hypervisor
●   Integrated into the linux kernel
Image Backends
●   Regular file
●   Qcow2
    –   snapshots (beware of the sync!)
    –   additonal layer, less performance
    –   compressed
●   LVM → block device (san storage)
●   Virt-alignment-scan
●   Cache
    –   Writethrough (host page on, guest disk write cache off)
    –   Writeback (host page on, guest disk write cache on)
        ●   Good overall I/O Performance
    –   None (host page off, guest disk write cache on)
        ●   Good write performance
CPU - pinning
●   NUMA (Non-Uniform Memory Access)
●   Shared cache ~ 15% improvement
●   Bad for different tasks
●   Processor availability
●   virsh capabilities
●   lstopo
CPU - pinning

<cputune>
  <vcpupin   vcpu="0"   cpuset="1-4,^2"/>
  <vcpupin   vcpu="1"   cpuset="0,1"/>
  <vcpupin   vcpu="2"   cpuset="2,3"/>
  <vcpupin   vcpu="3"   cpuset="0,4"/>
  ...
</cputune>
CPU - cgroups
●   Limit, account and
●   Bad for different tasks
●   Processor availability
●   virsh capabilities
●   lstopo
CPU - cgroups
●   Limit, account and isolate resource usage



     <cputune>
       <shares>2048</shares>
       ...
     </cputune>
CPU - model
● Subset of features
● Increase performance

● Nested virtualization

● /usr/share/libvirt/cpu_map.xml




    <cpu match='exact'>
       <model fallback='allow'>core2duo</model>
       <vendor>Intel</vendor>
       <topology sockets='1' cores='2' threads='1'/>
       <feature policy='disable' name='lahf_lm'/>
     </cpu>
Memory - KSM
●   Kernel Samepage Merging
●   Combines memory private pages
●   Very useful for VMs !!!!
●   Increases VM density
●   Enable by default
Networking7
●   MacVTap (direct)... NO!
●   Bridged networking... way to go!
●   Disable STP
●   Optimal configuration
●   NAT (masquerading)
●   iptables
Desktop Sharing
●   VNC
●   SPICE
    –   qlx driver
    –   Redirect printers
    –   Usb (mass-storage)
    –   Audio
Cloud Challenges
●   Virt-manager + LVM + DHCP => 10 hosts
●   Live-migration to any host
●   Image layout – don't waste space
●   Contextualization: install once and deploy
    many
●   Network Isolation
●   Mac/IP spoofing
●   Multi-tenancy
Cloud Challenges - ebtables
●   Isolate
●   Prevent mac spoofing

    # Drop packets that don't match the network's MAC Address
    -s ! <mac_address>/ff:ff:ff:ff:ff:0 -o <tap_device> -j DROP

    # Prevent MAC spoofing
    -s ! <mac_address> -i <tap_device> -j DROP
Cloud Challenges – What storage?
●   No Storage Holy Grail

                        LVM2   iSCSI   Qcow2 + DFS

    Provisioning time   +      ++      +++

    I/O                 ++     ++      +

    Network             ++     +++     +

    Snapshotting        -      -       +

    Livemigration       +      -       +
Thanks for attending!

jmelis - #opennebula @Freenode

More Related Content

What's hot (20)

PDF
Achieving the ultimate performance with KVM
ShapeBlue
 
ODP
Disk Performance Comparison Xen v.s. KVM
nknytk
 
PDF
Achieving the ultimate performance with KVM
StorPool Storage
 
PDF
SiteGround Tech TeamBuilding
Marian Marinov
 
PPTX
Cinder Live Migration and Replication - OpenStack Summit Austin
Ed Balduf
 
ODP
Gluster volume snapshot
Rajesh Joseph
 
ODP
Libvirt and bhyve under FreeBSD
Craig Rodrigues
 
PDF
Control your service resources with systemd
Marian Marinov
 
PDF
Dave Gilbert - KVM and QEMU
Danny Abukalam
 
PDF
OSv at Usenix ATC 2014
Don Marti
 
PDF
XPDS14: libvirt support for libxenlight - James Fehlig, SUSE
The Linux Foundation
 
PDF
XPDS14 - Scaling Xen's Aggregate Storage Performance - Felipe Franciosi, Citrix
The Linux Foundation
 
PDF
KVM tools and enterprise usage
vincentvdk
 
PDF
XPDS14 - OSv - A Modern Semi-POSIX LibraryOS - Glauber Costa, Cloudius Systems
The Linux Foundation
 
PDF
Improve your storage with bcachefs
Marian Marinov
 
PDF
OpenNebulaConf2015 2.02 Backing up your VM’s with Bacula - Alberto García
OpenNebula Project
 
PPTX
Vagrant
Denys Haryachyy
 
PDF
s6196-chris-huybregts-microsoft-new-gpu-virtualization-technologies
Chris Huybregts
 
PDF
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...
OpenNebula Project
 
PDF
OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBIT
OpenNebula Project
 
Achieving the ultimate performance with KVM
ShapeBlue
 
Disk Performance Comparison Xen v.s. KVM
nknytk
 
Achieving the ultimate performance with KVM
StorPool Storage
 
SiteGround Tech TeamBuilding
Marian Marinov
 
Cinder Live Migration and Replication - OpenStack Summit Austin
Ed Balduf
 
Gluster volume snapshot
Rajesh Joseph
 
Libvirt and bhyve under FreeBSD
Craig Rodrigues
 
Control your service resources with systemd
Marian Marinov
 
Dave Gilbert - KVM and QEMU
Danny Abukalam
 
OSv at Usenix ATC 2014
Don Marti
 
XPDS14: libvirt support for libxenlight - James Fehlig, SUSE
The Linux Foundation
 
XPDS14 - Scaling Xen's Aggregate Storage Performance - Felipe Franciosi, Citrix
The Linux Foundation
 
KVM tools and enterprise usage
vincentvdk
 
XPDS14 - OSv - A Modern Semi-POSIX LibraryOS - Glauber Costa, Cloudius Systems
The Linux Foundation
 
Improve your storage with bcachefs
Marian Marinov
 
OpenNebulaConf2015 2.02 Backing up your VM’s with Bacula - Alberto García
OpenNebula Project
 
s6196-chris-huybregts-microsoft-new-gpu-virtualization-technologies
Chris Huybregts
 
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...
OpenNebula Project
 
OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBIT
OpenNebula Project
 

Similar to Kvm optimizations (20)

PDF
Libvirt/KVM Driver Update (Kilo)
Stephen Gordon
 
PDF
Rmll Virtualization As Is Tool 20090707 V1.0
guest72e8c1
 
PDF
RMLL / LSM 2009
Franck_Villaume
 
PDF
VMware compute driver for OpenStack
openstackindia
 
PDF
Virtualization Primer for Java Developers
Richard McDougall
 
PPTX
Xen Project Update LinuxCon Brazil
The Linux Foundation
 
PDF
Aplura virtualization slides
The Linux Foundation
 
PDF
Storage-Performance-Tuning-for-FAST-Virtual-Machines_Fam-Zheng.pdf
aaajjj4
 
PDF
RunningFreeBSDonLinuxKVM
Takeshi HASEGAWA
 
PDF
OpenVZ Linux Containers
Kirill Kolyshkin
 
PDF
S4 xen hypervisor_20080622
Todd Deshane
 
PDF
3. configuring a compute node for nfv
videos
 
PDF
Joyent's Bryan Cantrill: Experiences Porting KVM to SmartOS at KVM Forum, Aug...
Peter Tripp
 
PDF
Experiences porting KVM to SmartOS
bcantrill
 
PDF
LCA 2013 - Baremetal Provisioning with Openstack
Devananda Van Der Veen
 
ODP
UDS 2012 Xen
George Dunlap
 
PPTX
virtualization and hypervisors
Gaurav Suri
 
PDF
Virtualization with KVM (Kernel-based Virtual Machine)
Novell
 
PDF
CCNA17 KVM and CloudStack
ShapeBlue
 
PDF
Advancedperformancetroubleshootingusingesxtop 101110131727-phpapp02
Suresh Kumar
 
Libvirt/KVM Driver Update (Kilo)
Stephen Gordon
 
Rmll Virtualization As Is Tool 20090707 V1.0
guest72e8c1
 
RMLL / LSM 2009
Franck_Villaume
 
VMware compute driver for OpenStack
openstackindia
 
Virtualization Primer for Java Developers
Richard McDougall
 
Xen Project Update LinuxCon Brazil
The Linux Foundation
 
Aplura virtualization slides
The Linux Foundation
 
Storage-Performance-Tuning-for-FAST-Virtual-Machines_Fam-Zheng.pdf
aaajjj4
 
RunningFreeBSDonLinuxKVM
Takeshi HASEGAWA
 
OpenVZ Linux Containers
Kirill Kolyshkin
 
S4 xen hypervisor_20080622
Todd Deshane
 
3. configuring a compute node for nfv
videos
 
Joyent's Bryan Cantrill: Experiences Porting KVM to SmartOS at KVM Forum, Aug...
Peter Tripp
 
Experiences porting KVM to SmartOS
bcantrill
 
LCA 2013 - Baremetal Provisioning with Openstack
Devananda Van Der Veen
 
UDS 2012 Xen
George Dunlap
 
virtualization and hypervisors
Gaurav Suri
 
Virtualization with KVM (Kernel-based Virtual Machine)
Novell
 
CCNA17 KVM and CloudStack
ShapeBlue
 
Advancedperformancetroubleshootingusingesxtop 101110131727-phpapp02
Suresh Kumar
 
Ad

More from OpenNebula Project (20)

PDF
OpenNebulaConf2019 - Welcome and Project Update - Ignacio M. Llorente, Rubén ...
OpenNebula Project
 
PDF
OpenNebulaConf2019 - Building Virtual Environments for Security Analyses of C...
OpenNebula Project
 
PDF
OpenNebulaConf2019 - CORD and Edge computing with OpenNebula - Alfonso Aureli...
OpenNebula Project
 
PDF
OpenNebulaConf2019 - 6 years (+) OpenNebula - Lessons learned - Sebastian Man...
OpenNebula Project
 
PDF
OpenNebulaConf2019 - Performant and Resilient Storage the Open Source & Linux...
OpenNebula Project
 
PDF
OpenNebulaConf2019 - Image Backups in OpenNebula - Momčilo Medić - ITAF
OpenNebula Project
 
PDF
OpenNebulaConf2019 - How We Use GOCA to Manage our OpenNebula Cloud - Jean-Ph...
OpenNebula Project
 
PDF
OpenNebulaConf2019 - Crytek: A Video gaming Edge Implementation "on the shoul...
OpenNebula Project
 
PDF
Replacing vCloud with OpenNebula
OpenNebula Project
 
PDF
NTS: What We Do With OpenNebula - and Why We Do It
OpenNebula Project
 
PDF
OpenNebula from the Perspective of an ISP
OpenNebula Project
 
PDF
NTS CAPTAIN / OpenNebula at Julius Blum GmbH
OpenNebula Project
 
PDF
Performant and Resilient Storage: The Open Source & Linux Way
OpenNebula Project
 
PDF
NetApp Hybrid Cloud with OpenNebula
OpenNebula Project
 
PPTX
NSX with OpenNebula - upcoming 5.10
OpenNebula Project
 
PDF
Security for Private Cloud Environments
OpenNebula Project
 
PDF
CheckPoint R80.30 Installation on OpenNebula
OpenNebula Project
 
PDF
DE-CIX: CloudConnectivity
OpenNebula Project
 
PDF
DDC Demo
OpenNebula Project
 
PDF
Cloud Disaggregation with OpenNebula
OpenNebula Project
 
OpenNebulaConf2019 - Welcome and Project Update - Ignacio M. Llorente, Rubén ...
OpenNebula Project
 
OpenNebulaConf2019 - Building Virtual Environments for Security Analyses of C...
OpenNebula Project
 
OpenNebulaConf2019 - CORD and Edge computing with OpenNebula - Alfonso Aureli...
OpenNebula Project
 
OpenNebulaConf2019 - 6 years (+) OpenNebula - Lessons learned - Sebastian Man...
OpenNebula Project
 
OpenNebulaConf2019 - Performant and Resilient Storage the Open Source & Linux...
OpenNebula Project
 
OpenNebulaConf2019 - Image Backups in OpenNebula - Momčilo Medić - ITAF
OpenNebula Project
 
OpenNebulaConf2019 - How We Use GOCA to Manage our OpenNebula Cloud - Jean-Ph...
OpenNebula Project
 
OpenNebulaConf2019 - Crytek: A Video gaming Edge Implementation "on the shoul...
OpenNebula Project
 
Replacing vCloud with OpenNebula
OpenNebula Project
 
NTS: What We Do With OpenNebula - and Why We Do It
OpenNebula Project
 
OpenNebula from the Perspective of an ISP
OpenNebula Project
 
NTS CAPTAIN / OpenNebula at Julius Blum GmbH
OpenNebula Project
 
Performant and Resilient Storage: The Open Source & Linux Way
OpenNebula Project
 
NetApp Hybrid Cloud with OpenNebula
OpenNebula Project
 
NSX with OpenNebula - upcoming 5.10
OpenNebula Project
 
Security for Private Cloud Environments
OpenNebula Project
 
CheckPoint R80.30 Installation on OpenNebula
OpenNebula Project
 
DE-CIX: CloudConnectivity
OpenNebula Project
 
Cloud Disaggregation with OpenNebula
OpenNebula Project
 
Ad

Kvm optimizations

  • 1. CentOS Dojo 2013 KVM Optimizations and Best Practices From the Desktop to the Datacenter Jaime Melis (opennebula.org)
  • 2. Summary ● Tools ● Drivers ● Image Backends ● CPU ● Memory ● Networking ● Desktop Sharing ● Cloud point-of-view Challenges
  • 3. Tools ● Libvirtd !!!!!! ● virt-manager ● virt-install ● qemu-img ● virt-alignment-scan (libguestfs) ● lstopo ● OpenNebula
  • 4. Drivers ● Easy... use virtio!!! ● Huge performance gain ● No emulation ● Cooperation with the hypervisor ● Integrated into the linux kernel
  • 5. Image Backends ● Regular file ● Qcow2 – snapshots (beware of the sync!) – additonal layer, less performance – compressed ● LVM → block device (san storage) ● Virt-alignment-scan ● Cache – Writethrough (host page on, guest disk write cache off) – Writeback (host page on, guest disk write cache on) ● Good overall I/O Performance – None (host page off, guest disk write cache on) ● Good write performance
  • 6. CPU - pinning ● NUMA (Non-Uniform Memory Access) ● Shared cache ~ 15% improvement ● Bad for different tasks ● Processor availability ● virsh capabilities ● lstopo
  • 7. CPU - pinning <cputune> <vcpupin vcpu="0" cpuset="1-4,^2"/> <vcpupin vcpu="1" cpuset="0,1"/> <vcpupin vcpu="2" cpuset="2,3"/> <vcpupin vcpu="3" cpuset="0,4"/> ... </cputune>
  • 8. CPU - cgroups ● Limit, account and ● Bad for different tasks ● Processor availability ● virsh capabilities ● lstopo
  • 9. CPU - cgroups ● Limit, account and isolate resource usage <cputune> <shares>2048</shares> ... </cputune>
  • 10. CPU - model ● Subset of features ● Increase performance ● Nested virtualization ● /usr/share/libvirt/cpu_map.xml <cpu match='exact'> <model fallback='allow'>core2duo</model> <vendor>Intel</vendor> <topology sockets='1' cores='2' threads='1'/> <feature policy='disable' name='lahf_lm'/> </cpu>
  • 11. Memory - KSM ● Kernel Samepage Merging ● Combines memory private pages ● Very useful for VMs !!!! ● Increases VM density ● Enable by default
  • 12. Networking7 ● MacVTap (direct)... NO! ● Bridged networking... way to go! ● Disable STP ● Optimal configuration ● NAT (masquerading) ● iptables
  • 13. Desktop Sharing ● VNC ● SPICE – qlx driver – Redirect printers – Usb (mass-storage) – Audio
  • 14. Cloud Challenges ● Virt-manager + LVM + DHCP => 10 hosts ● Live-migration to any host ● Image layout – don't waste space ● Contextualization: install once and deploy many ● Network Isolation ● Mac/IP spoofing ● Multi-tenancy
  • 15. Cloud Challenges - ebtables ● Isolate ● Prevent mac spoofing # Drop packets that don't match the network's MAC Address -s ! <mac_address>/ff:ff:ff:ff:ff:0 -o <tap_device> -j DROP # Prevent MAC spoofing -s ! <mac_address> -i <tap_device> -j DROP
  • 16. Cloud Challenges – What storage? ● No Storage Holy Grail LVM2 iSCSI Qcow2 + DFS Provisioning time + ++ +++ I/O ++ ++ + Network ++ +++ + Snapshotting - - + Livemigration + - +
  • 17. Thanks for attending! jmelis - #opennebula @Freenode