Saturday, January 14, 2012

Virt-manager KVM with Ubuntu 11.10 server

After deciding on Virt-manager to manage KVM based virtual machines on my local computer and servers, I started in on the task as follows:

Hardware
Desktop: Linux Mint 11 x64  AMD triple core
Server 1: HP DL365 two dual core, 8 gig ram, raid 1 with 500gb drives
Server 2: AMD dual core desktop with 4 Gig ram, 1 T hard drive

Setup Desktop for KVM


I decided to also prepare my main personal computer to host virtual machines and be controlled by ConVirt 2.0, that way I can move VM's from the server to my PC and vice-versa.  Generally followed: https://help.ubuntu.com/community/KVM/Installation  
  • sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils
    • This installed core packages needed for KVM
  • sudo apt-get install virt-viewer
    • so I can view virtual machine instances on computer outside of ConVirt
  • log out, then back in to affect new user group changes
  • virsh -c qemu:///system list
    • this command shows virtual machines (currently none), but also verifies the install went well
  • sudo apt-get install virt-manager
    • installs GUI tool to manage virtual machines (outside of ConVirt)
    • Works well, setup a 
To Allow Bridging to my Computer, needed to do the following based on:  http://doc.ubuntu.com/ubuntu/serverguide/C/network-configuration.html#bridging
  • gksudo gedit /etc/network/interfaces
    • allows editing of interfaces to define bridge interface
    • Added the following lines to the file
      • auto eth0
      • iface eth0 inet manual
      • #this line was changed from ...inet dhcp

      • auto br0
      • iface br0 inet dhcp
        • bridge_ports eth0
        • bridge_stp off
        • bridge_fd 0
        • bridge_maxwait 0
  • sudo /etc/init.d/networking restart 
  • This is not working yet!!!  found the following:  https://help.ubuntu.com/community/KVM/Networking   *** most helpful article found ****
    • Bridged networking does not work by default, so need to do some further setup
    • sudo apt-get install libcap2-bin
    • sudo setcap cap_net_admin=ei /usr/bin/qemu-system-x86_64   (did not work, hmmm)
    • sudo setcap cap_net_admin=ei /usr/bin/qemu
    • gksudo gedit /etc/security/capability.conf
      • add line
        • cap_net_admin    chenier
  • This was not successful, though bridge br0 is now working properly, the network is not working properly for the main computer.  --- br0 and eth0 are showing same ip address and MAC address. also, the computer says the wired network device is not managed.  Also, running /etc/init.d/networking restart gives message "...is deprecated because it may not enable again some interfaces... *Reconfiguring network interfaces... RTNETLINK answers: No such process; ssh stop/waiting; ssh start/running, process 3083; ssh stop/waiting; ssh start/running, process 3199"
  • Still problems, tried the cap_net 64 command and it did not throw an error this time
  • also changed /etc/network/interfaces line 
    • from..... iface eth0 inet dhcp
    • to....       iface eth0 inet manual
  • Now when rebooting everything works fine, with the exception that the gui networking device manager does not see or control eth0... is this now a function of capnet???  Actually, probably part of the following two points.
  • no prob, the command ifconfig shows proper connetion of br0 and any new VM I create connects to my DHCP server to get its IP address, so it appears all is well.  
  • Also, now ifconfig shows no IP address for my eth0, just for my br0, but I usnderstand that this is normal and that eth0 automatically gets its traffic through br0.
  • I am a bit concerned about the SSH information given when restarting networking (sudo /etc/init.d/networking restart) still gives the messages "ssh stop/waiting; ssh start/running, process 3083; ssh stop/waiting; ssh start/running, process 3199".  Maybe this is part of the RTNETLINK process and is normal.  I will want to verify this on the server install.
  • In this guide: https://help.ubuntu.com/8.04/serverguide/C/libvirt.html it states that the dhcdb daemon will need to be stopped and disabled if used (Desktop installs like mine).  It says to do the following:
    • sudo /etc/init.d/dhcdbd stop
      • this supposedly shuts down the service, but the command was not found on my computer and indeed the file /etc/init.d/dhcdbd is non-existent so apparently my version of Linux Mint uses something else.
  • Alternate configuration of /etc/network/interfaces was found at the bottom of this page: https://help.ubuntu.com/community/BridgingNetworkInterfaces
    • /etc/network/interfaces to read:
      • auto eth1
      • iface eth1 inet manual
      • up ip link set eth1 up
      •  
      • auto br0
      • iface br0 inet manual
        • bridge_ports eth1
        • bridge_fd 0
        • bridge_hello 2
        • bridge_maxage 12
        • bridge_stp off
    • I have not tried this yet, but suppose it will work also, since I currently have not problems, no need to try here.  The point of my eth0 having no separate IP address listed is normal as eth0 traffic routes through br0.
  • THE COMMAND brctl gives access to control and view bridge functions.  Typing brctl directly gives command options and man brctl give more help :)
  • more on bridging: http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge#Does_DHCP_work_over.2Fthrough_a_bridge.3F
  • For advanced bridging info and vlan bridging: http://blog.loftninjas.org/2008/08/11/enterprise-networking-with-kvm-and-libvirt/
    • discusses configuring libvert networking in the following files
      • /etc/libvirt/qemu/networks/default.xml
        • opening this file on my computer shows the network configuration of my virtual bridge virbr0, which I have not yet used for a virtual machine as I am mostly using the standard bridged network
      • /etc/libvirt/qemu/domain.xml
        • on my computer, this is blank
      • Discusses how to define vlan bridges
      • Shows examples of configuration of domain.xml and /etc/newtork/interfaces files for  creating subinterfaces
    • points to libvirt networking documentation: http://libvirt.org/formatnetwork.html


Setup Servers for KVM


Main Help References




Server 1 HP DL365 
  • Server Info:
    • Raid Controller: HP Smart Array E200i, 500 Gb sata drives in bay 2 & 4 mirrored with spare drive in bay 1
    • dual CPU each with dual core 1.8 GHz, 2MB L2 Cache
    • 8 Gb RAM
  • Install Ubuntu 10.10 AMD x64 as usual with the following notable configurations
    • choose software
      • OpenSSH server
      • Virtual Machine host
    • test SSH login from Desktop
      • ssh user@serverip
      • successfully logged in
    • Update and upgrade
      • sudo apt-get update
      • sudo apt-ger upgrade
    • Install a few additional packages
      • sudo apt-get install ubuntu-vm-builder
        • to install custom Ubuntu servers ad-hoc
      • sudo apt-get install libcap2-bin
        • needed for proper bridge networking
      • sudo setcap cap_net_admin=ei /usr/bin/qemu-system-x86_64
        • gives qemu cap net admin inheritability
    • configure cap_net_admin
      • sudo nano /etc/security/capability.conf
        • edit the capability file to give inheritance to cap_net_admin for the listed user by adding the following line
        • cap_net_admin     chenier
    • Change networking to allow bridged networking
      • sudo nano /etc/networking/interfaces
        • modify the interfaces configuration by modifying as follows
          • #primary networking interface
          • auto eth0
          • iface eth0 inet manual
          •  
          • #bridged networking using eth0
          • auto br0
          • iface br0 inet dhcp
            • bridge_ports eth0
            • bridge_stp off
            • bridge_fd 0
            • bridge_maxwait 0
      • sudo /etc/init.d/networking restart
        • this re-initializes the networking interfaces
        • now shows br0 bridge with IP address and same MAC address as eth0 and eth0 without ip address... looks correct
      • ADD ETH1 INTERFACE... not showing up....
    • Add virt-manager so it can be started via ssh -X from a workstation
      • sudo apt-get install virt-manager
      • this is optional as remote access to the server can be obtained from a local copy of virt-viewer on a workstation, however this allows desktops without virt-viewer, but an X windows environment to manage virtual machines
    • reboot to test everything and allow for proper groups to be created (libvirtd)
    • Test that KVM is working properly
      • virsh -c qemu:///system list
      • this shows an output of all running VMs, currently none, but the headers appear
    • Test starting virt-manager remotely.  Run the following from a desktop with X windows installed
      • ssh -X user@server
      • virt-manager
  • I got an error when trying to install a Virtual machine on this host via virt-manager local on my desktop (remote host):
    • After setting up  install parameters and selecting begin install, I got the following message:
      • Unable to complete install: 'Requested operation is not valid: domain is already running'  Traceback (most recent call last):    File "/usr/share/virt-manager/virtManager/asyncjob.py", line 45, in cb_wrapper   callback(asyncjob, *args, **kwargs)  File "/usr/share/virt-manager/virtManager/create.py", line 1633, in do_install      vm.startup()  File "/usr/share/virt-manager/virtManager/domain.py", line 1048, in startup    self._backend.create()  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 330, in create    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)libvirtError: Requested operation is not valid: domain is already running
      • I backed out and discovered that the machine was already running and I was able to open a virt-console instance of it.
    • this error was not a show stopper, just annoying and I see it is documented:  





Server 2: AMD 

  •   error
    • kvm: 1502: cpu0 unhandled rdmsr: 0xc0010001
    • kvm: 1614: cpu0 unhandled rdmsr:









Converting VirtualBox VMs for use by  KVM & Virt-manager
I had a couple VirtualBox based virtual machines I wanted to keep.  Following are the steps I took:

  • Resources: 
  • From Terminal, convert hard drive first to raw format, then to qcow format
    • VBoxManage clonehd --format RAW /directory/of/diskimage/imagename.vdi /new/directory/and/name.img
      • command that converts from VDI format to raw, takes a while
    • qemu-img convert -f raw /directory/ofraw/file/filename.img -O qcow2 /new/directory/filename.qcow
      • command to convert raw image to qcow format, takes a while also with no update in terminal, but you can see the file size grow if you open a file browser
  • Create virtual machine from the image
    • move image to location of disk images on the local machine
      • /var/lib/libvirt/images on my machine
  • THIS STEP NO LONGER NEEDED.... bug was aparently fixed. --  Change hard drive settings - virt-manager hard drive settings need to be changed for the hard drive before it will read it correctly, done as follows
    • virsh -c qemu:///session list --all
    • virsh -c qemu:///system list --all
      • depends on if your virtual machine is in the system account or your user account
      • this shows a list of your VMs
    • virsh -c qemu:///system dumpxml XP2 > ~/Desktop/XP2.xml
      • this placed the config file on my desktop
      • upon inspection, I found that the file correctly identifies the image as qcow2, so this bug must be fixed and my error starting the virtual machine is another problem
    • virsh -c qemu:///system define ~/Desktop/XP2.xml
      • uploads the VM config file
    • Windows XP give me the BSOD upon start right after loading d344bus.sys which seems to be part of daemon tools (virtual CD emulator I use for XP).  Perahaps this program is incompatible with new hardware required for or selected for KVM, but not sure if it is worth messing with at this point.



Various Hints in Virt-Manager Interface

  • Share Desktop Files with Servers via virt-manger (useful for mounting iso images from your desktop)
    • Getting Desktop Shares available via Virt-manager GUI
    • http://www.techotopia.com/index.php/Sharing_Ubuntu_11.04_Folders_with_Remote_Linux_and_UNIX_Systems
    •  on Desktop
      • sudo apt-get istall nfs-kernel-server   =installs nfs server package, which is not installed by default on Linux Mint... go figure
      • Service nfs-kernel-server status   = showed "nfsd running" message ... if not see above document to fix
      • define shares from the /etc/exports file
        • gksudo gedit /etc/exports
        • add share lines as per documentation included in comments in file or via above reference
      • sudo exportfs -a    
      • sudo exportfs
        • returns exported files information (to verify)
      • Firewall configuration changes if needed (not needed in default setup of Linux Mint)
    • on Server
      • log into server
      • sudo apt-get install nfs-common   =installs nfs-common which was not installed on my default server setup

Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)