Friday, May 18, 2012

Ubuntu 12.04 KVM server

To get my main server up to date, decided to push everything to the backup server and start fresh with 12.04 KVM server which I will manage remotely via virt-manager, libvirt, SSH, etc.

Hardware

HP DL365 with Two dual-core AMD Opteron 2x 1.8 GHz processors, 8 Gig ram, Raid hard drives

Base Install

Basically, get the core OS up from newly burned Ubuntu 12.04 Server x64 disk

  • Partitioning: Guided - use entire disk and set up LVM
  • Security updates automatically
  • Select software to install:
    • OpenSSH server
    • Virtual Machine host
  • Reboot
  • logon remotely from terminal on main computer
    • ssh user@ipaddressofserver
  • apply any updates
    • sudo apt-get update
    • sudo apt-get upgrade
  • peg DHCP address on home router so I always have the same address then do the following
    • sudo /etc/init.d/networking restart
Virsh and Virt-manager

Getting Virsh and Virt-manager working
  • Test at this point to see if I can list virtual machines
  • Check for and install missing packages
    • sudo apt-get install ubuntu-vm-builder
    • virsh -c qemu:///system list
      • still error
    • Performed the following to fix group permissions, then signed out and back in
      • sudo adduser [user] kvm (not sure if needed)
      • sudo adduser [user] libvirtd
    • virsh -c qemu:///system list
      • this shows (an empty) list of machines and their state, so everything appears to be in place
    • Install virt-manager locally on the server to allow ssh windowing into it.  This would not be needed for simply accessing from virt-manager installed on another machine.
      • sudo apt-get install virt-manager
Bridge Networking
  • Use the following to show bridges
    • brctl show
      • shows virbr0, but this is not the bridge I need to be connected to eth0
  • setup proper bridged network as follows:
    • sudo nano /etc/network/interfaces
    • change the following lines
      • auto eth0
      • iface eth0 inet dhcp
        • to
      • auto eth0
      • iface eth0 inet manual
    • add the following lines to the file
      • auto br0
      • iface br0 inet dhcp
        • bridge_ports eth0
        • bridge_fd 0
        • bridge_stp off
        • bridge_maxwait 0
  • Save file then test as follows
    • sudo /etc/init.d/networking restart
    • ifconfig
      • shows proper bridge setup as expected

Second NIC
The second physical NIC is not showing up when I run ifconfig and I am unable to configure it.   That is I see eth0, but no eth1.  (answer, was just command it up via: sudo ifconfig eth1 up)

  • run command: sudo lshw
    • it shows first NIC as follows:
 *-network
                description: Ethernet interface
                product: NetXtreme II BCM5708 Gigabit Ethernet
                vendor: Broadcom Corporation
                physical id: 0
                bus info: pci@0000:0a:00.0
                logical name: eth0
                version: 12
                serial: 00:1b:78:ce:e6:88
                size: 100Mbit/s
                capacity: 1Gbit/s
                width: 64 bits
                clock: 66MHz
                capabilities: pcix pm vpd msi bus_master cap_list rom ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
                configuration: autonegotiation=on broadcast=yes driver=bnx2 driverversion=2.1.11 duplex=full firmware=bc 1.9.6 latency=64 link=yes mingnt=64 multicast=yes port=twisted pair speed=100Mbit/s
                resources: irq:49 memory:f8000000-f9ffffff memory:fc000000-fc000
    • Second NIC as:
*-network DISABLED
                description: Ethernet interface
                product: NetXtreme II BCM5708 Gigabit Ethernet
                vendor: Broadcom Corporation
                physical id: 0
                bus info: pci@0000:0c:00.0
                logical name: eth1
                version: 12
                serial: 00:1b:78:ce:e6:8a
                capacity: 1Gbit/s
                width: 64 bits
                clock: 66MHz
                capabilities: pcix pm vpd msi bus_master cap_list rom ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
                configuration: autonegotiation=on broadcast=yes driver=bnx2 driverversion=2.1.11 duplex=half firmware=bc 1.9.6 latency=64 link=no mingnt=64 multicast=yes port=twisted pair
                resources: irq:28 memory:fa000000-fbffffff memory:fc100000-fc1007ff
  • the command: sudo lspci     gives the following lines
0a:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12)
0b:00.0 PCI bridge: Broadcom EPB PCI-Express to PCI-X Bridge (rev c3)
0c:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12)
0d:00.0 PCI bridge: Broadcom EPB PCI-Express to PCI-X Bridge (rev b4)
0e:04.0 PCI bridge: Broadcom BCM5785 [HT1000] PCI/PCI-X Bridge (rev b2)

  • sudo ifconfig eth1 up   ... to try and start eth1
    • success, now eth1 is shown in list of interfaces with ifconfig with it's MAC address, no networking yet, so edit interfaces as follows:
  • Network need for DRBL/Clonezilla server:  need br1 to connect physical nic 2 (eth1) to internal bridge and allow DRBL/Clonezilla server to act as DHCP server.   I believe this requires me to setup eth1 as DHCP client or pegged IP address.... I will first try to setup as DHCP client and see if br1 can get IP address from virtual computer (versus external DHCP server).  If this works it would be option A below, otherwise I would need something like option B, but with a different ip address on the same subnet and no DHCP server stuff. 
  • OPTION A: Setup eth1 and br1 as DHCP client
    , so I will use the br1 to connect DRBL DHCP server to external clients on seperate physical switch connected to nic 2
    • Quick test is to setup eth1 and br1 just as I have eth0 and br0
    • sudo nano /etc/network/interfaces  -- add the following lines
      • # secondary network interface (nic 2)
      • auto eth1
      • iface eth1 inet manual

      • #Bridged network for eth1
      • auto br1
      • iface br1 inet dhcp
        • bridge_ports eth1
        • bridge_fd 0
        • bridge_stp off
        • bridge_maxwait 0
    • sudo /etc/init.d/networking restart  -- to restart interfaces
    • sudo ifconfig  -- to view interfaces
      • looks correct with br1 getting ip address from my DHCP server and eth1 being linked with the same MAC address
  • OPTION B: Setup eth 1 and br1 as DHCP server    -- not doing for now since I now recall DRBL/Clonezilla prefers to have DHCP server setup directly on second interface, so I do not need DHCP service from KVM server
    • I want to have nic 2 of the server be on a natted subnet for purposes such as testing and clonezilla reformatting of computers, so the setup will look a bit different from the primary interface.....
    • sudo nano /etc/network/interfaces     -- modify to add eth1 as second bridged interface so I can us it from virtual machines and connect a switched physical subnet as well -- add the lines:
      • # secondary network interface (nic 2)
      • auto eth1
      • iface eth1 inet manual

      • #Bridged network for eth1
      • auto br1
      • iface br1 inet static
        • address 172.30.50.1
        • network 172.30.50.0
        • netmask 255.255.255.0
        • broadcast 172.30.50.1
        • gateway 172.30.50.1
        • bridge_ports eth1
        • bridge_fd 9
        • bridge_hello 2
        • bridge_maxage 12
        • bridge_stp off
    • sudo /etc/init.d/networking/restart  -- 
      • gives error: Failed to bring up br1
      • RTNETLINK aswers: File exists
    • tried to fix as follows:
      • sudo ifdown eth1
      • sudo /etc/init.d/networking restart (still gives same error)
      • sudo ifup eth1 (states already up)
    • sudo ifconfig  
      • shows that things look as they should:
      • br1 and eth1 have same MAC address
      • br1 has address given, etc
    • OK, so now I need to install DHCP server and define it
      •  (install dhcp3-server)
      • configure /etc/default/dhcp3-server)
Adding Storage 
FreeNas Server setup - Setup NFS shares on freenas server to be shared with KVM servers

  • log into FreeNas web interface using the admin password
  • Go to Services -- NFS
    • Settings
      • set number of servers (depending on maximum concurrent clients)
    • Shares - add new  -- this director for isos of install disks
      • Path = /mnt/store/isosserver (wherever you have the files you want to share)
      • Map all users to root = whatever you need
      • Authorized network = portion of my internal network I want to have access
      • comment = isoserver
      • all dirs = unchecked
      • Read only = what you need
      • Quiet = unchecked
    • Shares - add new   --this directory for existing virtual machine images 
      • Path = /mnt/store/vms
      • comment = virtual machines
      • other settings same as above
    • click on apply changes button !

Virt-manager Settings - adding shared drive

  • Right click a KVM host and select details
  • Select the Storage tab
  • click the plus icon in lower left of window to add a new pool
    • Name = shared-virtual-machines
    • Type = netfs: Network Exported Directory
    • Target Path = keep default = /var/lib/libvirt/images/shared-virtual-machines
    • Format = auto
    • Host Name: [IP address of server]
    • Source Path = /mnt/store/isosserver
  • repeat for /mnt/store/vms


Move VMs from other Servers
I have some virtual machines on older serves I want to move here.  Following are steps I took:







.........


Testing Out

Connecting from Another Machine
  • Do I need to do this???????  I did for 11.04 server
    • on Server
      • log into server
      • sudo apt-get install nfs-common   =installs nfs-common which was not installed on my default server setup
Good References:


Posted by at

1 comment:

  1. Pau10September 18, 2013 at 5:01 PM

    instead of: sudo lshw
    I used this: sudo lshw -class network

    ReplyDelete

Subscribe to: Post Comments (Atom)