Virtualized Openstack install with Fuel on KVMThis document contains instructions that will do a virtualized install of openstack with Fuel on KVM. It allows Fuel to run with all defaults. This procedure is done on Ubuntu 16.04 host. This will work with all versions of Fuel from Fuel 6 on through Fuel 9.
MotivationsThe primary motivation is to gain increased performance by taking advantage of nested virtualization. Based on VirtualBox feature request that has been open for many years, it's safe to conclude that waiting for VirtualBox to get nested virtualization is not an option.
Developing applications on Openstack with out the need for a stack of computers is more efficient. Running a minimal stack of four computers (Fuel, controller, two compute), a switch and some type of NAT device(s) all on physical devices is expensive to purchase, takes lots of space, consumes lots of electricity, requires cooling, and is time consuming to install and operate. This is alot of overhead for a proof of concept or for a one person development environment.
Minimizing the difference between development and deployment reduces errors. "It worked in devstack" but not in production is sometimes a problem.
Configuration needed for default Fuel deploymentThe Fuel network is the network segment that contains 10.20.0.x IP address. The Fuel documentation reefers to this network as the 'Admin' network. I guess it's full name could be Fuel Administration Network. I like using just Fuel Network or Fuel for shortest name. Fuel Network also assumes that Fuel server is at 10.20.0.2. Fuel server performs PXE boot service. In addition, Fuel defaults assumes vlan 100 and 101 for Storage and Management networks respectively on the same network segment.
Fuel needs Network Address Translation(NAT) of 10.20.0.x and 172.16.0.x networks to the public Internet. This is the default behavior for virsh and virt-manager with 192.168.122.x network. virsh appropriately names this network 'default'. We will leverage virsh and virt-manager to do the NAT and Virtual Machine(VM) management of our nodes.
Secret SauceAfter much digging around the net, it became apparent that KVM, Linux Bridge, virsh, and virtio all work together with vlan tagged frames. To test this, run the following works in side of a KVM instance.
To get linux bridge to recognize vlan tagged traffic, we need to explicitly tell it what vlan are allowed. This is done with vconfig command ran on the host.
# vconfig add eth1 100 # ifconfig eth1.100 127.16.0.128 netmask 255.255.255.0
# vconfig add br1 101
br1 will now be able to have untagged traffic and tagged traffic. This enables the Fuel server to use un-tagged traffic for PXE and the tagged traffic for Openstack Management, Openstack Storage and Openstack Neutron controlled tenant networks.
The how to, for installing Fuel in KVM via virt-managerThis is the sequence of steps to get it installed and working. Steps 3, 4 and 5 have more details on how to do each of their respective steps in latter sections.
- Install a host computer with Ubuntu 16.04
- the user stack is on Ubuntu Host
- Host Processor supports virtualization and it's enabled in the BIOS
- virtualization is installed. sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager vlan
- Define two bridges in virsh, br1 and br2
- Add needed vlans to br1
- Create soft allocated files for vm images
- Install Fuel in first vm
- Create nodes, each nodes needs:
- 2 network adaptors set to br1 and br2
- network adaptors use virtio device model
- boot options have NIC for br1 selected. This will pxe boot from br1
- select soft allocated files for VM VirtIO Disk, node-1, node-2 etc...
- With Fuel running, nodes will pxe boot from Fuel Server
Define Network Bridges in virshCreate the following two files:
<network> <name>br1</name> <forward mode='nat'/> <bridge name='br1' stp='on' delay='0'/> <ip address='10.20.0.1' netmask='255.255.255.0'> </ip> </network>
<network> <name>br2</name> <forward mode='nat'/> <bridge name='br2' stp='on' delay='0'/> <ip address='172.16.0.1' netmask='255.255.255.0'> </ip> </network>
virsh net-define br1.xml virsh net-define br2.xml virsh net-start br1 virsh net-start br2
Add vlan to br1
for i in 1 2 100 101 102 103 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 do echo $i vconfig add br1 $i done
Create soft allocated files for vm images
to check file for actual disk usage run:
for i in 0 1 2 3 do truncate -s 512G $HOME/virt_image/node-$i.img done
du -h $HOME/virt_image/node-*
Use virt-manager to create VMrun virt-manager on the command line
do the following in virt-manager
- Configure first node for the fuel
- Only one Nic is needed
- set network interface to use br1
- use a soft allocated file node-0.img for VirtIO Disk
- download Fuel ISO from here or here or here
- Under "IDE CDROM 1" connect the device to the file download above, typically /home/stack/Downloads/*.iso
- Under Boot Options, select IDE CDROM 1 as the Boot device
- start the VM
- install Fuel with all defaults.
- de-select IDE CDROM 1 as the boot device, select only VirtIO Disk 1
- reboot VM
- Configure nodes 1 through N (at least 3)
- each node needs to have two nics. First NIC port is set to br1(with NAT), second NIC port is set to br2(with NAT)
- to use a soft allocated file for VirtIO Disk, select one the files created from above $HOME/virt_image/node-?.img
- Under Boot Options, select NIC that is on br1, this will enable this vm to pxe boot from fuel server.
- start node
- repeat for the desired number of nodes
using the cloudOpen web browser to 10.20.0.2
login as admin password admin
My prior blog post has a detailed instructions on using Fuel web page to configure an environment.
using the cloud from another hostIt's not always convient to get on the host. Port forwarding can be used to gain access to Fuel server VM.
do the following command on the host ubuntu to setup port forwarding to fuel dashboard ( http )
in your web browser, put IP address or hostname of your host in the web browser using the port 8443. i.e 192.168.0.100:8443
# ssh -L 8443:10.20.0.2:8443 email@example.com
for fuel api
# ssh -L 8000:10.20.0.2:8000 firstname.lastname@example.org
# ssh -L 8773:10.20.0.2:8773 email@example.com