Skip to content

XEN_server_configuration

Jump to bottom
Carolina Fernández edited this page Mar 16, 2015 · 1 revision

Home > Configuration > XEN server configuration

This wiki page contains a step-by-setup configuration for a typical XEN server networking setup using multiple linux bridges with VLAN transparent tagging support. This configuration is roughly the one used within OFELIA for the VM service.

##Targeted configuration

XEN server bridges configuration

Example will setup a bridge (eth1.999 in the example) to transparently tag/untag (VLAN 999 in the example), and the rest of the bridges (eth2..ethN in the example) to forward unmodified traffic. VMs in the server, with the appropriate .conf file, can have one interface in each of the bridges.

##Requirements

  • GNU/Linux physical dedicated Server

  • Following packages are required (Debian-based distros):

    apt-get install xen-linux-system-2.6-xen-amd64 vlan python-libvirt libvirt-dev

    or according to server architecture:

    apt-get install xen-linux-system-2.6-xen-686 vlan python-libvirt libvirt-dev

##Procedure

  1. Install packages shown in requirements. Make the following changes

    mv /etc/grub.d/10_linux /etc/grub.d/50_linux
update-grub2
reboot

  2. Edit /etc/modules and add:

    8021q
loop max_loop=64

  3. Tune the maximum virtual memory to be used by XEN (e.g. 1G, 512Mb, etc.), as preferred.

    1. Edit /etc/default/grub to set it:
    # Xen boot parameters for all Xen boots
GRUB_CMDLINE_XEN="dom0_mem=1G,max:1G"
    1. Run update-grub to update the GRUB configuration file with the previous changes.

  4. Edit the XEN file configuration at /etc/xen/xend-config.sxp:

    1. Uncomment the following lines:
    (xend-http-server yes)
(xend-port 8000)
#...
(network-script 'network-multi-bridge-vlan')
    1. Set the minimum virtual memory to be used by XEN (you can take previous step as reference). For instance:
    (dom0-min-mem 1024)

  5. Disable the ballooning so as to avoid undesired reboots when VM requests are attended after the host's virtual memory capacity is exceeded.

    (enable-dom0-ballooning no)

See the procedure in [Xen wiki at Debian](https://wiki.debian.org/Xen/#Configure_Domain_0_Memory).

  1. Copy network-bridge-vlan and network-multi-bridge-vlan to /etc/xen/scripts.

  2. Grant execution permissions to all scripts under /etc/xen/scripts

    chmod +x /etc/xen/scripts/*

  3. Edit /etc/xen/scripts/network-multi-bridge-vlan for your particular setup details.

  • Specify your control (called management in the script) interfaces (eth1.999 in the example).
  • Add your data or experimental interfaces (eth2..ethN in the example).

  1. If some of those dom0 interfaces should have an IP, simply edit /etc/network/interfaces normally (note that renaming on interfaces, e.g. eth1->peth1, is applied after /etc/network/interfaces configuration is set).

  2. [optional] You can disable VM state saving/restore at boot time if you desire, by editing file: /etc/default/domains, and disable XENDOMAINS_SAVE, and put false value to XENDOMAINS_RESTORE

    XENDOMAINS_SAVE=””
XENDOMAINS_RESTORE=false

  3. Reboot

    reboot

##Verification

As root, issue brctl show command:

bridge name         bridge id       STP enabled         interfaces
eth1.999	8000.00259031a9e3	no		peth1.999
							vif<A>.0 # More vif* interfaces may follow below
eth2		8000.00259033b3ff	no		peth2
							vif<A>.1 # More vif* interfaces may follow below
eth3		8000.00259033b3fe	no		peth3
							vif<A>.2 # More vif* interfaces may follow below

####Important notes on brctl show

  • You should see the bridges defined in /etc/xen/scripts/network-multi-bridge-vlan
  • Each bridge shall have as many vif<A>.N interfaces as the number of defined bridges (N = 0..#bridges-1)
  • XEN will generate one physical interface per bridge after system reboot

Something went wrong?

When tmpbridge or pan0 appear, revise the procedure specially concerning permissions in the scripts

  • /etc/xen/scripts/network-multi-bridge-vlan configuration
  • /etc/network/interfaces configuration.

A sample configuration might shed some light on

But nevertheless note that this is just a configuration suited for some testing needs and may not reflect what you need.

/etc/xen/scripts/network-multi-bridge-vlan

Leave this at it is if you don't need to add an specific interface as a bridge other than eth1..3

/etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0
iface eth0 inet dhcp

auto eth1
auto eth2
auto eth3

auto eth1.999 # User experimentation interface
iface eth1.999 inet manual

auto eth1.1000 # Control interface
iface eth1.1000 inet static
        address www.xxx.yyy.zzz # Set your own IP
        netmask www3.xxx3.yyy3.zzz3 # Set your own network mask
        vlan_raw_device eth1
~$ ifconfig -a

After the reboot you may type this command to check how many interfaces are up and its data.

Note the peth* interfaces added by XEN.

eth0      Link encap:Ethernet  HWaddr 00:25:90:31:a9:e2  
          inet addr:aaa.bbb.ccc.ddd  Bcast:aaa2.bbb2.ccc2.ddd2  Mask:aaa3.bbb3.ccc3.ddd3
          inet6 addr: fe80::225:90ff:fe31:a9e2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9360735 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13516181 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2031758077 (1.8 GiB)  TX bytes:19308060049 (17.9 GiB)
          Interrupt:16 Memory:fbce0000-fbd00000 

eth1      Link encap:Ethernet  HWaddr 00:25:90:31:a9:e3  
          inet6 addr: fe80::225:90ff:fe31:a9e3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:59604947 errors:0 dropped:0 overruns:0 frame:0
          TX packets:54030695 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:6285647485 (5.8 GiB)  TX bytes:5648916109 (5.2 GiB)
          Interrupt:17 Memory:fbde0000-fbe00000 

eth2      Link encap:Ethernet  HWaddr 00:25:90:33:b3:ff  
          inet6 addr: fe80::225:90ff:fe33:b3ff/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:979069 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:83604043 (79.7 MiB)  TX bytes:468 (468.0 B)

eth3      Link encap:Ethernet  HWaddr 00:25:90:33:b3:fe  
          inet6 addr: fe80::225:90ff:fe33:b3fe/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:973969 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:82536874 (78.7 MiB)  TX bytes:468 (468.0 B)

eth4      Link encap:Ethernet  HWaddr 00:25:90:33:b3:fd  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:fad60000-fad80000 

eth5      Link encap:Ethernet  HWaddr 00:25:90:33:b3:fc  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:fade0000-fae00000 

eth1.999  Link encap:Ethernet  HWaddr 00:25:90:31:a9:e3  
          inet6 addr: fe80::225:90ff:fe31:a9e3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16456594 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:691796579 (659.7 MiB)  TX bytes:468 (468.0 B)

eth1.1000 Link encap:Ethernet  HWaddr 00:25:90:31:a9:e3  
          inet addr:www.xxx.yyy.zzz  Bcast:www2.xxx2.yyy2.zzz2  Mask:www3.xxx3.yyy3.zzz3
          inet6 addr: fe80::225:90ff:fe31:a9e3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:92008 errors:0 dropped:0 overruns:0 frame:0
          TX packets:48930 errors:0 dropped:3 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:26617770 (25.3 MiB)  TX bytes:8756491 (8.3 MiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:2207 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2207 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:205580 (200.7 KiB)  TX bytes:205580 (200.7 KiB)

peth2     Link encap:Ethernet  HWaddr 00:25:90:33:b3:ff  
          inet6 addr: fe80::225:90ff:fe33:b3ff/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:979068 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:97310918 (92.8 MiB)  TX bytes:936 (936.0 B)
          Memory:fae60000-fae80000 

peth3     Link encap:Ethernet  HWaddr 00:25:90:33:b3:fe  
          inet6 addr: fe80::225:90ff:fe33:b3fe/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:1026888 errors:0 dropped:0 overruns:0 frame:0
          TX packets:105 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:102890458 (98.1 MiB)  TX bytes:15478 (15.1 KiB)
          Memory:faee0000-faf00000 

peth1.999 Link encap:Ethernet  HWaddr 00:25:90:31:a9:e3  
          inet6 addr: fe80::225:90ff:fe31:a9e3/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:59512939 errors:0 dropped:0 overruns:0 frame:0
          TX packets:53939858 errors:0 dropped:3 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:5424560457 (5.0 GiB)  TX bytes:5637142278 (5.2 GiB)

vif<A>.0   Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:43231611 errors:0 dropped:0 overruns:0 frame:0
          TX packets:39090744 errors:0 dropped:15 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:4212713774 (3.9 GiB)  TX bytes:4109200072 (3.8 GiB)

vif<A>.1   Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:61 errors:0 dropped:921208 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:0 (0.0 B)  TX bytes:8112 (7.9 KiB)

vif<A>.2   Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:61 errors:0 dropped:921153 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:0 (0.0 B)  TX bytes:8262 (8.0 KiB)

####Important notes on ifconfig -a

  • There may be several vif<A>.N interfaces, as many as the ones shown at brctl show

If this didn't solve your problem you might be interested in checking networking examples from XEN website.

Clone this wiki locally