Virtual machines and docker containers. General information

In addition to the software EcoRouterOS the third part software can also be launched on the router's platform. Two type virtualization technology are used fro this purpose:

• full QEMU/KVM-based virtualization;
• Docker-based container virtualization.

Full virtualization allows to launch opertion systems and emulate paltforms which are supported by QEMU/KVM. If third-party software runs on Linux and does not require emulation of additional hardware, then a more suitable option will be container virtualization based on one OS.

The virtual machine and container functionality allows to avoid purchasing and supporting additional servers and to deploy software for various network services directly on the router.

Two interaction ways must be distinguished when configuriong virtual machines and containers:

• management of a virtual machine made by external means (creation, launch, stop, deleting);
• configuring the connection of the virtual machine interfaces to the EcoRouter ports, which is done from the EcoRouterOS command line.

Attention! The TCP offload engine must be disabled when using network interfaces with virtio driver since an error in the TCP header checksum calculation occurs currently.

There are two way to disable the TCP offload engine:

1. In OS on the virtual machine execute the following command:

ethtool --offload eth0 tx off

2. In virsh edit the network interface parameters by addyng the following lines:

<host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/>

<guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/>

To do it, follow these steps:

2.1. connect to remote host:

virsh -c qemu+tls://admin@ecorouter/system

2.2. shut down the virtual machine:

shutdown virt_name

2.3. enter the edit mode of xml-configuration file of the machine:

edit virt_name

2.4. add the following lines to the interface section:

<driver>
 <host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/>
 <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/>
</driver>

2.5. save the file and exit;

2.6. restart the virtual machine and check if the options applied:

ethtool -k ifname

Configuring virtual machine interfaces connect to EcoRouter

The EcoRouter provides virtual ports for virtual machines that can be mapped to physical ones, or routable L3 interfaces can be connected to them.

Use the enable container command in configuraton mode to enable the virtual container and machine functionality.

Use the show virtual-network vm command and the show virtual-network container command in administration mode to display existing virtual networks which are used by virtual machines and containers correspondingly.

Use the port virt.<NUM> command in configuration mode to create and configure virtual machine ports, where <NUM> is the virtual port number.

Use the virtual-network vm <IDENTIFIER> command in the context virtual machine port configuration mode to link virtual port to virtual network, where virtual interface identifier from show virtual-network vm command output is used. Use the virtual-network container <IDENTIFIER> context command for containers, where virtual interface identifier from show virtual-network container command output is used.

Use the service-instance <NAME> command in virtual mchine port configuration mode to configure service instances.

The further configuration by means of service instances is similar to conventional ports one (see section "Service Interfaces").

Configuring access of external tools for container management

Containers are managed using external managers which support the docker container clusters API. For example, the standard docker client version 1.12 and higher can be used. Access of external container management tools is possible only through the management port. Authentication and connection security are provided by using TLS and the cluster token.

To manage containers, it is necessary to include EcoRouter in the cluster (also known as "swarm"). Use the virtual-container join-swarm <TOKEN> <IP> <PORT> command in administration mode in the EcoRouter CLI to do this, where:

• <TOKEN> is 85-char cluster token;
• <IP> is manager's IP address;
• <PORT> is manager's TCP-port.

Use the docker swarm join-token worker command to display the needed parameters on cluster manager.

After the router is included in the cluster, further control is performed by the standard commands of the docker client of the swarm mode. TLS-connection is formed automatically and does not require configuration.

Use the no virtual-container join-swarm command in administration mode to exit the cluster.

Virtual disk copying

The EcoRouterOS supports virtual disks copying for virtual machines. Use the copy <ftp | tftp> virtual-disk <АДРЕС> <mgmt | vr default | vr <VR NAME>> command in configuration mode to perform such action.

ecorouter#copy ftp virtual-disk ftp://ftpuser:ftpuser@192.168.255.2:/ubuntu-14.04.qcow2 mgmt 
Download of virtual disk ubuntu-14.04.qcow2 complete   

The modifications of this command for FTP and TFTP servers are shown in the table below.

copy ftp virtual-disk ftp://user:password@xxx.xxx.xxx.xxx/filename mgmt

copy ftp virtual-disk ftp://user:password@xxx.xxx.xxx.xxx/filename vr default

copy tftp virtual-disk tftp://xxx.xxx.xxx.xxx/filename vr vrname

copy tftp virtual-disk tftp://xxx.xxx.xxx.xxx/filename mgmt

Core distribution between virtual routers and data-plane

The EcoROuterOS supports core allocation for virtual machines. The number of allocated cores may be varied from 0 to 4.

Use the hw reserved-cores {0 | 4} command in configuration mode to allocate cores, where 0 means that no cores will be allocated, 4 means that 4 cores will be allocated.

ATTENTION: The result of this command will be available only after saving the configuration and restarting the router.

ecorouter(config)#hw reserved-cores 4

Changes will be applied after reboot. Please save config and reload.

ecorouter(config)#write

ecorouter(config)#reload

reboot system? (y/n): y

As a result after executing the hw reserved-cores command, saving the configuration and rebooting the router for the virtual machines, 4 cores will be allocated.

Use the show platform cpu detail command to display the number of cores allocated for virtual machines.

Connection to virtual machine

Preparing client machine

To connect to the built-in EcoRouter virtualization system QEMU/KVM, a Linux/Unix based client machine must be properly configured in advance. The instruction is made and tested on the basis of the client under CentOS 7.

Install the LibVirt library and OpenSSL which are needed to manage the machine.

yum install libvirt openssl

Install the virt-manager and its dependencies in order to manage the machine with the GUI.

yum install qemu-kvm python-virtinst libvirt libvirt-python virt-manager libguestfs-tools

Use the following command sequence to install GUI in CentOS7.

yum -y groups install "GNOME Desktop" 

startx

Configuring an access of external tools for virtual machine management

The libvirt program is used for virtual machines management. Access to external virtual machine management tools is possible only through the management port. Authentication and connection security are provided by using the TLS protocol and the public key infrastructure (PKI). To obtain the certificate from the CA, the user certificate, and the user private key, see the "Public Key Infrastructure" section. The certificates and the private key must be saved to the files named cacert.pem, clientcert.pem and clientkey.pem, respectively, and put in the directory on the management machine intended for its storage. The example of configuration for Unix/Linux operating systems is shown below.

#mv cacert.pem /etc/pki/CA/

#mv clientcert.pem /etc/pki/libvirt/

#mv clientkey.pem /etc/pki/libvirt/private/

#chmod 444 /etc/pki/CA/cacert.pem

#chmod 440 /etc/pki/libvirt/clientcert.pem /etc/pki/libvirt/private/clientkey.pem

It is also necessary to provide the router domain name permission specified in the certificates Subject: CN = ecorouter certificates. To do this, the DNS system should be used or the name should be registered in the /etc/hosts file.

If previously the host settings on the machine were not executed, the file will look as shown below:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

127.2.2.2   ecorouter

Hypervisor management

The connection to hypervisor can be done from client machine by using management tool supporting libvirt, for example, virsh or virt-manager:

virsh -c qemu+tls://admin@ecorouter/system 

For example, the following command is used to display the virtual processor state of the virtual machine show_debian.

[root@localhost ~]# virsh -c qemu+tls://admin@ecorouter/system vcpuinfo show_debian | grep State

State:          running

A direct access to the desctop or to the command line of the virtual machine is done, for example, using virt-manager or virt-viewer:

$virt-viewer -c qemu://ecorouter/system <VM_name> &

If the graphic shell is used open the Virtual Machine Manager console. Go to the File - Add Connection section, fill the appeared form as shown in the figure below, and click Connect.

Virtual Machines Quick Configuration

To quickly configure virtual machines in EcoRouter the following steps shall be performed.

1. Enable virtual machine support in EcoRouter using the enable vm configuration mode command.

By default, all VMs use the same kernel. In case you need to load a virtual machine with resource-intensive applications, the number of cores can be increased to 4.

To do this, use the configuration mode command hw reserved-cores <N>, where N is the number of cores reserved for virtual machines.

Example:

ecorouter(config)#hw reserved-cores 4       

2. Copy the virtual machine image to EcoRouter using the administration mode command copy {ftp | tftp} virtual-disk.

ecorouter#copy ftp virtual-disk ftp://user:password@xxx.xxx.xxx.xxx/filename

ecorouter#copy tftp virtual-disk tftp://xxx.xxx.xxx.xxx/filename

3. Verify that libvirt and openssl are installed on the local computer from which the virtual machines will be managed.

To connect to virtual machines on EcoRouter, use the virsh command-line utility or the graphical analog virt-manager. The version of virt-manager must be at least 1.3.

4. Export to the local machine user certificates to connect to libvirt on EcoRouter.  An example of export for Linux machines is shown in the table below.

crypto ca export 

/etc/pki/CA/cacert.pem

crypto certificate export 

/etc/pki/libvirt/clientcert.pem

crypto key export 

/etc/pki/libvirt/private/clientkey.pem

All commands specified in the table are entered in the administration mode.

For correct operation it is necessary to set the following access rights to files:

chmod 444 /etc/pki/CA/cacert.pem

chmod 440 /etc/pki/libvirt/clientcert.pem /etc/pki/libvirt/private/clientkey.pem

5. Add an entry in the /etc/hosts file about the EcoRouter IP address with the host name - ecorouter.

6. Connect to libvirt on EcoRouter. In the console for this, enter the command virsh -c qemu+tls://admin@ecorouter/system.

In case you use the graphical shell, you must open the Virtual Machine Manager console. Go to the File – Add Connection, fill out the appeared form, as shown in the figure below, and click Connect.

7. Create a new virtual machine using the hard disk image that was previously copied to the EcoRouter (see step 2).

8. Virtual machines network interfaces must be connected to isolated networks. To create such a network, you must go to the details of connecting to EcoRouter and create a virtual network with the type Isolated virtual network.

9. If necessary, add network interfaces. Each interface connects to one of the previously created virtual networks.

10. In the Display Spice field, in the Address field, select All interfaces.

11. Turn on the machine and make sure that the operating system has loaded on the virtual monitor.

12.  Virtual ports are used to connect the virtual machine to EcoRouter. On the router, you need to create a virtual port using the port virt.0 configuration mode command. This port is attached to one of the virtual networks created through virt-manager. Then the interface of the virtual machine and the virtual port of the router will be connected through a virtual network. After that, you can work with this port as a normal port on the router. For example, you can configure a stream that will connect the real port of the router and virtual at the L2 level, thereby all the virtual machine packets will pass through the real port of the router.

Example:

Configuring the virtual port.

ecorouter#conf t

Enter configuration commands, one per line.  End with CNTL/Z.

ecorouter(config)#port virt.0

ecorouter(config-port-virt)#service-instance virt0

ecorouter(config-service-instance)#encapsulation untagged

ecorouter(config-service-instance)#connect port ge1

Configuring the external EcoRouter port.

ecorouter#conf t

Enter configuration commands, one per line.  End with CNTL/Z.

ecorouter(config)#port ge1

ecorouter(config-port-virt)#service-instance ge1

ecorouter(config-service-instance)#encapsulation untagged

ecorouter(config-service-instance)#connect port virt.0

After these settings, the following entries appear in the router configuration.

ecorouter#show running-config

!

...

!

port ge1

 lacp-priority 32767

 mtu 9728

 service-instance ge1

  encapsulation untagged

!

...

!

port virt.0

 virtual-network vm uplink

 service-instance virt0

  encapsulation untagged

!

...

!

flow port ge1 service-instance ge1 port virt.0

!

flow port virt.0 service-instance virt0 port ge1

!

end

In order to verify the correct configuration of the connection between the external and virtual EcoRouter port, you need to enter the show virtual-network vm administrative mode command.

ecorouter#show virtual-network vm

Virtual network uplink

 bridge virbr1

 port virt.0

13. Next, all the IP addressing settings will be made in the virtual machine.