Running VMs in XEN if HVM support is not there

I have seen this problem a lot many times when people try to run linux VMs on Xen host which does not have Intel VT feature. Once Xen is installed, they are unable to run any VM since by default Xen keeps the virtualization type to HVM.

However you can run VMs on this host by following the following trick:-
You need to get VMs which are Paravirtualized. You can get them by creating them on a host which has VT feature, then making them Paravirtualized and then exporting them from that host and importing on the host which does not have intel VT feature. Using this, you will be able to run Paravirtualized VMs on hosts which does not have Intel VT feature. I have tried this and it worked in my case. Thought of sharing this with larger audience as it might help someone in need. To create paravirtualized Ubuntu VM, see my post link: Creating Paravirtualize Ubuntu VM

Setup FTP Server on Ubuntu and precautions to take

Most of the times I face issue in configuring FTP on Linux platform as a lot many types of FTP servers are available and each one has different properties. In this post, I am going to share info about one such FTP server which is most common and using simple steps, we can configure it. So here it goes.

I am going to discuss about vsftpd. First you need to install it onto your ubuntu box. For that, enter the following command:-

rahul@ubuntu$ sudo apt-get install vsftpd

Once its installed, you can see its conf file in /etc with name vsftpd.conf . There you can specify the access_type ie, whether you want to allow anonymous access or local_access. By default, local access is enabled, so anyone with the username "anonymous" can login to ftp server. To allow only authenticated users to be able to access FTP server, you need to change the access_type to local_access. One can do this by editing the vsftpd.conf file, commenting the "anonymous_enable=NO" and adding "local_enable=YES".

rahul@ubuntu$ sudo vi /etc/vsftpd.conf

Once you are done with the changes, restart the service.

rahul@ubuntu$ sudo /etc/init.d/vsftpd restart

Once its done, are done with the setup of FTP server. Now you need to keep following things in mind:-
You cannot connect to FTP server by using the username as "root" as it might give error.
For that, use username "ftp" or any user which is present on that machine except root.
Use the default credentials of that account to log in.

Default Directory:-
Default directory in this case is the home directory of the user with whose name you have logged in. So if you have a user "rahul" and you have logged in via that username, then the default directory of ftp will be "/home/rahul" . If you will do "ls" once you are connected to that server, it will list all the files present in that directory.

If you want to allow user to write to that directory, then you have to accordingly change the setting in vsftpd.conf file and enable that flag.

Hope this will solve some of the problems which users face due to vsftpd.

Using proxy-server to access Blocked Websites

A proxy server is a server which gets the request from the client, evaluates the request and then sends back the response to the client. For example, if one wants to access a webpage, the request is sent to proxy-server, which reads that page and then sends the contents back to the client.

This comes handy when someone has blocked some sites and you want to access them. In that case, the request to those sites will be first sent to proxy server, who will fetch those results and will then return them back to you.

Recently some of the websites are blocked by the ISP's. If you want to access them, you need to have a proxy server. There is a site called www.proxy.org which lists a whole lot many proxy servers which one can use. Just select any proxy-server you want, based on the country and then it will ask for what address you want to go to. Type the address and you will be redirected to that website through proxy-server.

One thing to note is that don't use proxy-server for sites like facebook or gmail as they require authentication and you would not be able to do that using the proxy-servers.

Creating Ubuntu 12.04 Paravirtualized VM for Xen

Creating an Ubuntu VM:-

First of all, you need to create an Ubuntu VM. For that, there is no default template in XENServer 6.0. Therefore, select the "Other Install Media" template. Make sure that the VM-Name which you write should not contain any spaces. This is because we are going to use a script at later stages and it gives error if vmname contains space.

XenServer 5.5's pygrub(the bootloader for PV) is unable to read Ext4 filesystem. This could be a limitation in XenServer 6.0 as well although I am not sure about it. Therefore to be on the safer side, make sure that your /boot partition uses Ext3 or Ext2 filesystem. Here is one sample partition table which you can make. Make 3 partitions and keep the rest of space unallocated which we can allocate later.

Partition Table:-

Type: /boot              Filesystem:   Ext3/Ext2                Size:    5 GB

Type: /                     Filesystem:   Ext4/Ext3/Ext2        Size:  10 GB

Type: Swap                                                                Size: 2 * Size of RAM

Unallocated Space: XX GB

If you don't want to take advantage of Ext4, then you can format using entire disk and using filesystem type as anything except Ext4.

Once done with this, just install Ubuntu12.04 on the VM. Also install openssh-server on that machine.

rahul@ubuntu$ sudo apt-get install openssh-server

Xen uses hvc0 console. So one needs to change tty1 to hvc0. For that perform the following steps:-

step1:-

rahul@ubuntu$ sudo cp /etc/init/tty1.conf /etc/init/hvc0.conf

Step2:-

Open hvc0.conf and replace all tty1 occurences by hvc0.

rahul@ubuntu$ sudo vi /etc/init/hvc0.conf

That file looked like this on my Ubuntu12.04 VM.

Now there are a fixed set of commands which need to be run to make the VM paravirtualized. Here on this link, they have listed those commands.

Commands to run on Xen's Dom0 to make a VM PV

These commands are used(are present) in the script which we are going to run at later stage. Now it requires some parameters as input which are present in the grub.cfg file. So we will copy that part from grub.cfg and keep it in some text file with us so that we can refer to it at later stages when required.Copy these parameters from VM and keep with you so that its available when the VM is off.

We need to copy the "Default Menu Entry" and copy its entire stuff(kernel and ramdisk name) in a different text file for later use. Here is example of the grub.cfg of my machine "Ubuntu12.04". Please note that the location of kernel and initrd will change in your cases as it depends on which partition your /boot is. So just copy the default menu entry and its whole content till the start of next entry.

Xen modules are not loaded into initramfs. We can correct that by adding the entry in modules file. Enter the following command for that:

rahul@ubuntu$ sudo echo "xen-blkfront" >> /etc/initramfs-tools/modules 

rahul@ubuntu$ sudo update-initramfs -u

Shutdown the VM. Create a snapshot if you want to revert back if some error occurs.

 Now you need to SSH to XenServer ie. to Dom0 machine and configure the properties of DomU machine which you want to paravirtualize.
Perform "xe vm-list" to list all the vm's present on xen host. Identify the name of vm which you want to paravirtualize.

root@xen# xe vm-list

  Now download the script available here.

The above mentioned script contains nothing but the steps which we have pointed out above on the blog with the link to that site.

Change the execution permissions of the script.

root@openstack-xen1# chmod +x makepv.sh

Execute the script. It will ask for vmname. Provide the name of VM which you want to para-virtualize.

root@openstack-xen1# ./makepv.sh <vmname>

Then it will ask for other parameters like kernel path and kernel parameters.
Provide them from the boot_config we have earlier copied and saved in a file. Copy the parameters from that file and provide them here.
If you don't have the script, then manually execute the commands as listed in the blog-post provided above.

Once its done, you need to install xentools onto the VM which you are paravirtualizing.
So connect xen-tools.iso to the ubuntu12.04 vm and power-on the VM.
SSH to the vm and install xen-tools on it.

Steps to install xentools:-

ssh <vm_ip>
sudo mount /dev/cdrom1 /media/cdrom
cd /media/cdrom/Linux
sudo dpkg -i xe-guest-utilities_6.0.0-743_amd64.deb

If you try to access vm-console using xencenter and the default console is hvc0, then you might see the error that you are unable to type anything  there. This is one of the bugs of xencenter. You just need to close xencenter and reopen it again to get it fixed.

Reboot the VM.
You need to change the run-level of the services. For that, enter the following commands:-

ssh <vm ip address>

sudo update-rc.d -f xe-linux-distribution remove
sudo update-rc.d xe-linux-distribution defaults

Reboot the VM.

Now your ubuntu12.04 VM. Your VM is paravirtualized now.

Run xe vm-param-list command and you will see that now it uses pygrub instead of HVM.

Behaviour of FTP incase of mls command

General syntax of mls command is as follows:-

mls <RemoteFile>[…] <LocalFile>

Details can be found on the link: http://technet.microsoft.com/en-us/library/ff687677%28WS.10%29.aspx

Now when we give multiple remote files/folders to be listed, for each file/folder a separate ftp data connection is opened. So by firing a single command, one can see multiple ftp data connections created. 

 

Normally when a ftp command is fired, only one data connection is made. But in case of mls, multiple connections are created and terminated. So if you provide some command with some correct and some wrong paths, then it will list all the correct paths and provide error message in case of wrong paths.

One can get the pcap file containing such packets at the link provided below:-

https://docs.google.com/open?id=0B2XSUdG3GnDmY2M4ZmJhZDgtMGQwYi00NzdmLTg2MzktMTZjMDBjZDI5ZmZh

Diagrammatic representation of PORT and PASV commands in FTP

Hi

Here is the diagrammatic representation of PORT and PASV commands in FTP. Diagram represents the various connections that are opened and port numbers used.

Basics of FTP Protocol

Some short FTP details:-

FTP consists of 2 connections:-

1.    Control Connection on port 21(Standard, can be changed)

2.    Data Connection on port 20(Standard, can be changed)

Control connection is established once, data connection is established everytime data is transferred.

So, for Control connection, source and destination port remains same….

For every Data connection, port no. change. That port no. is negotiated on the control connection. 

So all the FTP commands doesn’t use the Data Connection…

Here is the list of FTP commands. Commands with (√ ) indicates that the command uses data connection and commands with (X) indicates that the command doesn’t use data connection.

 

FTP commands are of 4 types:-

A.   PORT commands (Also known as Active)

B.   PASV commands (Also known as Passive)

C.   EPRT commands (Also known as Extended PORT)

D. EPSV commands (Also known as Extended PASV)

PORT commands:-

        In this, Client tells to server on which client port the data connection will be made. So client sends the port no. with the command. In PORT commands, port 20 is used by server for Data transfer.

PASV commands:-

        Sometimes client might be behind the firewall, for such cases, server can’t directly connect to client and the port negotiated. So for that Passive mode is used. In Passive mode, any random port is used by server for Data connection.

        In this, Server sends the data connection port no. to client and then client connects to that portnumber for data connection.

EPRT and EPSV commands:-

        These are to support FTP in case of IPv6 and NATing. See RFC of FTP for more details.      

Now lets see what traffic flows in the ls command…

Kindly note:-

1.   FTP server IP is: 192.168.10.10

2.   Clients IP is: 192.168.10.11