Allowing FTP through Firewall by Inspecting traffic

Problem statement:-

FTP traffic needs to be allowed through the firewall. Basic details about FTP can be found here. This cannot be done by simply allowing the traffic for control connection ( by default port 21 ) and not taking care for the data-connection. For this, one needs to analyze the FTP control path (control connection) data to get the details about the data-path (data connection). Once the data-path details are known, one just needs to create embryonic flow entries for the data-connection.

Embryonic flow: A flow entry created for the traffic-flow before there is actual traffic flowing which matches the flow specifications.

flowID: This is a five-tuple field to differentiate the various connections.

Control path activities for a particular FTP request are as follows :-

For Active Mode

For Passive Mode 

  

Out of these 8 packets on control-path, ignoring ACK, we have ‘Five’ generic packets in our flow. We can list these requests/responses in terms of states.

These states are:-
1.    Opening
2.    Opened
3.    command_sent
4.    execution_started
5.    execution_complete

One needs to keep track of FTP control connection’s state and the data-connection details negotiated. This can be done by storing the details in a table called “flow table”.

Flow table column’s are as follows :-
1.    Command Type
2.    IP Address exchanged for data-connection( required for creating embryonic flow )
3.    Port number exchanged for data-connection( required for creating embryonic flow )
4.    Response_code of the command
5.    State of the FTP connection

When a new FTP connection is initiated (is in opening state), one entry is made in the flow table. It then goes through all the states; when it reaches to the state of execution_complete, that entry is removed.

STATE TRANSITIONS FOR PORT COMMANDS:-

Now lets consider the case for PORT command. Five request/response actions occur in this. These are:-

( Client[XYZ] to Server[FTP 21] ) – FTP Request: PORT          -> OPENING
( Server[FTP 21] to Client[XYZ] ) – Response: 200 PORT      ->  OPENED
( Client[XYZ] to Server[FTP 21] ) – FTP Request: <CMD>    ->  CMD_SENT
( Server[FTP 21] to Client[XYZ] ) – Response: 150                ->  EXEC_STARTED
( Server[FTP 21] to Client[XYZ] ) – Response: 226                ->  EXEC_COMP

Steps followed for each of the above requests/responses:-

Packet1:-

First packet sent is the packet with PORT command from client to server.

 

When this packet is received, a flow-entry is made and IP_Address and PORT negotiated are calculated from the message. Also the state is changed to OPENING.

Packet 2:-

Second packet is the response to the PORT command.

Once it is received, Response code is calculated from the message. If  the response-code is positive ( like 200), a flow-entry is present in the flow_table for this packet’s flowID and state is OPENING, then the state is changed from OPENING to OPENED.

Packet 3:-

On the next packet if flow table has a entry for its flowID and is in the OPENED state with the direction from Client to server, then it’s a request from Client to server to perform a specified operation.

After receiving this packet, the state is now changed from OPENED to CMD_SENT.

In case of PORT commands, TCP 3-way handshake for the data-connection occurs after this packet is received by the server. Therefore, embryonic-flow-entry is made in the allowed-connections table on the firewall so that this data-connection traffic is not blocked by the firewall.

Packet4:-

Next packet for the particular flow is the response from the server. Based on the response code received, the state will be changed.

If the response from the server is positive preliminary reply and the state for the particular flow-entry is CMD_SENT, then the state is changed to EXEC_STARTED. If the response is negative, then the flow-entry is deleted, embryonic-flow-entry is also deleted.

Packet5:-

This is the last packet received indicating the status of the FTP command.

If the response code is ‘226’, this means the transfer has occurred successfully. If the previous state was EXEC_STARTED, the state is now changed to EXEC_COMP and the embryonic-flow-entry and flow-entry can be deleted from the respective tables. If some negative reply is received, even then the embryonic-flow-entry and the flow-entry are deleted from the tables.

STATE TRANSITIONS FOR PASV COMMANDS:-

For passive commands, it again goes through the five states mentioned below:-
( Client[XYZ] to Server[FTP 21] ) – FTP Request: PASV                   ->   OPENING
( Server[FTP 21] to Client[XYZ] ) – Response: 227 (port_no)         ->   OPENED
( Client[XYZ] to Server[FTP 21] ) – FTP Request: <CMD>             ->   CMD_SENT
( Server[FTP 21] to Client[XYZ] ) – Response: 150                         ->   EXEC_STARTED
( Server[FTP 21] to Client[XYZ] ) – Response: 226                         ->   EXEC_COMP

Steps followed for each of the above requests/responses:-

Packet1:-

First packet sent is the packet with PASV command.

Once it is received, a flow-entry is made in the flow_table. Also the state is changed to OPENING.

Packet 2:-

Second packet is the response to the PASV command. If a flow-entry is present in the flow_table for the packet’s flowID and its state is OPENING and the direction is from server to client, then its a response from the server.

Once it is received, Response code and PORT negotiated is calculated. According to the response code received, a particular action is taken. Suppose if its 227, then the state is changed from OPENING to OPENED.

In case of PASV commands, TCP 3-way handshake on the data-connection occurs after this packet is received by the client. Therefore, an embryonic flow-entry is made in the firewall’s allowed-connections table with the data-connection details.

Packet 3:-

If the next packet for the particular flowID is from Client to server and state is OPENED, then it is a request from Client to server to perform some operation.

After receiving this packet, the state is now changed from OPENED to CMD_SENT.

Packet4:-

Next packet for the particular flowID would be response from the server.

If the response from the server is positive and state is CMD_SENT, then the state is changed from CMD_SENT to EXEC_STARTED. If the response is negative, then the flow-entry and embryonic-flow-entry are deleted  from the respective tables.

Packet5:-

This is the last packet received indicating the status of the FTP command.

If the response code is ‘226’ and state is EXEC_STARTED, this means the transfer has occurred successfully, state is changed to EXEC_COMP and the flow-entry and embryonic-flow-entry can now be deleted from the respective tables. In case of negative replies also, the same action is taken.

Conclusion:-

This approach can be followed in all the cases where you have the data-path created dynamically and its information is negotiated on the control-path. Above example listed how this can be applied to FTP. Similarly, one can use it for RSH, MSRPC with different set of columns in flow_table( as per the protocol needs).

 

How FTP works

FTP stands for File Transfer Protocol which is used to transfer files from one host to another over TCP. RFC 959 lists all the details of FTP. It is an application layer protocol and is used widely across the world for transferring files.

FTP connections (what we read and know):-

FTP consists of 2 connections:-

1.    Control Connection on port 21 of server (Standard, can be changed)

2.    Data Connection on port 20 of server (Standard, can be changed)

Control connection is established between port 21 of server and random port of client. Once opened, this connection remains there unless closed by user or idle timeout occurs.

Data connection is established every time client sends a request using control connection to server for data transfer. For every new Data connection, port number changes. The new port number is negotiated on the control connection. 

NOTE:-

Not all the FTP commands use the Data Connection…

Here is the list of FTP commands. Commands with (√ ) indicates that the command uses data connection and commands with (X) indicates that the command doesn’t use data connection.

 

Looking at the above diagram, one could see that of all the commands available for FTP, ratio of commands which use data connection to all commands is very less.

FTP data transfer modes are of 4 types:-

·        1. Active Mode (Also known as PORT based)

·        2. Passive Mode (Also known as PASV based)

·        3. Extended active mode to support NAT (EPRT)

·        4. Extended passive mode to support NAT (EPSV)

Active Mode:-
In this, client tells to the server on which client-port the data connection will be made. So client sends the port number with the command to the server. In PORT commands, port 20 is used by server for Data transfer.

Passive Mode:-

Sometimes client might be behind the firewall. In such cases, server cannot directly connect to the client on the negotiated port. So for that Passive mode is used. In Passive mode, any random port is used by server for Data connection.

        

In this, Server sends the data connection port no. to client and then client connects to that portnumber for data connection.

EPRT and EPSV modes:-
These modes are to support FTP in case of IPv6 and NATing. More details about them can be found in FTP’s RFC document.      

Traffic flow examples for PORT and PASV modes:-

Now we will try to see what traffic flows in case of FTP connection.

Server’s IP: 192.168.10.10

Client’s IP:  192.168.10.11

Now let us see what traffic flows when some request is sent by FTP-client to FTP-server which uses data-connection for both Active and passive modes. Thing to note here is that control connection is already established between client and server. We will discuss what happens after that.

­For Active mode:-

Once the user enters the FTP request, ftp client needs to create a data connection to get the output from the server. It first needs to negotiate the port. So it sends the command “PORT” with client machine’s IP and port to which the data connection should be made.

As seen from the above diagram, client sends request to server and server responds back. Port is successfully negotiated between them.

Next client sends the FTP request on the control channel. Now server performs 3-way handshake with the data port negotiated and then responds back with the output on the data channel. Once the data-transfer is complete, the data-connection is terminated.

For each new FTP request, a new data-connection is created.

For Passive mode:-

In passive mode, the client first informs server that it wants to open a passive mode data-connection with the server. It sends “PASV” command to server on control channel. Server responds back by sending the port to which it will open the data connection.

As we can see from the above diagram, server responds back with the IP and port number. 3-way handshake occurs at this time for the data-connection.

Next client sends the FTP request on the control-connection and server sends back the output on the data-connection. Once the data-transfer is complete for that request, the data-connection is terminated.

Conclusion:-

Considering the active and passive modes, we can re-design the basic diagram of FTP to more advanced and appropriate one as follows:-

This diagram now appropriately lists about the FTP and the various connections in FTP.

Setup FTP Server on Ubuntu and precautions to take

Most of the times I face issue in configuring FTP on Linux platform as a lot many types of FTP servers are available and each one has different properties. In this post, I am going to share info about one such FTP server which is most common and using simple steps, we can configure it. So here it goes.

I am going to discuss about vsftpd. First you need to install it onto your ubuntu box. For that, enter the following command:-

rahul@ubuntu$ sudo apt-get install vsftpd

Once its installed, you can see its conf file in /etc with name vsftpd.conf . There you can specify the access_type ie, whether you want to allow anonymous access or local_access. By default, local access is enabled, so anyone with the username "anonymous" can login to ftp server. To allow only authenticated users to be able to access FTP server, you need to change the access_type to local_access. One can do this by editing the vsftpd.conf file, commenting the "anonymous_enable=NO" and adding "local_enable=YES".

rahul@ubuntu$ sudo vi /etc/vsftpd.conf

Once you are done with the changes, restart the service.

rahul@ubuntu$ sudo /etc/init.d/vsftpd restart

Once its done, are done with the setup of FTP server. Now you need to keep following things in mind:-
You cannot connect to FTP server by using the username as "root" as it might give error.
For that, use username "ftp" or any user which is present on that machine except root.
Use the default credentials of that account to log in.

Default Directory:-
Default directory in this case is the home directory of the user with whose name you have logged in. So if you have a user "rahul" and you have logged in via that username, then the default directory of ftp will be "/home/rahul" . If you will do "ls" once you are connected to that server, it will list all the files present in that directory.

If you want to allow user to write to that directory, then you have to accordingly change the setting in vsftpd.conf file and enable that flag.

Hope this will solve some of the problems which users face due to vsftpd.

Behaviour of FTP incase of mls command

General syntax of mls command is as follows:-

mls <RemoteFile>[…] <LocalFile>

Details can be found on the link: http://technet.microsoft.com/en-us/library/ff687677%28WS.10%29.aspx

Now when we give multiple remote files/folders to be listed, for each file/folder a separate ftp data connection is opened. So by firing a single command, one can see multiple ftp data connections created. 

 

Normally when a ftp command is fired, only one data connection is made. But in case of mls, multiple connections are created and terminated. So if you provide some command with some correct and some wrong paths, then it will list all the correct paths and provide error message in case of wrong paths.

One can get the pcap file containing such packets at the link provided below:-

https://docs.google.com/open?id=0B2XSUdG3GnDmY2M4ZmJhZDgtMGQwYi00NzdmLTg2MzktMTZjMDBjZDI5ZmZh

Diagrammatic representation of PORT and PASV commands in FTP

Hi

Here is the diagrammatic representation of PORT and PASV commands in FTP. Diagram represents the various connections that are opened and port numbers used.

Basics of FTP Protocol

Some short FTP details:-

FTP consists of 2 connections:-

1.    Control Connection on port 21(Standard, can be changed)

2.    Data Connection on port 20(Standard, can be changed)

Control connection is established once, data connection is established everytime data is transferred.

So, for Control connection, source and destination port remains same….

For every Data connection, port no. change. That port no. is negotiated on the control connection. 

So all the FTP commands doesn’t use the Data Connection…

Here is the list of FTP commands. Commands with (√ ) indicates that the command uses data connection and commands with (X) indicates that the command doesn’t use data connection.

 

FTP commands are of 4 types:-

A.   PORT commands (Also known as Active)

B.   PASV commands (Also known as Passive)

C.   EPRT commands (Also known as Extended PORT)

D. EPSV commands (Also known as Extended PASV)

PORT commands:-

        In this, Client tells to server on which client port the data connection will be made. So client sends the port no. with the command. In PORT commands, port 20 is used by server for Data transfer.

PASV commands:-

        Sometimes client might be behind the firewall, for such cases, server can’t directly connect to client and the port negotiated. So for that Passive mode is used. In Passive mode, any random port is used by server for Data connection.

        In this, Server sends the data connection port no. to client and then client connects to that portnumber for data connection.

EPRT and EPSV commands:-

        These are to support FTP in case of IPv6 and NATing. See RFC of FTP for more details.      

Now lets see what traffic flows in the ls command…

Kindly note:-

1.   FTP server IP is: 192.168.10.10

2.   Clients IP is: 192.168.10.11