Prevention of IP Spoofing in TCP/IP Connection: Vulnerability Analysis and Probable

Solutions. A Reverse Engineering Approach

M.Karthik Narayanan 1
R.Srikanth 2
1

Department of Computer Science and Engineering

2
Department of Information Technology
Sri Venkateswara College of Engineering,
Tamil Nadu, India.
e-mail: karthiknarayanan_m@yahoo.com
e-mail: srikanthr_in@yahoo.com

ABSTRACT
In todays network environment the computer facilities are interconnected by using the Transmission Control
Protocol Internet Protocol [TCP/IP] suite [1]. The TCP specification consists of several vulnerabilities and
an equal number of weaknesses in its implementation. These drawbacks may allow an intruder to break into the
network, and enable him to take over a connection, thus denying service to legitimate user.
This paper tries to analyse the TCP code a REVERSE ENGINEERING technique and identifies several of
these vulnerabilities especially those which are related to TCP state transition diagram. It also discusses many
flaws in the TCP implementation that has occurred in various OS based systems. This paper also discusses TCP
attack IP SPOOFING and recommends steps to improve the security state of a TCP based system.
Keywords:
State Transition, TCP/IP, State Transition Diagrams, IP Spoofing, Timers.

INTRODUCTION
The basis of working of Internet is to provide an approach that allows dissimilar computers to communicate with
one another by hiding the underlying network hardware. The Transmission Control Protocol / Internet Protocol
[TCP/IP] suite is most widely used for this form of communication.
There are some inherent security problems in TCP/IP suite which allows several security breakdowns. TCP
sequence number prediction [3], IP spoofing, use of Internet Control Messaging Protocols [ICMP] for denial of
service are some of the methods of exploiting networks vulnerabilities. As most widely used network
applications like SMTP, telnet, FTP have TCP in their transport layer, security flaws in TCP can be dangerous.
The main objective of this paper is to identify and analyse vulnerabilities in TCP/IP and suggest enhancements
to over come them. This work is based on analysing the state transition diagram of TCP and identifying
improperly defined transition between different states in the state transition diagram of many widely used TCP
code implementations.
A Reverse Engineering technique is employed to analyse the transition information from the TCP code which
is used to locate extraneous state transition present in some implementations of TCP. This process enables us to
identify various sequences of packets that can be hazardous to the security state of the system.

TCP STATE TRANSITION DIAGRAMS AND TIMERS

The TCP state transition diagram governs the setup, establishment and termination of a connection. It consists of
well defined states and transition arcs between these states.

Narayanan, Srikanth (Paper #2)

1st Australian Computer, Network & Information Forensics Conference 2003

25 November 2003, Perth, Western Australia

Page 1

Normal Transition for Client

-------------

Normal Transition for Server

Figure 1: Tcp State Transition Diagrams.

The TCP state transition diagram is closely monitored by a series of timers. There are various timers associated
with connection establishment/termination, flow control or retransmission of data. The timers that are associated
with the area under considerations are:

A connection establishment timer is set when the SYN packet is sent in the connection establishment
phase. If a response is not received with in a particular time period, the connection establishment is
aborted.

A FIN_WAIT_2 timer is set when a when a connection is moved from the FIN_WAIT_1 state to
FIN_WAIT_2 state. This is used to drop a connection when no FIN packet arrives with in a
considerable period of time.

A TIME_WAIT timer is set when a connection enters a TIME_WAIT state. When the timer expires the
kernel data blocks related to that connection is deleted and the connection is terminated.

A keep-alive timer can be set for periodically checks whether the other end of the connection is still
active.

The weakness of the TCP [9] is due to flaws in its state transition diagram and timer operation which are
discussed in detail in the following sections.

Narayanan, Srikanth (Paper #2)

1st Australian Computer, Network & Information Forensics Conference 2003

25 November 2003, Perth, Western Australia

Page 2

ATTACK SCENARIO
IP Spoofing Instances
The concept of attack on TCP/IP such as TCP sequence number guessing was first developed by Morris [3]. The
Computer Emergency and Response Team (CERT) Coordination Centre [5] received reports of attacks in which
intruders created packets with spoofed IP source addresses. The attacks explore applications that use
authentication based on IP addresses. Intruder activity in spoofing IP addresses can lead to unauthorised remote
root access to systems behind a filtering router firewall. An instance of such an attack is the process of breaking
into THE WELL, a private network of some residence of San Francisco [6] [7].
Methodology
Let us assume that there are three hosts, hosts A and B and an intruder controlled host X. Let us assume that B
grants A some special privileges, and thus A can get some actions performed from B. The goal of X is to get that
same action performed from B for itself. In order to achieve this goal, the X has to perform two operations

Establish a forged connection with B.

Prevent A from informing B of any malfunction of Network Monitoring System.

The Host X in order to achieve these objectives has to Spoof the IP address of A in order to make B believe that
the packet being sent from X are actually from Host A. Let us assume that host A and B communicate with each
other by following the 3- way handshake mechanism of TCP/IP. The handshake method [2] is
A B: SYN (Seq. No. = M)
B A: SYN (Seq. No. = N): ACK (Seq. No.=M+1)
A B: ACK (Seq. No. = N+1)
The host X does the following to perform IP Spoofing

It sends a SYN packet to host B with some random number as sequence number, posing as host A. Host
B responds to it by sending a SYN+ACK back to host A with an acknowledgement number equal to the
original sequence number + 1.At the same time the Host B generates it own sequence number and
sends it along with the acknowledgement number. In order to complete the three way handshake, host
X sends an ACK to host B with an acknowledgement number equal to sequence number send by host B
to host A plus 1. If we assume that the host X is not present in the same subnet as A or B, and is not in
a position to get Bs packet, then the Host X has to figure out the sequence number in order to create
the TCP connection. These steps are
X B: SYN (Seq. No. = M), SRC = A
B A: SYN (Seq. No. = N): ACK (Seq. No. =M+1)
X B: ACK (Seq. No. = N+1), SRC = A

At the same time the Host X should take away the host As ability to respond to host B. For achieving
this, the host X may wait for host A to go down or block the protocol part of the Operating system so
that it does not respond back to host B. For example flooding A with incomplete messages.

Once host X establishes a connection with host B, it can send any commands to host B, which will be executed
by B with an assumption that they are being sent by their trusted host A.

PROBLEMS WITH TCP STATE TRANSITIONS

Let us try to consider the above scenario, where the intruder controlled host is able to take away the ability of
host A to respond to host B. The Host X is able to stall the login port of host A by sending a series of SYN
packets but not sending ACK packets corresponding to the SYN_ACK packets from A to X. As already
explained, TCP maintains a connection establishment timer. If a connection does not get established within a
stipulated time, TCP resets the connection.
Extraneous State Transition
Consider a sequence of packets between host X and A. X sends a packet to A with both SYN and FIN flag set.
A responds by sending a ACK packet back to X, as illustrated below
X A: SYN FIN (Seq. No. = m)
A X: ACK (Ack. No. = m+1)
On examining the state transition diagram (shown in the figure 2), we observe that A is initially in state
LISTEN. When it receives a packet from X, it starts processing the packets. There is ambiguity in the TCP
specification regarding the method to handle packets when both SYN and FIN bits are set. In the available
Narayanan, Srikanth (Paper #2)
1st Australian Computer, Network & Information Forensics Conference 2003

25 November 2003, Perth, Western Australia

Page 3

implementations the SYN flag is processed first and then proceeds towards the SYN_RCVD state, and then it
processes the FIN flag and performs a transition to the state CLOSE_WAIT. The only possibility for a normal
transition (CLOSE_WAIT) depends on the previous state to be established. But a transition from SYN_RCVD
state to the CLOSE_WAIT state is not defined in the TCP specification of many operating systems.
Thus contrary to the specification, there exist several TCP implementations a transition arc from the state
SYN_RCVD to the CLOSE_WAIT, as shown in the figure 2.

SECURITY RELEVENCE
In the explained attack scenario, The TCP connection is not fully established as the three way handshake is not
fully complete and thus the corresponding network application never got the connection from the kernel.
However A is in CLOSE_WAIT state and is expecting the application to send a close signal so it can send a FIN
packet to X and terminate the connection. This half open connection remains in the socket listen queue and the
application does not send any message to help TCP perform any state transition. Thus As machine gets struck
in the CLOSE_WAIT state. Suppose the keep-alive timer is enabled, TCP will be able to reset the connection
and perform a transition to the CLOSED state after some time that is specified by the operating system.
Intruder controlled X needs to perform the following steps repeatedly:

X sends a packet to host A with SYN and FIN flag set. A responds with ACK packet. A changes its
state from CLOSED/LISTEN to SYNC_RCVD and then to CLOSE_WAIT.

X does not send any more packets to A, thus preventing any TCP state transition in A.

Host A allocates a memory block to each of the half-open connections and will eventually run out of memory as
a result. It will not be able to respond to unexpected SYN_ACKs from other hosts for a long period of time.
Thus we observe that the extraneous state transitions exist in several implementations of TCP thereby leading to
several security violations of the systems.

Narayanan, Srikanth (Paper #2)

1st Australian Computer, Network & Information Forensics Conference 2003

25 November 2003, Perth, Western Australia

Page 4

Normal Transition for Client

-------------

Normal Transition for Server

Extraneous Transition

Figure 2: Extraneous State Transition In Tcp State Transition Diagram.

PROBLEM WITH TIMERS

As we have discussed before, whenever the process of connection setup is in progress, a connection
establishment timer is turned on. If the connection does not get established within a stipulated time, TCP reverts
back to the CLOSED state.
Simultaneous Open
During the establishment of a simultaneous open connection between the two hosts, it has been found that
connection establishment timer behaves in a different way. Let us consider an example of hosts A and X. Host A
sends a SYN packet to X, expecting a SYN_ACK packet back in response. Let us assume that, almost at the
same time, host X wants to start a connection with A and sends a SYN packet to A, Both A and X send a
SYN_ACK packet to each other when they receive the SYN packet from the other party. When each receives
Narayanan, Srikanth (Paper #2)
1st Australian Computer, Network & Information Forensics Conference 2003

25 November 2003, Perth, Western Australia

Page 5

the SYN_ ACK packet from the other party it assumes that the connection is established. In this example the
connection establishment timer is switched off when the host receives the SYN packet from the other host [4].
X A: SYN (Seq. No. = M)
A X: SYN (Seq. No. = N)
X A: SYN (Seq. No. = M): ACK (Ack. No. =N+1)
A X: SYN (Seq. No. = N): ACK (Ack No. =M+1)
Security Relevence
Let us consider the sequence of steps followed by intruder controlled host X and host A.

The host X sends a connection request to host A. A TCP connection is established between X and A to
transfer control signals. Host A sends a SYN packet to X in order to start a TCP connection for data
transfer and perform a state transition to the state SYN_SEND.

When X receives a SYN packet from A, it sends a SYN packet back in response.

When Host A receives this packet, it assumes that a simultaneous open connection is in progress. It
sends out a SYN_ACK packet to X. At the same time it switches off the timer and makes a state
transition to state SYN_RCVD.

Host X receives the SYN_ACK from A but does not send back any packet. A gets stalled in the state
SYN_RCVD.

Thus X is able to stall a port of the host A which is clear case of denial of service.

OBSERVATIONS
The transition from state SYN_RCVD to the state CLOSE_WAIT can be utilised by any intruder to stall the
TCP machine of the host computer for a particular port. The host being in CLOSE_WAIT state, expects a signal
from the application program, so that it can send a FIN segment and performs a transition to state LAST_ACK.
However the TCP connection has not been established, thus the application is not in a position to send a close
signal to the TCP machine. Thus the TCP machine remains stalled until the keep_alive timer resets the
connection to the CLOSED state. This approach can be effectively utilised by an intruder to start the process of
IP spoofing.
In the simultaneous open setup, the timer is disabled when the host receives an ACK and performs a state
transition to the ESTABLISHED state. This uncustomary close of the timer facilitates the intruder to stall the
TCP machine of the host computer for a particular port. The host is stalled in the state SYN_RCVD which is a
clear case of denial of service.
The transitions from the ESTABLISHED, FIN_WAIT_1 and FIN_WAIT_2 to the closed state occur when the
TCP machine receives a RST segment from the peer host. These transitions are important as they reset the TCP
machine and drop the network connection. However, since the incoming data segment is authenticated only with
respect to source IP address and the current sequence number window, an intruder performing IP spoofing can
pose as one of the hosts of a valid connection already in progress and sent an RST segment with proper
sequence number to the host, thus terminating the connection.

RECOMMENDATIONS
To prevent the spurious state transition from the SYN_RCVD state to the CLOSE_WAIT state, we should
request the OS vendors to modify the relevant part of the source code in their TCP implementation. In other
words, when the TCP machine is in the SYN_RCVD state it should neglect any FIN packets it might receive
from the peer host.
The connection establishment timer should be disabled only when the connection is established. In other words,
during the simultaneous open connection setup, the timer should be disabled when the host as received an ACK
and performed a state transition to state ESTABLISHED. Also for the each state in the state transition diagram
there should be timer escape route. Presently it has been observed that only few states like FIN_WAIT does not
have a timer associated with it. It may be possible for an intruder controlled machine to perform a state transition
to this state , as these states do not have proper timer back-offs. If the intruder does not send proper packets, the
host may get stalled in this state.
There is no easy way to prevent IP spoofing. We may perform this sort of task to prevent such kind of attacks.
First, we may configure the routers and gateways in the network such that they do not allow connections from
outside with the source IP address as same as that of any system with local subnet. Also they should not route
the packets from a host in the local subnet to the outside when the source IP address of the packet is something
Narayanan, Srikanth (Paper #2)
1st Australian Computer, Network & Information Forensics Conference 2003

25 November 2003, Perth, Western Australia

Page 6

not present in the local subnet. Second, we may encrypt the packets before sending it to the network. Although
the process of encryption requires extensive change in the present networking environment, it will ensure
integrity and authenticity of data.

CONCLUSION
The main objective of this paper was to identify and analyse some of the vulnerabilities of TCP/IP. Also it is
anticipated that our article may lead to creation of vendor patches for TCP code to plug security holes. We have
also provided several recommendations to plug some of these defects in TCP and its implementations. In the
future we would like to create a test bed to test our recommendations. We would also like to detect more
vulnerability in TCP, and to investigate automated detection of vulnerabilities in privileged programs using TCP
as their transport layer.

REFERENCES
J.POSTEL, Transmission Control Protocol IETF RFC 793, 1981
S.M.Bellavin, Security Problems In TCP/IP protocol Comp. Commune. Rev., Vol. 19 No. 2, Apr. 1989
R.T.Morris, A Weakness in the 4.2BSD UNIX TP/IP software Comp. Commune. Rev., Vol. 15 No. 5, Apr.
1991
W.R. Stevens AND G.R.Wright, TCP/IP illustrated volume 2.0- the implementation and reading vol. 2.0.
IP Spoofing attacks and hijacked terminal connections, CERT advisory, Jan. 1998.
J.Markoff, Thief took bite out of security, hackers breach show vulnerability of computer networks, the New
York Times Jan. 1995.
T.Shimamura, Technical details of the attack specified by Markoff in NYT usense newsgroup:
comp.protocol.tcp/ip, comp.security.misc Jan. 1995.

COPYRIGHT
[ M.Karthik Narayanan and R.Srikanth ] 2003. The author/s assign the We-B Centre & Edith Cowan
University a non-exclusive license to use this document for personal use provided that the article is used in full
and this copyright statement is reproduced. The authors also grant a non-exclusive license to the We-B Centre &
ECU to publish this document in full in the Conference Proceedings. Such documents may be published on the
World Wide Web, CD-ROM, in printed form, and on mirror sites on the World Wide Web. Any other usage is
prohibited without the express permission of the authors.

Narayanan, Srikanth (Paper #2)

1st Australian Computer, Network & Information Forensics Conference 2003

25 November 2003, Perth, Western Australia

Page 7