1
1
Network Traffic Analysis
Using tcpdump
Judy Novak
Judy Novak

Real World Analysis
All material Copyright  Novak, 2000, 2001. All rights reserved.
2
2
Real World Examples
•
Introduction to tcpdump
•
Writing tcpdump Filters
•
Examination of Datagram Fields
•
Beginning Analysis
•
Real World Examples
•
Step by Step Analysis
This page intentionally left blank.
3
3
Objectives
•
Learn advanced analysis through examining real
world examples

Before looking at output from the tcpdump output from the break-in,
let’s examine typical output from a telnet session
boulder.myplace.com.38060 > aspen.myplace.com.telnet: S 3774957990:
3774957990(0) win 8760 <mss 1460> (DF)
aspen.myplace.com.telnet > boulder.myplace.com.38060: S 2009600000:
2009600000(0) ack 3774957991 win 1024 <mss 1460>
boulder.myplace.com.38060 > aspen.myplace.com.telnet:. ack 1 win 8760 (DF)
aspen.myplace.com.telnet > boulder.myplace.com.38060: F 4289:4289(0) ack 92
win 1024
boulder.myplace.com.38060 > aspen.myplace.com.telnet: .ack 4290 win 8760 (DF)
boulder.myplace.com.38060 > aspen.myplace.com.telnet: F 92:92(0) ack 4290
win 8760(DF)
aspen.myplace.com.telnet > boulder.myplace.com.38060: .ack 93 win 1024
boulder.myplace.com.38060 > aspen.myplace.com.telnet: P 1:28(27) ack 1 win 8760
(DF)
aspen.myplace.com.telnet > boulder.myplace.com.38060: P 1:14(13) ack 1 win 1024
aspen.myplace.com.telnet > boulder.myplace.com.38060: P 14:23(9) ack 28 win 1024
A TCP session where connectivity has been established and data has been exchanged has the pattern
of the above records.
Namely:
• The connection is established by:
• The client (boulder) sending a SYN to the server (aspen) to the desired port
• The server (aspen) sending a SYN/ACK back to the client (boulder)
acknowledging the request
• The client (boulder) sending an ACK to the server (aspen) to complete the three-way
handshake
• Data is exchanged:
• The PUSH flag is set for data transmission
• Actual data bytes are sent (this can be seen before the ACK on records with the PUSH set,
you see values in parentheses, i.e. ( 27), (13), (9))

If no RESET/ACK is seen that means that the traffic never actually made it to the inactive port to be
rejected. In other words it was blocked by an exterior router. The sensor for this site resides outside the
filtering router.
This, of course, assumes that there is no backdoor entry to the network/host that the sensor does not see.
It turns out that this incident was a communication problem among the CERT organizations. One
reported it as a scan and somehow several phone calls later, it got reported as a break-in. Because we
had the tcpdump output, we were confident that a break-in did not occur.
Had we not had tcpdump or some other software that had an audit trail of activity, we would not have
any concrete evidence to refute the break-in claim. You should have an IDS or software that is capable
of maintaining a history of activity such as tcpdump.
7
7
Scan of High-Numbered Port
•
Hostile Activity:
•
Scan to TCP port 32775
•
Site router blocked much of activity
•
Internal sensor detected initial SYN activity to
many hosts
Examining tcpdump output for a given hour, a scan appeared of high-numbered port 32775. The
sensor that picked this up was inside the packet-filtering device signaling that some traffic got
through to internal hosts. At this point, all we could see was that there were initial SYN connections
attempted. We would need to further investigate to discover if there were any responses to these
connections.
8
8
Network View of Activity

16:46:43.276139 scanner.net.2097 > 192.168.5.132.32775: S
2950485912:2950485912(0) win 32120 (DF)
This is the type of traffic that appeared on the sensor. Many connections were allowed through the
filtering router destined for TCP port 32775. It can be surmised that the scanner actually probed the
entire class B 192.168 subnet, but since our vantage point is from the inside, we can only guess this.
We do know that all subnets and hosts that had access rules that allowed access to ports greater than
1023 were scanned. That is why it is suspected that the entire class B network was probed. Either
that or the scanner did some prior reconnaissance and knew that there were some subnets that
allowed traffic through to high-numbered ports.
TCP port 32775 doesn’t have a well-known service associated with it, but it is in the range of ports
considered to be Remote Procedure Call (RPC) services. This range is 32770 – 34000 both TCP and
UDP. So, we begin to surmise that the prober is looking for some sort of RPC service being offered
– either one that we know is vulnerable or an entirely new exploit that has yet to be reported.
10
10
Conventional inetd Services
telnet server x
listens on
port 23
telnet server q
listens on
port 23
telnet server z
listens on
port 23
A moment of digression to explain the difference in the way conventional internet services started by
the Unix inetd daemon are identified versus the way RPC services must be identified. A server can
start any given internet service on any port, but convention dictates that if we want to offer a
universal service, we must listen on a standard well-known port. For telnet, this is port 23. Almost
all servers offering the telnet service will listen on port 23 so the clients will know where to find the

•
Uses TCP port 111 to query
•
Prudent to block this port on packet-filtering device
•
For each RPC service running, query returns:
•
RPC service number
•
Port number
•
Service name
The Unix portmapper is the program that manages RPC services on a host. It registers each RPC
service as it starts and unregisters it when it stops. It can be queried to list all the RPC services a
host is running. So, a malicious user trying to exploit a vulnerable RPC service is most likely to
attempt to query the portmapper of a host as reconnaissance to discover the port at which the service
is listening. After gleaning that information, the attack usually takes place.
The problem with this approach is that most sites except those that are totally security unaware will
block TCP port 111 which is the port at which the portmapper listens for queries. If this is done, the
only other recourse for the hacker is to scan the ports in the RPC range to see if hosts are listening.
13
13
Sample rpcinfo Output
program vers proto port
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
100232 10 tcp 32775 sadmind
100011 1 udp 32773 rquotad
100002 2 udp 32772 rusersd
100024 1 udp 32774 status

16:47:01.242766 host4.net.32775 > scanner.net.2490: S 70452835:70452835(0)
ack 2961275373 win 9576 <nop,nop,timestamp 72210076
262468250,mss[|tcp]> (DF)
Back to the original scan! If the portmapper port TCP port 111 is blocked by the packet filtering
device, it might still be possible to scan RPC ports (32770 – 34000) directly. This won’t tell you
what RPC service is running, it will just tell you that something is listening in this range of ports and
it is most likely some RPC server. Additional scanning can be done to determine exactly what RPC
service is running on a responding port.
Since the scan made it through the packet filtering device, the concern then became – did any host
respond to the 32775 probe. If none responded, we could just dismiss this as our good fortune and a
relatively harmless scan. But if any answered, we have more investigation to do.
Unfortunately, many hosts responded with a SYN/ACK indicating that they listened with some RPC
service on port 32775. We don’t know exactly what service each host listened for at this point, we
only know that something was listening. The next step after seeing this was to look at the hosts that
responded and see if they pushed (most often the P flag will be set) data from the probed host to the
scanner or vice versa. Fortunately, this did not occur. This indicates that the scanner most likely
was doing reconnaissance only at this phase. Exchanged data would elevate the level of concern
because this means the scanned host either divulged some information or suffered some indignity at
the hands of the prober.
15
15
What RPC Services Were
Running on Port 32775?
rpcinfo -p hostname | grep "tcp 32775"
+ rpcinfo -p host1.net | grep "tcp 32775"
100083
1 tcp 32775
+ rpcinfo -p host20.net| grep "tcp 32775"
100083
1 tcp 32775

Very dangerous, very widespread
The CERT advisory can be found at:

Here is an excerpt from this advisory:
“The ToolTalk service allows independently developed applications to communicate with each other
by exchanging ToolTalk messages. Using ToolTalk, applications can create open protocols which
allow different programs to be interchanged, and new programs to be plugged into the system with
minimal reconfiguration.
The ToolTalk database server (rpc.ttdbserverd) is an ONC RPC service which manages objects
needed for the operation of the ToolTalk service. ToolTalk-enabled processes communicate with
each other using RPC calls to this program, which runs on each ToolTalk-enabled host. This
program is a standard component of the ToolTalk system, which ships as a standard component of
many commercial Unix operating systems. The ToolTalk database server runs as root.
Due to an implementation fault in rpc.ttdbserverd, it is possible for a malicious remote client to
formulate an RPC message that will cause the server to overflow an automatic variable on the stack.
By overwriting activation records stored on the stack, it is possible to force a transfer of control into
arbitrary instructions provided by the attacker in the RPC message, and thus gain total control of the
server process.”
17
17
Recommended Immediate
Action
32775
We cannot be absolutely sure that the scanner was trolling for reconnaissance about hosts that might
be running ToolTalk. But, we see enough evidence on this network that if the scanner returns to do
some damage, it is very possible that we have unpatched versions of ToolTalk that might be
susceptible to this exploit. There could be other RPC services running on other hosts that have
exploits associated with them such as sadmind.
The immediate reaction was to block this port. Granted, this is a stop-gap solution for this particular
event. The hacker can return and probe other RPC ports and probably find other ports that listen and

What is this curious one byte?
•
Examine records in hex and using tcpshow
•
What scripts can we use to check out this one
byte?
It was fairly unusual to see UDP traffic that purported to be sending only one byte of UDP data.
This traffic was examined further to gain some insight into what that one byte of data might be.
A fair question might be “what made the analyst see or examine this traffic?”. Basically, curiosity
was the motivation. This means that the analyst had the “luxury” of being curious. The point is if
analysts are overloaded, they will not have the time or mindset to be curious. If analysts operate in a
chaotic environment where phones are continually ringing, deadlines are bearing down on them,
they’ve got multiple sites, and a lot of traffic to evaluate, they may not have an atmosphere that
nurtures curiosity.
20
20
Examining Port 31789
Records in Hex
•
Execute following command:
tcpdump –r tcpdumpfile –x ‘dst port 31789’
12:13:10.570000 1.2.3.4.31790 > 192.168.21.63.31789: udp 1
4500 001d a932 0000 7311 10cf 0102 0304
c0a4 153f 7c2e 7c2d 0009 38b1 413b 0000
0001 0000 0000 0000 0356 4752 0341
12:13:10.570000 1.2.3.4.31790 > 192.168.21.90.31789: udp 1
4500 001d c432 0000 7311 f5b3 0102 0304
c0a4 155a 7c2e 7c2d 0009 3896 41cc 8000
0001 0000 0002 0002 0236 3003 3132
This will dump tcpdump data in hexadecimal that contains a particular pattern, in this case

23
23
Examining the tcpshow Output
Packet 1
IP Header
Version: 4
Header Length: 20 bytes
Service Type: 0x00
Datagram Length: 29
bytes
Identification: 0xA932
Flags: MF=off, DF=off
Fragment Offset: 0
TTL: 115
Encapsulated Protocol: UDP
Header Checksum: 0x10CF
Source IP Address: 1.2.3.4
Destination IP Address: 192.169.21.63
The above tcpshow interpretation of the IP header shows nothing unusual. Note the 29 byte datagram
length.
Packet 1
IP Header
Version: 4
Header Length: 20 bytes
Service Type: 0x00
Datagram Length: 29 bytes
Identification: 0xA932
Flags: MF=off, DF=off
Fragment Offset: 0
TTL: 115

command doesn’t know how to interpret.
Sometimes, as you can see, the hex output is more accurate because it reveals the actual bytes.
tcpshow doesn’t always interpret unconventionally crafted traffic correctly.
What this mutant packet shows is that the UDP header was probably crafted. Again, the reason is
unknown; it is worthwhile noting that this probably is not for benevolent purposes. It could be an
attempted denial of service against a particular operating system that will not be able to handle this
packet. Or, it can be an attempt to send a covert message – one that is not to be discovered by
anyone other than the intended recipient.
25
25
Examining The Length Values
4500 001d a932 0000 7311 10cf 0102 0304
IP HEADER
c0a4 153f 7c2e 7c2d 0009 38b1 413b 0000
UDP HEADER UDP DATA
0001 0000 0000 0000 0356 4752 0341
IP datagram total length UDP length
The IP datagram total length agrees with the crafted UDP header
because it is 29 bytes (20 IP header + 8 UDP header + 1 UDP data).
The UDP length has been crafted to be 9 bytes (8 UDP header + 1
UDP data).
This is the hex record that was dumped before. The focus now becomes the fields that contain length
values.
It turns out that the minimum size of an IP datagram over Ethernet is 46 bytes which is the actual
length of this IP datagram. However, a normal datagram that had one byte of UDP data would be 29
bytes (20 IP header + 8 UDP header + 1 UDP data) and would require 17 bytes of padding. The
padding would be zeroes. For instance, in the above packet, we should see:
4500 001d a932 0000 7311 10cf 0102 0304
c0a4 153f 7c2e 7c2d 0009 38b1 4100 0000
0000 0000 0000 0000 0000 0000 0000