Experiences in Design and Implementation
of a High Performance Transport Protocol
Yunhong Gu, Xinwei Hong, and Robert L. Grossman
National Center for Data Mining

Outline
•
TCP’s inefficiency in grid applications
•
UDT
•
Design issues
•
Implementations issues
•
Conclusion and future work

TCP and AIMD
•
TCP has been very successful in the
Internet
–
AIMD (Additive Increase Multiplicative
Decrease)
•
Fair: max-min fairness
•
Stable: globally asynchronously stable
•
But, inefficient and not scalable

The throughput is limited by the
slowest stream!
Amsterdam
Chicago 2
Chicago 1

UDT – UDP-based Data Transfer Protocol
•
Application level transport protocol built
above UDP
•
Reliable data delivery
•
End-to-end approach
•
Bi-directional
•
General transport API; not a (file transfer)
tool.
•
Open source

UDT Architecture
DATA
ACK
ACK2
NAK
Sender
Recver
Sender

Congestion/Flow Control
•
Performance evaluation
–
Efficiency
–
Fairness and friendliness
–
Stability

Reliability/Acknowledging
•
Acknowledging is expensive
–
Packet processing at end hosts and routers
–
Buffer processing
•
Timer-based selective acknowledgement
–
Send acknowledgement per constant time (if
there are packets to be acknowledged)
•
Explicit negative acknowledgement

Congestion Control
•
AIMD with decreasing increases
•
Increase formula

UDT: Efficiency and Fairness Characteristics
•
Takes 7.5 seconds to reach 90% of the link
capacity, independent of BDP
•
Satisfies max-min fairness if all the flows
have the same end-to-end link capacity
–
Otherwise, any flow will obtain at least half of
its fair share
•
Does not take more bandwidth than
concurrent TCP flow as long as
6/10
822
⋅≤⋅
SYNLRTT

Efficiency
0 10 20 30 40 50 60 70 80 90 100
0
200
400
600
800
1000
Time (s)
Throughput (Mbps)
to Chicago, 1Gbps, 0.04ms
to Canarie, OC-12, 16ms


Fairness
10
-2
10
-1
10
0
10
1
10
2
10
3
0.8
0.85
0.9
0.95
1
RTT (ms)
Fairness Index
UDT
TCP
•
Fairness index
–
Simulation: Jain’s Fairness Index for 10 UDT
and TCP flows over 100Mb/s link with
different RTTs


The 4 UDT flows
have similar
performance and
leave enough space
for TCP flows

TCP Friendliness
0 1 2 3 4 5 6 7 8 9 10
20
30
40
50
60
70
80
Number of UDT flows
TCP Throughput (Mbps)
•
Impact on short life TCP flows
–
500 1MB TCP flows with 1-10 bulk UDT
flows, over 1Gb/s link between Chicago and
Amsterdam

Stability
10
-2
10
-1
10

API
•
Conformance

Efficiency and CPU utilization
•
Efficiency = Mbps/MHz
•
Maximize throughput
–
Use CPU time as little as possible, so that CPU
won’t be used up before network bottleneck is
reached
–
Remove CPU burst, which can cause packet
loss: even distribution of processing
•
Minimize CPU utilization

Loss Processing
•
On high BDP networks, the number of lost
packets can be very large during a loss event
•
Access to the loss information may take long time
•
Acknowledge may take several packets
0 10 20 30 40 50 60 70 80 90 100
0
1000

•
Speculation of next packet
Protocol Buffer Protocol Buffer
User Buffer
Data
New Data