Making HawkNL UDP connections more robust

Or, why was it not done right the first time :)

By Phil Frisbie, Jr.

Background

I am writing this as I prepare to rip out small sections of code from HawkNL's 7800 lines, and add large sections of code in their place. But first, before I get into details, let me give you a background on how UDP (or NL_UNRELIABLE) connections came about and how they currently work. Some of this discussion assumes you have an understanding of UDP, TCP, and the socket API.

One of my early ideas behind HawkNL (NL) was to not only provide a low level porting layer for HawkNLU, but also hide most, if not all, of the differences of UDP and TCP. I wanted HawkNLU to be able to use either TCP or UDP, and to use the same code for both. I wanted this layer to be thin and simple, since simple algorithms are easier to debug and maintain. However, sometimes the simple algorithms will not provide what you need, so you need to add the complex algorithms after all. TCP packets have already gone through this change from simple to complex and they now work well, and they are virtually interchangeable with UDP packets. But UDP connections do not work as well as they should.

Current UDP connection algorithm

Since UDP is a connectionless protocol, NL must provide the code to emulate TCP connections. The current algorithm is basically unchanged since version 0.3 alpha. When you call nlListen it simply sets a flag so that calling nlAcceptConnection will not give you an error. The exchange between nlConnect and nlAcceptConnection is fairly simple, and goes like this:

Client                            Server
nlConnect-sends connect request
                                  nlAcceptConnection-checks for connection request
                                                    -opens up a new socket
                                                    -calls connect() on new socket
                                                    -sends a message back with new
                                                     port number
                                                    -sends message back from new
                                                     socket to open up firewall
                                                    -returns new socket to app
-waits for reply, and if none
then resends the request
-receive new port number
-call connect() to remote address
and new port

Now, with a non-blocking socket, a thread is created for the nlConnect. nlRead and nlWrite will return an error until the connection is completed.

Some good points about this algorithm are that it is simple, it helps to eliminate forged packets since the sockets are connected and will only accept packets from the connected remote address, and it works well on a LAN.

Some bad points are that the server opens up a new port with each new socket, so firewalls and NAT cause problems. This is a major difference from TCP, which uses the same server port for all the accepted connections. Sometimes the client does not get the connection confirmation with the new port number, so it will request another connection. So the same client can be accepted twice by the server since the server does not check for a duplicated connection request. And finally, there is no way to gracefully close the connection when the other side closes.

New UDP connection algorithm

These are my goals for the new algorithm:

Add more error checking for duplicated connections.
Add graceful connection closing.
Use only one port on the server, just like TCP.

The first goal, adding more error checking for duplicated connections, is relatively simple. When nlListen is called on a socket, it should create a thread to listen and accept connections. After the thread receives a connection request, it will add the IP and port number to a list of pending connections. It will then send the connection confirmation, wait for another message from the client acknowledging the connection, and resend if the client sends another connection request. So, a second message needs to be added to the client code. The new connection will be stored until nlAcceptConnection is called, just like TCP.

The second goal, adding graceful connection closing, is a little more difficult. Yes, sometimes a socket error will occur when the remote port is closed, but not always. And what if the remote port is closed and quickly reopened? No error will occur, and you will be sending the remote port packets it is not expecting. So, NL will need to send a special message to the remote client or server when we close the socket. The message must be something unlikely to be sent by a real game. There must also be an acknowledgment message to confirm the close. When the socket is closed, a thread must take care of sending the close message, waiting for the reply close message, and sending an acknowledgment. On the other end, each call to nlRead must look at the packet received to see if it is the connection close message. If it is, then discard the packet and have a thread send a close message back, wait for the acknowledgment, and then send an acknowledgment. Further calls to nlWrite will need to fail when the connection is in a state of closing.

The third goal, using only one port on the server just like TCP, is the most important, and it ties right in with the first two. Just like in the first goal, when nlListen is called, it will create a thread. This thread, however, will not only accept connections, it will handle the graceful closing needed for the second goal, and it will also sort out the incoming packets and forward them to the right connection. You see, each accepted connection will not be connected to a real socket, it will simply have a buffer to hold incoming packets. For speed and scalability, hashing will be used on the remote IP and port number to look up the NLsocket that the packet needs to be forwarded to. So a call to nlRead will simply check a buffer for a packet, but nlWrite will call sendto() using the 'connected' address.

Conclusion

This change will require a lot of work, but it will be worth it to finally have robust UDP connections that really do act like TCP. I wish I had taken the time to address this problem earlier, but it is only in the last few months that the HawkNL UDP connections have really gotten some serious use in varied projects. So now they must be fixed before I can devote more time to HawkNLU.

Phil Frisbie, Jr.