Teeworlds 0.6
home overview fundamentals packet_layout map_download snap_items system_messages game_messages ctrl_messages

Fundamentals

There are a few base concepts that you need to understand to fully grasp the whole network protocol from scratch.

Tokens

Not all 0.6 versions do have tokens only the last release 0.6.5 introduced tokens. It uses no token when connecting to 0.6.4 or older 0.6 versions to stay backwards compatible. Every game related packet (ignoring server info for now) has a packet header with a token field. The token field has to be the token of the receiving party. So when the client sends a packet to the server the token field contains the token from the server and vice versa.
Client and server generate a random token and then exchange it.
Start the server or client with the config debug 1 to see error messages in the log if a wrong token was sent. This check happens in network_conn.cpp Feed()

Sequence numbers

Every chunk header can contain a vital flag (MSGFLAG_VITAL). Note that one teeworlds packet can contain multiple chunks. If said vital flag is set to true the receiving party has to increment a counter. This counter is called the sequence or acknowledge number. Client and server will send the amount of vital chunks they received as sequence number in the packet header.

The CNetConnection class keeps track of these sequence numbers. 

                
    unsigned short m_Sequence;
    unsigned short m_Ack;
    unsigned short m_PeerAck;
Using these three variables client and server keep track of: Be warned that the client and server silently drop all packets with wrong sequence numbers set in the packet header.
Quick debug tip if you suspect sequence number bugs in your implemenation. Comment out or add dbg_msgs in these lines.

Message queueing, chunks and flushing

Control messages are sent immediately. All ctrl messages go through the SendControlMsg() method which calls SendPacket() directly. In SendPacket() the teeworlds packet header gets added and the packet is sent over the network. Note that control messages always have 0 chunks because of the Construct.m_NumChunks = 0; line 
            
    void CNetBase::SendControlMsg(const NETADDR *pAddr, TOKEN Token, int Ack, int ControlMsg, const void *pExtra, int ExtraSize)
    {
        CNetPacketConstruct Construct;
        Construct.m_Token = Token;
        Construct.m_Flags = NET_PACKETFLAG_CONTROL;
        Construct.m_Ack = Ack;
        Construct.m_NumChunks = 0;
        Construct.m_DataSize = 1+ExtraSize;
        Construct.m_aChunkData[0] = ControlMsg;
        if(ExtraSize > 0)
            mem_copy(&Construct.m_aChunkData[1], pExtra, ExtraSize);

        // send the control message
        SendPacket(pAddr, &Construct);
    }
But Game messages and system messages usually go through SendMsg() -> m_NetClient.Send(&Packet); where they end up in 
            
    if(pChunk->m_Flags&NETSENDFLAG_CONNLESS)
    {
        // [..]
    }
    else
    {
        // [..]
        m_Connection.QueueChunk(Flags, pChunk->m_DataSize, pChunk->m_pData);
    }
And m_Connection.QueueChunk() as the name suggests only puts these messages into a queue of so called chunks. So no packet header is created and no data is being sent over the network yet. For now the messages only get collected in a queue. This queue is then sent as soon as it is full or a message with the flag MSGFLAG_FLUSH is being send. When it is being send all messages in the queue get their own chunk header and in the very beginning of the udp packet there is the packet header which also includes one field that says how many chunks are in the packet.

This is probably done for performance reasons to not waste bandwith on teeworlds/udp/ip headers when non time critical messages are being sent that are followed by other messages anyways. So they are bundled up and sent together.

So if you are calling SendMsg() and care about time delay make sure to use the MSGFLAG_FLUSH flag otherwise it might be slightly delayed. The delay is not very dramatic since flushes happen quite frequently so only use it for super time sensitive things such as user inputs or tee positions.

Lets illustrate chunks and flushing via a real world example. When a client connects to a server the first packet with multiple chunks is Motd, Server Settings, Ready. This is happening because when a client connects OnClientConnected() is being called. Where the motd and server settings are being sent. 
            
    void CGameContext::OnClientConnected(int ClientID, bool Dummy, bool AsSpec)
    {
        // [..]

        // send motd
        SendMotd(ClientID);

        // send settings
        SendSettings(ClientID);
    }
            
        
            
    void CGameContext::SendMotd(int ClientID)
    {
        CNetMsg_Sv_Motd Msg;
        Msg.m_pMessage = Config()->m_SvMotd;
        Server()->SendPackMsg(&Msg, MSGFLAG_VITAL, ClientID);
    }

    void CGameContext::SendSettings(int ClientID)
    {
        CNetMsg_Sv_ServerSettings Msg;
        Msg.m_KickVote = Config()->m_SvVoteKick;
        Msg.m_KickMin = Config()->m_SvVoteKickMin;
        Msg.m_SpecVote = Config()->m_SvVoteSpectate;
        Msg.m_TeamLock = m_LockTeams != 0;
        Msg.m_TeamBalance = Config()->m_SvTeambalanceTime != 0;
        Msg.m_PlayerSlots = Config()->m_SvPlayerSlots;
        Server()->SendPackMsg(&Msg, MSGFLAG_VITAL, ClientID);
    }
Note that the call to SendPackMsg() does contain the flag MSGFLAG_VITAL but no MSGFLAG_FLUSH so they are not sent and just added to the queue. Directly after OnClientConnected() is called SendConnectionReady() gets called. 
            
    GameServer()->OnClientConnected(ClientID, ConnectAsSpec);
    SendConnectionReady(ClientID);
            
        
            
    void CServer::SendConnectionReady(int ClientID)
    {
        CMsgPacker Msg(NETMSG_CON_READY, true);
        SendMsg(&Msg, MSGFLAG_VITAL|MSGFLAG_FLUSH, ClientID);
    }
Note that this time the flag MSGFLAG_FLUSH is used which causes the whole queue (Motd, Settings) to be sent inclusing the newly added CON_READY. So basically we send 3 messages here but they just get queued and then sent together. But this is all happening in the same tick so it feels instant.

So these three pseudo code method calls 
            
    Send(CNetMsg_Sv_Motd, MSGFLAG_VITAL);                   // nothing is sent
    Send(CNetMsg_Sv_ServerSettings, MSGFLAG_VITAL);         // nothing is sent
    Send(NETMSG_CON_READY, MSGFLAG_VITAL|MSGFLAG_FLUSH);    // all 3 are sent
cause one udp packet to be sent that looks like this 
            
    +-------------------------+---------------+-------+---------------+-----------------+---------------+-----------+
    | teeworlds packet header | chunk header  | motd  | chunk header  | server settings | chunk header  | CON_READY |
    |        chunks = 3       |     vital     |       |     vital     |                 | vital & flush |           |
    +-------------------------+---------------+-------+---------------+-----------------+---------------+-----------+

Int packing

Note there is also documentation on ints in the libtw2 docs.
There is also a more technical section with examples about int packing.

All numbers (integers) sent via the network are packed using a custom teeworlds specific packer.
Even though it being a custom packer the numbers 0-63 are pretty much the standard binary representation.

So binary 00000001 is decimal 1
and binary 00000010 is decimal 2
and so on..

If we exceed 63 it starts to differ.
The int packer code can be found here. Together with this annotation comment. 

                
    Format: ESDDDDDD EDDDDDDD EDD... Extended, Data, Sig
What it means is that the second bit of the the first byte of the integer is its sign bit. Meaning that if the second bit of the first byte is 1 it is a negative number. If the second bit of the first byte is a 0 it is a positive number. And the first bit of every byte is the extension bit. If it is set to 1 it means another byte is following if it is set to 0 the current byte is the last. This extension bit allows to send small numbers (-62 to 63) as a single byte over the network. This extension bit also means that technically the protocol allows numbers of any size. Which you should not send tho since they are read into a C++ int which is only 4 bytes.

Also note that the byte order is little endian. The smallest positive two byte number 64 is represented as those bits: 10000000 00000001
Which can be dissected as: 
                
    ESDDDDDD EDDDDDDD
    10000000 00000001
    ^^^    ^ ^^    ^
    ||\   /  | \   /
    || \ / not  \ /
    ||  \ extended
    ||   \      /
    ||    \    /
    ||     \  /
    ||      \/
    ||      /\
    ||     /  \
    ||    /    \
    || 0000001 000000
    ||       |
    ||       v
    ||       64
    ||
    |positive
    extended

There is a rust crate named teeint which soley focusses on int packing.

String packing

When strings are sent via the network. They are sent as plain C strings. They do not contain a length. And have to be null terminated.

The receiver then can use diffent types of sanitizers when unpacking the string:

Raw packing

If you want to send raw bytes over the network. The teeworlds protocol got you covered.
There is void CPacker::AddRaw(const void *pData, int Size) and its counter part const unsigned char *CUnpacker::GetRaw(int Size)

It will just send and receive byte by byte the raw data you give it. Note that while reading you have to know the size. The raw data does not include a size field or is terminated in any way.
So raw can only be used in contexts where the length is known to the recipient. This usually happens by a leading integer field holding the size.

Huffman compression

Packets can be compressed using huffman compression. The teeworlds packet header is never compressed only its payload. The packet header sets the COMPRESSION flag if the payload is compressed. Usually the same packets are compressed. But technically every packet could be compressed or uncompressed.

Huffman compression is an alorithm not specific to teeworlds. To understand its fundamentals you can simply checkout any non teeworlds related huffman documentation. Like this youtube video. Huffman compression depends on a weight tree. So if you reimplement huffman your self make sure it uses the same weight tree as the official teeworlds server and client. The official teeworlds frequence table can be found here.
But chances are you do not have to reimplement the huffman alorithm even if you want to build a server or client in a new programming language. There are community implemenations for python, javascript, ruby and rust already.