[Code] Solution of TCP packet sticking/coalescing (sticky packet) - Designing a protocol step by step

First Post:

Last Update:

Word Count:
1k

Read Time:
6 min

Introduction

This article explains how to solve a common issue in network programming—TCP packet sticking (also known as packet coalescing or sticky packets).
In addition, it demonstrates how to design a custom application-layer protocol for socket communication.
All examples in this article are implemented in C#.

What Is Sticky Packet?

If you are not familiar with socket programming, please read the following article first:

Socket Initialization with different languages.

Let us start with a simple receiver implementation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
using System;
using System.Net.Sockets;
using System.Text;

class Client
{
    static void Main()
    {
        TcpClient client = new TcpClient();

        client.Connect("127.0.0.1", 8080);

        NetworkStream stream = client.GetStream();

        string message = "Hello from C# client";
        byte[] data = Encoding.UTF8.GetBytes(message);
        stream.Write(data, 0, data.Length);

        byte[] buffer = new byte[1024];
        int bytesRead = stream.Read(buffer, 0, buffer.Length);
        string reply = Encoding.UTF8.GetString(buffer, 0, bytesRead);
        Console.WriteLine("Server reply: " + reply);

        client.Close();
    }
}

The root cause of packet sticking lies in the following line:
1
byte[] buffer = new byte[1024];

Here, the receive buffer size is fixed at 1024 bytes.
However, the size of the data sent by the server (or client—it does not matter) is not guaranteed to be exactly 1024 bytes.

  • If the actual message size is 1025 bytes, one byte will be lost.
  • If the actual message size is 512 bytes, it may appear safe at first.

In practice, the remote host may send messages frequently and continuously.
As a result, you may receive a 1024-byte buffer that actually contains two consecutive 512-byte messages.

If the receiver cannot distinguish between these two messages, the handler may incorrectly treat them as a single message, which can lead to logic errors or corrupted data.

This phenomenon is known as TCP packet sticking.

Solution

To solve this problem, the sender must explicitly inform the receiver of the message length.

The most common and effective approach is to prepend a message header to each payload.
In this section, we will design a custom application-layer protocol (OSI Layer 7) to achieve this.

Fundamental Design

We define the protocol format as follows:

1
| Header | Payload |

| Field | Description |
| —- | —- |
| Header | Constant length. |
| Payload | Your message. |

The header structure is defined as:

1
| Command | Parameter | Length |

| Field | Size |Description |
| —- | —- | —- |
| Command | 1 byte | What the receiver should do. |
| Parameter | 1 byte | Parameters of the command. |
| Length | 4 byte | Payload’s length |

Example command definitions:
| Command | Parameter | Meaning |
| —- | —- | —- |
| 0 | 0 | Disconnect. |
| 1 | 0 | Message handler. |

At this point, we have successfully defined our protocol.
The next step is implementation.

Implementation

We will apply object-oriented programming (OOP) concepts in C#.

Notice: Hungarian notation is used in this implementation.

  1. Designing a class

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    class clsMyProtocol
    {
        public const int MAX_SIZE = 65535;
        public const int HEADER_SIZE = 6;

        private byte _nCommand = 0;
        public byte m_nCommand => _nCommand;

        private byte _nParam = 0;
        public byte m_nParam => _nParam;

        private int _nDataLength = 0;
        public int m_nDataLength => _nDataLength;

        private byte[] _abMessageData = Array.Empty<byte>();
        public byte[] m_abMessageData => _abMessageData;

        private byte[] _abMoreData = Array.Empty<byte>();
        public byte[] m_abMoreData => _abMoreData;

        // Constructor for parsing received buffer
        public clsMyProtocol(byte[] abBuffer)
        {
            if (abBuffer == null || abBuffer.Length < HEADER_SIZE)
                return;

            using (var ms = new MemoryStream(abBuffer))
            using (var br = new BinaryReader(ms))
            {
                _nCommand = br.ReadByte();
                _nParam = br.ReadByte();
                _nDataLength = br.ReadInt32(); // <-- little-endian by default

                if (abBuffer.Length - HEADER_SIZE >= _nDataLength && _nDataLength > 0)
                    _abMessageData = br.ReadBytes(_nDataLength);

                int remaining = (int)(abBuffer.Length - HEADER_SIZE - _nDataLength);
                if (remaining > 0)
                    _abMoreData = br.ReadBytes(remaining);
            }
        }

        // Constructor for building packets to send
        public clsMyProtocol(byte nCmd, byte nParam, byte[] abMsg)
        {
            _nCommand = nCmd;
            _nParam = nParam;
            _abMessageData = abMsg;
            _nDataLength = _abMessageData.Length;
        }

        public byte[] fnabGetBytes()
        {
            try
            {
                using (var ms = new MemoryStream())
                using (var bw = new BinaryWriter(ms))
                {
                    bw.Write(_nCommand);
                    bw.Write(_nParam);
                    bw.Write(_nDataLength);
                    bw.Write(_abMessageData);

                    return ms.ToArray();
                }
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message, "fnabGetBytes()", MessageBoxButtons.OK, MessageBoxIcon.Error);
                return Array.Empty<byte>();
            }
        }

        public (byte nCommand, byte nParam, int nLength, byte[] abMsg) fnGetMsg()
            => (_nCommand, _nParam, _nDataLength, _abMessageData);

        public static (byte nCommand, byte nParam, int nLength) fnGetHeader(byte[] abBuffer)
        {
            if (abBuffer == null || abBuffer.Length < HEADER_SIZE)
                return (0, 0, 0);

            byte nCommand = abBuffer[0];
            byte nParam = abBuffer[1];
            int nLength = BitConverter.ToInt32(abBuffer, 2);

            return (nCommand, nParam, nLength);
        }
    }

  2. Receiver handler:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69

    byte[] fnabCombineBytes(byte[] abFirstBytes, int nFirstIndex, int nFirstLength, byte[] abSecondBytes, int nSecondIndex, int nSecondLength)
    {
        byte[] abBytes = new byte[nFirstLength + nSecondLength];
        using (MemoryStream ms = new MemoryStream())
        {
            ms.Write(abFirstBytes, nFirstIndex, nFirstLength);
            ms.Write(abSecondBytes, nSecondIndex, nSecondLength);

            abBytes = ms.ToArray();
        }

        return abBytes;
    }

    void fnMsgHandler(string szMsg)
    {
        //todo: Message handler.
    }

    void fnHandler(Socket skt)
    {
        clsMyProtocol myProto = null;

        int nRecvLength = 0;
        byte[] abStaticRecvBuffer = new byte[clsMyProtocol.MAX_SIZE];
        byte[] abDynamicRecvBuffer = { };

        do
        {
            abStaticRecvBuffer = new byte[clsMyProtocol.MAX_SIZE];
            nRecvLength = skt.Receive(abStaticRecvBuffer);

            if (nRecvLength <= 0)
                break;
            else if (abDynamicRecvBuffer.Length < clsMyProtocol.HEADER_SIZE)
                continue;
            else
            {
                var headerInfo = clsMyProtocol.fnGetHeader(abDynamicRecvBuffer);
                while (abDynamicRecvBuffer.Length - clsMyProtocol.HEADER_SIZE >= headerInfo.nLength)
                {
                    myProtocol = new clsMyProtocol(abDynamicRecvBuffer);
                    abDynamicRecvBuffer = myProtocol.m_abMoreData;
                    headerInfo = clsMyProtocol.fnGetHeader(abDynamicRecvBuffer);

                    byte[] abBuffer = myProtocol.fnGetMsg().abMsg;

                    if (myProtocol.m_nCommand == 0)
                    {
                        if (myProtocol.m_nParam == 0)
                        {
                            skt.Close();
                        }
                    }
                    else if (myProtocol.m_nCommand == 1)
                    {
                        if (myProtocol.m_nParam == 0)
                        {
                            string szMsg = Encoding.UTF8.GetString(abBuffer);

                            fnMsgHandler(szMsg);
                        }
                    }
                }
            }
        }
        while (nRecvLength > 0);
    }

The sender:

Important: Both sender and receiver must share the same protocol definition.

1
2
3
4
5
6
7
8
9
10
11
void fnSendHelloWorld(Socket skt)
{
    uint nCmd = 1;
    uint nParam = 0;
    string szMsg = "Hello world!";
    byte[] abMsg = Encoding.UTF8.GetBytes(szMsg);

    clsMyProtocol myProtocol = new clsMyProtocol((byte)nCmd, (byte)nParam, abMsg);

    skt.Send(myProtocol.fnabGetBytes());
}

With a clearly defined message header and proper buffer management, the original problem—TCP packet sticking—is effectively resolved.

Conclusion

By designing a simple yet structured application-layer protocol, you can now establish robust and reliable socket communication, even under high-throughput or bursty network conditions.

≡