Why is the package protocol provided by Skynet uses 2 bytes to indicate a package length.

Skynet provides a Lua library NETPACK to resolve data in the TCP stream to the length + content package. Although classmates using SkyNet can do not use it, they will also implement a set of parsed protocols to parse external TCP data streams (such as the REDIS Driver parsing of the Skynet), the data stream of the Redis Server is used, but there are still a lot. Classmates inquiry, can not customize the length of the header.

Among the protocols defined here, the packet length is represented by the 2 bytes of BIG-Endian, that is, the length of a package must not exceed 64K. This makes many people very difficult. Has a few times have been suggested, relaxing the length into 4 bytes, because most packages in their application environment will not exceed 64K, but there is always a small amount exceeding this limit.

Historically, when Skynet Gate is implemented with C (that version can still be used) can indeed customizing the use of 2 bytes or 4 bytes to represent the package length. But after some considering, I still remained this option.

A good library should be concise, and guide users to do the right thing with the right way; not to provide opportunities for users to make mistakes. When communicating with the game client, if you only use a TCP connection, the allowable packet itself is wrong. Even 64K is too big.

The game usually requires a faster response speed, if you allow a large data block in a single TCP connection, such as 100K, then under a relatively weak network condition (such as a mobile network), processing this package may need more than 1 Minute time. And such a large data block, most of the business logic is not expected to be immediately issued or received. A typical application scenario is that the user queries all the shelves in the auction line. If all returned data is placed in a packet, it is easy to become large. And querying a lot of this operation, the user itself will not expect immediately.

In a single TCP connection, such a large data block will block the entire channel, and the data that is rapidly delivered later will be delayed.

If you want to make a heartbeat package in the business layer to detect if the network is timeout, it is easy to brush the heartbeat. And usually the network processing layer does not provide an interface to detect whether the business layer is accepting data (Skynet’s GateServer does not provide such an interface, although it is easy to provide, only need to modify a few lines of Lua code), only in a complete data The package will be handed over to the business layer after receiving.

The correct approach is to add a level on the protocol of the length + content. Add a protocol called big data block to allow a large data block to be sent several segments. The data block ID can be added to this protocol, and the data block ID is attached to the package of this data block later.

Why use this way around, not directly change the header from 2 bytes to 4 bytes? When you do this design, you have shown that you pay attention to the questions mentioned above.

When you split the packets, you can carry multiple channels of multiple channels on a single TCP connection. For online games, not all packets are related to the context, you can see that there are multiple clues. For example, information and scene synchronization of chat channels are independent of each other. Skynet also provides additional Socket API support for this scenario. Socket.LWRITE can write a string (a packet) to a low priority channel. Only when the default channel (high priority) package is all sent, the package on the low priority channel is at least one (single package ensures atomic).

For example, you can use it to send chat information, you will not put other important packets because of the flooding of chat information. Again, you can use to send a split big data block. If you still have many other important data to be transferred to the client, then these data blocks will be scattered interpolated.

Of course, you can also send all the data to the client with lwrite, and only put your heartbeat in a conventional high priority channel, you can ensure that the heartbeat frequency is more stable.

In addition, there is a security vulnerability with 4-byte packet lengths that may be used.

The code of the general segmentation package, when receiving the header, will pre-assign a relatively long space according to the length information, and wait for the rear data to reach after filling. If an attacker continues to send a malicious length information on a newly established connection, such as 2G, server memory is easily removed quickly.

Early Skynet’s GATE implementation, uses the implementation of a fixed length RingBuffer to avoid this attack. But the new version is not particularly handled because there is no longer allowed 4 bytes of length.