Foreword to the preface
The server model involves threading mode and IO mode, and it is clear that these can be targeted for various scenarios. The series is divided into three parts:
Single-threaded/multi-threaded blocking I/O model
Single-threaded non-blocking I/O model
Multi-threaded non-blocking I/O model, Reactor and its improvements
Foreword
The server model discussed here mainly refers to the server-side I/O processing model. There are different classifications from different dimensions. Here we discuss the server model from the perspective of I/O blocking and non-blocking, I/O processing for single-threading and multi-threading.
For I/O, it can be divided into two types: blocking I/O and non-blocking I/O. Blocking I/O causes the current thread to enter a blocking state while doing I/O read and write operations, and does not enter a blocking state instead of blocking I/O.
For threads, one thread is responsible for all client-side I/O operations in a single-threaded case, while multi-threaded threads handle all client-side I/O operations together.
Single-threaded blocking I/O model
The single-threaded blocking I/O model is the simplest server model, and almost all programmers start with this simple model when they first started working on network programming. This model can only handle one client access at the same time, and is blocked on I/O operations, and the thread will wait all the time without doing anything else. For multiple client accesses, you must wait until the previous client access is complete before proceeding to the next access process, requesting one queue, providing only one question and one answer service.
First, the server must initialize a socket server and bind a port number to listen for client access. Then, the client 1 calls the server's service, and the server processes the request after receiving the request, and writes the data back to the client 1 after the processing, and the whole process is completed in one thread. Finally, the client 2's request is processed and the data is written back to the client 2, even if the client 2 makes a request before the server processes the client 1, and waits for the server to respond to the client 1 before the client 2 Perform response processing.
This model is characterized by single-threading and blocking I/O. Single-threaded, that is, only one thread on the server side handles all requests from the client. The ratio of the client-side connection to the server-side processing thread is n:1. It cannot handle multiple connections at the same time, and only serializes the connection. Blocking I/O means that the server is blocked when reading and writing data. When reading client data, it waits for the client to send data and copy the operating system kernel to the user process. At this time, the blocking state is released. When writing data back to the client, wait for the user process to write data to the kernel and send it to the client before unblocking. This blocking poses a problem for network programming. The server must wait until the client successfully receives to continue processing another client's request, during which the thread will not be able to respond to any client requests.
The characteristics of the model: it is the simplest server model, the whole running process has only one thread, can only support the processing of a client's request at the same time (if there are multiple client access, you must wait in line), the server system resource consumption is more Small, but low concurrency and poor fault tolerance.
Multi-threaded blocking I/O model
For the shortcomings of the single-threaded blocking I/O model, we can use multi-threading to improve it so that it can respond to multiple clients concurrently. The core of the multi-threaded model is to use the multi-threading mechanism to assign one thread to each client. The server starts to listen to the client's access. If two clients send a request, the server creates two threads to process them after receiving the client request. Each thread is responsible for one client connection until the response is completed. During the process, two threads concurrently process requests for their respective clients, including reading client data, processing client data, and writing data back to the client.
The I/O operation of this model is also blocked, because each thread will enter a blocking state when it performs a read or write operation, and will not be unblocked until the data or data read to the client is successfully written to the client. . Although the I/O operation is blocked, this mode is significantly better than the single-threaded processing. It does not wait until the first request is processed before processing the second one, but concurrently processing the client request, the client connection and the server. The proportion of end-processing threads is 1:1.
Multi-threaded blocking I/O model features: Support for concurrent response to multiple clients, greatly improved processing power, large concurrency, but large server system resource consumption, and thread switching between multiple threads Cost, while having a more complex structure.
Single-threaded non-blocking I/O model
The multi-threaded blocking I/O model does increase the concurrent processing power of the server by introducing multiple threads, but each connection requires a thread to be responsible for I/O operations. When the number of connections is large, the number of machine threads may be too large, and these threads are in a wait state most of the time, causing a great waste of resources. Given the shortcomings of multi-threaded blocking I/O models, is it possible to maintain multiple client connections with one thread and not block read and write operations? The single-threaded non-blocking I/O model is described below.
One of the most important features of the single-threaded non-blocking I/O model is that it returns immediately after a read or write interface is called, without entering a blocking state. Before discussing a single-threaded non-blocking I/O model, you must first understand the detection mechanism of socket events in non-blocking situations, because the most important thing for a single-threaded non-blocking model is to detect which connections have events of interest. Generally there are three detection methods as follows.
Application traversal socket event detection
When multiple clients request from the server, the server side saves a list of socket connections, and the application layer thread polls the socket list for an attempt to read or write. For a read operation, if a number of data is successfully read, the read data is processed; if the read fails, the next loop continues to try. For write operations, first try to write data to a specified socket. If the write fails, the next loop will continue.
It seems that no matter how many socket connections there are, they can be managed by one thread. A thread is responsible for traversing these socket lists and constantly trying to read or write data. This makes good use of the blocking time and the processing power is improved. However, this model needs to traverse all the socket lists in the application, and needs to process the splicing of data. When the connection is idle, it may also occupy more CPU resources, which is not suitable for practical use. The improved approach is to use an event-driven, non-blocking approach.
Kernel traversal socket event detection
This approach hands the traversal of the socket to the operating system kernel, organizes the results of the socket traversal into a series of event lists and returns to the application layer processing. For the application layer, the objects they need to deal with are these events, which is one of the event-driven, non-blocking implementations.
There are multiple client connections on the server side, and the application layer requests a list of read and write events from the kernel. The kernel traverses all sockets and generates a corresponding readable list readList and writable list writeList. The readList indicates whether each socket is readable. For example, the value of socket 1 is 1, indicating readability, and the value of socket2 is 0, indicating that it is unreadable. The writeList indicates whether each socket is writable. The application layer traverses the read and write event lists readList and writeList to perform corresponding read and write operations.
When the kernel traverses the socket, it is no longer necessary to traverse all the sockets at the application layer, and the traversal work is moved down to the kernel layer, which helps to improve the detection efficiency. However, it needs to pass all connected readable event lists and writable event lists to the application layer. If the number of socket connections becomes large, copying the list from the kernel to the application layer is not a small overhead. In addition, when there are fewer active connections, there are many invalid copies of the data between the kernel and the application layer because it copies both active and inactive connection states into the application layer.
Kernel callback based event detection
Detecting whether a socket is readable and writable by traversing is a less efficient way, whether it is traversing in the application layer or traversing in the kernel. So another way to optimize the traversal is to call back the function. The sockets in the kernel correspond to a callback function. When the client sends data to the socket, the kernel will call the callback function after receiving the data from the network card. The event list is maintained in the callback function, and the application layer obtains the event list. All events of interest are available.
There are two ways in which the kernel can detect events based on callbacks. The first is to read and write events with a readable list readList and a writable list writeList. The number of sockets is the same as the length of the two lists readList and writeList. The first element of readList is marked with 1 to indicate that socket 1 can be used. Read, in the same way, the second element of writeList is marked as 1 to indicate that socket 2 is writable. As shown in the figure, multiple clients connect to the server. When the client sends data, the kernel calls the callback function after the data is successfully copied from the network card. The first element of the readList is set to 1, and the application layer sends a request to read and write the event list. The return kernel contains a list of readList and writeList events of the event identifier, and then the table traverses the read event list readList and the write event list writeList, and the socket corresponding to the element corresponding to 1 is read or written. This avoids the operation of traversing the socket, but there is still a lot of useless data (elements with a status of 0) copied from the kernel to the application layer. Then there is a second way of detecting events.
The kernel based callback based event detection method is shown in the figure. There are multiple client socket connections on the server side. First, the application layer tells the kernel what events each socket is interested in. Then, when the client sends the data, there will be a callback function. After the kernel successfully copies the data from the network card, the callback function is added to the event list as the readable event event1. Similarly, the kernel finds that socket 2 is added to the event list as a writable event event2 when the network card is writable. Finally, the application layer requests the kernel to read and write the event list. The kernel returns the event list containing event1 and event2 to the application layer. The application layer learns that the socket 1 has data to be read by traversing the event list, so that the read operation is performed. Socket 2 can write data.
In the above two ways, the operating system kernel maintains all the connections of the client and continuously updates the event list through the callback function, and the application layer thread can read the list of events to know the readable or writable connections, and then read the connections. The write operation greatly improves the detection efficiency and the natural processing capability is also stronger.
For Java, the implementation of non-blocking I/O is based entirely on the operating system kernel's non-blocking I/O, which shields the operating system's non-blocking I/O differences and provides a unified API so that we don't have to care about the operating system. . The JDK will help us choose the implementation of non-blocking I/O. For example, for Linux systems, JDK will prefer to implement non-blocking I/O for Java with epoll when epoll is supported. This non-blocking event detection mechanism is the second most efficient way in "kernel callback-based event detection."
After learning about the event detection method in non-blocking mode, return to the discussion of the single-threaded non-blocking I/O model. Although there is only one thread, it can realize the timely processing of multiple connections by combining the non-blocking read and write operations with the above several detection mechanisms, without causing other connections to be unprocessed due to the blocking operation of a certain connection. In the case that most of the client connections remain active, this thread will continue to process these connections all the time, it makes good use of the blocking time, greatly improving the efficiency of this thread.
The main advantage of the single-threaded non-blocking I/O model is the management of multiple connections. Generally, non-blocking NIO mode is used in the scenario where multiple connections need to be processed at the same time. In this model, only one thread is used to maintain and process. Connections, which greatly improve the efficiency of the machine. In general, the server side uses the NIO mode, and for the client, for convenience and habit, the blocking mode socket can be used for communication.
Multi-threaded non-blocking I/O model
The single-threaded non-blocking I/O model has greatly improved the efficiency of the machine, while on multi-core machines it is possible to continue to increase machine efficiency through multithreading. The simplest and most natural way is to assign client connections to groups of threads by group, each thread handling the connections within the corresponding group. As shown, there are 4 client access servers, the server manages socket 1 and socket 2 by thread 1, and thread 2 manages socket 3 and socket 4, through event detection and non- Blocking reads and writes allows each thread to process efficiently.
The most classic multi-threaded non-blocking I/O model approach is the Reactor mode. First look at the Reactor under a single thread, Reactor divides the entire processing of the server into several events, such as receiving events, reading events, writing events, executing events, and so on. Reactor distributes these events to different processors for processing through event detection mechanisms. As shown, several clients connect to the server. Reactor is responsible for detecting various events and distributing them to the processor. These processors include the accept processor that receives the connection, the read processor that reads the data, and the write processor that writes the data. The logical process processor that executes the logic. In the whole process, as long as there are pending events, the Reactor thread can be executed continuously without blocking somewhere, so the processing efficiency is very high.
Based on the single-threaded Reactor model, it is improved to multi-threaded mode according to the actual usage scenario. There are two common ways: one is to introduce multiple threads in the time-consuming process processor, such as using a thread pool; the other is to directly use multiple Reactor instances, one for each Reactor instance.
An improved way of Reactor mode is shown in the figure. Its overall structure is basically similar to the single-threaded Reactor, but introduces a thread pool. Since operations such as receiving connections, reading data, and writing data are basically less time consuming, they are all handled in the Reactor thread. However, for logical processing that may be time consuming, a thread pool can be introduced in the process processor. The process processor does not execute the task itself, but instead hands it to the thread pool, thereby avoiding time consuming operations in the Reactor thread. After moving time-consuming operations into the thread pool, Reactor is guaranteed to be efficient even though Reactor has only one thread.
Another improvement to the Reactor mode is shown in the figure. There are multiple Reactor instances, one for each Reactor instance. Because the receiving event is relative to the server, the client's connection receiving work is handled by an accept handler. The accept handler will evenly distribute the received client connections to all Reactor instances. Each Reactor instance is responsible for processing the allocation. Client connections to the Reactor, including read data, write data, and logical processing of the connection. This is the principle of multiple Reactor instances.
Multi-threaded non-blocking I/O mode greatly improves server-side processing power. It takes full advantage of the machine's CPU and is suitable for handling high-concurrency scenarios, but it also makes the program more complex and prone to problems.
The material of this product is PC+ABS. All condition of our product is 100% brand new. OEM and ODM are avaliable of our products for your need. We also can produce the goods according to your specific requirement.
Our products built with input/output overvoltage protection, input/output overcurrent protection, over temperature protection, over power protection and short circuit protection. You can send more details of this product, so that we can offer best service to you!
Led Adapter,Mini Led Adapter,Security Led Adapter,Waterproof Led Adapter
Shenzhen Waweis Technology Co., Ltd. , https://www.huaweishiadapter.com