[5.10] [Feature] : SMC-R connection optimization and data path general optimization · Pull Request !577 · anolis/cloud-kernel

bugzilla: https://bugzilla.openanolis.cn/show_bug.cgi?id=1461

Description of problem:

1. Backport upstream
2. Connect

There are two main issue, one is that the establishment of a single connection takes longer than that of the TCP, another is that the degree of concurrency
is low under multi-connection processing. This patch set is mainly
optimized for the first issue, and the follow-up of the second issue
will be synchronized in the future.

The internal bottleneck of Linux SMC implementation must be resolved,
and there are two different issues around our request.

through the off-CPU flame graphs, SMC workers' off-CPU as that:

__mutex_lock.isra.11 (47.96%)
    smc_listen_work (48.17%)
    process_one_work (48.23%)
    work_thread (48.23%)
    kthread (48.23%)
    ret_from_fork (48.23%)
    kworker (48.23%)

We can clearly see two issues:

1. Workers spend a lot of waiting time on locks in smc_listen_work, which is a global lock, named
smc_server_lgr_pending. It makes almost only one SMC connection can complete the handshake at the same time,
and other has to wait. During benchmark, we can easily observe this through PS command, lots of workers are
suspend on D state rather than S state.

2. At present, The handshake process of the SMC connection is handled by Linux workqueue,
which is essentially a fixed size kthread pool. In the implementation of Linux, an SMC connection will
occupy a kthread in workqueue’s pool until the handshake is completed. In off-CPU flame graphs above,
all kworker in workqueue are suspend in smc_listen_work, which means no worker are free.
Once the number of concurrent connections exceeds the upper limit of workqueue, new incoming connections has to wait.
The upper limit is 256 * N cores by default, and the maximum is 512 * N cores. We have plans to give up the workqueue
and implement SMC with a fully asynchronous approach.

3. general optimization of data path

类型	指派人员	状态
审查	Wen Gu	已完成 (0/0人)
测试	小龙已测试通过	已完成 (1/1人)

GVPanolis/cloud-kernel

搜索帮助

GVP anolis/cloud-kernel