【OLK-6.6】回合6.7至6.6-mm: PCP high auto-tuning

【特性描述】
不同工作负载的页面分配性能要求通常是不同的。因此，我们需要自动调整每个 CPU 的 pcp(每个 CPU 页面集)的大小以优化页面分配性能。

引入config 配置：PCP_BATCH_SCALE_MAX ， default 5， range 0 6
该系列的补丁如下：
[1/9] mm, pcp: avoid to drain PCP when process exit
[2/9] cacheinfo: calculate per-CPU data cache size
[3/9] mm, pcp: reduce lock contention for draining high-order pages
[4/9] mm: restrict the pcp batch scale factor to avoid too long latency
[5/9] mm, page_alloc: scale the number of pages that are batch allocated
[6/9] mm: add framework for PCP high auto-tuning
[7/9] mm: tune PCP high automatically
[8/9] mm, pcp: decrease PCP high if free pages < high watermark
[9/9] mm, pcp: reduce detecting time of consecutive high order page freeing

补丁 [1/9]、[2/9]、[3/9] 优化了连续释放高阶页时 PCP 的耗尽。

补丁 [4/9]、[5/9] 优化了批量释放和分配。

补丁 [6/9]、[7/9]、[8/9] 实现并优化了 PCP 高自动调整方法。

补丁 [9/9] 在 PCP 高自动调整的基础上，优化了连续高阶页面释放的 PCP 的耗尽，减少zone 锁的争用。

The test results for patches with performance impact are as follows,

kbuild
======

On a 2-socket Intel server with 224 logical CPU, we run 8 kbuild
instances in parallel (each with `make -j 28`) in 8 cgroup.  This
simulates the kbuild server that is used by 0-Day kbuild service.

build time   lock contend%	free_high	alloc_zone
	----------	----------	---------	----------
base	     100.0	      14.0          100.0            100.0
patch1	      99.5	      12.8	     19.5	      95.6
patch3	      99.4	      12.6	      7.1	      95.6
patch5	      98.6	      11.0	      8.1	      97.1
patch7	      95.1	       0.5	      2.8	      15.6
patch9	      95.0	       1.0	      8.8	      20.0

The PCP draining optimization (patch [1/9], [3/9]) and PCP batch
allocation optimization (patch [5/9]) reduces zone lock contention a
little.  The PCP high auto-tuning (patch [7/9], [9/9]) reduces build
time visibly.  Where the tuning target: the number of pages allocated
from zone reduces greatly.  So, the zone contention cycles% reduces
greatly.

With PCP tuning patches (patch [7/9], [9/9]), the average used memory
during test increases up to 18.4% because more pages are cached in
PCP.  But at the end of the test, the number of the used memory
decreases to the same level as that of the base patch.  That is, the
pages cached in PCP will be released to zone after not being used
actively.

netperf SCTP_STREAM_MANY
========================

On a 2-socket Intel server with 128 logical CPU, we tested
SCTP_STREAM_MANY test case of netperf test suite with 64-pair
processes.

score   lock contend%	free_high	alloc_zone  cache miss rate%
	     -----	----------	---------	----------  ----------------
base	     100.0	       2.1          100.0            100.0	         1.3
patch1	      99.4	       2.1	     99.4	      99.4		 1.3
patch3	     106.4	       1.3	     13.3	     106.3		 1.3
patch5	     106.0	       1.2	     13.2	     105.9		 1.3
patch7	     103.4	       1.9	      6.7	      90.3		 7.6
patch9	     108.6	       1.3	     13.7	     108.6		 1.3

The PCP draining optimization (patch [1/9]+[3/9]) improves
performance.  The PCP high auto-tuning (patch [7/9]) reduces
performance a little because PCP draining cannot be triggered in time
sometimes.  So, the cache miss rate% increases.  The further PCP
draining optimization (patch [9/9]) based on PCP tuning restore the
performance.

lmbench3 UNIX (AF_UNIX)
=======================

On a 2-socket Intel server with 128 logical CPU, we tested UNIX
(AF_UNIX socket) test case of lmbench3 test suite with 16-pair
processes.

score   lock contend%	free_high	alloc_zone  cache miss rate%
	     -----	----------	---------	----------  ----------------
base	     100.0	      51.4          100.0            100.0	         0.2
patch1	     116.8	      46.1           69.5	     104.3	         0.2
patch3	     199.1	      21.3            7.0	     104.9	         0.2
patch5	     200.0	      20.8            7.1	     106.9	         0.3
patch7	     191.6	      19.9            6.8	     103.8	         2.8
patch9	     193.4	      21.7            7.0	     104.7	         2.1

The PCP draining optimization (patch [1/9], [3/9]) improves
performance much.  The PCP tuning (patch [7/9]) reduces performance a
little because PCP draining cannot be triggered in time sometimes.
The further PCP draining optimization (patch [9/9]) based on PCP
tuning restores the performance partly.

The patchset adds several fields in struct per_cpu_pages.  The struct
layout before/after the patchset is as follows,

base
====

struct per_cpu_pages {
	spinlock_t                 lock;                 /*     0     4 */
	int                        count;                /*     4     4 */
	int                        high;                 /*     8     4 */
	int                        batch;                /*    12     4 */
	short int                  free_factor;          /*    16     2 */
	short int                  expire;               /*    18     2 */

/* XXX 4 bytes hole, try to pack */

struct list_head           lists[13];            /*    24   208 */

/* size: 256, cachelines: 4, members: 7 */
	/* sum members: 228, holes: 1, sum holes: 4 */
	/* padding: 24 */
} __attribute__((__aligned__(64)));

patched
=======

struct per_cpu_pages {
	spinlock_t                 lock;                 /*     0     4 */
	int                        count;                /*     4     4 */
	int                        high;                 /*     8     4 */
	int                        high_min;             /*    12     4 */
	int                        high_max;             /*    16     4 */
	int                        batch;                /*    20     4 */
	u8                         flags;                /*    24     1 */
	u8                         alloc_factor;         /*    25     1 */
	u8                         expire;               /*    26     1 */

/* XXX 1 byte hole, try to pack */

short int                  free_count;           /*    28     2 */

/* XXX 2 bytes hole, try to pack */

struct list_head           lists[13];            /*    32   208 */

/* size: 256, cachelines: 4, members: 11 */
	/* sum members: 237, holes: 2, sum holes: 3 */
	/* padding: 16 */
} __attribute__((__aligned__(64)));

The size of the struct doesn't changed with the patchset.

【特性竞争力】
1）优化进程退出时高级页面释放导致PCP 的耗尽
【硬件架构】
NoArch
【特性约束】
NA
【涉及仓库】

【交付个人/团队】

GVP openEuler/kernel

Content Risk Flag

Comments (2)

GVPopenEuler/kernel

Content Risk Flag