402 Star 1.4K Fork 1.3K

GVPopenEuler / kernel

 / 详情

Spark SQL场景IO bpf readahead 优化 同步OLK-6.6

Done
Task
Opened this issue  
2023-12-13 17:41

【特性描述】
通过ebpf实现可编程内核,针对应用的读取方式调整内核的预读行为,达到spark sql场景的读性能优化,
【特性竞争力】
通过ebpf实现可编程内核,针对应用的读取方式调整内核的预读行为,避免反复下发小IO磁头跳动带来开销。
【硬件架构】
NoArch
【特性约束】
NA
【涉及仓库】
OLK-6.6
【验证记录】

打上验证补丁:

diff --git a/mm/readahead.c b/mm/readahead.c
index c3f45b710043..8599bb5147eb 100644
--- a/mm/readahead.c
+++ b/mm/readahead.c
@@ -717,7 +717,10 @@ void page_cache_sync_ra(struct readahead_control *ractl,
 		req_count = 1;
 		do_forced_ra = true;
 	}
-
+	if (ractl->file->f_ctl_mode & FMODE_CTL_RANDOM)
+		printk("FMODE_CTL_RANDOM was set\n");
+	if (ractl->file->f_ctl_mode & FMODE_CTL_WILLNEED)
+		printk("FMODE_CTL_WILLNEED was set\n");
 	/* try to read first READAHEAD_FIRST_SIZE into pagecache */
 	if (ractl->file && (ractl->file->f_ctl_mode & FMODE_CTL_WILLNEED) &&
 		page_cache_readahead_from_head(ractl->mapping, ractl->file,

执行测试用例:

root@debian:~/readahead# sh testcase00.sh
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks: done                            
Creating filesystem with 2621440 4k blocks and 655360 inodes
Filesystem UUID: bf06d5b8-7e4b-4913-89e0-1dc1a80a8c1a
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

sleep 3s for file_read_pattern launch
2+0 records in
2+0 records out
2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0220223 s, 95.2 MB/s
test FMODE_WILLNEED
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00451005 s, 232 MB/s
[ 1455.728379] FMODE_CTL_WILLNEED was set
test set FMODE_RANDOM
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.0714635 s, 147 MB/s
[ 1455.967379] FMODE_CTL_RANDOM was set
[ 1455.969469] FMODE_CTL_RANDOM was set
[ 1455.971596] FMODE_CTL_RANDOM was set
......
[ 1457.781893] FMODE_CTL_RANDOM was set
[ 1457.783646] FMODE_CTL_RANDOM was set
test successed!

root@debian:~/readahead# sh testcase01.sh
meta-data=/dev/sda               isize=512    agcount=4, agsize=655360 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=1
         =                       reflink=1    bigtime=1 inobtcount=1 nrext64=1
data     =                       bsize=4096   blocks=2621440, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=16384, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
Discarding blocks...Done.
sleep 3s for file_read_pattern launch
2+0 records in
2+0 records out
2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0200794 s, 104 MB/s
test FMODE_WILLNEED
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0050548 s, 207 MB/s
[ 1581.737010] FMODE_CTL_WILLNEED was set
test set FMODE_RANDOM
testtest: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.36
Starting 1 thread
testtest: Laying out IO file (1 file / 50MiB)
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.502477 s, 209 MB/s

testtest: (groupid=0, jobs=1): err= 0: pid=3508: Mon Jan 15 14:08:20 2024
  read: IOPS=685, BW=2743KiB/s (2809kB/s)(50.0MiB/18665msec)
    clat (usec): min=389, max=18254, avg=1454.09, stdev=339.92
     lat (usec): min=389, max=18255, avg=1454.49, stdev=339.96
    clat percentiles (usec):
     |  1.00th=[ 1074],  5.00th=[ 1188], 10.00th=[ 1254], 20.00th=[ 1336],
     | 30.00th=[ 1369], 40.00th=[ 1401], 50.00th=[ 1434], 60.00th=[ 1467],
     | 70.00th=[ 1516], 80.00th=[ 1549], 90.00th=[ 1614], 95.00th=[ 1696],
     | 99.00th=[ 2024], 99.50th=[ 2114], 99.90th=[ 2671], 99.95th=[ 9503],
     | 99.99th=[16909]
   bw (  KiB/s): min= 2272, max= 3016, per=100.00%, avg=2747.03, stdev=144.66, samples=37
   iops        : min=  568, max=  754, avg=686.76, stdev=36.16, samples=37
  lat (usec)   : 500=0.01%, 750=0.05%, 1000=0.23%
  lat (msec)   : 2=98.50%, 4=1.12%, 10=0.05%, 20=0.03%
  cpu          : usr=0.14%, sys=57.88%, ctx=12800, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=12800,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=2743KiB/s (2809kB/s), 2743KiB/s-2743KiB/s (2809kB/s-2809kB/s), io=50.0MiB (52.4MB), run=18665-18665msec

Disk stats (read/write):
  sda: ios=12680/80, sectors=101440/204800, merge=0/0, ticks=7575/984, in_queue=8560, util=98.67%
[ 1582.126897] FMODE_CTL_RANDOM was set
[ 1582.129017] FMODE_CTL_RANDOM was set
[ 1582.131326] FMODE_CTL_RANDOM was set
......
[ 1600.774171] FMODE_CTL_RANDOM was set
[ 1600.775609] FMODE_CTL_RANDOM was set
[ 1600.776854] FMODE_CTL_RANDOM was set
test clear FMODE_RANDOM
test successed!

Comments (1)

Hi wangzhaolong1, welcome to the openEuler Community.
I'm the Bot here serving you. You can find the instructions on how to interact with me at Here.
If you have any questions, please contact the SIG: Kernel, and any of the maintainers.

WangZhaoLong created任务
openeuler-ci-bot added
 
sig/Kernel
label
zhengzengkai set branch to OLK-6.6
WangZhaoLong changed description
WangZhaoLong changed description
WangZhaoLong changed description
WangZhaoLong changed description
WangZhaoLong changed description
WangZhaoLong changed description
ci-robot throughopeneuler/kernel Pull Request !3895 changed issue state from 待办的 to 已完成
Expand operation logs

Sign in to comment

Status
Assignees
Projects
Milestones
Pull Requests
Successfully merging a pull request will close this issue.
Branches
Planed to start   -   Planed to end
-
Top level
Priority
Duration (hours)
参与者(2)
5329419 openeuler ci bot 1632792936
C
1
https://gitee.com/openeuler/kernel.git
git@gitee.com:openeuler/kernel.git
openeuler
kernel
kernel

Search