Fetch the repository succeeded.
【特性描述】
通过ebpf实现可编程内核,针对应用的读取方式调整内核的预读行为,达到spark sql场景的读性能优化,
【特性竞争力】
通过ebpf实现可编程内核,针对应用的读取方式调整内核的预读行为,避免反复下发小IO磁头跳动带来开销。
【硬件架构】
NoArch
【特性约束】
NA
【涉及仓库】
OLK-6.6
【验证记录】
打上验证补丁:
diff --git a/mm/readahead.c b/mm/readahead.c
index c3f45b710043..8599bb5147eb 100644
--- a/mm/readahead.c
+++ b/mm/readahead.c
@@ -717,7 +717,10 @@ void page_cache_sync_ra(struct readahead_control *ractl,
req_count = 1;
do_forced_ra = true;
}
-
+ if (ractl->file->f_ctl_mode & FMODE_CTL_RANDOM)
+ printk("FMODE_CTL_RANDOM was set\n");
+ if (ractl->file->f_ctl_mode & FMODE_CTL_WILLNEED)
+ printk("FMODE_CTL_WILLNEED was set\n");
/* try to read first READAHEAD_FIRST_SIZE into pagecache */
if (ractl->file && (ractl->file->f_ctl_mode & FMODE_CTL_WILLNEED) &&
page_cache_readahead_from_head(ractl->mapping, ractl->file,
执行测试用例:
root@debian:~/readahead# sh testcase00.sh
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks: done
Creating filesystem with 2621440 4k blocks and 655360 inodes
Filesystem UUID: bf06d5b8-7e4b-4913-89e0-1dc1a80a8c1a
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
sleep 3s for file_read_pattern launch
2+0 records in
2+0 records out
2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0220223 s, 95.2 MB/s
test FMODE_WILLNEED
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00451005 s, 232 MB/s
[ 1455.728379] FMODE_CTL_WILLNEED was set
test set FMODE_RANDOM
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.0714635 s, 147 MB/s
[ 1455.967379] FMODE_CTL_RANDOM was set
[ 1455.969469] FMODE_CTL_RANDOM was set
[ 1455.971596] FMODE_CTL_RANDOM was set
......
[ 1457.781893] FMODE_CTL_RANDOM was set
[ 1457.783646] FMODE_CTL_RANDOM was set
test successed!
root@debian:~/readahead# sh testcase01.sh
meta-data=/dev/sda isize=512 agcount=4, agsize=655360 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=1
= reflink=1 bigtime=1 inobtcount=1 nrext64=1
data = bsize=4096 blocks=2621440, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=16384, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Discarding blocks...Done.
sleep 3s for file_read_pattern launch
2+0 records in
2+0 records out
2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0200794 s, 104 MB/s
test FMODE_WILLNEED
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0050548 s, 207 MB/s
[ 1581.737010] FMODE_CTL_WILLNEED was set
test set FMODE_RANDOM
testtest: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.36
Starting 1 thread
testtest: Laying out IO file (1 file / 50MiB)
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.502477 s, 209 MB/s
testtest: (groupid=0, jobs=1): err= 0: pid=3508: Mon Jan 15 14:08:20 2024
read: IOPS=685, BW=2743KiB/s (2809kB/s)(50.0MiB/18665msec)
clat (usec): min=389, max=18254, avg=1454.09, stdev=339.92
lat (usec): min=389, max=18255, avg=1454.49, stdev=339.96
clat percentiles (usec):
| 1.00th=[ 1074], 5.00th=[ 1188], 10.00th=[ 1254], 20.00th=[ 1336],
| 30.00th=[ 1369], 40.00th=[ 1401], 50.00th=[ 1434], 60.00th=[ 1467],
| 70.00th=[ 1516], 80.00th=[ 1549], 90.00th=[ 1614], 95.00th=[ 1696],
| 99.00th=[ 2024], 99.50th=[ 2114], 99.90th=[ 2671], 99.95th=[ 9503],
| 99.99th=[16909]
bw ( KiB/s): min= 2272, max= 3016, per=100.00%, avg=2747.03, stdev=144.66, samples=37
iops : min= 568, max= 754, avg=686.76, stdev=36.16, samples=37
lat (usec) : 500=0.01%, 750=0.05%, 1000=0.23%
lat (msec) : 2=98.50%, 4=1.12%, 10=0.05%, 20=0.03%
cpu : usr=0.14%, sys=57.88%, ctx=12800, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=12800,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=2743KiB/s (2809kB/s), 2743KiB/s-2743KiB/s (2809kB/s-2809kB/s), io=50.0MiB (52.4MB), run=18665-18665msec
Disk stats (read/write):
sda: ios=12680/80, sectors=101440/204800, merge=0/0, ticks=7575/984, in_queue=8560, util=98.67%
[ 1582.126897] FMODE_CTL_RANDOM was set
[ 1582.129017] FMODE_CTL_RANDOM was set
[ 1582.131326] FMODE_CTL_RANDOM was set
......
[ 1600.774171] FMODE_CTL_RANDOM was set
[ 1600.775609] FMODE_CTL_RANDOM was set
[ 1600.776854] FMODE_CTL_RANDOM was set
test clear FMODE_RANDOM
test successed!
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
Sign in to comment