11 Star 1 Fork 27

src-openEuler / bpftrace

 / 详情

bpftrace安装后执行出现crash与coredump

已完成
缺陷
创建于  
2022-09-19 16:42

【标题描述】能够简要描述问题:说明什么场景下,做了什么操作,出现什么问题(尽量使用正向表达方式)
【环境信息】
硬件信息:
1) 裸机场景提供出问题的硬件信息;
2) 虚机场景提供虚机XML文件或者配置信息
软件信息:
1) OS版本及分支 22.09
2) 内核信息 5.10.0-106.12.0.64.oe2209.x86_64/aarch64
3) 发现问题的组件版本信息
如果有特殊组网,请提供网络拓扑图
【问题复现步骤】
具体操作步骤
1、安装bpftrace
输入图片说明
2、执行bpftrace /usr/share/bpftrace/tools/bashreadline.bt
出现概率(是否必现,概率性错误)
必现
【预期结果】
描述预期结果,可以通过对比新老版本获取
程序挂载成功,无coredump和crash产生
输入图片说明
【实际结果】
描述出问题的结果
出现coredump
输入图片说明
出现crash
[ 4404.801962] CPU: 2 PID: 2509 Comm: bpftrace Kdump: loaded Not tainted 5.10.0-106.9.0.62.oe2209.x86_64 #1
[ 4404.801963] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58-20220525_182517-szxrtosci10000 04/01/2014
[ 4404.801972] RIP: 0010:uprobe_unregister+0x46/0x50
[ 4404.801976] Code: 89 c5 4c 89 ef e8 7a 42 7e 00 4c 89 e6 48 89 ef e8 ff fe ff ff 4c 89 ef e8 37 ce eb ff 48 89 ef 5d 41 5c 41 5d e9 5a ec ff ff <0f> 0b 5d 41 5c 41 5d c3 66 90 0f 1f 44 00 00 41 57 41 56 49 89 d6
[ 4404.801977] RSP: 0018:ffffafdb80823c90 EFLAGS: 00010246
[ 4404.801979] RAX: 0000000000000000 RBX: ffffa02a07f89f70 RCX: ffffa02a01865510
[ 4404.801979] RDX: 0000000000000001 RSI: 00000000000f6240 RDI: ffffffffbeae7c00
[ 4404.801980] RBP: ffffa02a07f89f70 R08: 0000000000000000 R09: ffffa02a00a51ae8
[ 4404.801981] R10: 0000000000223bbf R11: 000000000022362d R12: ffffa02a07f89f18
[ 4404.801982] R13: ffffa02a03ae2008 R14: ffffafdb80823ad8 R15: dead000000000100
[ 4404.801987] FS: 0000000000000000(0000) GS:ffffa02a3bd00000(0000) knlGS:0000000000000000
[ 4404.801988] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4404.801989] CR2: 00007f5cfd718288 CR3: 000000000e40a001 CR4: 0000000000170ee0
[ 4404.801993] Call Trace:
[ 4404.802012] __probe_event_disable+0x3d/0x70
[ 4404.802015] probe_event_disable+0x40/0x50
[ 4404.802017] trace_uprobe_register+0x5c/0x60
[ 4404.802022] perf_trace_event_unreg.isra.0+0x26/0x90
[ 4404.802024] perf_uprobe_destroy+0x38/0x60
[ 4404.802027] _free_event+0xfc/0x1e0
[ 4404.802029] perf_event_release_kernel+0x287/0x2d0
[ 4404.802032] perf_release+0xc/0x10
[ 4404.802036] __fput+0xcb/0x240
[ 4404.802041] task_work_run+0x5f/0x90
[ 4404.802046] do_exit+0x22f/0x3f0
[ 4404.802048] do_group_exit+0x33/0xa0
[ 4404.802051] get_signal+0x158/0x520
[ 4404.802056] arch_do_signal+0x2a/0x1d0
[ 4404.802059] exit_to_user_mode_loop+0xa9/0xc0
[ 4404.802061] exit_to_user_mode_prepare+0x79/0x80
[ 4404.802066] syscall_exit_to_user_mode+0x12/0x40
[ 4404.802070] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 4404.802081] RIP: 0033:0x556c9e7ad241
[ 4404.802083] Code: Unable to access opcode bytes at RIP 0x556c9e7ad217.
[ 4404.802084] RSP: 002b:00007ffd72207328 EFLAGS: 00000202
[ 4404.802085] RAX: 0000556c9e7ad240 RBX: 00007f677808b710 RCX: 0000000000000031
[ 4404.802086] RDX: 00007f678030db00 RSI: 0000000000000000 RDI: 00007ffd722070a0
[ 4404.802087] RBP: 00007ffd72207340 R08: 00007ffd722070b0 R09: 0000556c9f9df684
[ 4404.802088] R10: 00000000ffffffff R11: 2fe9fc0345d00c1d R12: 00007ffd72208510
[ 4404.802089] R13: 00007ffd72207500 R14: 00007ffd72208858 R15: 00007f677808b5e0
[ 4404.802095] ---[ end trace 116485cf0ea41210 ]---

【附件信息】
比如系统message日志/组件日志、dump信息、图片等

评论 (3)

wangting 创建了缺陷

Hi wangting112, welcome to the openEuler Community.
I'm the Bot here serving you. You can find the instructions on how to interact with me at Here.
If you have any questions, please contact the SIG: dev-utils, and any of the maintainers: @jingxiaolu , @ethan848 , @myeuler , @fcwicky , @yanan-rock , @hexiaowen , @love_hangzhou , @forrest_ly , @Monday , @caodongxia

openeuler-ci-bot 添加了
 
sig/dev-utils
标签
wangting 修改了描述
wangting 修改了描述
wangting 修改了描述
zhangpanting 优先级设置为主要
zhangpanting 计划截止日期设置为2022-09-22
zhangpanting 计划开始日期设置为2022-09-21

附加dmesg完整日志
[ 98.936016] ------------[ cut here ]------------
[ 98.936419] WARNING: CPU: 0 PID: 1981 at kernel/events/uprobes.c:1114 uprobe_unregister+0x68/0x80
[ 98.937100] Modules linked in: xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT nf_nat_tftp nft_objref nf_conntrack_tftp tun bridge stp llc nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_tables ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_mangle iptable_raw iptable_security rfkill ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sunrpc nls_cp437 vfat fat sch_fq_codel fuse ext4 mbcache jbd2 virtio_gpu virtio_net net_failover virtio_dma_buf virtio_blk virtio_console failover ghash_ce sha2_ce sha256_arm64 sha1_ce virtio_pci virtio_mmio virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod aes_neon_bs aes_neon_blk aes_ce_blk crypto_simd cryptd aes_ce_cipher
[ 98.942895] CPU: 0 PID: 1981 Comm: bpftrace Kdump: loaded Not tainted 5.10.0-106.12.0.64.oe2209.aarch64 #1
[ 98.943634] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[ 98.944164] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO BTYPE=--)
[ 98.944635] pc : uprobe_unregister+0x68/0x80
[ 98.944990] lr : uprobe_unregister+0x20/0x80
[ 98.945339] sp : ffff80001435bad0
[ 98.945629] x29: ffff80001435bad0 x28: ffff8000119f9dc8
[ 98.946056] x27: 0000000000000005 x26: 0000000000000000
[ 98.946489] x25: ffff8000119f9dc8 x24: ffff0000fe908af4
[ 98.946935] x23: ffff0000f8ba2a20 x22: ffff80001435b9d8
[ 98.947359] x21: ffff0000004c8c70 x20: ffff0000004c8c18
[ 98.947784] x19: ffff0000004c8c70 x18: 0000000000000020
[ 98.948208] x17: 0000000000000000 x16: 0000000000000000
[ 98.948633] x15: ffffffffffffffff x14: ffff00000cc83ec0
[ 98.949059] x13: ffffffffffffffff x12: ffff00000cc83833
[ 98.949482] x11: 000000948dfddd8c x10: 0000000000029027
[ 98.949913] x9 : ffff8000102d2670 x8 : 0000000000c8987d
[ 98.950346] x7 : 00000000000003e2 x6 : 0000000000027a69
[ 98.950785] x5 : 0000000000000000 x4 : 0000000000000000
[ 98.951215] x3 : ffff800012159000 x2 : ffff0000fca61080
[ 98.951639] x1 : 0000000000000000 x0 : 0000000000000000
[ 98.952061] Call trace:
[ 98.952285] uprobe_unregister+0x68/0x80
[ 98.952613] __probe_event_disable+0x50/0x90
[ 98.952962] probe_event_disable+0x68/0x90
[ 98.953301] trace_uprobe_register+0x8c/0xb0
[ 98.953651] perf_trace_event_unreg.isra.0+0x40/0xb0
[ 98.954052] perf_uprobe_destroy+0x44/0x64
[ 98.954407] _free_event+0xd4/0x200
[ 98.954720] put_event+0x44/0x50
[ 98.955004] perf_event_release_kernel+0x250/0x2dc
[ 98.955392] perf_release+0x14/0x24
[ 98.955699] __fput+0xbc/0x240
[ 98.955967] ____fput+0x18/0x24
[ 98.956248] task_work_run+0xd0/0x230
[ 98.956563] do_exit+0x1b4/0x3f4
[ 98.956847] do_group_exit+0x40/0x114
[ 98.957160] get_signal+0x190/0x70c
[ 98.957458] do_signal+0x13c/0x1f0
[ 98.957746] do_notify_resume+0x144/0x27c
[ 98.958078] work_pending+0xc/0xa4
[ 98.958381] ---[ end trace f6b851c1bac867b3 ]---
[ 296.254171] Unable to handle kernel paging request at virtual address fecffffa00000087
[ 296.254788] Mem abort info:
[ 296.255027] ESR = 0x96000004
[ 296.255276] EC = 0x25: DABT (current EL), IL = 32 bits
[ 296.255668] SET = 0, FnV = 0
[ 296.255911] EA = 0, S1PTW = 0
[ 296.256166] Data abort info:
[ 296.256419] ISV = 0, ISS = 0x00000004
[ 296.256714] CM = 0, WnR = 0
[ 296.256959] [fecffffa00000087] address between user and kernel address ranges
[ 296.257467] Internal error: Oops: 96000004 [#1] SMP
[ 296.257828] Modules linked in: xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT nf_nat_tftp nft_objref nf_conntrack_tftp tun bridge stp llc nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_tables ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_mangle iptable_raw iptable_security rfkill ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sunrpc nls_cp437 vfat fat sch_fq_codel fuse ext4 mbcache jbd2 virtio_gpu virtio_net net_failover virtio_dma_buf virtio_blk virtio_console failover ghash_ce sha2_ce sha256_arm64 sha1_ce virtio_pci virtio_mmio virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod aes_neon_bs aes_neon_blk aes_ce_blk crypto_simd cryptd aes_ce_cipher
[ 296.263393] CPU: 0 PID: 2008 Comm: systemd-hostnam Kdump: loaded Tainted: G W 5.10.0-106.12.0.64.oe2209.aarch64 #1
[ 296.264272] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[ 296.264814] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO BTYPE=--)
[ 296.265289] pc : build_probe_list+0x90/0x210
[ 296.265652] lr : uprobe_mmap+0x100/0x220
[ 296.265980] sp : ffff800012e23890
[ 296.266264] x29: ffff800012e23890 x28: 0000000000000000
[ 296.266707] x27: 0000aaaab1860000 x26: 0000000000000000
[ 296.267140] x25: ffff800012159588 x24: ffff8000119f9dc8
[ 296.267568] x23: 0000000000020fff x22: ffff0000036fc1d8
[ 296.267996] x21: ffff800012e23948 x20: ffff0000006905a0
[ 296.268433] x19: fecffff9ffffffff x18: 0000000000000000
[ 296.268876] x17: 0000000000000000 x16: 0000000000000000
[ 296.269316] x15: 0000aaab08a80770 x14: 0000000000000200
[ 296.269747] x13: 13b0000000000000 x12: ffff0000facc0000
[ 296.270177] x11: ffff0000fbade280 x10: 0000000000000000
[ 296.270613] x9 : ffff8000102d2be0 x8 : ffff000000690658
[ 296.271048] x7 : 0000000000000000 x6 : 000000000000003f
[ 296.271475] x5 : 0000000000000020 x4 : 0000000000000000
[ 296.271898] x3 : ffff800012159000 x2 : ffffffffffffffff
[ 296.272338] x1 : 0000000000000000 x0 : 0000000000000000
[ 296.272771] Call trace:
[ 296.272997] build_probe_list+0x90/0x210
[ 296.273323] uprobe_mmap+0x100/0x220
[ 296.273631] __mmap_region+0x1dc/0x690
[ 296.273952] __do_mmap_mm+0x33c/0x560
[ 296.274264] do_mmap+0x84/0x220
[ 296.274554] vm_mmap_pgoff+0xc4/0x1e4
[ 296.274869] vm_mmap+0x34/0x4c
[ 296.275141] elf_map+0x90/0x130
[ 296.275414] load_elf_binary+0x44c/0xc90
[ 296.275742] search_binary_handler+0x128/0x2ec
[ 296.276108] exec_binprm+0x5c/0x1c0
[ 296.276404] bprm_execve.part.0+0xec/0x150
[ 296.276750] bprm_execve+0x7c/0xb0
[ 296.277040] do_execveat_common+0x1c4/0x240
[ 296.277392] __arm64_sys_execve+0x48/0x60
[ 296.277725] el0_svc_common.constprop.0+0x7c/0x1bc
[ 296.278117] do_el0_svc+0x34/0xd0
[ 296.278403] el0_svc+0x20/0x30
[ 296.278678] el0_sync_handler+0xb8/0xc0
[ 296.279009] el0_sync+0x160/0x180
[ 296.279293] Code: 35000440 f9400733 b40001b3 d503201f (f9404662)
[ 296.281688] Starting crashdump kernel...

经过排查属于 intel 相关提交引入的问题
https://gitee.com/openeuler/kernel/commit/180b1ed46e4d8c01c762f6399b108eae138a37ef
引入问题对应pr:引入问题的PR:
https://gitee.com/openeuler/kernel/pulls/82

后续待kernel修复后 进行验证

zhangpanting 负责人设置为mdche

bpftrace BEGIN相关命令都有此类问题
输入图片说明
输入图片说明
输入图片说明

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(4)
5329419 openeuler ci bot 1632792936
1
https://gitee.com/src-openeuler/bpftrace.git
git@gitee.com:src-openeuler/bpftrace.git
src-openeuler
bpftrace
bpftrace

搜索帮助