【fuzz】ABRT on unknown address 0x00000000000d

【环境信息】
系统：x86
【测试版本】
Name: dovecot
Version: 2.3.10.1
Release: 5
openEuler-20.03-LTS
【注意】
受影响版本排查（受影响/不受影响）：
1、master
2、openEuler-LTS-20.03
3、openEuler-LTS-20.03-SP1
4、openEuler-LTS-20.03-Next
5、openEuler-20.09
【测试步骤】
1、编译
python infra/helper.py build_fuzzers --sanitizer=address dovecot
2、执行
python infra/helper.py run_fuzzer dovecot fuzz-imap-bodystructure -rss_limit_mb=0
【报错信息】

==13==ERROR: AddressSanitizer: ABRT on unknown address 0x00000000000d (pc 0x7f536e86d428 bp 0x7ffe75d3f830 sp 0x7ffe75d3f688
SCARINESS: 10 (signal)
    #0 0x7f536e86d428 in raise (/lib/x86_64-linux-gnu/libc.so.6+0x35428)
    #1 0x7f536e86f029 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x37029)
    #2 0x568f4b in default_fatal_finish /src/dovecot/src/lib/failures.c:459:3
    #3 0x5655d3 in fatal_handler_real /src/dovecot/src/lib/failures.c:471:2
    #4 0x564d7d in default_fatal_handler /src/dovecot/src/lib/failures.c:479:2
    #5 0x564f10 in i_panic /src/dovecot/src/lib/failures.c:523:2
    #6 0x55642a in part_write_bodystructure_common /src/dovecot/src/lib-imap/./imap-bodystructure.c:95:3
    #7 0x55179a in part_write_body /src/dovecot/src/lib-imap/./imap-bodystructure.c:239:2
    #8 0x55107a in imap_bodystructure_write /src/dovecot/src/lib-imap/./imap-bodystructure.c:248:3
    #9 0x5558d1 in LLVMFuzzerTestOneInput /src/dovecot/src/lib-imap/fuzz-imap-bodystructure.c:31:3
    #10 0x459441 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/f
    #11 0x458b85 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/llvm-per/FuzzerLoop.cpp:470:3
    #12 0x45ac57 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:698:19
    #13 0x45b6d5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFilepiler-rt/lib/fuzzer/FuzzerLoop.cpp:832:5
    #14 0x44a6ae in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiver.cpp:826:6
    #15 0x472e82 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10
    #16 0x7f536e85882f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #17 0x41edb8 in _start (/out/fuzz-imap-bodystructure+0x41edb8)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: ABRT (/lib/x86_64-linux-gnu/libc.so.6+0x35428) in raise
==13==ABORTING
MS: 1 CopyPart-; base unit: 0bb4ed75b15ac8c06e403f9ac3ff1b0b2400b81f
0x22,0x22,0x40,0x20,0x28,0x29,0x2d,0x20,0x60,0x20,0x3a,0x20,0x30,0x20,0x39,0x20,0x28,0x6d,0x70,0x20,0x28,0x29,0x2d,0x20,0x29x20,0x60,0x20,0x0,0x0,0x0,0x0,
\"\"@ ()- ` : 0 9 (mp ()- ) ()-  ` \x00\x00\x00\x00
artifact_prefix='./'; Test unit written to ./crash-1907479da020c54ab62e638a3bb35aa569d67ca3

【问题复现】
python infra/helper.py reproduce dovecot fuzz-imap-bodystructure crash-1907479da020c54ab62e638a3bb35aa569d67ca3

【测试步骤】
1、编译
python infra/helper.py build_fuzzers --sanitizer=address dovecot
2、执行
python infra/helper.py run_fuzzer dovecot fuzz-smtp-server -rss_limit_mb=0
【报错信息】

==13==ERROR: AddressSanitizer: ABRT on unknown address 0x00000000000d (pc 0x7f5c09bf8428 bp 0x7ffec4f54650 sp 0x7ffec4f544a8 T0)
SCARINESS: 10 (signal)
    #0 0x7f5c09bf8428 in raise (/lib/x86_64-linux-gnu/libc.so.6+0x35428)
    #1 0x7f5c09bfa029 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x37029)
    #2 0x60d9cb in default_fatal_finish /src/dovecot/src/lib/failures.c:459:3
    #3 0x60a053 in fatal_handler_real /src/dovecot/src/lib/failures.c:471:2
    #4 0x6097fd in default_fatal_handler /src/dovecot/src/lib/failures.c:479:2
    #5 0x609990 in i_panic /src/dovecot/src/lib/failures.c:523:2
    #6 0x5a0dbf in smtp_server_cmd_ehlo_reply_create /src/dovecot/src/lib-smtp/smtp-server-cmd-helo.c:158:2
    #7 0x5a0ec0 in smtp_server_cmd_ehlo_reply_default /src/dovecot/src/lib-smtp/smtp-server-cmd-helo.c:192:10
    #8 0x5a0611 in smtp_server_cmd_helo_run /src/dovecot/src/lib-smtp/smtp-server-cmd-helo.c:109:3
    #9 0x5a0895 in smtp_server_cmd_helo /src/dovecot/src/lib-smtp/smtp-server-cmd-helo.c:127:2
    #10 0x588388 in smtp_server_command_execute /src/dovecot/src/lib-smtp/smtp-server-command.c:256:3
    #11 0x5746b8 in smtp_server_connection_handle_command /src/dovecot/src/lib-smtp/smtp-server-connection.c:303:2
    #12 0x5739bd in smtp_server_connection_handle_input /src/dovecot/src/lib-smtp/smtp-server-connection.c:440:9
    #13 0x5732fe in smtp_server_connection_input /src/dovecot/src/lib-smtp/smtp-server-connection.c:589:2
    #14 0x63744a in io_loop_call_io /src/dovecot/src/lib/ioloop.c:713:2
    #15 0x638641 in io_loop_call_pending /src/dovecot/src/lib/ioloop.c:751:5
    #16 0x638141 in io_loop_handler_run /src/dovecot/src/lib/ioloop.c:766:2
    #17 0x637d87 in io_loop_run /src/dovecot/src/lib/ioloop.c:738:3
    #18 0x569192 in LLVMFuzzerTestOneInput /src/dovecot/src/lib-smtp/fuzz-smtp-server.c:41:2
    #19 0x471231 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:556:15
    #20 0x470975 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:470:3
    #21 0x472a47 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:698:19
    #22 0x4734c5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:832:5
    #23 0x46249e in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:826:6
    #24 0x48ac72 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10
    #25 0x7f5c09be382f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #26 0x436ba8 in _start (/out/fuzz-smtp-server+0x436ba8)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: ABRT (/lib/x86_64-linux-gnu/libc.so.6+0x35428) in raise
==13==ABORTING
MS: 4 CopyPart-PersAutoDict-ShuffleBytes-CrossOver- DE: "HELO"-; base unit: 513b1662549244fec20aa388b74160d4c0ff89f9
0x48,0x45,0x4c,0x4f,0x20,0x2b,0x2b,0x2b,0xa,0x2b,0x2b,
HELO +++\x0a++
artifact_prefix='./'; Test unit written to ./crash-d99ce042065033ae7516d197d0d4c920b55a52bb

【问题复现】
python infra/helper.py reproduce dovecot fuzz-imap-bodystructure crash-d99ce042065033ae7516d197d0d4c920b55a52bb

Hey yanglijin, Welcome to openEuler Community.
All of the projects in openEuler Community are maintained by @openeuler-ci-bot.
That means the developers can comment below every pull request or issue to trigger Bot Commands.
Please follow instructions at https://gitee.com/openeuler/community/blob/master/en/sig-infrastructure/command.md to find the details.

修复的PR如下：
master分支：!26:Resolve fuzz-test about ABRT error
openEuler-21.03分支：!27:Resolve fuzz-test about ABRT error
openEuler-20.09分支：!28:Resolve fuzz-test about ABRT error
openEuler-20.03-LTS分支：!29:Resolve fuzz-test about ABRT error
openEuler-20.03-LTS-Next分支：!30:Resolve fuzz-test about ABRT error
openEuler-20.03-LTS-SP1分支：!31:Resolve fuzz-test about ABRT error

问题1修复后，测试31分钟后，出现新的问题，如下：

#42495969       REDUCE cov: 563 ft: 2079 corp: 451/44Kb lim: 4096 exec/s: 22773 rss: 77Mb L: 510/3506 MS: 1 EraseBytes-
#42502266       REDUCE cov: 563 ft: 2079 corp: 451/44Kb lim: 4096 exec/s: 22777 rss: 77Mb L: 275/3506 MS: 2 InsertByte-EraseBytes-
Panic: file imap-quote.c: line 20 (imap_append_string): assertion failed: (src != NULL)
Error: Raw backtrace: /out/fuzz-imap-bodystructure(backtrace+0x5b) [0x4dd28b] -> /out/fuzz-imap-bodystructure() [0x5c273c] -> /out/fuzz-imap-bodystructure() [0x5c25f1] -> /out/fuzz-imap-bodystructure() [0x5c29ea] -> /out/fuzz-imap-bodystructure() [0x568f01] -> /out/fuzz-imap-bodystructure() [0x565604] -> /out/fuzz-imap-bodystructure() [0x564dae] -> /out/fuzz-imap-bodystructure() [0x564f41] -> /out/fuzz-imap-bodystructure() [0x55fcc3] -> /out/fuzz-imap-bodystructure() [0x55601f] -> /out/fuzz-imap-bodystructure() [0x5514a9] -> /out/fuzz-imap-bodystructure() [0x55107b] -> /out/fuzz-imap-bodystructure() [0x5558d2] -> /out/fuzz-imap-bodystructure() [0x459442] -> /out/fuzz-imap-bodystructure() [0x458b86] -> /out/fuzz-imap-bodystructure() [0x45ac58] -> /out/fuzz-imap-bodystructure() [0x45b6d6] -> /out/fuzz-imap-bodystructure() [0x44a6af] -> /out/fuzz-imap-bodystructure() [0x472e83] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f8d1246c830] -> /out/fuzz-imap-bodystructure() [0x41edb9]
AddressSanitizer:DEADLYSIGNAL
=================================================================
==13==ERROR: AddressSanitizer: ABRT on unknown address 0x00000000000d (pc 0x7f8d12481428 bp 0x7ffdd91a3e50 sp 0x7ffdd91a3ca8 T0)
SCARINESS: 10 (signal)
    #0 0x7f8d12481428 in raise (/lib/x86_64-linux-gnu/libc.so.6+0x35428)
    #1 0x7f8d12483029 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x37029)
    #2 0x568f7b in default_fatal_finish /src/dovecot/src/lib/failures.c:459:3
    #3 0x565603 in fatal_handler_real /src/dovecot/src/lib/failures.c:471:2
    #4 0x564dad in default_fatal_handler /src/dovecot/src/lib/failures.c:479:2
    #5 0x564f40 in i_panic /src/dovecot/src/lib/failures.c:523:2
    #6 0x55fcc2 in imap_append_string /src/dovecot/src/lib-imap/imap-quote.c:20:2
    #7 0x55601e in params_write /src/dovecot/src/lib-imap/./imap-bodystructure.c:47:3
    #8 0x5514a8 in part_write_body /src/dovecot/src/lib-imap/./imap-bodystructure.c:185:2
    #9 0x55107a in imap_bodystructure_write /src/dovecot/src/lib-imap/./imap-bodystructure.c:248:3
    #10 0x5558d1 in LLVMFuzzerTestOneInput /src/dovecot/src/lib-imap/fuzz-imap-bodystructure.c:31:3
    #11 0x459441 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:556:15
    #12 0x458b85 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:470:3
    #13 0x45ac57 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:698:19
    #14 0x45b6d5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:832:5
    #15 0x44a6ae in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:826:6
    #16 0x472e82 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10
    #17 0x7f8d1246c82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #18 0x41edb8 in _start (/out/fuzz-imap-bodystructure+0x41edb8)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: ABRT (/lib/x86_64-linux-gnu/libc.so.6+0x35428) in raise
==13==ABORTING
MS: 1 CrossOver-; base unit: fb60fa6c9f7264247dda8c98e6da0f6c04449f5f
0x22,0x22,0x22,0x22,0x28,0x6e,0x69,0x4c,0x20,0x6e,0x69,0x4c,0x29,0x22,0x22,0x7c,0x20,0x61,0x20,0x30,0x20,0x30,0x20,0x28,0x22,0x22,0x28,0x29,0x29,0x28,0x22,0x22,0x22,0x22,0x30,0x20,0x2a,0x20,0x3c,0x20,0x22,0x22,0x22,0x22,0x3c,0x20,0x3c,0x20,0x3c,0x20,0x3c,0x20,0x3c,0x20,0x3c,0x20,0x30,0x20,0x5d,0x20,0x3f,0x20,0x3c,0x20,0x20,0x3c,0x29,
\"\"\"\"(niL niL)\"\"| a 0 0 (\"\"())(\"\"\"\"0 * < \"\"\"\"< < < < < < 0 ] ? <  <)
artifact_prefix='./'; Test unit written to ./crash-531d2960ad7b603ecedc783b3eb96754ae5a6e68
Base64: IiIiIihuaUwgbmlMKSIifCBhIDAgMCAoIiIoKSkoIiIiIjAgKiA8ICIiIiI8IDwgPCA8IDwgPCAwIF0gPyA8ICA8KQ==

real    31m15.034s
user    0m0.351s
sys     0m0.118s
[root@23 oss-fuzz]#

备注：目前官方未修复

问题再分析
关于修复后出现的新问题，实际上从社区中可以发现这个问题是因为出现了不合条件的数据而导致的，也可以说之前并没有修复，具体分析如下：

版本2.3.13中增加i_assert(text == ((part->flags & MESSAGE_PART_FLAG_TEXT) != 0));

用来校验text标志，理论上两处的标识是一致的，但实际上会出现不一致，这就是问题中出现的情况。

参考链接：https://github.com/dovecot/core/commit/84cde48e5859b1d20f2fd98b57b7410b4d3727fa

master分支用返回错误取代断言

发生text标志不同的情况有：损坏的message_parts来自缓存，message_part->data 是从邮件输入中新读取的。
所以这里不能使用断言，请注意这句话"text flag mismatch"

参考链接：https://github.com/dovecot/core/commit/0f66865e0e3dae3e902abd8bb559d7706dbff437

再测试

基于master分支代码测试（因为最新版本并未解决问题）
build.sh需要修改为下面所示，即使用autogen.sh脚本，而不是autoreconf，否则编译失败

./autogen.sh
#autoreconf -I . -fiv

测试结果如下图：
输入图片说明
焦点请对准text flag mismatch

参考社区对imap-bodystructure.c文件的多出修改，适配当前版本，测试如下：

和社区maser分支的测试结果吻合

结论
fuzz构造的数据，有些是不符合预期数据格式的，该软件包中很多地方都是通过断言来保证数据的预期，如果真的不满足预期，那么就会发生断言错误。所以软件包中在真有可能发生非预期的地方，进行返回值处理，而非断言，这也是上面text标志判断处逻辑修改的原因。
综上所术，fuzz测试中出现的问题，是因为数据不合理，而代码上通过断言来处理，所以导致错误，根据官方将代码改成返回值的方式即可。

处理的PR如下：
master分支：!32:backport some patches about imap-bodystructure
openeuler-20.03-LTS-Next分支：!33:backport some patches about imap-bodystructure
openeuler-21.03分支：!34:[sync] PR-32: backport some patches about imap-bodystructure
openeuler-20.03-LTS-SP1分支：!35:[sync] PR-33: backport some patches about imap-bodystructure
openeuler-20.03-LTS-SP2分支：!36:[sync] PR-33: backport some patches about imap-bodystructure

和开发对齐为fuzz变异后数据格式不合理导致的断言异常，目前上游社区代码也存在这个问题，可以跟随社区~

src-openEuler/dovecot

内容风险标识

评论 (5)

src-openEuler/dovecot .gitee-modal { width: 500px !important; }

内容风险标识