7 Star 0 Fork 19

src-openEuler / libhugetlbfs

 / 详情

【20.03 LTS】【x86】libhugetlbfs 报错Segments would overlap

已完成
缺陷
创建于  
2020-12-09 11:24

执行
#配置大页数为0
echo 0 > /proc/sys/vm/nr_hugepages
cat /proc/sys/vm/nr_hugepages
export HUGETLB_ELFMAP='R:W'
export LD_LIBRARY_PATH='./'
export HUGETLB_VERBOSE=3
export HUGETLB_SHARE=1
./../common/test >log 2>&1 & pid=$!

其中test.c为
#include <stdio.h>

int main()
{
int i =0;
for (i = 0; i < 3; i++)
{
printf("Hello World");
sleep (3);

    }

}

Makefile:

CC = $(CROSS_COMPILE)gcc

ifeq ("$(ARCH)", "arm64")
CFLAGS += -DCONFIG_ARM64=1
endif

#$(MAKE_TARGETS): | $(MAKE_DEPS)
all:
[ -e $(BINDIR) ] || mkdir -p $(BINDIR)
$(CC) -g test.c -o test -Wl,-zcommon-page-size=0x200000 -Wl,-zmax-page-size=0x200000 -Wl,-Ttext-segment=0x200000 -lhugetlbfs

log文件:

libhugetlbfs: INFO: Found pagesize 524288 kB
libhugetlbfs: INFO: Parsed kernel version: [4] . [19] . [90]
libhugetlbfs: INFO: Feature private_reservations is present in this kernel
libhugetlbfs: INFO: Feature noreserve_safe is present in this kernel
libhugetlbfs: INFO: Feature map_hugetlb is present in this kernel
libhugetlbfs: INFO: Kernel has MAP_PRIVATE reservations. Disabling heap prefaulting.
libhugetlbfs: INFO: Kernel supports MAP_HUGETLB
libhugetlbfs: INFO: HUGETLB_SHARE=1, sharing enabled for only read-only segments
libhugetlbfs: INFO: HUGETLB_NO_RESERVE=no, reservations enabled
libhugetlbfs: INFO: Segment 0 (phdr 2): 0x200000-0x2008e0 (filesz=0x8e0) (prot = 0x5)
libhugetlbfs: INFO: Segment 1 (phdr 3): 0x5ffdd8-0x600040 (filesz=0x260) (prot = 0x3)
libhugetlbfs: WARNING: Layout problem with segments 0 and 1:
Segments would overlap
libhugetlbfs: INFO: No segments were appropriate for remapping

段重叠
请确认

评论 (9)

buaixuebi 创建了缺陷
buaixuebi 关联仓库设置为src-openEuler/libhugetlbfs
展开全部操作日志

Hey ruihongw, Welcome to openEuler Community.
All of the projects in openEuler Community are maintained by @openeuler-ci-bot.
That means the developers can comment below every pull request or issue to trigger Bot Commands.
Please follow instructions at https://gitee.com/openeuler/community/blob/master/en/sig-infrastructure/command.md to find the details.

Charlie_Li 优先级设置为主要
陈亚强 负责人设置为吴旭

非问题;
该流程是在加载elf时,读取elf段,判断是否可以支持大页加载。从test的elf结构中读到了多个segment,并将多个段起始、结束地址做对齐,最后根据对齐地址判断是否会段重叠,若重叠则不使用大页。

输入图片说明

输入图片说明

输入图片说明
@buaixuebi

问题根因:编译出的elf文件包含多个段,其中两段在做页对齐时,对齐到了同一个页。出现段重叠
问题影响:编译指定的2M大页程序,实际arm是64K页表,只支持512M大页,编译时按2M对齐,实际2个段映射到同一个页;
问题处理:非问题,将Makefile的2M对齐,修改为512M页对齐;

[root@openEuler common]# readelf -l test

Elf file type is EXEC (Executable file)
Entry point 0x2005c0
There are 9 program headers, starting at offset 64

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000200040 0x0000000000200040
0x00000000000001f8 0x00000000000001f8 R 0x8
INTERP 0x0000000000000238 0x0000000000200238 0x0000000000200238
0x000000000000001b 0x000000000000001b R 0x1
[Requesting program interpreter: /lib/ld-linux-aarch64.so.1]
LOAD 0x0000000000000000 0x0000000000200000 0x0000000000200000
0x00000000000008e0 0x00000000000008e0 R E 0x200000
LOAD 0x00000000001ffdd8 0x00000000005ffdd8 0x00000000005ffdd8
0x0000000000000260 0x0000000000000268 RW 0x200000
DYNAMIC 0x00000000001ffde8 0x00000000005ffde8 0x00000000005ffde8
0x00000000000001e0 0x00000000000001e0 RW 0x8
NOTE 0x0000000000000254 0x0000000000200254 0x0000000000200254
0x0000000000000044 0x0000000000000044 R 0x4
GNU_EH_FRAME 0x00000000000007ac 0x00000000002007ac 0x00000000002007ac
0x0000000000000044 0x0000000000000044 R 0x4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 0x10
GNU_RELRO 0x00000000001ffdd8 0x00000000005ffdd8 0x00000000005ffdd8
0x0000000000000228 0x0000000000000228 R 0x1

export HUGETLB_ELFMAP='R:W';export LD_LIBRARY_PATH='./;export HUGETLB_VERBOSE=3;export HUGETLB_SHARE=1

Starting program: /opt/ETS/testcases/bin/package_test/libhugetlbfs/common/test
libhugetlbfs: INFO: Found pagesize 524288 kB
libhugetlbfs: INFO: Parsed kernel version: [4] . [19] . [90]
libhugetlbfs: INFO: Feature private_reservations is present in this kernel
libhugetlbfs: INFO: Feature noreserve_safe is present in this kernel
libhugetlbfs: INFO: Feature map_hugetlb is present in this kernel
libhugetlbfs: INFO: Kernel has MAP_PRIVATE reservations. Disabling heap prefaulting.
libhugetlbfs: INFO: Kernel supports MAP_HUGETLB
libhugetlbfs: INFO: HUGETLB_SHARE=1, sharing enabled for only read-only segments
libhugetlbfs: INFO: HUGETLB_NO_RESERVE=no, reservations enabled
libhugetlbfs: INFO: Segment 0 (phdr 2): 0x200000-0x2008e0 (filesz=0x8e0) (prot = 0x5)

Breakpoint 2, parse_elf_normal (info=0xfffffffff2f8, size=64, data=0x0) at elflink.c:742
742 segments[num_segs].page_size = seg_psize;
(gdb) p i
$15 = 2
(gdb) p info->dlpi_phdr[2]
$16 = {p_type = 1, p_flags = 5, p_offset = 0, p_vaddr = 2097152, p_paddr = 2097152, p_filesz = 2272, p_memsz = 2272, p_align = 2097152}
(gdb) p /x info->dlpi_phdr[2]
$17 = {p_type = 0x1, p_flags = 0x5, p_offset = 0x0, p_vaddr = 0x200000, p_paddr = 0x200000, p_filesz = 0x8e0, p_memsz = 0x8e0, p_align = 0x200000}
(gdb) c
Continuing.
libhugetlbfs: INFO: Segment 1 (phdr 3): 0x5ffdd8-0x600040 (filesz=0x260) (prot = 0x3)

Breakpoint 2, parse_elf_normal (info=0xfffffffff2f8, size=64, data=0x0) at elflink.c:742
742 segments[num_segs].page_size = seg_psize;
(gdb) c
Continuing.

Breakpoint 1, verify_segment_layout (segs=0xfffffffff198, num_segs=2) at elflink.c:654
654 long base_size = getpagesize();
(gdb) bt
#0 verify_segment_layout (segs=0xfffffffff198, num_segs=2) at elflink.c:654
#1 0x0000fffff7f54e3c in parse_elf_normal (info=0xfffffffff2f8, size=64, data=0x0) at elflink.c:747
#2 0x0000fffff7edf870 in dl_iterate_phdr () from /lib64/libc.so.6
#3 0x0000fffff7f576c4 in parse_elf () at elflink.c:1292
#4 0x0000fffff7f577c4 in __lh_hugetlbfs_setup_elflink () at elflink.c:1309
#5 0x0000fffff7f5c08c in setup_libhugetlbfs () at init.c:36
#6 0x0000fffff7fcd584 in ?? () from /lib/ld-linux-aarch64.so.1
#7 0x0000fffff7fcd688 in ?? () from /lib/ld-linux-aarch64.so.1
#8 0x0000fffff7fc1204 in ?? () from /lib/ld-linux-aarch64.so.1
Backtrace stopped: not enough registers or memory available to unwind further
(gdb) f 1
#1 0x0000fffff7f54e3c in parse_elf_normal (info=0xfffffffff2f8, size=64, data=0x0) at elflink.c:747
747 if (verify_segment_layout(segments, num_segs))
(gdb) p /x info->dlpi_phdr[2]
$18 = {p_type = 0x1, p_flags = 0x5, p_offset = 0x0, p_vaddr = 0x200000, p_paddr = 0x200000, p_filesz = 0x8e0, p_memsz = 0x8e0, p_align = 0x200000}
(gdb) p /x segments[0]
$19 = {start = 0x0, end = 0x20000000, page_size = 0x20000000}
(gdb) p /x info->dlpi_phdr[3]
$20 = {p_type = 0x1, p_flags = 0x6, p_offset = 0x1ffdd8, p_vaddr = 0x5ffdd8, p_paddr = 0x5ffdd8, p_filesz = 0x260, p_memsz = 0x268, p_align = 0x200000}
(gdb) p /x segments[1]
$21 = {start = 0x0, end = 0x20000000, page_size = 0x20000000}
(gdb) p /x info ->dlpi_addr
$22 = 0x0
(gdb) p /x seg_psize
$23 = 0x20000000

将Makefile修改为按512M对齐后,无段重叠的问题
$(CC) -g test.c -o test -Wl,-zcommon-page-size=0x20000000 -Wl,-zmax-page-size=0x20000000 -Wl,-Ttext-segment=0x20000000 -lhugetlbfs

[root@openEuler 2m]# readelf -l test

Elf file type is EXEC (Executable file)
Entry point 0x200005c0
There are 9 program headers, starting at offset 64

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000020000040 0x0000000020000040
0x00000000000001f8 0x00000000000001f8 R 0x8
INTERP 0x0000000000000238 0x0000000020000238 0x0000000020000238
0x000000000000001b 0x000000000000001b R 0x1
[Requesting program interpreter: /lib/ld-linux-aarch64.so.1]
LOAD 0x0000000000000000 0x0000000020000000 0x0000000020000000
0x00000000000008e0 0x00000000000008e0 R E 0x20000000
LOAD 0x000000001ffffdd8 0x000000005ffffdd8 0x000000005ffffdd8
0x0000000000000260 0x0000000000000268 RW 0x20000000
DYNAMIC 0x000000001ffffde8 0x000000005ffffde8 0x000000005ffffde8
0x00000000000001e0 0x00000000000001e0 RW 0x8
NOTE 0x0000000000000254 0x0000000020000254 0x0000000020000254
0x0000000000000044 0x0000000000000044 R 0x4
GNU_EH_FRAME 0x00000000000007ac 0x00000000200007ac 0x00000000200007ac
0x0000000000000044 0x0000000000000044 R 0x4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 0x10
GNU_RELRO 0x000000001ffffdd8 0x000000005ffffdd8 0x000000005ffffdd8
0x0000000000000228 0x0000000000000228 R 0x1

[root@openEuler 2m]# ./test
libhugetlbfs: INFO: Found pagesize 524288 kB
libhugetlbfs: INFO: Parsed kernel version: [4] . [19] . [90]
libhugetlbfs: INFO: Feature private_reservations is present in this kernel
libhugetlbfs: INFO: Feature noreserve_safe is present in this kernel
libhugetlbfs: INFO: Feature map_hugetlb is present in this kernel
libhugetlbfs: INFO: Kernel has MAP_PRIVATE reservations. Disabling heap prefaulting.
libhugetlbfs: INFO: Kernel supports MAP_HUGETLB
libhugetlbfs: WARNING: Hugepage size 536870912 unavailablelibhugetlbfs: WARNING: Hugepage size 536870912 unavailablelibhugetlbfs: INFO: HUGETLB_SHARE=1, sharing enabled for only read-only segments
libhugetlbfs: INFO: HUGETLB_NO_RESERVE=no, reservations enabled
libhugetlbfs: INFO: No segments were appropriate for remapping
Hello WorldHello WorldHello World[root@openEuler 2m]#

陈亚强 任务状态待办的 修改为修复中
陈亚强 任务状态修复中 修改为已确认

@吴旭
改成512对齐后 有什么影响吗?

@吴旭
改成512对齐后 有什么影响吗?

@Charlie_Li 没影响,arm 64k页表,只支持512M大页

吴旭 任务状态已确认 修改为待办的
吴旭 添加协作者吴旭
吴旭 负责人吴旭 修改为Charlie_Li
吴旭 取消协作者吴旭
Charlie_Li 任务状态待办的 修改为已完成

2M的话设置一下这个 export HUGETLB_DEFAULT_PAGE_SIZE=2048kB

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(6)
5329419 openeuler ci bot 1632792936 5474000 wuxu wu 1578987273 5603730 yaqiangchen 1607954772
1
https://gitee.com/src-openeuler/libhugetlbfs.git
git@gitee.com:src-openeuler/libhugetlbfs.git
src-openeuler
libhugetlbfs
libhugetlbfs

搜索帮助