[OLK-6.6] RDMA/hns: Support flexible wqe buffer page size

【特性描述】
支持hns ROCE自适应wqe buffer页面大小
【特性竞争力】
NA
【硬件架构】
鲲鹏服务器RoCE板载网卡
【特性约束】
NA
【涉及仓库】
driver/infiniband/hw/hns
【交付个人/团队】
鲲鹏服务器板载网卡RoCE团队

Hi stinft, welcome to the openEuler Community.
I'm the Bot here serving you. You can find the instructions on how to interact with me at Here.
If you have any questions, please contact the SIG: Kernel, and any of the maintainers.

测试基本业务：

[root@localhost ~]# ib_send_bw -d hns_0 &
[1] 7134
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0

************************************
* Waiting for client to connect... *
************************************
ib_send_bw -d hns_0 192.168.1.20&
[2] 7135
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0
---------------------------------------------------------------------------------------
                    Send BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
---------------------------------------------------------------------------------------
                    Send BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : OFF
 ibv_wr* API     : OFF
 TX depth        : 128
 RX depth        : 512
 CQ Moderation   : 1
 CQ Moderation   : 1
 Mtu             : 1024[B]
 Mtu             : 1024[B]
 Link type       : Ethernet
 Link type       : Ethernet
 GID index       : 3
 GID index       : 3
 Max inline data : 0[B]
 Max inline data : 0[B]
 rdma_cm QPs     : OFF
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x0004 PSN 0x4ae93d
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 local address: LID 0000 QPN 0x0002 PSN 0xa6934d
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 remote address: LID 0000 QPN 0x0002 PSN 0xa6934d
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 remote address: LID 0000 QPN 0x0004 PSN 0x4ae93d
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 65536      1000             0.00               6265.13            0.100242
---------------------------------------------------------------------------------------
 65536      1000             6237.20            6227.74            0.099644
---------------------------------------------------------------------------------------

[1]-  Done                    ib_send_bw -d hns_0
[2]+  Done                    ib_send_bw -d hns_0 192.168.1.20
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ib_send_bw -d hns_0 -c UD&
[1] 7136
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0

************************************
* Waiting for client to connect... *
************************************

[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ib_send_bw -d hns_0 -c UD 192.168.1.20&
[2] 7137
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0
 Max msg size in UD is MTU 1024
 Max msg size in UD is MTU 1024
 Changing to this MTU
 Changing to this MTU
---------------------------------------------------------------------------------------
                    Send BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : UD           Using SRQ      : OFF
 PCIe relax order: ON
---------------------------------------------------------------------------------------
                    Send BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : UD           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : OFF
 TX depth        : 128
 CQ Moderation   : 100
 Mtu             : 1024[B]
 Link type       : Ethernet
 GID index       : 3
 Max inline data : 0[B]
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 ibv_wr* API     : OFF
 RX depth        : 1000
 CQ Moderation   : 100
 Mtu             : 1024[B]
 Link type       : Ethernet
 GID index       : 3
 Max inline data : 0[B]
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x000c PSN 0xf5d53e
 local address: LID 0000 QPN 0x000a PSN 0xf1ff6f
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 remote address: LID 0000 QPN 0x000a PSN 0xf1ff6f
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 remote address: LID 0000 QPN 0x000c PSN 0xf5d53e
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 1024       1000             0.00               3104.55            3.179058
---------------------------------------------------------------------------------------
 1024       1000             3150.10            2742.60            2.808426
---------------------------------------------------------------------------------------

[1]-  Done                    ib_send_bw -d hns_0 -c UD
[2]+  Done                    ib_send_bw -d hns_0 -c UD 192.168.1.20
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ib_send_bw -d hns_0 -c XRC&
[1] 7138
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0

************************************
* Waiting for client to connect... *
************************************

[root@localhost ~]#
[root@localhost ~]# ib_send_bw -d hns_0 -c XRC 192.168.1.20&
[2] 7139
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0
hns_0: verify_qp_create_cap:1720: change sq depth from 0 to minimum 64.
---------------------------------------------------------------------------------------
                    Send BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : XRC          Using SRQ      : ON
 PCIe relax order: ON
---------------------------------------------------------------------------------------
                    Send BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : XRC          Using SRQ      : ON
 PCIe relax order: ON
 ibv_wr* API     : OFF
 ibv_wr* API     : OFF
 TX depth        : 128
 RX depth        : 512
 CQ Moderation   : 1
 CQ Moderation   : 1
 Mtu             : 1024[B]
 Mtu             : 1024[B]
 Link type       : Ethernet
 Link type       : Ethernet
 GID index       : 3
 GID index       : 3
 Max inline data : 0[B]
 Max inline data : 0[B]
 rdma_cm QPs     : OFF
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x0012 PSN 0x93acfd SRQn 00000000
 local address: LID 0000 QPN 0x0003 PSN 0x1d3dbc SRQn 0x000001
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 remote address: LID 0000 QPN 0x0003 PSN 0x1d3dbc SRQn 0x000001
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 remote address: LID 0000 QPN 0x0012 PSN 0x93acfd SRQn 00000000
---------------------------------------------------------------------------------------
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 65536      1000             0.00               6299.14            0.100786
---------------------------------------------------------------------------------------
 65536      1000             6274.51            6263.28            0.100212
---------------------------------------------------------------------------------------

[1]-  Done                    ib_send_bw -d hns_0 -c XRC
[2]+  Done                    ib_send_bw -d hns_0 -c XRC 192.168.1.20
[root@localhost ~]#
[root@localhost ~]# ib_write_bw -d hns_0 &
[1] 7142
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0

************************************
* Waiting for client to connect... *
************************************

[root@localhost ~]#
[root@localhost ~]# ib_write_bw -d hns_0 192.168.1.20&
[2] 7143
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0

[root@localhost ~]# hns_0: verify_qp_create_cap:1728: change rq depth from 1 to minimum 64.
hns_0: verify_qp_create_cap:1728: change rq depth from 1 to minimum 64.
---------------------------------------------------------------------------------------
                    RDMA_Write BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
---------------------------------------------------------------------------------------
                    RDMA_Write BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : OFF
 ibv_wr* API     : OFF
 TX depth        : 128
 CQ Moderation   : 1
 CQ Moderation   : 1
 Mtu             : 1024[B]
 Mtu             : 1024[B]
 Link type       : Ethernet
 Link type       : Ethernet
 GID index       : 3
 GID index       : 3
 Max inline data : 0[B]
 Max inline data : 0[B]
 rdma_cm QPs     : OFF
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x001a PSN 0x23edb RKey 0x000300 VAddr 0x00ffffa7e0e000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 local address: LID 0000 QPN 0x0014 PSN 0x5a366a RKey 0x000400 VAddr 0x00ffff97d38000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 remote address: LID 0000 QPN 0x0014 PSN 0x5a366a RKey 0x000400 VAddr 0x00ffff97d38000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 remote address: LID 0000 QPN 0x001a PSN 0x23edb RKey 0x000300 VAddr 0x00ffffa7e0e000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 65536      5000             6350.94            6347.35            0.101558
---------------------------------------------------------------------------------------
 65536      5000             6350.94            6347.35            0.101558
---------------------------------------------------------------------------------------

[1]-  Done                    ib_write_bw -d hns_0
[2]+  Done                    ib_write_bw -d hns_0 192.168.1.20
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ib_read_bw -d hns_0 &
[1] 7144
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0

************************************
* Waiting for client to connect... *
************************************

[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ib_read_bw -d hns_0 192.168.1.20&
[2] 7145
[root@localhost ~]# hns_0: query_dev_attr:369: device link_type: 0
hns_0: verify_qp_create_cap:1728: change rq depth from 1 to minimum 64.
hns_0: verify_qp_create_cap:1728: change rq depth from 1 to minimum 64.
---------------------------------------------------------------------------------------
                    RDMA_Read BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
---------------------------------------------------------------------------------------
                    RDMA_Read BW Test
 Dual-port       : OFF          Device         : hns_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : OFF
 ibv_wr* API     : OFF
 CQ Moderation   : 1
 TX depth        : 128
 Mtu             : 1024[B]
 CQ Moderation   : 1
 Link type       : Ethernet
 Mtu             : 1024[B]
 GID index       : 3
 Link type       : Ethernet
 Outstand reads  : 128
 GID index       : 3
 rdma_cm QPs     : OFF
 Outstand reads  : 128
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x0022 PSN 0x27418e OUT 0x80 RKey 0x000300 VAddr 0x00ffffa3a18000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 local address: LID 0000 QPN 0x001c PSN 0x8312e2 OUT 0x80 RKey 0x000400 VAddr 0x00ffff842ab000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 remote address: LID 0000 QPN 0x001c PSN 0x8312e2 OUT 0x80 RKey 0x000400 VAddr 0x00ffff842ab000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
 remote address: LID 0000 QPN 0x0022 PSN 0x27418e OUT 0x80 RKey 0x000300 VAddr 0x00ffffa3a18000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:01:20
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 65536      1000             3352.62            3351.66            0.053627
---------------------------------------------------------------------------------------
 65536      1000             3352.62            3351.66            0.053627
---------------------------------------------------------------------------------------

[1]-  Done                    ib_read_bw -d hns_0
[2]+  Done                    ib_read_bw -d hns_0 192.168.1.20
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]#

```bash

GVP openEuler/kernel

内容风险标识

评论 (2)

GVPopenEuler/kernel

内容风险标识

[OLK-6.6] RDMA/hns: Support flexible wqe buffer page size

评论 (2)

搜索帮助

GVP openEuler/kernel