Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

SPDK NVMe-oF TCP functionality verification, performance test and optimization

Description

SPDK NVMe-oF TCP functionality verification, performance test and optimization.

Official performance report reference:

NVMe-of TCP performance related references：

CPU micro Top-Down analysis for IO interface libaio, io_uring, SPDK:

Activity

Xinliang Liu 
September 7, 2022 at 8:57 AM

x86 1 node switch to SPDK initiator

3 ssd numjobs=16, gets the expected IOPS numbers

read: IOPS=2545k, BW=9941MiB/s (10.4GB/s)(194GiB/20011msec)

Jianming Yang 
September 6, 2022 at 8:29 AM

SPDK NVMe-oF TCP fio test performance result

1 target 2 initiator

target: RH2288H V5 Intel(R) Xeon(R) Platinum 8180

nvme disk: 3.2T*3

基线数据：1个nvme裸盘：4k randwrite IOPS=719k，4k randread IOPS=850k

网络：100Gbits/s

initiator1: RH1288 V3 Intel(R) Xeon(R) CPU E5-2695 v4

initiator2: TaiShan 2280 v2 Kunpeng 920-4826

SPDK NVMe-oF TCP测试配置：

target cpu核数：16 max_queue_depth：256 mode：spdk

initiator1 mode：kernel

initiator2 mode：kernel

fio qd:4096 num_jobs:32 iodepth:128

4k randwrite IOPS=2045k BW=7988MB/s

Xinliang Liu 
August 31, 2022 at 8:17 AM

SPDK configure options almost the same as Mayastor does:

Xinliang Liu 
August 30, 2022 at 2:56 AM

SPDK NVMe-oF TCP fio test running steps:

Test steps run on server node

(1) switch to nvme spdk user space driver

(2) setup config, for parameter description please reference to

(3) run test

Xinliang Liu 
August 23, 2022 at 2:09 AM

Setup a two node 100G bandwidth x86 environment will have a test soon. 

Done

Details

Assignee

Reporter

Parent

Original estimate

Time tracking

No time logged4w remaining

Sprint

Priority

Checklist

Sentry

Created August 23, 2022 at 1:48 AM
Updated October 26, 2022 at 3:37 AM
Resolved October 26, 2022 at 3:37 AM