（9）ceph pool 性能测试

Ceph包含在rados bench RADOS存储集群上执行性能基准测试的命令。该命令将执行写测试和两种类型的读测试。--no-cleanup在测试读写性能时，该选项很重要。默认情况下，该rados bench命令将删除已写入存储池的对象。留在这些对象后面允许两个读取测试来测量顺序和随机读取性能。

（1）准备工作，清除所有的文件系统缓存
[root@node141 mnt]# echo 3 > /proc/sys/vm/drop_caches
[root@node141 mnt]# sync

（2）创建一个新的存储池
[root@node141 mnt]# ceph osd pool create rbd 128 128

（3）测试一下rbd的写性能，添加 --no-cleanup参数

[root@node141 mnt]# rados bench -p rbd 10 write  --no-cleanup hints = 1Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objectsObject prefix: benchmark_data_node141_28363  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)    0       0         0         0         0         0           -           0    1      16        81        65   259.999       260    0.107901    0.200157    2      16       161       145   289.976       320    0.190659    0.215092    3      16       252       236   314.632       364   0.0807696    0.193549    4      16       351       335   334.958       396    0.191582    0.185652    5      16       445       429   343.157       376    0.158618    0.178393    6      16       548       532   354.621       412    0.162274    0.178178    7      16       652       636   363.378       416    0.190742    0.173784    8      16       753       737   368.448       404     0.15392    0.170866    9      16       863       847   376.391       440    0.103038    0.169129   10      16       964       948   379.144       404    0.138741    0.167306Total time run:         10.1023Total writes made:      965Write size:             4194304Object size:            4194304Bandwidth (MB/sec):     382.09Stddev Bandwidth:       53.3267Max bandwidth (MB/sec): 440Min bandwidth (MB/sec): 260Average IOPS:           95Stddev IOPS:            13.3317Max IOPS:               110Min IOPS:               65Average Latency(s):     0.167422Stddev Latency(s):      0.0765119Max latency(s):         0.668499Min latency(s):         0.0569634

（4）测试一下，10秒的顺序读取：

[root@node141 mnt]# rados bench -p rbd 10 seqhints = 1  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)    0       0         0         0         0         0           -           0    1      16       257       241   963.679       964   0.0607106   0.0623843    2      16       549       533   1065.73      1168   0.0450988   0.0583824    3      16       835       819   1091.76      1144    0.044447    0.057111Total time run:       3.47417Total reads made:     965Read size:            4194304Object size:          4194304Bandwidth (MB/sec):   1111.06Average IOPS:         277Stddev IOPS:          27.8747Max IOPS:             292Min IOPS:             241Average Latency(s):   0.056581Max latency(s):       0.388605Min latency(s):       0.015351

（5）测试10秒随机读写性能

[root@node141 mnt]# rados bench -p rbd 10 randhints = 1  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)    0       0         0         0         0         0           -           0    1      16       408       392   1566.56      1568   0.0715088    0.038812    2      16       794       778   1554.08      1544   0.0219942   0.0396005    3      16      1211      1195   1591.93      1668    0.125987   0.0389289    4      15      1604      1589    1587.9      1576   0.0497586   0.0392412    5      16      2013      1997   1596.67      1632  0.00512184   0.0388821    6      16      2415      2399   1598.52      1608    0.146983   0.0389842    7      16      2806      2790   1593.56      1564    0.110503   0.0391283    8      16      3186      3170   1584.34      1520   0.0226104   0.0393014    9      15      3598      3583   1591.83      1652   0.0117805   0.0392624   10      16      3965      3949   1579.03      1464   0.0752829   0.0395739Total time run:       10.0536Total reads made:     3965Read size:            4194304Object size:          4194304Bandwidth (MB/sec):   1577.54Average IOPS:         394Stddev IOPS:          15.6308Max IOPS:             417Min IOPS:             366Average Latency(s):   0.0397174Max latency(s):       0.353027Min latency(s):       0.00414538

（6）要增加并发读取和写入的数量，请使用-t 默认为16个现场的选项，-b参数可以调整正在写入的对象的大小，默认对象大小为4MB。安全的最大对象为16MB.
添加 --run-name 选项以控制在基准测试期间写入的对象的名称。多个rados bench命令可以同时通过改变跑出--run-name标签每个运行命令的实例。这可以防止在多个客户端尝试访问同一对象并允许不同客户端访问不同对象时可能发生的潜在I / O错误。--run-name在尝试模拟真实世界的工作负载时，该选项也很有用。例如：

[root@node141 mnt]# rados bench -p rbd 10 write  -t 4 --run-name zabbix71hints = 1Maintaining 4 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objectsObject prefix: benchmark_data_node141_28450  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)    0       0         0         0         0         0           -           0    1       4        40        36   143.992       144    0.065704    0.103059    2       4        86        82   163.981       184   0.0763531   0.0955013    3       4       126       122   162.641       160    0.106746   0.0953542    4       4       173       169   168.972       188   0.0872597   0.0934176    5       4       218       214   171.171       180    0.170851   0.0927783    6       4       255       251   167.305       148   0.0943133    0.093864    7       4       294       290   165.686       156   0.0976048   0.0959294    8       4       340       336   167.971       184   0.0882485   0.0946923    9       4       386       382   169.748       184   0.0703278   0.0933611   10       4       430       426    170.37       176   0.0623777   0.0933391Total time run:         10.1109Total writes made:      431Write size:             4194304Object size:            4194304Bandwidth (MB/sec):     170.51Stddev Bandwidth:       16.6747Max bandwidth (MB/sec): 188Min bandwidth (MB/sec): 144Average IOPS:           42Stddev IOPS:            4.16867Max IOPS:               47Min IOPS:               36Average Latency(s):     0.0935851Stddev Latency(s):      0.0433054Max latency(s):         0.579434Min latency(s):         0.0533891Cleaning up (deleting benchmark objects)Removed 431 objectsClean up completed and total clean up time :1.62203

(7) 删除rados bench 命令创建的数据
[root@node141 mnt]# rados -p rbd cleanup
Removed 965 objects