ceph-deploy 添加/删除 mon节点

1、简介

mon 操作 我还真是比较少，关于 mon 添加删除 操作 我反正 从来 没有 在线上环境执行过；哈哈哈，为了以备 不时之需，还是 练练手 了解下（有备无患 嘛）；本文先 基于 ceph-deploy 工具 操作（手动的 反正 我也试过了 -- 没难度，有兴趣的 可以 自己 手动 尝试下，需要的话 留言 后期 补上 手动 操作文档）

1.1 操作系统

## 本文操作系统 Ubuntu 16.04 amd64# lsb_release -a

1.2 ceph 版本

ceph Luminous 12.2.5

2、删除 mon

为什么 先来 删除 mon，因为 我得 环境 刚好 部署了 三个 mon，先删除 两个 mon

2.1 删除

# ceph-deploy --username cstor mon destroy mon2[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cstor/.cephdeploy.conf[ceph_deploy.cli][INFO  ] Invoked (1.5.39): /usr/bin/ceph-deploy --username cstor mon destroy mon2[ceph_deploy.cli][INFO  ] ceph-deploy options:[ceph_deploy.cli][INFO  ]  username                      : cstor[ceph_deploy.cli][INFO  ]  verbose                       : False[ceph_deploy.cli][INFO  ]  overwrite_conf                : False[ceph_deploy.cli][INFO  ]  subcommand                    : destroy[ceph_deploy.cli][INFO  ]  quiet                         : False[ceph_deploy.cli][INFO  ]  cd_conf                       : [ceph_deploy.cli][INFO  ]  cluster                       : ceph[ceph_deploy.cli][INFO  ]  mon                           : ['mon2'][ceph_deploy.cli][INFO  ]  func                          : [ceph_deploy.cli][INFO  ]  ceph_conf                     : None[ceph_deploy.cli][INFO  ]  default_release               : False[ceph_deploy.mon][DEBUG ] Removing mon from mon2cstor@mon2's password: [mon2][DEBUG ] connection detected need for sudocstor@mon2's password: [mon2][DEBUG ] connected to host: cstor@mon2 [mon2][DEBUG ] detect platform information from remote host[mon2][DEBUG ] detect machine type[mon2][DEBUG ] find the location of an executable[mon2][DEBUG ] get remote short hostname[mon2][INFO  ] Running command: sudo ceph --cluster=ceph -n mon. -k /var/lib/ceph/mon/ceph-mon2/keyring mon remove mon2[mon2][WARNIN] removing mon.mon2 at XXX.XXX.XXX.XXX:6789/0, there will be 2 monitors[mon2][INFO  ] polling the daemon to verify it stopped[mon2][INFO  ] Running command: sudo systemctl stop ceph-mon@mon2.service[mon2][INFO  ] Running command: sudo mkdir -p /var/lib/ceph/mon-removed[mon2][DEBUG ] move old monitor data

2.2 配置 修改

我也不知道 不修改 配置 会不会 有什么 影响（最好还是 修改一下吧）-- mon_initial_members、mon_host 这两个参数（修改 集群 中 所有得配置 -- 虽然麻烦，但是 还需要操作哦）

# cat /etc/ceph/ceph.conf    ... ...    mon_initial_members = mon0, mon1    mon_host = XXX.XXX.XXX.XXX,XXX.XXX.XXX.XXX    ... ...

3、添加 mon

哈哈哈，删除 mon 之后，现在 开始 添加 mon （开始埋坑）

3.1 配置修改

## 修改配置文件时，需要 将集群 得配置文件 统一# cat /etc/ceph/ceph.conf    ... ...    mon_initial_members = mon0, mon1,mon2    mon_host = XXX.XXX.XXX.XXX,XXX.XXX.XXX.XXX,XXX.XXX.XXX.XXX        ... ...

3.2 添加 mon

# ceph-deploy --username cstor mon create mon2[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cstor/.cephdeploy.conf[ceph_deploy.cli][INFO  ] Invoked (1.5.39): /usr/bin/ceph-deploy --username cstor mon create mon2[ceph_deploy.cli][INFO  ] ceph-deploy options:[ceph_deploy.cli][INFO  ]  username                      : cstor[ceph_deploy.cli][INFO  ]  verbose                       : False[ceph_deploy.cli][INFO  ]  overwrite_conf                : False[ceph_deploy.cli][INFO  ]  subcommand                    : create[ceph_deploy.cli][INFO  ]  quiet                         : False[ceph_deploy.cli][INFO  ]  cd_conf                       : [ceph_deploy.cli][INFO  ]  cluster                       : ceph[ceph_deploy.cli][INFO  ]  mon                           : ['mon2'][ceph_deploy.cli][INFO  ]  func                          : [ceph_deploy.cli][INFO  ]  ceph_conf                     : None[ceph_deploy.cli][INFO  ]  keyrings                      : None[ceph_deploy.cli][INFO  ]  default_release               : False[ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts mon2[ceph_deploy.mon][DEBUG ] detecting platform for host mon2 ...cstor@mon2's password: [mon2][DEBUG ] connection detected need for sudocstor@mon2's password: [mon2][DEBUG ] connected to host: cstor@mon2 [mon2][DEBUG ] detect platform information from remote host[mon2][DEBUG ] detect machine type[mon2][DEBUG ] find the location of an executable[ceph_deploy.mon][INFO  ] distro info: Ubuntu 16.04 xenial[mon2][DEBUG ] determining if provided host has same hostname in remote[mon2][DEBUG ] get remote short hostname[mon2][DEBUG ] deploying mon to mon2[mon2][DEBUG ] get remote short hostname[mon2][DEBUG ] remote hostname: mon2[mon2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf[mon2][DEBUG ] create the mon path if it does not exist[mon2][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-mon2/done[mon2][DEBUG ] done path does not exist: /var/lib/ceph/mon/ceph-mon2/done[mon2][INFO  ] creating keyring file: /var/lib/ceph/tmp/ceph-mon2.mon.keyring[mon2][DEBUG ] create the monitor keyring file[mon2][INFO  ] Running command: sudo ceph-mon --cluster ceph --mkfs -i mon2 --keyring /var/lib/ceph/tmp/ceph-mon2.mon.keyring --setuser 64045 --setgroup 64045[mon2][INFO  ] unlinking keyring file /var/lib/ceph/tmp/ceph-mon2.mon.keyring[mon2][DEBUG ] create a done file to avoid re-doing the mon deployment[mon2][DEBUG ] create the init path if it does not exist[mon2][INFO  ] Running command: sudo systemctl enable ceph.target[mon2][INFO  ] Running command: sudo systemctl enable ceph-mon@mon2[mon2][INFO  ] Running command: sudo systemctl start ceph-mon@mon2[mon2][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon2.asok mon_status[mon2][DEBUG ] ********************************************************************************[mon2][DEBUG ] status for monitor: mon.mon2[mon2][DEBUG ] {[mon2][DEBUG ]   "election_epoch": 0, [mon2][DEBUG ]   "extra_probe_peers": [[mon2][DEBUG ]     "XXX.XXX.XXX.XXX:6789/0", [mon2][DEBUG ]     "XXX.XXX.XXX.XXX:6789/0"[mon2][DEBUG ]   ], [mon2][DEBUG ]   "feature_map": {[mon2][DEBUG ]     "mon": {[mon2][DEBUG ]       "group": {[mon2][DEBUG ]         "features": "0x1ffddff8eea4fffb", [mon2][DEBUG ]         "num": 1, [mon2][DEBUG ]         "release": "luminous"[mon2][DEBUG ]       }[mon2][DEBUG ]     }[mon2][DEBUG ]   }, [mon2][DEBUG ]   "features": {[mon2][DEBUG ]     "quorum_con": "0", [mon2][DEBUG ]     "quorum_mon": [], [mon2][DEBUG ]     "required_con": "0", [mon2][DEBUG ]     "required_mon": [[mon2][DEBUG ]       "kraken", [mon2][DEBUG ]       "luminous"[mon2][DEBUG ]     ][mon2][DEBUG ]   }, [mon2][DEBUG ]   "monmap": {[mon2][DEBUG ]     "created": "2019-01-04 15:43:39.644529", [mon2][DEBUG ]     "epoch": 3, [mon2][DEBUG ]     "features": {[mon2][DEBUG ]       "optional": [], [mon2][DEBUG ]       "persistent": [[mon2][DEBUG ]         "kraken", [mon2][DEBUG ]         "luminous"[mon2][DEBUG ]       ][mon2][DEBUG ]     }, [mon2][DEBUG ]     "fsid": "79fabbdf-b031-4621-952a-3fe880068501", [mon2][DEBUG ]     "modified": "2019-02-22 15:47:19.502869", [mon2][DEBUG ]     "mons": [[mon2][DEBUG ]       {[mon2][DEBUG ]         "addr": "XXX.XXX.XXX.XXX:6789/0", [mon2][DEBUG ]         "name": "mon0", [mon2][DEBUG ]         "public_addr": "XXX.XXX.XXX.XXX:6789/0", [mon2][DEBUG ]         "rank": 0[mon2][DEBUG ]       }[mon2][DEBUG ]     ][mon2][DEBUG ]   }, [mon2][DEBUG ]   "name": "mon2", [mon2][DEBUG ]   "outside_quorum": [], [mon2][DEBUG ]   "quorum": [], [mon2][DEBUG ]   "rank": -1, [mon2][DEBUG ]   "state": "probing", [mon2][DEBUG ]   "sync_provider": [][mon2][DEBUG ] }[mon2][DEBUG ] ********************************************************************************[mon2][INFO  ] monitor: mon.mon2 is currently at the state of probing[mon2][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon2.asok mon_status[mon2][WARNIN] monitor mon2 does not exist in monmap[mon2][WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors[mon2][WARNIN] monitors may not be able to form quorum

4、mon 状态 查询

# ceph -s# ceph quorum_status [--format json-pretty]# ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.{mon_name}.asok mon_status