MongoDB分布式集群分片

MongoDB高可用集群搭建

一、环境准备

# 启动时需要使用非root用户，所有创建一个mongo用户：useradd mongo# 为mongo用户添加密码：echo 123456 | passwd --stdin mongo# 将mongo添加到sudoersecho "mongo ALL = (root) NOPASSWD:ALL" | tee /etc/sudoers.d/mongochmod 0440 /etc/sudoers.d/mongo#解决sudo: sorry, you must have a tty to run sudo问题，在/etc/sudoer注释掉 Default requiretty 一行sudo sed -i 's/Defaults    requiretty/Defaults:chiansun !requiretty/' /etc/sudoers# 创建一个mongo目录mkdir /mongo# 给相应的目录添加权限chown -R mongo:mongo /mongo# 配置mongo的yum源cat >> /etc/yum.repos.d/mongodb-org-4.0.repo << EOF[mongodb-org-4.0]name=MongoDB Repositorybaseurl=http://repo.mongodb.org/yum/redhat/\$releasever/mongodb-org/4.0/x86_64/gpgcheck=1enabled=1gpgkey=https://www.mongodb.org/static/pgp/server-4.0.ascEOF# 关闭selinuxsed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/configsetenforce 0# 关闭防火墙systemctl disable firewalldsystemctl stop firewalld

二、主机规划

192.168.33.14   node-1192.168.33.15   node-2192.168.33.16   node-3node-1    node-2       node-3 mongos    mongos       mongos     路由服务器，寻址config    config       config     配置服务器，保存配置shard1主  shard2主     shard3主    分片：保存数据shard2从  shard3从     shard1从    副本集：备份数据，可以配置读写分离（主负责写，从负责同步数据和读）shard3从  shard1从     shard2从

三、安装部署

# 分别在多台机器上使用mongo用户登录sudo yum install -y mongodb-org # 分别在多台机器上创建mongo config server对应的目录mkdir -p /mongo/config/{log,data,run}# 分别在多台机器上修改config server的配置文件cat > /mongo/config/mongod.conf << EOFsystemLog:  destination: file  logAppend: true  path: /mongo/config/log/mongod.logstorage:  dbPath: /mongo/config/data  journal:    enabled: trueprocessManagement:  fork: true  pidFilePath: /mongo/config/run/mongod.pidnet:  port: 27100  bindIp: 0.0.0.0replication:  replSetName: configsharding:  clusterRole: configsvrEOF# 启动所有的mongo config server服务mongod --config /mongo/config/mongod.conf# 登录任意一台配置服务器，初始化配置副本集mongo --port 27100# 创建配置# id名要和replSetName名保持一致config = {   _id : "config",    members : [        {_id : 0, host : "192.168.33.14:27100" },        {_id : 1, host : "192.168.33.15:27100" },        {_id : 2, host : "192.168.33.16:27100" }    ]}# 初始化副本集配置rs.initiate(config)# 查看分区状态rs.status()# 注意:其中，"_id" : "config"应与配置文件中配置的 replicaction.replSetName 一致，"members" 中的 "host" 为三个节点的ip和port

1.配置第一个分片和副本集

# 修改mongo shard1 server的配置文件mkdir -p /mongo/shard1/{log,data,run}# 分别在多台机器上修改shard1 server的配置文件cat > /mongo/shard1/mongod.conf << EOFsystemLog:  destination: file  logAppend: true  path: /mongo/shard1/log/mongod.logstorage:  dbPath: /mongo/shard1/data  journal:    enabled: trueprocessManagement:  fork: true  pidFilePath: /mongo/shard1/run/mongod.pidnet:  port: 27001  bindIp: 0.0.0.0replication:  replSetName: shard1sharding:  clusterRole: shardsvrEOF# 启动所有的shard1 servermongod --config /mongo/shard1/mongod.conf# 登陆任意一台shard1服务器，初始化副本集mongo --port 27001# 使用admin数据库use admin# 定义副本集配置config = {   _id : "shard1",    members : [        {_id : 0, host : "192.168.33.14:27001" },        {_id : 1, host : "192.168.33.15:27001" },        {_id : 2, host : "192.168.33.16:27001" }    ]}# 初始化副本集配置rs.initiate(config);# 查看分区状态rs.status()

2.配置第二个分片和副本集

# 修改mongo shard2 server的配置文件mkdir -p /mongo/shard2/{log,data,run}# 分别在多台机器上修改shard2 server的配置文件cat > /mongo/shard2/mongod.conf << EOFsystemLog:  destination: file  logAppend: true  path: /mongo/shard2/log/mongod.logstorage:  dbPath: /mongo/shard2/data  journal:    enabled: trueprocessManagement:  fork: true  pidFilePath: /mongo/shard2/run/mongod.pidnet:  port: 27002  bindIp: 0.0.0.0replication:  replSetName: shard2sharding:  clusterRole: shardsvrEOF# 启动所有的shard2 servermongod --config /mongo/shard2/mongod.conf# 登陆任意一台shard2服务器，初始化副本集mongo --port 27002# 使用admin数据库use admin# 定义副本集配置config = {   _id : "shard2",    members : [        {_id : 0, host : "192.168.33.14:27002" },        {_id : 1, host : "192.168.33.15:27002" },        {_id : 2, host : "192.168.33.16:27002" }    ]}# 初始化副本集配置rs.initiate(config)# 查看分区状态rs.status()

3.配置第三个分片和副本集

# 修改mongo shard3 server的配置文件mkdir -p /mongo/shard3/{log,data,run}# 分别在多台机器上修改shard3 server的配置文件cat > /mongo/shard3/mongod.conf << EOFsystemLog:  destination: file  logAppend: true  path: /mongo/shard3/log/mongod.logstorage:  dbPath: /mongo/shard3/data  journal:    enabled: trueprocessManagement:  fork: true  pidFilePath: /mongo/shard3/run/mongod.pidnet:  port: 27003  bindIp: 0.0.0.0replication:  replSetName: shard3sharding:  clusterRole: shardsvrEOF# 启动所有的shard3 servermongod --config /mongo/shard3/mongod.conf# 登陆任意一台的shard3服务器，初始化副本集mongo --port 27003# 使用admin数据库use admin# 定义副本集配置config = {   _id : "shard3",    members : [        {_id : 0, host : "192.168.33.14:27003" },        {_id : 1, host : "192.168.33.15:27003" },        {_id : 2, host : "192.168.33.16:27003" }    ]}# 初始化副本集配置rs.initiate(config)# 查看分区状态rs.status()

4.配置mongos路由器

##### 注意：启动mongos是守候进程是因为/mongo/mongos/mongod.conf缺少了fork: true这个配置#######------------------------------------------------------------------------------------------mkdir -p /mongo/mongos/{log,data,run}# 添加mongs的配置文件cat > /mongo/mongos/mongod.conf << EOFsystemLog:  destination: file  logAppend: true  path: /mongo/mongos/log/mongod.logprocessManagement:  fork: true  pidFilePath: /mongo/mongos/run/mongod.pidnet:  port: 27200  bindIp: 0.0.0.0sharding:  configDB: config/192.168.33.14:27100,192.168.33.15:27100,192.168.33.16:27100EOF# 注意，这里configDB后面的config要与配置服务器的_id保持一致# 启动路由服务器mongos --config /mongo/mongos/mongod.conf# 登录其中的一台路由节点，手动启用分片mongo --port 27200# 添加分片到mongossh.addShard("shard1/192.168.33.14:27001,192.168.33.15:27001,192.168.33.16:27001")sh.addShard("shard2/192.168.33.15:27002,192.168.33.16:27002,192.168.33.14:27002")sh.addShard("shard3/192.168.33.16:27003,192.168.33.14:27003,192.168.33.15:27003")# 设置slave可读rs.slaveOk()

5.常用操作

#没有分片是因为没有开启分片规则# 对bike这个数据库开启分片功能use admindb.runCommand({"enablesharding":"bike"}) # 对bike数据库下的users集合按id的hash进行分片db.runCommand({"shardcollection":"bike.users","key":{_id:'hashed'}})# 启动所有的config servermongod --config /mongo/config/mongod.conf# 启动所有的shard1mongod --config /mongo/shard1/mongod.conf# 启动所有的shard2mongod --config /mongo/shard2/mongod.conf# 启动所有的shard3mongod --config /mongo/shard3/mongod.conf# 启动所有的mongosmongos --config /mongo/mongos/mongod.conf# 关闭服务mongod --shutdown --dbpath /mongo/shard3/datamongod --shutdown --dbpath /mongo/shard2/datamongod --shutdown --dbpath /mongo/shard1/datamongod --shutdown --dbpath /mongo/config/data