Centos7下Redis6.2.6 cluster集群搭建

下载解压编译

sudo wget https://download.redis.io/releases/redis-6.2.6.tar.gz 
sudo tar -zxvf redis-4.0.11.tar.gz 
cd redis-6.2.6 
sudo make test 
sudo make PREFIX=/usr/local/redis install //安装到指定目录 
//安装完成在安装目录下的bin目录执行 
./redis-server

配置文件

[vagrant@localhost bin]$ sudo cp /tmp/redis-6.2.6/redis.conf /usr/local/redis/bin/redis_6300.conf 
[vagrant@localhost bin]$ sudo vim redis_6300.conf 
    # 注释本地访问 # bind 127.0.0.1 
    # 指定端口 port 6300 
    # 启动后台运行 
    daemonize yes 
    # 存放进程Id 
    pidfile /var/run/redis_6300.pid 
    # 日志文件 
    logfile "6300.log" 
    # 快照存放文件 
    dbfilename 6300.rdb 
    # 是否启动保存操作日志 
    appendonly yes 
    # 操作日志存放文件 
    appendfilename "6300.aof" 
    # 是否开启集群 
    cluster-enabled yes 
    # 集群配置文件（启动后自动生成） 
    cluster-config-file nodes-6300.conf 
    # 集群节点ping、pong超时时间 
    cluster-node-timeout 5000

根据上面的配置文件生成另外5个配置文件

[vagrant@localhost bin]$ sed 's/6300/6301/' redis_6300.conf > redis_6301.conf 
[vagrant@localhost bin]$ sed 's/6300/6302/' redis_6300.conf > redis_6302.conf 
[vagrant@localhost bin]$ sed 's/6300/6303/' redis_6300.conf > redis_6303.conf 
[vagrant@localhost bin]$ sed 's/6300/6304/' redis_6300.conf > redis_6304.conf 
[vagrant@localhost bin]$ sed 's/6300/6305/' redis_6300.conf > redis_6305.conf

创建批量启动的脚本

[vagrant@localhost bin]$ sudo vim start_all_node.sh 
    #!/bin/bash 
    ./redis-server ./redis_6300.conf 
    ./redis-server ./redis_6301.conf 
    ./redis-server ./redis_6302.conf 
    ./redis-server ./redis_6303.conf 
    ./redis-server ./redis_6304.conf 
    ./redis-server ./redis_6305.conf

赋予执行权限

[vagrant@localhost bin]$ chmod +x start_all_node.sh

启动服务

[vagrant@localhost bin] ./start_all_node.sh

查看进程，Redis以cluster集群模式启动，此时各节点间还没建立关系

[vagrant@localhost bin]$ ps -ef|grep redis 
vagrant 6459 1 0 19:55 ? 00:00:00 ./redis-server *:6300 [cluster]           vagrant 6465 1 0 19:55 ? 00:00:00 ./redis-server *:6301 [cluster]
vagrant 6467 1 0 19:55 ? 00:00:00 ./redis-server *:6302 [cluster] 
vagrant 6473 1 0 19:55 ? 00:00:00 ./redis-server *:6304 [cluster] 
vagrant 6479 1 0 19:55 ? 00:00:00 ./redis-server *:6305 [cluster] 
vagrant 6485 1 0 19:55 ? 00:00:00 ./redis-server *:6303 [cluster]

集群管理

Redis Cluster 将全部的键空间划分为16384块，每一块空间称之为槽（slot），又将这些槽及槽所对应的 k-v 划分给集群中的每个主节点负责

集群的创建

使用redis-cli –cluster create 命令创建集群（将节点加入集群，指定节点主从关系，同时执行均匀分槽，将16384个槽位分配给3个主节点）

[vagrant@localhost bin]$ ./redis-cli --cluster create 127.0.0.1:6300 127.0.0.1:6301 127.0.0.1:6302 127.0.0.1:6303 127.0.0.1:6304 127.0.0.1:6305 --cluster-replicas 1

执行上述命令后，集群就创建成功了，此时架构如下：

那么Redis Cluster 数据如何读写？

使用了虚拟槽分区：使用分散度良好的hash函数将数据映射到一个固定范围的整数集合【CRC16(key) mod 16383，结果便是对应的slot】。

集群模式下，客户端去操作集群是直连到一个具体的节点上操作的。当该节点接收到任何键操作命令时，会先计算键对应的slot，然后根据slot找出对应节点，如果对应的节点是自身，则执行键操作命令，返回结果；如果不是自身，会返回给客户端MOVED重定向错误，告诉客户端应该请求具体哪个节点，由客户端发起二次请求到正确的节点，完成本次键操作。MOVED错误信息如下所示:

[vagrant@localhost bin]$ redis-cli -p 6300 
127.0.0.1:6300> set key1 1 
(error) MOVED 9189 127.0.0.1:6301

集群扩容

新增6306、6307两个节点，一主一从，并为主节点分配384个槽位。

# 拷贝配置文件 
[vagrant@localhost bin]$ sed 's/6300/6306/' redis_6300.conf > redis_6306.conf 
[vagrant@localhost bin]$ sed 's/6300/6307/' redis_6300.conf > redis_6307.conf
# 编辑start_all_node.sh 批量启动脚本，新增两行
[vagrant@localhost bin]$ sudo vim start_all_node.sh 
    ./redis-server ./redis_6306.conf 
    ./redis-server ./redis_6307.conf

启动服务、查看进程

[vagrant@localhost bin]$ ./start_all_node.sh 
[vagrant@localhost bin]$ ps -ef|grep redis 
vagrant 6459 1 0 Apr06 ? 00:00:40 ./redis-server *:6300 [cluster] 
vagrant 6465 1 0 Apr06 ? 00:00:40 ./redis-server *:6301 [cluster] 
vagrant 6467 1 0 Apr06 ? 00:00:41 ./redis-server *:6302 [cluster] 
vagrant 6473 1 0 Apr06 ? 00:00:39 ./redis-server *:6304 [cluster] 
vagrant 6479 1 0 Apr06 ? 00:00:39 ./redis-server *:6305 [cluster] 
vagrant 6485 1 0 Apr06 ? 00:00:39 ./redis-server *:6303 [cluster] 
vagrant 7020 1 0 09:48 ? 00:00:00 ./redis-server *:6306 [cluster] 
vagrant 7022 1 0 09:48 ? 00:00:00 ./redis-server *:6307 [cluster]

将新节点6306作为主节点加入已有集群 127.0.0.1:6300【已在集群中的任意节点】

[vagrant@localhost bin]$ ./redis-cli -h 127.0.0.1:6300 --cluster add-node 127.0.0.1:6306 127.0.0.1:6300

查看6306节点ID

[vagrant@localhost bin]$ ./redis-cli -p 6300 
127.0.0.1:6300> cluster nodes 
a1c5689f94be7cdb318391d65cdffab411ad0fe3 127.0.0.1:6305@16305 slave 4dabcf15beb6c6000c41b959f03b752ca70a1330 0 1649297381158 1 connected
4dabcf15beb6c6000c41b959f03b752ca70a1330 127.0.0.1:6300@16300 myself,master - 0 1649297379000 1 connected 0-5460
5ffcd588ef362d77a689e404f7efe4ad9b52ae9e 127.0.0.1:6306@16306 master - 0 1649297381000 0 connected 
ef1e65ff26f30bad719ca6c1021fe4ccd7111e2c 127.0.0.1:6304@16304 slave 2beda757d603c320dbf3cb5c575f11e4d6caeb49 0 1649297380000 3 connected 
b45532e9f61f67a7cd57729a22755bb806b79207 127.0.0.1:6301@16301 master - 0 1649297381000 2 connected 5461-10922 
e213d20b674cb5aea1ae4972ef5413f5edc602d8 127.0.0.1:6303@16303 slave b45532e9f61f67a7cd57729a22755bb806b79207 0 1649297382165 2 connected
2beda757d603c320dbf3cb5c575f11e4d6caeb49 127.0.0.1:6302@16302 master - 0 1649297383241 3 connected 10923-16383

将新节点6307作为6306主节点的从节点加入集群

# –cluster-slave指定当前节点为从节点 # –cluster-master-id 指定master节点集群Id 
./redis-cli -h 127.0.0.1:6300 --cluster add-node 127.0.0.1:6307 127.0.0.1:6300 --cluster-slave --cluster-master-id 5ffcd588ef362d77a689e404f7efe4ad9b52ae9e

此时还未分配槽位的状态，需要手动分配槽位

[vagrant@localhost bin]$ ./redis-cli -p 6300 
127.0.0.1:6300> cluster nodes 
a1c5689f94be7cdb318391d65cdffab411ad0fe3 127.0.0.1:6305@16305 slave 4dabcf15beb6c6000c41b959f03b752ca70a1330 0 1649297655000 1 connected 
4dabcf15beb6c6000c41b959f03b752ca70a1330 127.0.0.1:6300@16300 myself,master - 0 1649297656000 1 connected 0-5460 
799899eee84e10e5a2fc78a742568b52c422668c 127.0.0.1:6307@16307 slave 5ffcd588ef362d77a689e404f7efe4ad9b52ae9e 0 1649297654996 0 connected 
5ffcd588ef362d77a689e404f7efe4ad9b52ae9e 127.0.0.1:6306@16306 master - 0 1649297655000 0 connected # 主节点，未分配状态 
ef1e65ff26f30bad719ca6c1021fe4ccd7111e2c 127.0.0.1:6304@16304 slave 2beda757d603c320dbf3cb5c575f11e4d6caeb49 0 1649297651000 3 connected 
b45532e9f61f67a7cd57729a22755bb806b79207 127.0.0.1:6301@16301 master - 0 1649297657047 2 connected 5461-10922 
e213d20b674cb5aea1ae4972ef5413f5edc602d8 127.0.0.1:6303@16303 slave b45532e9f61f67a7cd57729a22755bb806b79207 0 1649297652904 2 connected 
2beda757d603c320dbf3cb5c575f11e4d6caeb49 127.0.0.1:6302@16302 master - 0 1649297656012 3 connected 10923-16383

为主节点6306分配槽位

[vagrant@localhost bin]$ ./redis-cli -h 127.0.0.1 -p 6300 --cluster reshard 127.0.0.1:6306 
>>> Performing Cluster Check (using node 127.0.0.1:6306) 
M: 5ffcd588ef362d77a689e404f7efe4ad9b52ae9e 127.0.0.1:6306 
    slots: (0 slots) master 
    1 additional replica(s) 
S: a1c5689f94be7cdb318391d65cdffab411ad0fe3 127.0.0.1:6305 
    slots: (0 slots) slave 
    replicates 4dabcf15beb6c6000c41b959f03b752ca70a1330 
M: 4dabcf15beb6c6000c41b959f03b752ca70a1330 127.0.0.1:6300 
    slots:[0-5460] (5461 slots) master 
    1 additional replica(s) 
M: b45532e9f61f67a7cd57729a22755bb806b79207 127.0.0.1:6301 
    slots:[5461-10922] (5462 slots) master 
    1 additional replica(s) 
M: 2beda757d603c320dbf3cb5c575f11e4d6caeb49 127.0.0.1:6302 
    slots:[10923-16383] (5461 slots) master 
    1 additional replica(s) 
S: 799899eee84e10e5a2fc78a742568b52c422668c 127.0.0.1:6307 
    slots: (0 slots) slave 
    replicates 5ffcd588ef362d77a689e404f7efe4ad9b52ae9e 
S: e213d20b674cb5aea1ae4972ef5413f5edc602d8 127.0.0.1:6303 
    slots: (0 slots) slave 
    replicates b45532e9f61f67a7cd57729a22755bb806b79207 
S: ef1e65ff26f30bad719ca6c1021fe4ccd7111e2c 127.0.0.1:6304 
    slots: (0 slots) slave 
    replicates 2beda757d603c320dbf3cb5c575f11e4d6caeb49 
[OK] All nodes agree about slots configuration. 
>>> Check for open slots... 
>>> Check slots coverage... [OK] 
All 16384 slots covered. How many slots do you want to move (from 1 to 16384)? 384 # 分配多少槽位 
What is the receiving node ID? 5ffcd588ef362d77a689e404f7efe4ad9b52ae9e #分配给哪个节点【6306节点】 
Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs. Source node #1: 2beda757d603c320dbf3cb5c575f11e4d6caeb49 # 从哪个节点分割槽位【6302节点】 
Source node #2: done # done完成配置 ... 是否确认分配 
Do you want to proceed with the proposed reshard plan (yes/..no)? yes

扩容后的节点信息

[vagrant@localhost bin]$ ./redis-cli -p 6300 
127.0.0.1:6300> cluster nodes 
a1c5689f94be7cdb318391d65cdffab411ad0fe3 127.0.0.1:6305@16305 slave 4dabcf15beb6c6000c41b959f03b752ca70a1330 0 1649298565000 1 connected 
4dabcf15beb6c6000c41b959f03b752ca70a1330 127.0.0.1:6300@16300 myself,master - 0 1649298566000 1 connected 0-5460 799899eee84e10e5a2fc78a742568b52c422668c 127.0.0.1:6307@16307 slave 5ffcd588ef362d77a689e404f7efe4ad9b52ae9e 0 1649298567601 7 connected 
5ffcd588ef362d77a689e404f7efe4ad9b52ae9e 127.0.0.1:6306@16306 master - 0 1649298565000 7 connected 10923-11306 # 已经分配了384个槽位 
ef1e65ff26f30bad719ca6c1021fe4ccd7111e2c 127.0.0.1:6304@16304 slave 2beda757d603c320dbf3cb5c575f11e4d6caeb49 0 1649298565000 3 connected 
b45532e9f61f67a7cd57729a22755bb806b79207 127.0.0.1:6301@16301 master - 0 1649298566562 2 connected 5461-10922 
e213d20b674cb5aea1ae4972ef5413f5edc602d8 127.0.0.1:6303@16303 slave b45532e9f61f67a7cd57729a22755bb806b79207 0 1649298568616 2 connected 
2beda757d603c320dbf3cb5c575f11e4d6caeb49 127.0.0.1:6302@16302 master - 0 1649298569627 3 connected 11307-16383

集群缩容

移除6306、6307节点。将6306的384个槽位重新分配给6302

# --cluster-from 分配源节点ID 
# --cluster-to 分配目标节点ID 
# --cluster-slots 分配槽位数量和节点信息 
./redis-cli -h 127.0.0.1 -p 6300 --cluster reshard --cluster-from 5ffcd588ef362d77a689e404f7efe4ad9b52ae9e --cluster-to 2beda757d603c320dbf3cb5c575f11e4d6caeb49 --cluster-slots 384 127.0.0.1:6306

查看集群节点信息

127.0.0.1:6300> cluster nodes 
a1c5689f94be7cdb318391d65cdffab411ad0fe3 127.0.0.1:6305@16305 slave 4dabcf15beb6c6000c41b959f03b752ca70a1330 0 1649299460000 1 connected 
4dabcf15beb6c6000c41b959f03b752ca70a1330 127.0.0.1:6300@16300 myself,master - 0 1649299462000 1 connected 0-5460 
799899eee84e10e5a2fc78a742568b52c422668c 127.0.0.1:6307@16307 slave 2beda757d603c320dbf3cb5c575f11e4d6caeb49 0 1649299463000 8 connected 
5ffcd588ef362d77a689e404f7efe4ad9b52ae9e 127.0.0.1:6306@16306 master - 0 1649299462326 7 connected # 槽位已经重新分配 
ef1e65ff26f30bad719ca6c1021fe4ccd7111e2c 127.0.0.1:6304@16304 slave 2beda757d603c320dbf3cb5c575f11e4d6caeb49 0 1649299461000 8 connected 
b45532e9f61f67a7cd57729a22755bb806b79207 127.0.0.1:6301@16301 master - 0 1649299463461 2 connected 5461-10922 
e213d20b674cb5aea1ae4972ef5413f5edc602d8 127.0.0.1:6303@16303 slave b45532e9f61f67a7cd57729a22755bb806b79207 0 1649299464551 2 connected 
2beda757d603c320dbf3cb5c575f11e4d6caeb49 127.0.0.1:6302@16302 master - 0 1649299460264 8 connected 10923-16383

移除6306、6307节点

[vagrant@localhost bin]$ ./redis-cli --cluster del-node 127.0.0.1:6306 5ffcd588ef362d77a689e404f7efe4ad9b52ae9e 
>>> Removing node 5ffcd588ef362d77a689e404f7efe4ad9b52ae9e from cluster 127.0.0.1:6306 
>>> Sending CLUSTER FORGET messages to the cluster... 
>>> Sending CLUSTER RESET SOFT to the deleted node. 

[vagrant@localhost bin]$ ./redis-cli --cluster del-node 127.0.0.1:6307 799899eee84e10e5a2fc78a742568b52c422668c 
>>> Removing node 799899eee84e10e5a2fc78a742568b52c422668c from cluster 127.0.0.1:6307 
>>> Sending CLUSTER FORGET messages to the cluster... 
>>> Sending CLUSTER RESET SOFT to the deleted node

再次查看集群节点信息，6306、6307节点已经从集群中移除

[vagrant@localhost bin]$ ./redis-cli -c -p 6300 
127.0.0.1:6300> cluster nodes 
a1c5689f94be7cdb318391d65cdffab411ad0fe3 127.0.0.1:6305@16305 slave 4dabcf15beb6c6000c41b959f03b752ca70a1330 0 1649299625437 1 connected 
4dabcf15beb6c6000c41b959f03b752ca70a1330 127.0.0.1:6300@16300 myself,master - 0 1649299625000 1 connected 0-5460 
ef1e65ff26f30bad719ca6c1021fe4ccd7111e2c 127.0.0.1:6304@16304 slave 2beda757d603c320dbf3cb5c575f11e4d6caeb49 0 1649299624388 8 connected 
b45532e9f61f67a7cd57729a22755bb806b79207 127.0.0.1:6301@16301 master - 0 1649299626523 2 connected 5461-10922 
e213d20b674cb5aea1ae4972ef5413f5edc602d8 127.0.0.1:6303@16303 slave b45532e9f61f67a7cd57729a22755bb806b79207 0 1649299624000 2 connected 
2beda757d603c320dbf3cb5c575f11e4d6caeb49 127.0.0.1:6302@16302 master - 0 1649299622291 8 connected 10923-16383

如何保证高可用？

要保证高可用的前提是离不开从节点的，一旦某个主节点因为某种原因不可用后，就需要一个一直默默当备胎的从节点顶上来了。一般在集群搭建时最少都需要6个实例，其中3个实例做主节点，各自负责一部分槽位，另外3个实例各自对应一个主节点做其从节点，对主节点的操作进行复制

本作品采用《CC 协议》，转载必须注明作者和本文链接