• 周六. 10 月 12th, 2024

5G编程聚合网

5G时代下一个聚合的编程学习网

热门标签

redis手工分片

King Wang

1 月 3, 2022

目录

一、redis环境

二、redis手工分片步骤

1. 配置级联复制

2. 去掉老哨兵监控

3. 停止新实例从老实例的复制

4. 添加新哨兵监控

5. 重启新哨兵       

6. 添加老哨兵监控

7. 重启老哨兵

三、注意事项

四、自动化脚本


 

      随着数据量越来越大,一个redis实例可能需要分成多个以形成数据分片。此时通常可以采取两种方式操作:一是启用cluster模式自动完成数据分片;二是手工分片,即配置需要分片redis实例的副本,再修改应用程序按一定方式(如取模等)访问不同redis实例。cluster方式的配置可以参考“初学乍练redis:Redis 5.0.3单实例数据迁移到Cluster”。本文说明第二种方式的具体操作步骤。

一、redis环境

        假设现有三台物理机192.168.1.36、192.168.1.37、192.168.1.38,分别配置为redis一主两从,通过哨兵进行访问。192.168.1.36为主实例,端口为20007,master name为redis7。三台机器上分别启动一个哨兵进程,端口为30001。从哨兵查看复制信息如下:

[root~]#sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001 info | grep redis7
master1:name=redis7,status=ok,address= 192.168.1.36:20007,slaves=2,sentinels=3
[root~]#sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001 info | grep redis7
master1:name=redis7,status=ok,address= 192.168.1.36:20007,slaves=2,sentinels=3
[root~]#sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001 info | grep redis7
master1:name=redis7,status=ok,address= 192.168.1.36:20007,slaves=2,sentinels=3

        目标是将现有的一组redis(一主两从)拆分为四组(每组为一主两从),由应用程序实现数据分片策略。新增的三组redis实例所在物理机192.168.1.39、192.168.1.40、192.168.1.41,并且三台机器上分别启动一个哨兵进程,依然是通过哨兵进行访问。

二、redis手工分片步骤

1. 配置级联复制

(1)在192.168.1.39、192.168.1.40、192.168.1.41上配置三组标准一主两从redis复制,39为主,端口分别是20001、20002、20003。

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20001 -a 123456 slaveof 192.168.1.39 20001
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20002 -a 123456 slaveof 192.168.1.39 20002
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20003 -a 123456 slaveof 192.168.1.39 20003
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20001 -a 123456 slaveof 192.168.1.39 20001
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20002 -a 123456 slaveof 192.168.1.39 20002
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20003 -a 123456 slaveof 192.168.1.39 20003

(2)配置为老实例的slave,进行数据复制

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20001 -a 123456 slaveof 192.168.1.36 20007
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20002 -a 123456 slaveof 192.168.1.36 20007
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20003 -a 123456 slaveof 192.168.1.36 20007

        此步完成后形成一个级联复制,192.168.1.36:20007具有5个slave,分别是192.168.1.37:20007、192.168.1.38:20007、192.168.1.39:20001、192.168.1.39:20002、192.168.1.39:20003。而192.168.1.39上三个redis实例(20001、20002、20003)又分别带有192.168.1.40、192.168.1.42两个slave(端口同192.168.1.39)。可以使用以下命令查看各个实例上的键数:

# 查看key数量,在192.168.1.39执行
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 20007 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 20007 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 20007 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20001 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20002 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20003 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20001 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20002 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20003 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20001 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20002 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20003 -a 123456 info | grep db0:keys=

2. 去掉老哨兵监控

# 在192.168.1.36执行
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001 sentinel remove redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001 sentinel remove redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001 sentinel remove redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001 info | grep redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001 info | grep redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001 info | grep redis7

3. 停止新实例从老实例的复制

#  在192.168.1.39执行
cat ~/remove_repl.txt | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20003 -a 123456
cat ~/remove_repl.txt | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20004 -a 123456
cat ~/remove_repl.txt | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20005 -a 123456
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20003 -a 123456 info replication
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20004 -a 123456 info replication
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20005 -a 123456 info replication

~/remove_repl.txt文件内容:

slaveof no one
config rewrite

4. 添加新哨兵监控

# 在192.168.1.39执行
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 30001
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 30001
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 30001
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 30001 info | grep master
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 30001 info | grep master
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 30001 info | grep master

/home/redis/tmp_sentinel_monitor文件内容:

sentinel monitor redis1  192.168.1.39 20001 2
sentinel set redis3 auth-pass 123456
sentinel set redis3 down-after-milliseconds 5000
sentinel set redis3 failover-timeout 10000
sentinel monitor redis2  192.168.1.39 20002 2
sentinel set redis4 auth-pass 123456
sentinel set redis4 down-after-milliseconds 5000
sentinel set redis4 failover-timeout 10000
sentinel monitor redis3  192.168.1.39 20003 2
sentinel set redis5 auth-pass 123456
sentinel set redis5 down-after-milliseconds 5000
sentinel set redis5 failover-timeout 10000

5. 重启新哨兵       

# 分别在192.168.1.41、192.168.1.40、192.168.1.39执行
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 shutdown
sudo -u redis /home/redis/redis-5.0.3/src/redis-sentinel /home/redis/sentinel/sentinel.conf > /home/redis/sentinel/sentinel.log 2>&1 &
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 info | grep master

6. 添加老哨兵监控

# 在192.168.1.36执行
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001 info | grep redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001 info | grep redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001 info | grep redis7

/home/redis/tmp_sentinel_monitor文件内容:

sentinel monitor redis7  192.168.1.36 20007 2
sentinel set redis7 auth-pass 123456
sentinel set redis7 down-after-milliseconds 5000
sentinel set redis7 failover-timeout 10000

7. 重启老哨兵

# 在192.168.1.38、192.168.1.37、192.168.1.36执行
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 shutdown
sudo -u redis /home/redis/redis-5.0.3/src/redis-sentinel /home/redis/sentinel/sentinel.conf > /home/redis/sentinel/sentinel.log 2>&1 &
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 info

三、注意事项

  • 哨兵实例本身不能加密码,否则应用程序不能正常连接。
  • 不要在级联复制的中间层master上加哨兵监控,否则通过哨兵对顶层master的访问会出问题。
  • 在去掉中间层master对上层的复制时,需要先去掉哨兵对顶层master的监控,否则即使在中间层master实例上执行了slaveof no one,依然会保持顶层master与中间层master的主从关系。
  • 为保证哨兵工作正常,避免出现+sdown问题,在更改哨兵配置后需要重启哨兵进程。

四、自动化脚本

        manual_shard.sh 文件内容如下:

#!/bin/bash
if [ ! -n "$1" ] ;then
echo "请输入原集群端口号!"
exit
fi
if [ ! -n "$2" ] ;then
echo "请输入拆分后的新集群端口号!"
exit
fi
para=$@
new_repl()
{
case $1 in
1)
index=1
for a in $para
do
if [ $index -gt 1 ]; then
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.39 -p $a -a 123456 info 2>/dev/null | grep db1:keys=
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p $a -a 123456 info 2>/dev/null | grep db1:keys=
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.41 -p $a -a 123456 info 2>/dev/null | grep db1:keys=
fi
let index+=1
done
;;
2)
index=1
for a in $para
do
if [ $index -gt 1 ]; then
cat ~/remove_repl.txt | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p $a -a 123456 2>/dev/null
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p $a -a 123456 info replication 2>/dev/null
fi
let index+=1
done
;;
3)
index=1
for a in $para
do
if [ $index -gt 1 ]; then
cat /dev/null > /home/redis/tmp_sentinel_monitor
clustername=`expr $a - 20000`
echo "sentinel monitor redis${clustername} 192.168.1.40 $a 2" >> /home/redis/tmp_sentinel_monitor
echo "sentinel set redis${clustername} auth-pass 123456" >> /home/redis/tmp_sentinel_monitor
echo "sentinel set redis${clustername} down-after-milliseconds 5000" >> /home/redis/tmp_sentinel_monitor
echo "sentinel set redis${clustername} failover-timeout 10000" >> /home/redis/tmp_sentinel_monitor
cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.39 -p 30001
cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p 30001
cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.41 -p 30001
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.39 -p 30001 info | grep $a
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p 30001 info | grep $a
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.41 -p 30001 info | grep $a
fi
let index+=1
done
;;
esac
}
get_char()
{
SAVEDSTTY=`stty -g`
stty -echo
stty cbreak
dd if=/dev/tty bs=1 count=1 2> /dev/null
stty -raw
stty echo
stty $SAVEDSTTY
}
echo "脚本执行步骤:"
echo -e "1. 查看key数量\n2. 去掉老哨兵监控\n3. 停止新实例从老实例的复制\n4. 添加新哨兵监控\n5. 重启新哨兵\n6. 添加老哨兵监控\n7. 重启老哨兵"
echo
echo -e "\n按任意键继续 1. 查看key数量 ...,组合键 CTRL+C 终止命令!\n"
char=`get_char`
# 查看key数量
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p $1 -a 123456 info 2>/dev/null | grep db1:keys=
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p $1 -a 123456 info 2>/dev/null | grep db1:keys=
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p $1 -a 123456 info 2>/dev/null | grep db1:keys=
new_repl 1
echo
echo -e "\n按任意键继续 2. 去掉老哨兵监控 ...,组合键 CTRL+C 终止命令!\n"
char=`get_char`
# 去掉老哨兵监控并确认
master_info=`/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 info | grep $1`
master_name=`echo $master_info | awk -F"," '{print $1}' | awk -F"=" '{print $2}'`
master_ip=`echo $master_info | awk -F"," '{print $3}' | awk -F"=" '{print $2}' | awk -F":" '{print $1}'`
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 sentinel remove $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001 sentinel remove $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001 sentinel remove $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 info | grep $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001 info | grep $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001 info | grep $master_name
echo
echo -e "\n按任意键继续 3. 停止新实例从老实例的复制 ...,组合键 CTRL+C 终止命令!\n"
char=`get_char`
# 停止新实例从老实例的复制并确认
new_repl 2
echo
echo -e "\n按任意键继续 4. 添加新哨兵监控 ...,组合键 CTRL+C 终止命令!\n"
char=`get_char`
# 添加新哨兵监控并确认
new_repl 3
echo
echo -e "\n按任意键继续 5. 重启新哨兵 ...,组合键 CTRL+C 终止命令!\n"
char=`get_char`
# 重启新哨兵并确认
/root/restart_sentinel.sh
ssh 192.168.1.40 /root/restart_sentinel.sh
ssh 192.168.1.41 /root/restart_sentinel.sh
wait
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.39 -p 30001 info | grep master
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p 30001 info | grep master
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.41 -p 30001 info | grep master
echo
echo -e "\n按任意键继续 6. 添加老哨兵监控 ...,组合键 CTRL+C 终止命令!\n"
char=`get_char`
# 添加老哨兵监控并确认
cat /dev/null > /home/redis/tmp_sentinel_monitor
echo "sentinel monitor ${master_name} ${master_ip} $1 2" >> /home/redis/tmp_sentinel_monitor
echo "sentinel set ${master_name} auth-pass 123456" >> /home/redis/tmp_sentinel_monitor
echo "sentinel set ${master_name} down-after-milliseconds 5000" >> /home/redis/tmp_sentinel_monitor
echo "sentinel set ${master_name} failover-timeout 10000" >> /home/redis/tmp_sentinel_monitor
cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001
cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001
cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 info | grep $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001 info | grep $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001 info | grep $master_name
echo
echo -e "\n按任意键继续 7. 重启老哨兵 ...,组合键 CTRL+C 终止命令!\n"
char=`get_char`
# 重启老哨兵并确认
echo "重启老哨兵并确认,在192.168.1.36上执行/root/restart_all_sentinel.sh"

        restart_sentinel.sh 文件内容如下: 

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 -a 123456 shutdown
sudo -u redis /home/redis/redis-5.0.3/src/redis-sentinel /home/redis/sentinel/sentinel.conf > /home/redis/sentinel/sentinel.log 2>&1 &

        restart_all_sentinel.sh 文件内容如下: 

#!/bin/bash
# 重启本机哨兵
/root/restart_sentinel.sh
# 重启远程哨兵
ssh 192.168.1.37 /root/restart_sentinel.sh
ssh 192.168.1.38 /root/restart_sentinel.sh
wait
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 info | grep master
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001 info | grep master
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001 info | grep master

        如将20011拆分为两个新的redis实例20006、20007:

manual_shard.sh 20011 20006 20007

 

发表回复