redis cluster数据迁移（Redis数据迁移方案分享）

受疫情影响，公司业务下滑，公司内部开始开源节流IT基础设施这边要做的就是压缩服务器资源，对线上业务系统、服务等进行整合，我来为大家讲解一下关于redis cluster数据迁移?跟着小编一起来看一看吧!

redis cluster数据迁移

需求场景

受疫情影响，公司业务下滑，公司内部开始开源节流。IT基础设施这边要做的就是压缩服务器资源，对线上业务系统、服务等进行整合。

此次整合涉及到单机Redis数据的迁移。

本文模拟线上redis环境，进行数据进行迁移测试。

测试环境

Redis实例	角色	说明
10.100.210.33:6380	模拟线上生产环境
10.100.202.248:6381	模拟目标服务器上新部署的redis从库——Redis主从方案迁移数据	完成
10.100.202.248:6382	模拟目标服务器上新部署的redis实例——Redis BGSAVE方案迁移数据	完成
10.100.202.248:6383	模拟目标服务器上新部署的redis实例——redis-dump方案迁移数据	未测试

方案一：Redis从库同步主库的数据

线上环境中redis主实例配置了requirepass Abc@123，搭建从库实例，要想成功获取主库的数据，需在从库配置文件中配置masterauth Abc@123.

从库的配置文件中，requirepass Abc@123配置项目也需要配置，这里配置该参数的目的是当从库的数据与主库的数据保持同步后，会将其与主库断开连接，重启redis实例，使其使用从库本地的RDB文件进行启动，同时将应用中的redis相关配置项指向该实例，保证其与原redis实例的连接方式不变（连接redis密码与使用主库时一致。）

也可以暂时不配置该参数，待应用修改配置重新发布，从库与主库数据保持同步后，进行修改。

从库配置replicaof 10.100.210.33 6380

启动从库，关注日志信息是否从主库同步数据：

1990:C 07 May 2022 13:38:50.341 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 1990:C 07 May 2022 13:38:50.341 # Redis version=5.0.12, bits=64, commit=00000000, modified=0, pid=1990, just started 1990:C 07 May 2022 13:38:50.341 # Configuration loaded 1991:S 07 May 2022 13:38:50.342 * Increased maximum number of open files to 10032 (it was originally set to 1024). _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 5.0.12 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in standalone mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 6381 | `-._ `._ / _.-' | PID: 1991 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' 1991:S 07 May 2022 13:38:50.342 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core /somaxconn is set to the lower value of 128. 1991:S 07 May 2022 13:38:50.342 # Server initialized 1991:S 07 May 2022 13:38:50.342 # WARNING overcommit_memory is set to 0! Background save may fail under low memory cond ition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. 1991:S 07 May 2022 13:38:50.342 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This wi ll create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/tr ansparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Re dis must be restarted after THP is disabled. 1991:S 07 May 2022 13:38:50.343 * DB loaded from disk: 0.000 seconds 1991:S 07 May 2022 13:38:50.343 * Ready to accept connections 1991:S 07 May 2022 13:38:50.343 * Connecting to MASTER 10.100.210.33:6380 1991:S 07 May 2022 13:38:50.343 * MASTER <-> REPLICA sync started 1991:S 07 May 2022 13:38:50.343 * Non blocking connect for SYNC fired the event. 1991:S 07 May 2022 13:38:50.344 * Master replied to PING, replication can continue... 1991:S 07 May 2022 13:38:50.344 * Partial resynchronization not possible (no cached master) 1991:S 07 May 2022 13:38:50.363 * Full resync from master: dd1b86ecf9302bef9cd8de1daeffe3489ea9ef41:0 1991:S 07 May 2022 13:38:50.418 * MASTER <-> REPLICA sync: receiving 375972 bytes from master 1991:S 07 May 2022 13:38:50.494 * MASTER <-> REPLICA sync: Flushing old data 1991:S 07 May 2022 13:38:50.494 * MASTER <-> REPLICA sync: Loading DB in memory 1991:S 07 May 2022 13:38:50.497 * MASTER <-> REPLICA sync: Finished with success 1991:S 07 May 2022 13:43:51.087 * 10 changes in 300 seconds. Saving... 1991:S 07 May 2022 13:43:51.087 * Background saving started by pid 1998 1998:C 07 May 2022 13:43:51.173 * DB saved on disk 1998:C 07 May 2022 13:43:51.173 * RDB: 8 MB of memory used by copy-on-write 1991:S 07 May 2022 13:43:51.188 * Background saving terminated with success

上面的日志信息显示已经从主库同步完成数据。

接下来，修改从库配置文件，重启从库实例，使原从库独立运行。

vim /etc/redis/6381.conf # replicaof 10.100.210.33 6380 # 注释掉该配置项 :wq [root@c7-8 redis]# service redis_6381 restart Stopping ... Redis stopped Starting Redis server... [root@c7-8 redis]# redis-cli -p 6381 127.0.0.1:6381> keys * 1) "HzAccessToken:e2184a0878934858989d8caa34c621a1" 2) "boxState:860730050772392" 3) "wallet:trans_money_5614485601059840" 4) "HzAccessToken:8f3eb26f0d064703a90b97d9d552a5da" ... ... 127.0.0.1:6381> info replication # Replication role:master # 当前以master角色运行 connected_slaves:0 master_replid:97f166e9ab74833f1abc265cf6c946313d36709b master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

此时，将相关联的应用配置改为目标主机上的redis实例——10.100.202.248:6381，启动即可。

方案二：BGSAVE命令生成RDB文件进行迁移

通过redis-cli命令行工具连接redis实例，执行BGSAVE命令。

[root@develop ~]# redis-cli -p 6380 -h 10.100.210.33 10.100.210.33:6380> 10.100.210.33:6380> 10.100.210.33:6380> 10.100.210.33:6380> bgsave (error) NOAUTH Authentication required. 10.100.210.33:6380> auth Abc@123 OK 10.100.210.33:6380> bgsave Background saving started

redis_6380实例运行日志中输出信息如下：

26504:M 07 May 14:31:39.167 * Background saving started by pid 21451 21451:C 07 May 14:31:39.173 * DB saved on disk 21451:C 07 May 14:31:39.174 * RDB: 0 MB of memory used by copy-on-write 26504:M 07 May 14:31:39.259 * Background saving terminated with success

拷贝dump.rdb文件到目标服务器上：

[root@develop ~]# scp /var/lib/redis/6380/dump.rdb 10.100.202.248:/var/lib/redis/ The authenticity of host '10.100.202.248 (10.100.202.248)' can't be established. ECDSA key fingerprint is f4:f9:34:70:6c:85:9d:6c:65:ec:5f:1b:fd:62:9f:b3. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '10.100.202.248' (ECDSA) to the list of known hosts. root@10.100.202.248's password: dump.rdb

新建redis_6382实例：

[root@c7-8 utils]# ./install_server.sh Welcome to the redis service installer This script will help you easily set up a running redis server Please select the redis port for this instance: [6379] 6382 Please select the redis config file name [/etc/redis/6382.conf] Selected default - /etc/redis/6382.conf Please select the redis log file name [/var/log/redis_6382.log] Selected default - /var/log/redis_6382.log Please select the data directory for this instance [/var/lib/redis/6382] Selected default - /var/lib/redis/6382 Please select the redis executable path [/usr/local/bin/redis-server] Selected config: Port : 6382 Config file : /etc/redis/6382.conf Log file : /var/log/redis_6382.log Data dir : /var/lib/redis/6382 Executable : /usr/local/bin/redis-server Cli Executable : /usr/local/bin/redis-cli Is this ok? Then press ENTER to go on or Ctrl-C to abort. Copied /tmp/6382.conf => /etc/init.d/redis_6382 Installing service... Successfully added to chkconfig! Successfully added to runlevels 345! Starting Redis server... Installation successful!

停止redis_6382实例，避免拷贝过来的rdb文件被进程覆盖

service redis_6382 stop

[root@c7-8 utils]# mv /var/lib/redis/dump.rdb /var/lib/redis/6382/

修改配置文件，启动redis_6382实例：

vim /etc/redis/6382.conf requirepass Abc@123 :wq service redis_6382 start [root@c7-8 6382]# redis-cli -p 6382 127.0.0.1:6382> auth Abc@123 OK 127.0.0.1:6382> 127.0.0.1:6382> 127.0.0.1:6382> keys * 1) "GPS:{862370051630949}:20210527-1" 2) "GPS:{862370051630949}:20210531-1" 3) "HzAccessToken:0bc7db7327184fdda009f411fa2457b7" ... ...

该方法会存在差异数据，差异数据的多少取决于业务繁忙程度和执行BGSAVE命令后到启动新实例并加载完成之间的时间差。

时间越长，业务越繁忙，差异数据越大。

方案三：redis-dump进行数据迁移

redis-dump &redis-load工具目前仍为BETA版本。是一个将 Redis 数据导入/导出为 json 格式数据的小工具。

其数据导出/导入通过redis-dump、redis-load命令来实现，其中，redis-dump命令将redis数据输出到json格式的文件中；

redis-load命令则是将json格式数据导入到Redis数据库中。

gitee地址。

redis-dump的目的之一是能够将数据还原到已知状态。当从redis-dump文件还原数据到redis实例中时，expire 将会被重置为使用redis-dump命令创建转储文件时的值。这也是该命令与BGSAVE 生成本地RDB文件或者AOF文件进行还原数据的方案的最大不同点。

redis-dump工具支持按db进行转储，相比BGSAVE和主从模式更灵活。

此处不再进行测试，感兴趣的可以参考redis-dump作者的手册进行测试。

,