6 posts tagged with "redis"

背景​

使用k8s 搭建redis集群

root@redis-app-0:/data# redis-cli
127.0.0.1:6379> info cluster
# Cluster
cluster_enabled:1
127.0.0.1:6379> cluster meet 10.42.0.35 6379
OK
127.0.0.1:6379> cluster meet 10.42.0.36 6379
OK
127.0.0.1:6379> cluster meet 10.42.0.37 6379
OK
127.0.0.1:6379> cluster meet 10.42.0.38 6379
OK
127.0.0.1:6379> cluster meet 10.42.0.39 6379
OK

查看节点

27.0.0.1:6379> cluster nodes
f8d5dd6aef17c622f541ade32a95430421606f6c 10.42.0.39:6379 master - 0 1673791234512 0 connected
76af8c3c32cf535a3733ce75db2c3c6719c644fc 10.42.0.38:6379 master - 0 1673791234512 4 connected
7f803ec0f21e4382bb773285fd40286069b26075 10.42.0.36:6379 master - 0 1673791235012 2 connected
c48e86d680b74df9c70cb7369201fb2cbd8650be 10.42.0.34:6379 myself,master - 0 0 5 connected
3ab965513f345444689cdeb7358c51263772f454 10.42.0.35:6379 master - 0 1673791233512 1 connected
8e256e1614cf9f330404693b6f18785da5794fbc 10.42.0.37:6379 master - 0 1673791234011 3 connected
127.0.0.1:6379>

分配槽位：

10.42.0.39:6379> cluster nodes
f8d5dd6aef17c622f541ade32a95430421606f6c 10.42.0.39:6379 myself,slave 76af8c3c32cf535a3733ce75db2c3c6719c644fc 0 0 0 connected
7f803ec0f21e4382bb773285fd40286069b26075 10.42.0.36:6379 master - 0 1673791666954 2 connected
8e256e1614cf9f330404693b6f18785da5794fbc 10.42.0.37:6379 slave 7f803ec0f21e4382bb773285fd40286069b26075 0 1673791668456 3 connected
76af8c3c32cf535a3733ce75db2c3c6719c644fc 10.42.0.38:6379 master - 0 1673791667955 4 connected
3ab965513f345444689cdeb7358c51263772f454 10.42.0.35:6379 slave c48e86d680b74df9c70cb7369201fb2cbd8650be 0 1673791668955 5 connected
c48e86d680b74df9c70cb7369201fb2cbd8650be 10.42.0.34:6379 master - 0 1673791666954 5 connected
10.42.0.39:6379> redis-cli -h  10.42.0.34  -p 6379 cluster addslots {0..5461}
(error) ERR unknown command 'redis-cli'
10.42.0.39:6379> exit
root@redis-app-0:/data# redis-cli -h  10.42.0.34  -p 6379 cluster addslots {0..5461}
OK
root@redis-app-0:/data# redis-cli -h  10.42.0.36  -p 6379 cluster addslots  {5462..10922}
OK
root@redis-app-0:/data# redis-cli -h  10.42.0.38  -p 6379 cluster addslots   {10923..16383}

搭建结果：

127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:5
cluster_my_epoch:5
cluster_stats_messages_sent:3134
cluster_stats_messages_received:3134

相关阅读​

• https://blog.csdn.net/Hlroliu/article/details/105859200
• 集群配置
• 进入pod
• https://blog.csdn.net/Hlroliu/article/details/107067809
• 私有镜像 k8s

相关阅读​

• https://rtmsoft.me/Lettuce%E8%B0%83%E7%94%A8Lua%E8%84%9A%E6%9C%AC.html
• https://github.com/lettuce-io/lettuce-core/blob/main/src/main/templates/io/lettuce/core/api/RedisScriptingCommands.java

拉代码

git clone https://github.com/redis/redis.git
cd redis/
## 带上调试信息
make CFLAGS="-g -O0" 
## 创建一个目录
mkdir rediscluster 
mkdir 7000 7001 7002 7003 7004 7005

创建几个节点胡配置

tree
.
├── 7000
├── 7001
├── 7002
├── 7003
├── 7004
└── 7005

然后像这样启动6个：

src/redis-server rediscluster/7001/redis.conf

主动下线是一个命令

CLUSTER FAILOVER

相关阅读​

• https://redis.io/topics/cluster-tutorial
• https://redis.io/commands/cluster-failover

1 类型 string
list set zset hash

2 对应的命令

set 命令 edis/src/t_string.c
string 的过期不是实时的

list lpush lpop / 有ziplist 和双向链表组成

hash 有两个hashtable ， 一个是内容一个谁拿来扩容

set 谁由intset 和 hashtable组成

zset ziplist+ skiplist

3 aof/rdb

aof 谁append only file 都是可读的文本 rdb谁整个盘快照

4 redis 淘汰

noeviction:如果缓存数据超过了maxmemory限定值，并且客户端正在执行的命令(大部分的写入指令，但DEL和几个指令例外)会导致内存分配，则向客户端返回错误响应 allkeys-lru: 对所有的键都采取LRU淘汰 volatile-lru: 仅对设置了过期时间的键采取LRU淘汰 allkeys-random: 随机回收所有的键 volatile-random: 随机回收设置过期时间的键 volatile-ttl: 仅淘汰设置了过期时间的键---淘汰生存时间TTL(Time To Live)更小的键

5 HA​

Redis Cluster master-replica model

In order to remain available when a subset of master nodes are failing or are not able to communicate with the majority of nodes, Redis Cluster uses a master-replica model where every hash slot has from 1 (the master itself) to N replicas (N-1 additional replica nodes).

In our example cluster with nodes A, B, C, if node B fails the cluster is not able to continue, since we no longer have a way to serve hash slots in the range 5501-11000.

However when the cluster is created (or at a later time) we add a replica node to every master, so that the final cluster is composed of A, B, C that are master nodes, and A1, B1, C1 that are replica nodes. This way, the system is able to continue if node B fails.

Node B1 replicates B, and B fails, the cluster will promote node B1 as the new master and will continue to operate correctly.

However, note that if nodes B and B1 fail at the same time, Redis Cluster is not able to continue to operate.

failover 错误转移​

当master 失去连接后，slave会向master 发起一个paxos 选票

分片​

分片会路由到不同胡slot，运算方式要crc16（key）% 16384

每个分片会有特定slot

一致性​

不是强一致性，

Redis Cluster is not able to guarantee strong consistency. In practical terms this means that under certain conditions it is possible that Redis Cluster will lose writes that were acknowledged by the system to the client.

The first reason why Redis Cluster can lose writes is because it uses asynchronous replication. This means that during writes the following happens:

Your client writes to the master B. The master B replies OK to your client. The master B propagates the write to its replicas B1, B2 and B3.

因为是异步的

缓存穿透 雪崩​

穿透是指通过redis 雪崩就是指同时失效

限流​

https://segmentfault.com/a/1190000040570911

相关阅读

• https://www.cnblogs.com/aquester/p/10229415.html#:~:text=Redis%20%20%E9%9B%86%E7%BE%A4%E7%9A%84%E4%B8%BB%E4%BB%8E%E5%88%87%E6%8D%A2%E9%87%87%E5%8F%96%E9%80%89%E4%B8%BE%E6%9C%BA%E5%88%B6%EF%BC%8C%E8%A6%81%E6%B1%82%E5%B0%91%E6%95%B0%E6%9C%8D%E4%BB%8E%E5%A4%9A%E6%95%B0%EF%BC%8C%E8%80%8C%E5%8F%82%E4%B8%8E%E9%80%89%E4%B8%BE%E7%9A%84%E5%8F%AA%E8%83%BD%E4%B8%BAmaster%EF%BC%8C%E6%89%80%E4%BB%A5%E5%8F%AA%E6%9C%89%E5%A4%9A%E6%95%B0master%E5%AD%98%E6%B4%BB%E5%8A%A8%E6%97%B6%E6%89%8D%E8%83%BD%E8%BF%9B%E8%A1%8C%EF%BC%8C%E9%80%89%E4%B8%BE%E7%94%B1slave%E5%8F%91%E8%B5%B7%E3%80%82%20Redis%20%E7%94%A8%E4%BA%86%E5%92%8CRaft%E7%AE%97%E6%B3%95term%EF%BC%88%E4%BB%BB%E6%9C%9F%EF%BC%89%E7%B1%BB%E4%BC%BC%E7%9A%84%E7%9A%84%E6%A6%82%E5%BF%B5%EF%BC%8C%E5%9C%A8,Redis%20%E4%B8%AD%E5%8F%AB%E4%BD%9Cepoch%EF%BC%88%E7%BA%AA%E5%85%83%EF%BC%89%EF%BC%8C%20epoch%20%E6%98%AF%E4%B8%80%E4%B8%AA%E6%97%A0%E7%AC%A6%E5%8F%B7%E7%9A%8464%E6%95%B4%E6%95%B0%EF%BC%8C%E4%B8%80%E4%B8%AA%E8%8A%82%E7%82%B9%E7%9A%84%20epoch%E4%BB%8E0%E5%BC%80%E5%A7%8B%E3%80%82

跳表 skiplist​

Binary trees can be used for representing abstract data types such as dictionaries and ordered lists. They work well when the elements are inserted in a random order. 二叉树可以使用来表达一个词典或者一个有序列表.当二叉树的元素插入是随机顺序的时候,他可以工作很好

Some sequences of operations, such as inserting the elements in order, produce degenerate data structures that perform very poo.rly. 但是一些特定的顺序,举个例子,如果插入的数据本身已经是有序的,那么效果则会很差 If it were possible to randomly permute the list of items to be inserted, trees would work well with high probability for any input sequence. 如果可以产生随机的数据去插入,那么树会对输入的性能变得非常好 In most cases queries must be answered online, so randomly permuting the input is impractical. Balanced tree algorithms rearrange the tree as operations are performed to maintain certain balance conditions and assure good performance. 在大部分的情况必须实时查询,所以随机交换输入是不可能的,平衡树需要重排来保证某些平衡的条件和确保比较好的性能 Skip lists are a probabilistic alternative to balanced trees. 跳表相对于平衡树来说是另外一个选择 Skip lists are balanced by consulting a random number generator. 跳表的平衡是通过获取一个随机数 Although skip lists have bad worstcase performance, no input sequence consistently produces the worst-case performance (much like quicksort when the pivot element is chosen randomly). 虽然跳表也有bad case(举个例子每次随机的高度都一样) , 但是没有一个输入序列可以把这个bad case稳定的生成(因为是随机产生所以不会一直产生最坏情况) It is very unlikely a skip list data structure will be significantly unbalanced (e.g., for a dictionary of more than 250 elements, the chance that a search will take more than three-times the expeci.ed time is less than one in a million). Skip lists have balance properties similar to that of search trees built by random insertions, yet do not require insertions to be random. 这不太可能跳表变得非常显著地不平衡,跳表的平衡性质和一个随机插入的搜索树相类似,但是不需要随机去插入 It is easier to balance a data structure probabilistitally than to explicitly maintain the balance. 通过概率去平衡一个暑假结构比显式保持它的平衡要简单 For many applications, skip lists are a more natural representation than trees, and they lead to simpler algorithms. 对于很多应用,相对于树来说跳表会更加自然. The simplicity of skip list algorithms makes them easier to implement and provides significant constant factor speed improvements over balanced tree and self-adjusting tree algorithms. 跳表的简易型可以实现和提供一个显著的常量因子去比平衡树和自适应树更好 Skip lists are also very space efficient. 跳表的空间利用率很高 They can easily be configured to require an average of 1% pointers per element (or even less) and do not require balance or priority information to be stored with each node. 他们可以需要平均百分之1的指针和不需要平衡或者权重信息取存储每个节点

SKIP LISTS 跳表 We might need to examine every node of the list when searching a linked list (Figure la). 当我们使用链表这个结构来存储数据,如果我们要搜索某个节点,我们需要顺序遍历每个节点. If the list is stored in sorted order and every other node of the list also has a pointer to the node two ahead of it in the list (Figure lb), we have to examine no more than [n/21 +1 nodes (where n is the length of the list). 如果这个list是 Also giving every fourth node a pointer four ahead (Figure lc) requires that no more than rn/41 + 2 nodes be examined. If every (27th node has a pointer 2’ nodes ahead (Figure Id), the number of nodes that must be examined can be reduced to rlog,nl while only doubling the number of pointers. This data structure could be used for fast searching, but insertion and deletion would be impractical. A node that has k forward pointers is called a level k node. If every (2’)th node has a pointer 2’ nodes ahead, then levels of nodes are distributed in a simple pattern: 50 percent are level 1, 25 percent are level 2, 12.5 percent are level 3 and so on. What would happen if the levels of nodes were chosen randomly, but in the same proportions (e.g., as in Figure le)? A node’s ith forward pointer, instead of pointing 2’-’ nodes ahead, points to the next node of level i or higher. Insertions or deletions would require only local modifications; the level of a node, chosen randomly when the node is inserted, need never change. Some arrangements of levels would give poor execution times, but we will see that such arrangements are rare. Because these data structures are linked lists with extra pointers that s

相关阅读​

• https://homepage.divms.uiowa.edu/~ghosh/skip.pdf