5 Star 9 Fork 5

liyunfeng / redis-aware

forked from balloonwj / redis-6.0.3 
Create your Gitee Account
Explore and code with more than 6 million developers,Free private repositories !:)
Sign up
Clone or download
Cancel
Notice: Creating folder will generate an empty file .keep, because not support in Git
Loading...
README.md

redis-aware

1 . Under the LFU policy, the access heat of the key is accumulated. If it exceeds a certain threshold, it is pushed to the corresponding client through publish / subscribe, and the access heat is reduced

在LFU策略下, 累计key的访问热度。 如果超过某阈值,就通过发布/订阅 推送到对应的client,并降低访问热度

2 . The delpush command is added to delete the key and push it to the corresponding client in an asynchronous manner. In this way, we can ensure the consistency of redis and local cache.

增加了delpush 命令,用异步的方式,删除key,并且推送到对应的client。这样,我们就能保证redis和本地缓存的一致性。

change point

  1. int lfu_hot_threshold; 如果大于此阈值,将会发布到对应client default value = 50
factor 100 hits 1000 hits 100K hits 1M hits 10M hits
0 104 255 255 255 255
1 18 49 255 255 255
10 10 18 142 255 255
100 8 11 49 143 255

2 . 新的事件通知,包含HotKey和delKey,发布到client

/**
 * NOTIFY_HOT (1<<0) 热key
 * NOTIFY_DEL (1<<1) 删除
 * APP_KEY_SPIT "-"  连接符,如goods-skuId:123
 */
void notifySpecialKeyEvent(robj *key, int type) {
    sds chan;
    robj *chanobj, *eventobj;
    int vlen;
    sds *v = sdssplitlen(key->ptr,sdslen(key->ptr), APP_KEY_SPIT,1, &vlen);
    chan = type == NOTIFY_HOT ? sdsnewlen("_keyhot:",8) : sdsnewlen("_keydel:",8);
    chan = sdscatsds(chan, v[0]);
    chanobj = createObject(OBJ_STRING, chan);
    eventobj = createStringObject(v[1],strlen(v[1]));
    pubsubPublishMessage(chanobj, eventobj);
    decrRefCount(chanobj);
    decrRefCount(eventobj);
}

3 . 提高衰减速率,减少对突发热key的灵敏度

/**
 * 提高衰减速率,衰减到初始值5的时候,再恢复速率,防止内存不足等情况下导致的直接淘汰
 * 注:实际场景下 要监控redis内存 一般不会触发LFU淘汰
 */
unsigned long LFUDecrAndReturn(robj *o) {
    unsigned long ldt = o->lru >> 8;
    unsigned long counter = o->lru & 255;
    unsigned long num_periods = server.lfu_decay_time ?
                  LFUTimeElapsed(ldt) / server.lfu_decay_time : 0;
    if(num_periods){
        if(counter == LFU_INIT_VAL)
            return (num_periods> counter) ? LFU_INIT_VAL : counter - num_periods;
        return (num_periods * 30  > counter) ? LFU_INIT_VAL : counter - num_periods * 30;
    }
    return counter;
}

4 . 增加delpush指令 用于异步删除且发布到client,保证缓存一致性

  { 
     "delpush",delpushCommand,-2,
     "write fast @keyspace",
     0,NULL,1,-1,1,0,0,0
  }


/**
 * 指令执行
 */
void delpushGenericCommand(client *c) {
    int numdel = 0, j;

    for (j = 1; j < c->argc; j++) {
        expireIfNeeded(c->db,c->argv[j]);
        int deleted = dbAsyncDelete(c->db,c->argv[j]) ;
        if (deleted) {
            signalModifiedKey(c,c->db,c->argv[j]);
            notifySpecialKeyEvent(c->argv[j], NOTIFY_DEL);
            server.dirty++;
            numdel++;
        }
    }
    addReplyLongLong(c,numdel);
}

补充

本人gitee地址https://gitee.com/TangBoHo, 对订单 秒杀 QA redis略有学习,欢迎交流 微信号 18210601309

Comments ( 0 )

Sign in for post a comment

About

redis 6.0.3 版本代码。 1. 增加了LFU模式下 热点key 通过 pub/sub 推送到Local cache 2. 增加了delpush指令 使用异步方式删除key 并推送到client 保证redis和local cache的一致 spread retract
C
BSD-3-Clause
Cancel

Releases

No release

Contributors

All

Activities

load more
can not load any more
C
1
https://gitee.com/TangBoHo/redis-6.0.3.git
git@gitee.com:TangBoHo/redis-6.0.3.git
TangBoHo
redis-6.0.3
redis-aware
master

Search