技术头条 - 一个快速在微博传播文章的方式     搜索本站
您现在的位置首页 --> 系统架构 --> redis在大数据量下的压测表现

redis在大数据量下的压测表现

浏览:7391次  出处信息

    redis version:2.0.4 

    Java client for redis: http://code.google.com/p/jredis/

    随机产生一个10个字节的字符串key,value与key相同,写入redis,当数据量达到1900w时,出现了如下的一些状况:刚开始测试程序全部写入db0,后来修改了测试程序,均匀写入其它数据库

    master上面的日志信息:

    [457] 25 Mar 11:35:58 - DB 0: 18308251 keys (0 volatile) in 50331648 slots HT.

    [457] 25 Mar 11:35:58 - DB 1: 61384 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 2: 61203 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 3: 61195 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 4: 61227 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 5: 61208 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 6: 61207 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 7: 61218 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 8: 61189 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 9: 61223 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 10: 61179 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 11: 61361 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 12: 61227 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 13: 61231 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 14: 30792 keys (0 volatile) in 32768 slots HT.

    [457] 25 Mar 11:35:58 - DB 15: 30590 keys (0 volatile) in 32768 slots HT.

    slave开启来后,在slave的redis.log日志记录里看到如下的报错信息:

    [15029] 25 Mar 11:33:20 * Connecting to MASTER...

    [15029] 25 Mar 11:33:20 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?

    [15029] 25 Mar 11:33:21 * Connecting to MASTER...

    [15029] 25 Mar 11:33:21 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?

    [15029] 25 Mar 11:33:22 * Connecting to MASTER...

    [15029] 25 Mar 11:33:22 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?

    [15029] 25 Mar 11:33:23 * Connecting to MASTER...

    [15029] 25 Mar 11:33:23 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?

    这个报错信息不太友好,host与port肯定是对的

    查看slave的info信息:slave与master一直同步失败,同步到547w数据的时候就挂掉了

    redis> info

    redis_version:2.0.4

    redis_git_sha1:00000000

    redis_git_dirty:0

    arch_bits:64

    multiplexing_api:epoll

    process_id:15029

    uptime_in_seconds:933

    uptime_in_days:0

    connected_clients:1

    connected_slaves:0

    blocked_clients:0

    used_memory:1006633016

    used_memory_human:960.00M

    changes_since_last_save:0

    bgsave_in_progress:0

    last_save_time:1301023746

    bgrewriteaof_in_progress:0

    total_connections_received:1

    total_commands_processed:1

    expired_keys:0

    hash_max_zipmap_entries:64

    hash_max_zipmap_value:512

    pubsub_channels:0

    pubsub_patterns:0

    vm_enabled:0

    role:slave

    master_host:#.#.#.#

    master_port:6379

    master_link_status:down

    master_last_io_seconds_ago:-1

    db0:keys=5479113,expires=0

    db1:keys=3,expires=0

    db3:keys=2,expires=0

    db4:keys=1,expires=0

    然后查看redis master上面的日志信息:

    [457] 25 Mar 11:35:59 - Accepted #.#.#.#:38193

    [457] 25 Mar 11:35:59 * Slave ask for synchronization

    [457] 25 Mar 11:35:59 * Starting BGSAVE for SYNC

    [457] 25 Mar 11:35:59 # Can\'t save in background: fork: Cannot allocate memory

    [457] 25 Mar 11:35:59 * Replication failed, can\'t BGSAVE

    [457] 25 Mar 11:35:59 - Client closed connection

    再查看redis master使用内存的情况,使用3.3G内存

    redis> info

    redis_version:2.0.4

    redis_git_sha1:00000000

    redis_git_dirty:0

    arch_bits:64

    multiplexing_api:epoll

    process_id:457

    uptime_in_seconds:261600

    uptime_in_days:3

    connected_clients:2

    connected_slaves:0

    blocked_clients:0

    used_memory:3539375688

    used_memory_human:3.30G

    changes_since_last_save:19470395

    bgsave_in_progress:0

    last_save_time:1300764509

    bgrewriteaof_in_progress:0

    total_connections_received:7130

    total_commands_processed:22435670

    expired_keys:0

    hash_max_zipmap_entries:64

    hash_max_zipmap_value:512

    pubsub_channels:0

    pubsub_patterns:0

    vm_enabled:0

    role:master

    再看一下OS:原来是内存用光了

    $free -m

     total       used       free     shared    buffers     cached

    Mem:          3951       3900         50          0          0          8

    在有内存的情况下,每秒可以达到5w次tps get key[10bytes] value[10bytes],但19148251万object*(10bytes key length+10bytes value length)/1024/1024=365M,而redis-master用了3.3G的内存,管理结构成本相当大。

建议继续学习:

  1. redis源代码分析 - persistence    (阅读:31152)
  2. Redis消息队列的若干实现方式    (阅读:10714)
  3. 基于Redis构建系统的经验和教训    (阅读:9299)
  4. 浅谈redis数据库的键值设计    (阅读:8296)
  5. redis运维的一些知识点    (阅读:7432)
  6. Redis和Memcached的区别    (阅读:6789)
  7. Redis作者谈Redis应用场景    (阅读:6572)
  8. redis 运维实际经验纪录之一    (阅读:6476)
  9. 记Redis那坑人的HGETALL    (阅读:6323)
  10. Redis内存存储结构分析    (阅读:6182)
QQ技术交流群:445447336,欢迎加入!
扫一扫订阅我的微信号:IT技术博客大学习
© 2009 - 2024 by blogread.cn 微博:@IT技术博客大学习

京ICP备15002552号-1