IT技术博客大学习 共学习 共进步
全部 移动开发 后端 数据库 AI 算法 安全 DevOps 前端 设计 开发者

redis在大数据量下的压测表现

Data & Architecture DBA 2011-03-27 23:59:09 累计浏览 8,294 次
本机暂存

    redis version:2.0.4 

    Java client for redis: http://code.google.com/p/jredis/

    随机产生一个10个字节的字符串key,value与key相同,写入redis,当数据量达到1900w时,出现了如下的一些状况:刚开始测试程序全部写入db0,后来修改了测试程序,均匀写入其它数据库

    master上面的日志信息:

    [457] 25 Mar 11:35:58 - DB 0: 18308251 keys (0 volatile) in 50331648 slots HT.

    [457] 25 Mar 11:35:58 - DB 1: 61384 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 2: 61203 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 3: 61195 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 4: 61227 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 5: 61208 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 6: 61207 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 7: 61218 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 8: 61189 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 9: 61223 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 10: 61179 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 11: 61361 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 12: 61227 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 13: 61231 keys (0 volatile) in 65536 slots HT.

    [457] 25 Mar 11:35:58 - DB 14: 30792 keys (0 volatile) in 32768 slots HT.

    [457] 25 Mar 11:35:58 - DB 15: 30590 keys (0 volatile) in 32768 slots HT.

    slave开启来后,在slave的redis.log日志记录里看到如下的报错信息:

    [15029] 25 Mar 11:33:20 * Connecting to MASTER...

    [15029] 25 Mar 11:33:20 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?

    [15029] 25 Mar 11:33:21 * Connecting to MASTER...

    [15029] 25 Mar 11:33:21 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?

    [15029] 25 Mar 11:33:22 * Connecting to MASTER...

    [15029] 25 Mar 11:33:22 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?

    [15029] 25 Mar 11:33:23 * Connecting to MASTER...

    [15029] 25 Mar 11:33:23 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?

    这个报错信息不太友好,host与port肯定是对的

    查看slave的info信息:slave与master一直同步失败,同步到547w数据的时候就挂掉了

    redis> info

    redis_version:2.0.4

    redis_git_sha1:00000000

    redis_git_dirty:0

    arch_bits:64

    multiplexing_api:epoll

    process_id:15029

    uptime_in_seconds:933

    uptime_in_days:0

    connected_clients:1

    connected_slaves:0

    blocked_clients:0

    used_memory:1006633016

    used_memory_human:960.00M

    changes_since_last_save:0

    bgsave_in_progress:0

    last_save_time:1301023746

    bgrewriteaof_in_progress:0

    total_connections_received:1

    total_commands_processed:1

    expired_keys:0

    hash_max_zipmap_entries:64

    hash_max_zipmap_value:512

    pubsub_channels:0

    pubsub_patterns:0

    vm_enabled:0

    role:slave

    master_host:#.#.#.#

    master_port:6379

    master_link_status:down

    master_last_io_seconds_ago:-1

    db0:keys=5479113,expires=0

    db1:keys=3,expires=0

    db3:keys=2,expires=0

    db4:keys=1,expires=0

    然后查看redis master上面的日志信息:

    [457] 25 Mar 11:35:59 - Accepted #.#.#.#:38193

    [457] 25 Mar 11:35:59 * Slave ask for synchronization

    [457] 25 Mar 11:35:59 * Starting BGSAVE for SYNC

    [457] 25 Mar 11:35:59 # Can\'t save in background: fork: Cannot allocate memory

    [457] 25 Mar 11:35:59 * Replication failed, can\'t BGSAVE

    [457] 25 Mar 11:35:59 - Client closed connection

    再查看redis master使用内存的情况,使用3.3G内存

    redis> info

    redis_version:2.0.4

    redis_git_sha1:00000000

    redis_git_dirty:0

    arch_bits:64

    multiplexing_api:epoll

    process_id:457

    uptime_in_seconds:261600

    uptime_in_days:3

    connected_clients:2

    connected_slaves:0

    blocked_clients:0

    used_memory:3539375688

    used_memory_human:3.30G

    changes_since_last_save:19470395

    bgsave_in_progress:0

    last_save_time:1300764509

    bgrewriteaof_in_progress:0

    total_connections_received:7130

    total_commands_processed:22435670

    expired_keys:0

    hash_max_zipmap_entries:64

    hash_max_zipmap_value:512

    pubsub_channels:0

    pubsub_patterns:0

    vm_enabled:0

    role:master

    再看一下OS:原来是内存用光了

    $free -m

     total       used       free     shared    buffers     cached

    Mem:          3951       3900         50          0          0          8

    在有内存的情况下,每秒可以达到5w次tps get key[10bytes] value[10bytes],但19148251万object*(10bytes key length+10bytes value length)/1024/1024=365M,而redis-master用了3.3G的内存,管理结构成本相当大。

同分类推荐文章

  1. 使用deepseek进行Oracle恢复,引起重大故障 (2026-06-22 10:56:00)
  2. 接手一个只差临门一脚的数据库恢复 (2026-06-18 00:13:09)
  3. 我做了一个 AI 版的 StarRocks 升级风险扫描工具,直接帮我定位到一个风险 (2026-06-15 01:00:00)

查看更多 数据库 文章 →

建议继续学习

  1. redis源代码分析 - persistence (累计阅读 32,229)
  2. Linux 性能监控、测试、优化工具 (累计阅读 13,013)
  3. hbase介绍 (累计阅读 12,367)
  4. Redis消息队列的若干实现方式 (累计阅读 12,088)
  5. 海量数据面试题举例 (累计阅读 11,114)
  6. 基于Redis构建系统的经验和教训 (累计阅读 10,522)
  7. 浅谈redis数据库的键值设计 (累计阅读 9,354)
  8. 【2014年版】异地购房提取北京公积金 (累计阅读 9,148)
  9. 长连接(KeepAlive)在 http 连接中的性能影响 (累计阅读 8,783)
  10. redis运维的一些知识点 (累计阅读 8,685)