redis在大数据量下的压测表现
redis version:2.0.4
Java client for redis: http://code.google.com/p/jredis/
随机产生一个10个字节的字符串key,value与key相同,写入redis,当数据量达到1900w时,出现了如下的一些状况:刚开始测试程序全部写入db0,后来修改了测试程序,均匀写入其它数据库
master上面的日志信息:
[457] 25 Mar 11:35:58 - DB 0: 18308251 keys (0 volatile) in 50331648 slots HT.
[457] 25 Mar 11:35:58 - DB 1: 61384 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 2: 61203 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 3: 61195 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 4: 61227 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 5: 61208 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 6: 61207 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 7: 61218 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 8: 61189 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 9: 61223 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 10: 61179 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 11: 61361 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 12: 61227 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 13: 61231 keys (0 volatile) in 65536 slots HT.
[457] 25 Mar 11:35:58 - DB 14: 30792 keys (0 volatile) in 32768 slots HT.
[457] 25 Mar 11:35:58 - DB 15: 30590 keys (0 volatile) in 32768 slots HT.
slave开启来后,在slave的redis.log日志记录里看到如下的报错信息:
[15029] 25 Mar 11:33:20 * Connecting to MASTER...
[15029] 25 Mar 11:33:20 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?
[15029] 25 Mar 11:33:21 * Connecting to MASTER...
[15029] 25 Mar 11:33:21 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?
[15029] 25 Mar 11:33:22 * Connecting to MASTER...
[15029] 25 Mar 11:33:22 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?
[15029] 25 Mar 11:33:23 * Connecting to MASTER...
[15029] 25 Mar 11:33:23 # Bad protocol from MASTER, the first byte is not \'$\', are you sure the host and port are right?
这个报错信息不太友好,host与port肯定是对的
查看slave的info信息:slave与master一直同步失败,同步到547w数据的时候就挂掉了
redis> info
redis_version:2.0.4
redis_git_sha1:00000000
redis_git_dirty:0
arch_bits:64
multiplexing_api:epoll
process_id:15029
uptime_in_seconds:933
uptime_in_days:0
connected_clients:1
connected_slaves:0
blocked_clients:0
used_memory:1006633016
used_memory_human:960.00M
changes_since_last_save:0
bgsave_in_progress:0
last_save_time:1301023746
bgrewriteaof_in_progress:0
total_connections_received:1
total_commands_processed:1
expired_keys:0
hash_max_zipmap_entries:64
hash_max_zipmap_value:512
pubsub_channels:0
pubsub_patterns:0
vm_enabled:0
role:slave
master_host:#.#.#.#
master_port:6379
master_link_status:down
master_last_io_seconds_ago:-1
db0:keys=5479113,expires=0
db1:keys=3,expires=0
db3:keys=2,expires=0
db4:keys=1,expires=0
然后查看redis master上面的日志信息:
[457] 25 Mar 11:35:59 - Accepted #.#.#.#:38193
[457] 25 Mar 11:35:59 * Slave ask for synchronization
[457] 25 Mar 11:35:59 * Starting BGSAVE for SYNC
[457] 25 Mar 11:35:59 # Can\'t save in background: fork: Cannot allocate memory
[457] 25 Mar 11:35:59 * Replication failed, can\'t BGSAVE
[457] 25 Mar 11:35:59 - Client closed connection
再查看redis master使用内存的情况,使用3.3G内存
redis> info
redis_version:2.0.4
redis_git_sha1:00000000
redis_git_dirty:0
arch_bits:64
multiplexing_api:epoll
process_id:457
uptime_in_seconds:261600
uptime_in_days:3
connected_clients:2
connected_slaves:0
blocked_clients:0
used_memory:3539375688
used_memory_human:3.30G
changes_since_last_save:19470395
bgsave_in_progress:0
last_save_time:1300764509
bgrewriteaof_in_progress:0
total_connections_received:7130
total_commands_processed:22435670
expired_keys:0
hash_max_zipmap_entries:64
hash_max_zipmap_value:512
pubsub_channels:0
pubsub_patterns:0
vm_enabled:0
role:master
再看一下OS:原来是内存用光了
$free -m
total used free shared buffers cached
Mem: 3951 3900 50 0 0 8
在有内存的情况下,每秒可以达到5w次tps get key[10bytes] value[10bytes],但19148251万object*(10bytes key length+10bytes value length)/1024/1024=365M,而redis-master用了3.3G的内存,管理结构成本相当大。
建议继续学习:
- redis源代码分析 - persistence (阅读:31141)
- Redis消息队列的若干实现方式 (阅读:10704)
- 基于Redis构建系统的经验和教训 (阅读:9284)
- 浅谈redis数据库的键值设计 (阅读:8288)
- redis运维的一些知识点 (阅读:7418)
- Redis和Memcached的区别 (阅读:6776)
- Redis作者谈Redis应用场景 (阅读:6565)
- redis 运维实际经验纪录之一 (阅读:6472)
- 记Redis那坑人的HGETALL (阅读:6318)
- Redis内存存储结构分析 (阅读:6179)
扫一扫订阅我的微信号:IT技术博客大学习
- 作者:Data & Architecture DBA 来源: Data & Architecture DBA
- 标签: redis 压测
- 发布时间:2011-03-27 23:59:09
- [54] IOS安全–浅谈关于IOS加固的几种方法
- [52] android 开发入门
- [52] 如何拿下简短的域名
- [51] 图书馆的世界纪录
- [49] Go Reflect 性能
- [49] Oracle MTS模式下 进程地址与会话信
- [47] 【社会化设计】自我(self)部分――欢迎区
- [46] 读书笔记-壹百度:百度十年千倍的29条法则
- [36] 程序员技术练级攻略
- [29] 视觉调整-设计师 vs. 逻辑