MySQL服务启动脚本故障排查
上周cenalulu给大家十分详细地剖析了一下MySQL服务启动脚本的工作原理,我相信就在这个3个单词service mysql start后面,我们都会不止一次的面对红色的[FAILED],由于各种因素,具体的操作环境间的差别,各自也都会面对着不同的启动问题,下面我就来说说上周五,我遇到的关于MySQL服务启动的问题。
SHELL>service mysql start
Starting MySQL.Manager of pid-file quit without updating fi [FAILED]
遇到[FAILED]之前,我都做了什么呢?
大家一定还记得之前,我给大家介绍的一篇关于Percona XtraDB的文章。下面要讲的这个启动问题,就是接着那个实验之后。源码编译安装完成,启动正常,关于XtraDB的实验结构也都符合feature中的介绍。在这之后,我又进行了xtrabackup工具的备份操作,再往后,出事的当天上午,我通过mysqladmin shutdown正常关闭mysqld(我查看过err日志,normal shutdown),你可能在想,怎么没用service mysql stop呢,呵呵,其实,我是忘了mysqld已经做成了系统自启服务,就习惯的用了mysqldadmin去关闭,难道是这个微小的差别导致的吗?
问题出现了,开始排错吧。经过上次的troubleshooting的训练,我想所有看过文章和没有看过文章的聪明的朋友,都会本能的去查看error log。看了之后,没有[Error]信息,似乎看起来没有让人慌张的迹象,但是,不能正常启动,它也一定会有不同以往的信息提示我们,而且是很多很多,让我给大家稍微“秀”一下,可能明眼的朋友能看出些什么。
以下是引用片段:
091218 12:34:42 [Note] /usr/local/mysql/libexec/mysqld: Normal shutdown091218 12:34:42 [Note] Event Scheduler: Purging the queue. 0 events
091218 12:34:44 [Note] /usr/local/mysql/libexec/mysqld: Shutdown complete091218 12:34:44 mysqld_safe mysqld from pid file /usr/local/mysql/var/eiken-app.pid ended
091218 12:43:43 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/var
InnoDB: The InnoDB memory heap is disabled
InnoDB: Neither mutexes nor rw_locks use GCC atomic builtins.
091218 12:43:43 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
091218 12:43:43 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files…
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer…
InnoDB: Page directory corruption: infimum not pointed to
091218 12:43:43 InnoDB: Page dump in ascii and hex (16384 bytes):
@len 16384; hex 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
―omit――
InnoDB: End of page dump
091218 12:43:43 InnoDB: Page checksum 1575996416, prior-to-4.0.14-form checksum 1371122432
InnoDB: stored checksum 0, prior-to-4.0.14-form stored checksum 0
InnoDB: Page lsn 0 0, low 4 bytes of lsn at page end 0
InnoDB: Page number (if stored to page already) 0,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a freshly allocated page
InnoDB: Page directory corruption: supremum not pointed to
091218 12:43:43 InnoDB: Page dump in ascii and hex (16384 bytes):
―-omit――
InnoDB: End of page dump
091218 12:43:43 InnoDB: Page checksum 1575996416, prior-to-4.0.14-form checksum 1371122432
InnoDB: stored checksum 0, prior-to-4.0.14-form stored checksum 0
InnoDB: Page lsn 0 0, low 4 bytes of lsn at page end 0
InnoDB: Page number (if stored to page already) 0,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a freshly allocated page
091218 12:43:43 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 133306 K
bytes of memory
Hope that’s ok; if not, decrease some variables in the equation.
thd: 0×0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong…
stack_bottom = (nil) thread_stack 0×30000
/usr/local/mysql/libexec/mysqld(my_print_stacktrace+0×22) [0x849ddb2]
/usr/local/mysql/libexec/mysqld(handle_segfault+0×37f) [0x81b57df]
[0x31e420]
…..
就是这么个情况。通过上面那些日志,看到上面有些关于innodb(事实上,现在是xtradb)数据文件(ibdata1)的问题。这时ibdata1有100多M的数据,因为有sql dump文件,所以就大胆的把ibdata1,还有ib_logfile也一并给删掉了,然后再试着通过service mysql start启动,依然[FAILED]。
在进行上面的排错之前,我依然头脑发热地像往常一样,慌张地在网上搜寻解决答案,依然未果。提到最多的解决方法就是,删除目录/var/lock/subsys下的mysql文件(应该对于由于其他问题引起的FAILED错误有效)。关于这个文件有兴趣的朋友,可以看看本文开始提到的cenalulu的那篇文章以及后面的comment。
希望,听听大家的看法。
建议继续学习:
- Nginx启动初始化过程(一) (阅读:4485)
- MySQL服务启动脚本完全解析 (阅读:2567)
- Nginx启动初始化过程(二) (阅读:2466)
- 思考mysql内核之初级系列1--- mysql的启动过程 (阅读:2381)
- 由App的启动说起 (阅读:1508)
扫一扫订阅我的微信号:IT技术博客大学习
- 作者:zhang 来源: SQL部落
- 标签: 启动
- 发布时间:2009-12-21 12:21:30
- [54] android 开发入门
- [53] IOS安全–浅谈关于IOS加固的几种方法
- [51] Oracle MTS模式下 进程地址与会话信
- [51] 图书馆的世界纪录
- [50] 如何拿下简短的域名
- [50] Go Reflect 性能
- [48] 读书笔记-壹百度:百度十年千倍的29条法则
- [47] 【社会化设计】自我(self)部分――欢迎区
- [40] 程序员技术练级攻略
- [31] 视觉调整-设计师 vs. 逻辑