接手一个只差临门一脚的数据库恢复

刚刚进行了一个数据库恢复case,故障处理起来非常简单(设置一个参数启动库即可),大概回溯下故障经过,反馈是由于客户在没有停机的情况下复制虚拟机,然后启动虚拟机之后,数据库无法正常启动报ORA-00314 ORA-00312错误

Wed Jun 17 10:24:02 2026
ALTER DATABASE   MOUNT
Successful mount of redo thread 1, with mount id 3689345602
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE   MOUNT
Wed Jun 17 10:24:06 2026
ALTER DATABASE OPEN
Beginning crash recovery of 1 threads
 parallel recovery started with 32 processes
Started redo scan
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_60458.trc:
ORA-00314: log 2 of thread 1, expected sequence# 12271 doesn't match 9443
ORA-00312: online log 2 thread 1: '/oradata/orcl/redo02.log'
Aborting crash recovery due to error 314
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_60458.trc:
ORA-00314: log 2 of thread 1, expected sequence# 12271 doesn't match 9443
ORA-00312: online log 2 thread 1: '/oradata/orcl/redo02.log'
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_60458.trc:
ORA-00314: log 2 of thread 1, expected sequence# 12271 doesn't match 9443
ORA-00312: online log 2 thread 1: '/oradata/orcl/redo02.log'
ORA-314 signalled during: ALTER DATABASE OPEN...
Wed Jun 17 10:24:07 2026
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_m000_61170.trc:
ORA-00314: log 1 of thread 1, expected sequence# 12269 doesn't match 9441
ORA-00312: online log 1 thread 1: '/oradata/orcl/redo01.log'
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_m000_61170.trc:
ORA-00314: log 2 of thread 1, expected sequence# 12271 doesn't match 9443
ORA-00312: online log 2 thread 1: '/oradata/orcl/redo02.log'
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_m000_61170.trc:
ORA-00314: log 3 of thread 1, expected sequence# 12265 doesn't match 9444
ORA-00312: online log 3 thread 1: '/oradata/orcl/redo03.log'
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_m000_61170.trc:
ORA-00314: log 4 of thread 1, expected sequence# 12266 doesn't match 9438
ORA-00312: online log 4 thread 1: '/oradata/orcl/redo04.log'
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_m000_61170.trc:
ORA-00314: log 5 of thread 1, expected sequence# 12267 doesn't match 9439
ORA-00312: online log 5 thread 1: '/oradata/orcl/redo05.log'
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_m000_61170.trc:
ORA-00314: log 6 of thread 1, expected sequence# 12268 doesn't match 9440
ORA-00312: online log 6 thread 1: '/oradata/orcl/redo06.log'

这个错误的redo sequence差距有点大,个人感觉可能不是简单的复制引起的,由于没有第一现场不好溯源,不乱做评论,姑且认为是由于虚拟机复制引起的问题
客户使用隐含参数强制打开数据库，报ORA-1555错误

Wed Jun 17 10:42:50 2026
ALTER DATABASE OPEN RESETLOGS
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 1838398816
Resetting resetlogs activation ID 3463997979 (0xce786a1b)
ORA-344 signalled during: ALTER DATABASE OPEN RESETLOGS...
Wed Jun 17 10:44:30 2026
ALTER DATABASE OPEN RESETLOGS
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 1838398816
Resetting resetlogs activation ID 3463997979 (0xce786a1b)
Wed Jun 17 10:45:13 2026
Setting recovery target incarnation to 3
Wed Jun 17 10:45:13 2026
Assigning activation ID 3689297678 (0xdbe6370e)
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: /oradata/orcl/redo01.log
Successful open of redo thread 1
Wed Jun 17 10:45:14 2026
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Jun 17 10:45:14 2026
SMON: enabling cache recovery
ORA-01555 caused by SQL statement below (SQL ID: 4krwuz0ctqxdt, SCN: 0x0000.6d93bd66):
select ctime, mtime, stime from obj$ where obj# = :1
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_18038.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 35 with name "_SYSSMU35_3782695576$" too small
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_18038.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 35 with name "_SYSSMU35_3782695576$" too small
Error 704 happened during db open, shutting down database
USER (ospid: 18038): terminating the instance due to error 704
Instance terminated by USER, pid = 18038
ORA-1092 signalled during: ALTER DATABASE OPEN RESETLOGS...
opiodr aborting process unknown ospid (18038) as a result of ORA-1092
Wed Jun 17 10:45:16 2026
ORA-1092 : opitsk aborting process

这个是一个比较经典的错误,以往的文章中总结了这类错误可能涉及的具体sql语句
在数据库open过程中常遇到ORA-01555汇总
数据库open过程遭遇ORA-1555对应sql语句补充
客户尝试多次重启之后，数据库报ORA-600 2662错误

Wed Jun 17 10:55:34 2026
Thread 1 advanced to log sequence 3 (thread open)
Thread 1 opened at log sequence 3
  Current log# 3 seq# 3 mem# 0: /oradata/orcl/redo03.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Jun 17 10:55:34 2026
SMON: enabling cache recovery
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_6707.trc  (incident=117868):
ORA-00600: internal error code, arguments: [2662], [0], [1838441753], [0], [1838486139], [12583040], 
Incident details in: /u01/oracle/diag/rdbms/orcl/orcl/incident/incdir_117868/orcl_ora_6707_i117868.trc
Wed Jun 17 10:55:45 2026
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_6707.trc:
ORA-00600: internal error code, arguments: [2662], [0], [1838441753], [0], [1838486139], [12583040], 
Errors in file /u01/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_6707.trc:
ORA-00600: internal error code, arguments: [2662], [0], [1838441753], [0], [1838486139], [12583040], 
Error 600 happened during db open, shutting down database
USER (ospid: 6707): terminating the instance due to error 600
Instance terminated by USER, pid = 6707
ORA-1092 signalled during: alter database open...

客户后续多次重建ctl，强制拉库等操作，一直在ORA-600 2662上面循环，后面终于出现了ORA-600 4193/4194错误，数据库没有正常open成功,至此客户放弃恢复尝试。
我们接手故障之后,设置undo手工管理模式,然后直接启动库成功

然后使用expdp导出数据,完成本次恢复工作