Yesterday, I demonstrated a DataGuard Switchover from a 2node RAC(ASM) database to a SingleInstance(FileSystem) Database.
These are the messages in the alert log and drc log files during the Switchover (Messages from the start of the Switchover to the subsequent restart and shutdown of all three instances.
First on node 1 which is running database instance RAC1. The drcRAC1.log and alert_RAC1.log :
Next on node2 running instance RAC2 :
Next, the original Standby (SingleInstance) node : the drcSTBY.log and alert_STBY.log
Ignore the Fatal NI and TNS errrors. I plan to address them later.
.
.
.
These are the messages in the alert log and drc log files during the Switchover (Messages from the start of the Switchover to the subsequent restart and shutdown of all three instances.
First on node 1 which is running database instance RAC1. The drcRAC1.log and alert_RAC1.log :
04/10/2017 23:30:58
SWITCHOVER TO stby
Switchover to physical standby cannot be initiated from primary database; redirecting to the switchover target stby
using connect identifier: stby
04/10/2017 23:31:00
SWITCHOVER TO stby
04/10/2017 23:31:02
Notifying Oracle Clusterware to prepare primary database for switchover
04/10/2017 23:31:19
Target standby stby did not have Active Data Guard enabled at the time of switchover.
To maintain Active Data Guard license compliance Block Change Tracking will be disabled.
04/10/2017 23:31:20
Notifying Clusterware to restart this instance for Switchover
04/10/2017 23:31:23
Shutting down instance after CTL_SWITCH
04/10/2017 23:33:20
>> Starting Data Guard Broker bootstrap <<
Broker Configuration File Locations:
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
2017-04-10 23:33:20.858 DMON: Attach state object
2017-04-10 23:33:21.407 DMON: rfafoGetLocks reinitializing dubious PMYSHUT lock value block contents: sts=0, flags=0x0, spare1=0x0, spare2=0x0, cksm=0x0, rndm=0x0
2017-04-10 23:33:21.407 DMON: Broker state reconciled, version = 0, state = 00000000
2017-04-10 23:33:21.407 DMON: Broker State Initialized
2017-04-10 23:33:21.407 Version = 1
2017-04-10 23:33:21.407 State = 00000000
2017-04-10 23:33:21.407 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 2
2017-04-10 23:33:21.422 7fffffff 0 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 0
2017-04-10 23:33:24.580 7fffffff 0 DMON: start task execution: broker initialization
2017-04-10 23:33:24.580 DMON: Boot configuration (0.0.0), loading from "+DATA/RAC/dgbroker1.dat"
2017-04-10 23:33:24.856 DMON Registering service RAC_DGB with listener(s)
2017-04-10 23:33:24.857 DMON: Executing SQL [ALTER SYSTEM REGISTER]
2017-04-10 23:33:24.902 SQL [ALTER SYSTEM REGISTER] Executed successfully
04/10/2017 23:33:25
Broker Configuration: "rac"
Protection Mode: Maximum Performance
Fast-Start Failover (FSFO): Disabled, flags=0x0, version=0
Primary Database: stby (0x02010000)
Physical standby bootstrap requires Oracle Clusterware buildup
04/10/2017 23:33:39
Notifying Oracle Clusterware to buildup
04/10/2017 23:33:41
rac version check successfully completed
Broker configuration file is current on rac, completing initialization
Creating process RSM0
04/10/2017 23:42:33
Data Guard Broker shutting down
Data Guard Broker - notifying primary of shutdown
posting shutdown message to primary database 0x02001000
RSM0 successfully terminated
2017-04-10 23:42:36.437 >> DMON Process Shutdown <<
=============================================================================================
Mon Apr 10 23:30:59 2017
SWITCHOVER VERIFY: Send VERIFY request to switchover target stby
SWITCHOVER VERIFY COMPLETE
SWITCHOVER VERIFY: Send VERIFY request to switchover target stby
SWITCHOVER VERIFY COMPLETE
ALTER DATABASE SWITCHOVER TO 'stby'
Mon Apr 10 23:31:04 2017
Starting switchover [Process ID: 17791]
Mon Apr 10 23:31:04 2017
ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY [Process Id: 17791] (RAC1)
Waiting for target standby to receive all redo
Mon Apr 10 23:31:04 2017
Waiting for all non-current ORLs to be archived...
Mon Apr 10 23:31:04 2017
All non-current ORLs have been archived.
Mon Apr 10 23:31:04 2017
Waiting for all FAL entries to be archived...
Mon Apr 10 23:31:04 2017
All FAL entries have been archived.
Mon Apr 10 23:31:04 2017
Waiting for dest_id 2 to become synchronized...
Mon Apr 10 23:31:05 2017
Active, synchronized Physical Standby switchover target has been identified
Preventing updates and queries at the Primary
Generating and shipping final logs to target standby
Switchover End-Of-Redo Log thread 1 sequence 70 has been fixed
Switchover End-Of-Redo Log thread 2 sequence 39 has been fixed
Switchover: Primary highest seen SCN set to 0x0.0x3553b3
ARCH: Noswitch archival of thread 2, sequence 39
ARCH: End-Of-Redo Branch archival of thread 2 sequence 39
ARCH: LGWR is actively archiving destination LOG_ARCHIVE_DEST_2
ARCH: Standby redo logfile selected for thread 2 sequence 39 for destination LOG_ARCHIVE_DEST_2
ARCH: Noswitch archival of thread 1, sequence 70
ARCH: End-Of-Redo Branch archival of thread 1 sequence 70
ARCH: LGWR is actively archiving destination LOG_ARCHIVE_DEST_2
ARCH: Standby redo logfile selected for thread 1 sequence 70 for destination LOG_ARCHIVE_DEST_2
ARCH: Archiving is disabled due to current logfile archival
Primary will check for some target standby to have received all redo
Waiting for target standby to apply all redo
Backup controlfile written to trace file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_rsm0_17791.trc
Converting the primary database to a new standby database
Clearing standby activation ID 2519803190 (0x96312536)
The primary database controlfile was created using the
'MAXLOGFILES 192' clause.
There is space for up to 188 standby redo logfiles
Use the following SQL commands on the standby database to create
standby redo logfiles that match the primary database:
ALTER DATABASE ADD STANDBY LOGFILE 'srl1.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl2.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl3.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl4.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl5.f' SIZE 52428800;
Archivelog for thread 1 sequence 70 required for standby recovery
Archivelog for thread 2 sequence 39 required for standby recovery
Offline data file 5 marked as online during convert to standby or switchover to standby.
Restore of backup may be required if the file is not physically accessible.
Offline data file 7 marked as online during convert to standby or switchover to standby.
Restore of backup may be required if the file is not physically accessible.
Switchover: Primary controlfile converted to standby controlfile succesfully.
Mon Apr 10 23:31:14 2017
Reconfiguration started (old inc 4, new inc 6)
List of instances (total 1) :
1
Dead instances (total 1) :
2
My inst 1
publish big name space - dead or down/up instance detected, invalidate domain 0
Global Resource Directory frozen
* dead instance detected - domain 0 invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Apr 10 23:31:14 2017
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:31:14 2017
Post SMON to start 1st pass IR
Mon Apr 10 23:31:14 2017
Switchover: Complete - Database shutdown required
Mon Apr 10 23:31:14 2017
Sending request(convert to primary database) to switchover target stby
Mon Apr 10 23:31:14 2017
minact-scn: Inst 1 is now the master inc#:4 mmon proc-id:17203 status:0x7
Mon Apr 10 23:31:14 2017
Process (ospid 17189) is suspended due to switchover to physical standby operation.
Mon Apr 10 23:31:14 2017
minact-scn status: grec-scn:0x0000.00000000 gmin-scn:0x0000.0033ccde gcalc-scn:0x0000.0033ccee
minact-scn: master found reconf/inst-rec before recscn scan old-inc#:4 new-inc#:4
Mon Apr 10 23:31:14 2017
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:31:14 2017
Reconfiguration complete (total time 0.6 secs)
Mon Apr 10 23:31:19 2017
Switchover complete. Database shutdown required
Completed: ALTER DATABASE SWITCHOVER TO 'stby'
Target standby stby did not have Active Data Guard enabled at the time of switchover.
To maintain Active Data Guard license compliance Block Change Tracking will be disabled.
Mon Apr 10 23:31:21 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_rfs_6513.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_rfs_6518.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:23 2017
Clusterware restarting instance for Data Guard Broker operation; shutting down instance now
DMON (ospid: 17306): terminating the instance
Mon Apr 10 23:31:24 2017
Instance terminated by DMON, pid = 17306
Mon Apr 10 23:31:39 2017
Starting ORACLE instance (normal) (OS id: 6761)
Mon Apr 10 23:31:40 2017
CLI notifier numLatches:3 maxDescs:519
Mon Apr 10 23:31:41 2017
Due to limited space in shared pool (need 6094848 bytes, have 4194112 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:31:42 2017
**********************************************************************
Mon Apr 10 23:31:42 2017
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)
Mon Apr 10 23:31:42 2017
Per process system memlock (soft) limit = UNLIMITED
Mon Apr 10 23:31:42 2017
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 802M
Mon Apr 10 23:31:42 2017
Available system pagesizes:
4K, 2048K
Mon Apr 10 23:31:42 2017
Supported system pagesize(s):
Mon Apr 10 23:31:42 2017
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
Mon Apr 10 23:31:42 2017
4K Configured 4 204804 NONE
Mon Apr 10 23:31:42 2017
2048K 0 401 0 NONE
Mon Apr 10 23:31:42 2017
RECOMMENDATION:
Mon Apr 10 23:31:42 2017
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
Mon Apr 10 23:31:42 2017
**********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 2
Number of processor cores in the system is 2
Number of processor sockets in the system is 1
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
[name='eth1:1', type=1, ip=169.254.3.70, mac=08-00-27-6f-ef-ba, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
[name='eth0', type=1, ip=192.168.78.51, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:1' configured from GPnP for use as a public interface.
[name='eth0:1', type=1, ip=192.168.78.61, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:4' configured from GPnP for use as a public interface.
[name='eth0:4', type=1, ip=192.168.78.251, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:5' configured from GPnP for use as a public interface.
[name='eth0:5', type=1, ip=192.168.78.252, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is enabled
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Advanced Analytics
and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1
System name: Linux
Node name: collabn1.racattack
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Using parameter settings in server-side pfile /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initRAC1.ora
System parameters with non-default values:
processes = 300
sga_max_size = 800M
spfile = "+DATA/RAC/spfilerac.ora"
control_files = "+DATA/RAC/CONTROLFILE/current.282.931825275"
control_files = "+FRA/RAC/CONTROLFILE/current.256.931825275"
db_block_size = 8192
db_cache_size = 100M
compatible = "12.1.0.2.0"
log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
log_archive_dest_2 = "service="stby""
log_archive_dest_2 = "ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="stby" net_timeout=30"
log_archive_dest_2 = "valid_for=(online_logfile,all_roles)"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
log_archive_trace = 0
log_archive_config = "dg_config=(RAC,stby)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 4
archive_lag_target = 0
cluster_database = TRUE
db_create_file_dest = "+DATA"
db_recovery_file_dest = "+FRA"
db_recovery_file_dest_size= 4785M
standby_file_management = "MANUAL"
thread = 1
undo_tablespace = "UNDOTBS1"
instance_number = 1
remote_login_passwordfile= "EXCLUSIVE"
db_domain = "racattack"
dispatchers = "(PROTOCOL=TCP) (SERVICE=RACXDB)"
audit_file_dest = "/u01/app/oracle/admin/RAC/adump"
audit_trail = "DB"
db_name = "RAC"
open_cursors = 300
dg_broker_start = TRUE
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
diagnostic_dest = "/u01/app/oracle"
enable_pluggable_database= TRUE
Mon Apr 10 23:31:52 2017
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Mon Apr 10 23:31:52 2017
Cluster communication is configured to use the following interface(s) for this instance
169.254.3.70
cluster interconnect IPC version: Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0
Starting background process PMON
Mon Apr 10 23:31:53 2017
PMON started with pid=2, OS id=6880
Starting background process PSP0
Starting background process VKTM
Mon Apr 10 23:31:53 2017
PSP0 started with pid=3, OS id=6882
Mon Apr 10 23:31:55 2017
VKTM started with pid=4, OS id=6892 at elevated (RT) priority
Starting background process GEN0
Mon Apr 10 23:31:55 2017
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Starting background process MMAN
Mon Apr 10 23:31:55 2017
GEN0 started with pid=5, OS id=6896
Mon Apr 10 23:31:55 2017
MMAN started with pid=6, OS id=6898
Starting background process DIAG
Starting background process DBRM
Mon Apr 10 23:31:55 2017
DIAG started with pid=8, OS id=6910
Starting background process VKRM
Mon Apr 10 23:31:55 2017
DBRM started with pid=9, OS id=6912
Starting background process PING
Mon Apr 10 23:31:55 2017
VKRM started with pid=10, OS id=6914
Starting background process ACMS
Mon Apr 10 23:31:55 2017
PING started with pid=11, OS id=6916
Starting background process DIA0
Mon Apr 10 23:31:55 2017
ACMS started with pid=12, OS id=6918
Mon Apr 10 23:31:56 2017
DIA0 started with pid=13, OS id=6920
Starting background process LMON
Mon Apr 10 23:31:56 2017
LMON started with pid=14, OS id=6922
Starting background process LMD0
Starting background process LMS0
Mon Apr 10 23:31:56 2017
LMD0 started with pid=7, OS id=6929
Mon Apr 10 23:31:56 2017
* Load Monitor used for high load check
* New Low - High Load Threshold Range = [1920 - 2560]
Mon Apr 10 23:31:56 2017
LMS0 started with pid=15, OS id=6931 at elevated (RT) priority
Starting background process RMS0
Starting background process LMHB
Mon Apr 10 23:31:56 2017
RMS0 started with pid=16, OS id=6935
Mon Apr 10 23:31:56 2017
LMHB started with pid=17, OS id=6937
Starting background process LCK1
Starting background process DBW0
Mon Apr 10 23:31:56 2017
LCK1 started with pid=18, OS id=6939
Starting background process LGWR
Mon Apr 10 23:31:56 2017
DBW0 started with pid=19, OS id=6941
Starting background process CKPT
Mon Apr 10 23:31:56 2017
LGWR started with pid=20, OS id=6943
Mon Apr 10 23:31:56 2017
CKPT started with pid=21, OS id=6945
Starting background process SMON
Mon Apr 10 23:31:56 2017
SMON started with pid=23, OS id=6949
Starting background process RECO
Starting background process LREG
Mon Apr 10 23:31:56 2017
RECO started with pid=25, OS id=6953
Starting background process PXMN
Mon Apr 10 23:31:56 2017
LREG started with pid=26, OS id=6955
Mon Apr 10 23:31:56 2017
PXMN started with pid=27, OS id=6957
Starting background process RBAL
Mon Apr 10 23:31:56 2017
RBAL started with pid=28, OS id=6959
Starting background process ASMB
Starting background process MMON
Mon Apr 10 23:31:56 2017
ASMB started with pid=29, OS id=6961
Starting background process MMNL
Mon Apr 10 23:31:56 2017
MMON started with pid=30, OS id=6963
Mon Apr 10 23:31:56 2017
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Mon Apr 10 23:31:56 2017
MMNL started with pid=31, OS id=6965
starting up 1 shared server(s) ...
Mon Apr 10 23:31:57 2017
NOTE: ASMB registering with ASM instance as Standard client 0xffffffffffffffff (reg:839035202) (new connection)
NOTE: ASMB connected to ASM instance +ASM1 osid: 6971 (Flex mode; client id 0xffffffffffffffff)
Mon Apr 10 23:31:58 2017
lmon registered with NM - instance number 1 (internal mem no 0)
Mon Apr 10 23:31:58 2017
NOTE: initiating MARK startup
Starting background process MARK
Mon Apr 10 23:31:58 2017
MARK started with pid=35, OS id=6977
Mon Apr 10 23:31:58 2017
NOTE: MARK has subscribed
Mon Apr 10 23:31:58 2017
Reconfiguration started (old inc 0, new inc 2)
List of instances (total 1) :
1
My inst 1 (I'm a new instance)
Mon Apr 10 23:31:59 2017
Using default pga_aggregate_limit of 2048 MB
Mon Apr 10 23:32:18 2017
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Apr 10 23:32:26 2017
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:32:26 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:32:26 2017
Reconfiguration complete (total time 27.7 secs)
Starting background process LCK0
Mon Apr 10 23:32:29 2017
LCK0 started with pid=39, OS id=7084
Starting background process RSMN
Mon Apr 10 23:32:30 2017
RSMN started with pid=40, OS id=7094
Mon Apr 10 23:32:32 2017
Instance started by oraagent
Starting background process DMON
ORACLE_BASE from environment = /u01/app/oracle
Mon Apr 10 23:32:32 2017
DMON started with pid=41, OS id=7100
Mon Apr 10 23:32:34 2017
ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))' SCOPE=MEMORY SID='RAC1';
Mon Apr 10 23:32:35 2017
ALTER SYSTEM SET remote_listener=' collabn-cluster-scan.racattack:1521' SCOPE=MEMORY SID='RAC1';
ALTER DATABASE MOUNT /* db agent *//* {1:46642:1193} */
Mon Apr 10 23:32:36 2017
This instance was first to mount
Mon Apr 10 23:32:38 2017
NOTE: ASMB mounting group 1 (DATA)
NOTE: Assigning number (1,0) to disk (/dev/asm-disk1)
NOTE: Assigning number (1,1) to disk (/dev/asm-disk2)
NOTE: Assigning number (1,2) to disk (/dev/asm-disk5)
SUCCESS: mounted group 1 (DATA)
NOTE: grp 1 disk 0: DATA_0000 path:/dev/asm-disk1
NOTE: grp 1 disk 1: DATA_0001 path:/dev/asm-disk2
NOTE: grp 1 disk 2: DATA_0002 path:/dev/asm-disk5
NOTE: ASMB mounting group 2 (FRA)
Mon Apr 10 23:32:39 2017
NOTE: dependency between database RAC and diskgroup resource ora.DATA.dg is established
Mon Apr 10 23:32:39 2017
NOTE: Assigning number (2,0) to disk (/dev/asm-disk3)
SUCCESS: mounted group 2 (FRA)
NOTE: grp 2 disk 0: FRA_0000 path:/dev/asm-disk3
Mon Apr 10 23:32:40 2017
NOTE: dependency between database RAC and diskgroup resource ora.FRA.dg is established
Mon Apr 10 23:32:46 2017
WARNING: STANDBY_FILE_MANAGEMENT initialization parameter is
not set to the value "AUTO".
This may cause recovery of the standby database to terminate
prior to applying all available redo data.
It may be necessary to use the ALTER DATABASE CREATE DATAFILE
command to add datafiles created on the primary database.
Starting background process TMON
ARCH: STARTING ARCH PROCESSES
Starting background process ARC0
Mon Apr 10 23:32:48 2017
TMON started with pid=44, OS id=7628
Mon Apr 10 23:32:48 2017
ARC0 started with pid=45, OS id=7630
ARC0: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:32:48 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC1
Starting background process ARC2
Mon Apr 10 23:32:48 2017
ARC1 started with pid=46, OS id=7632
Starting background process ARC3
Mon Apr 10 23:32:48 2017
ARC2 started with pid=47, OS id=7634
ARC1: Archival started
ARC2: Archival started
Mon Apr 10 23:32:48 2017
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC1: Thread not mounted
Mon Apr 10 23:32:48 2017
ARC2: Becoming the heartbeat ARCH
Mon Apr 10 23:32:48 2017
ARC3 started with pid=48, OS id=7636
Mon Apr 10 23:32:48 2017
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
ARC0: Thread not mounted
Mon Apr 10 23:32:48 2017
ARC3: Thread not mounted
Mon Apr 10 23:32:49 2017
ARC2: Thread not mounted
Mon Apr 10 23:32:49 2017
Network Resource Management enabled for Process LGWR (pid 6943) for Exadata I/O
Successful mount of redo thread 1, with mount id 2528954836
Mon Apr 10 23:32:49 2017
Allocated 3981120 bytes in shared pool for flashback generation buffer
Starting background process RVWR
Mon Apr 10 23:32:49 2017
RVWR started with pid=49, OS id=7638
Physical Standby Database mounted.
Lost write protection disabled
Mon Apr 10 23:32:58 2017
Archiving previously deferred ORLs (RAC1)
ARCH: End-Of-Redo Branch archival of thread 1 sequence 70
Mon Apr 10 23:33:13 2017
Archived Log entry 136 added for thread 1 sequence 70 ID 0x96312536 dest 1:
ARCH: End-Of-Redo Branch archival of thread 2 sequence 39
Mon Apr 10 23:33:15 2017
Archived Log entry 137 added for thread 2 sequence 39 ID 0x96312536 dest 1:
Completed: ALTER DATABASE MOUNT /* db agent *//* {1:46642:1193} */
Mon Apr 10 23:33:20 2017
Starting Data Guard Broker (DMON)
Starting background process INSV
Mon Apr 10 23:33:21 2017
INSV started with pid=42, OS id=7855
Starting background process NSV1
Mon Apr 10 23:33:25 2017
NSV1 started with pid=52, OS id=7869
Mon Apr 10 23:33:26 2017
Decreasing number of real time LMS from 1 to 0
Starting background process RSM0
Mon Apr 10 23:33:41 2017
RSM0 started with pid=55, OS id=7929
Mon Apr 10 23:33:41 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Network Resource Management enabled for Process (pid 7909) for Exadata I/O
Primary database is in MAXIMUM PERFORMANCE mode
Mon Apr 10 23:33:46 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:33:46 2017
ALTER SYSTEM SET log_archive_dest_2='' SCOPE=BOTH;
Mon Apr 10 23:33:46 2017
ARC2: Becoming the active heartbeat ARCH
ARC2: Becoming the active heartbeat ARCH
RFS[1]: Assigned to RFS process (PID:7909)
RFS[1]: Selected log 5 for thread 1 sequence 72 dbid 2519807290 branch 931825279
Mon Apr 10 23:33:46 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:33:47 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='RAC1';
Mon Apr 10 23:33:47 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='RAC1';
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
Mon Apr 10 23:33:50 2017
Attempt to start background Managed Standby Recovery process (RAC1)
Starting background process MRP0
Mon Apr 10 23:33:50 2017
MRP0 started with pid=56, OS id=7988
Mon Apr 10 23:33:50 2017
MRP0: Background Managed Standby Recovery process started (RAC1)
Mon Apr 10 23:33:55 2017
Started logmerger process
Mon Apr 10 23:33:57 2017
Managed Standby Recovery starting Real Time Apply
Mon Apr 10 23:33:58 2017
RFS[2]: Assigned to RFS process (PID:8007)
RFS[2]: Opened log for thread 1 sequence 71 dbid 2519807290 branch 931825279
RFS[1]: Selected log 6 for thread 1 sequence 73 dbid 2519807290 branch 931825279
Mon Apr 10 23:33:59 2017
Reconfiguration started (old inc 2, new inc 4)
List of instances (total 2) :
1 2
New instances (total 1) :
2
My inst 1
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:33:59 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:34:00 2017
Reconfiguration complete (total time 0.5 secs)
Mon Apr 10 23:34:15 2017
Archived Log entry 138 added for thread 1 sequence 72 ID 0x96bd0722 dest 1:
Mon Apr 10 23:34:16 2017
Archived Log entry 139 added for thread 1 sequence 71 rlc 931825279 ID 0x96bd0722 dest 2:
RFS[2]: Opened log for thread 2 sequence 40 dbid 2519807290 branch 931825279
Mon Apr 10 23:34:23 2017
Archived Log entry 140 added for thread 2 sequence 40 rlc 931825279 ID 0x0 dest 2:
Mon Apr 10 23:34:28 2017
Parallel Media Recovery started with 2 slaves
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
Mon Apr 10 23:34:33 2017
Waiting for all non-current ORLs to be archived...
Mon Apr 10 23:34:33 2017
All non-current ORLs have been archived.
Clearing online redo logfile 1 +DATA/RAC/ONLINELOG/group_1.283.931825279
Clearing online log 1 of thread 1 sequence number 69
Mon Apr 10 23:34:50 2017
Clearing online redo logfile 1 complete
Clearing online redo logfile 2 +DATA/RAC/ONLINELOG/group_2.284.931825283
Clearing online log 2 of thread 1 sequence number 70
Mon Apr 10 23:34:51 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:35:00 2017
Increasing number of real time LMS from 0 to 1
Mon Apr 10 23:35:09 2017
Clearing online redo logfile 2 complete
Clearing online redo logfile 3 +DATA/RAC/ONLINELOG/group_3.290.931826413
Clearing online log 3 of thread 2 sequence number 39
Mon Apr 10 23:35:22 2017
Clearing online redo logfile 3 complete
Clearing online redo logfile 4 +DATA/RAC/ONLINELOG/group_4.291.931826417
Clearing online log 4 of thread 2 sequence number 38
Clearing online redo logfile 4 complete
Mon Apr 10 23:35:28 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_2_seq_38.357.940980599
Mon Apr 10 23:35:28 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_67.355.940978999
Mon Apr 10 23:35:29 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_68.356.940980597
Mon Apr 10 23:35:31 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_69.358.940980603
Mon Apr 10 23:35:32 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_2_seq_39.360.940980795
Mon Apr 10 23:35:35 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_70.359.940980783
Mon Apr 10 23:35:38 2017
Resetting standby activation ID 0 (0x0)
Mon Apr 10 23:35:39 2017
Media Recovery End-Of-Redo indicator encountered
Mon Apr 10 23:35:39 2017
Media Recovery Continuing
Mon Apr 10 23:35:39 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_71.361.940980839
Mon Apr 10 23:35:39 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_2_seq_40.363.940980861
Mon Apr 10 23:35:39 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_72.362.940980841
Mon Apr 10 23:35:51 2017
Media Recovery Waiting for thread 1 sequence 73 (in transit)
Mon Apr 10 23:35:51 2017
Recovery of Online Redo Log: Thread 1 Group 6 Seq 73 Reading mem 0
Mem# 0: +DATA/RAC/ONLINELOG/group_6.298.937936361
Mem# 1: +FRA/RAC/ONLINELOG/group_6.304.937936363
Mon Apr 10 23:41:26 2017
Shutting down instance (immediate)
Shutting down instance: further logons disabled
Stopping background process MMNL
Stopping background process MMON
Mon Apr 10 23:41:28 2017
MRP0: Background Media Recovery cancelled with status 16037
Mon Apr 10 23:41:28 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_pr00_8020.trc:
ORA-16037: user requested cancel of managed recovery operation
Mon Apr 10 23:41:28 2017
Managed Standby Recovery not using Real Time Apply
Mon Apr 10 23:41:29 2017
Recovery interrupted!
Recovered data files to a consistent state at change 3497232
Mon Apr 10 23:41:29 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 99
Mon Apr 10 23:41:29 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_pr00_8020.trc:
ORA-16037: user requested cancel of managed recovery operation
Mon Apr 10 23:41:29 2017
MRP0: Background Media Recovery process shutdown (RAC1)
Mon Apr 10 23:41:29 2017
NOTE: Deferred communication with ASM instance
License high water mark = 10
Mon Apr 10 23:41:29 2017
All dispatchers and shared servers shutdown
ALTER DATABASE CLOSE NORMAL
Stopping Emon pool
ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC0: Archival stopped
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC1: Archival stopped
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC2: Relinquishing active heartbeat ARCH role
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC3: Archival stopped
Mon Apr 10 23:41:30 2017
ARC2: Archival stopped
Mon Apr 10 23:41:31 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 5
Mon Apr 10 23:41:31 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 14
Mon Apr 10 23:41:31 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 2
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:41:37 2017
NOTE: De-assigning number (2,0) from disk (/dev/asm-disk3)
SUCCESS: diskgroup FRA was dismounted
NOTE: De-assigning number (1,0) from disk (/dev/asm-disk1)
NOTE: De-assigning number (1,1) from disk (/dev/asm-disk2)
NOTE: De-assigning number (1,2) from disk (/dev/asm-disk5)
SUCCESS: diskgroup DATA was dismounted
NOTE: Database dismounted; ASMB process exiting
NOTE: ASMB clearing idle groups before exit
Stopping background process MARK
Mon Apr 10 23:41:38 2017
NOTE: Shutting down MARK background process
Stopping background process RBAL
Mon Apr 10 23:41:56 2017
Reconfiguration started (old inc 4, new inc 6)
List of instances (total 1) :
1
Dead instances (total 1) :
2
My inst 1
publish big name space - dead or down/up instance detected, invalidate domain 0
Global Resource Directory frozen
* dead instance detected - domain 0 invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:41:56 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:41:56 2017
Reconfiguration complete (total time 0.2 secs)
Mon Apr 10 23:42:31 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:42:36 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:42:37 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:42:37 2017
Stopping background process VKTM
Mon Apr 10 23:42:42 2017
freeing rdom 0
Mon Apr 10 23:42:44 2017
Instance shutdown complete
Next on node2 running instance RAC2 :
04/10/2017 23:34:35
>> Starting Data Guard Broker bootstrap <<
Broker Configuration File Locations:
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
2017-04-10 23:34:35.584 DMON: Attach state object
2017-04-10 23:34:35.991 DMON: Broker State Initialized
2017-04-10 23:34:35.991 Version = 3
2017-04-10 23:34:35.991 State = 00000000
2017-04-10 23:34:35.992 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 2
2017-04-10 23:34:35.993 7fffffff 0 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 0
2017-04-10 23:34:39.031 7fffffff 0 DMON: start task execution: broker initialization
2017-04-10 23:34:39.130 DMON: Boot configuration (0.0.0), loading from "+DATA/RAC/dgbroker1.dat"
2017-04-10 23:34:39.192 DMON Registering service RAC_DGB with listener(s)
2017-04-10 23:34:39.192 DMON: Executing SQL [ALTER SYSTEM REGISTER]
2017-04-10 23:34:39.193 SQL [ALTER SYSTEM REGISTER] Executed successfully
04/10/2017 23:34:39
Broker Configuration: "rac"
Protection Mode: Maximum Performance
Fast-Start Failover (FSFO): Disabled, flags=0x0, version=0
Primary Database: stby (0x02010000)
04/10/2017 23:34:47
rac version check successfully completed
Broker configuration file is current on rac, completing initialization
Creating process RSM0
04/10/2017 23:41:41
Data Guard Broker shutting down
RSM0 successfully terminated
2017-04-10 23:41:44.988 >> DMON Process Shutdown <<
==========================================================================================================
Mon Apr 10 23:31:10 2017
Switchover in progress in another database instance - Database is shutdown automatically
LGWR (ospid: 3335): terminating the instance due to error 16456
Mon Apr 10 23:31:11 2017
Instance terminated by LGWR, pid = 3335
Mon Apr 10 23:33:50 2017
Starting ORACLE instance (normal) (OS id: 8233)
Mon Apr 10 23:33:51 2017
CLI notifier numLatches:3 maxDescs:519
Mon Apr 10 23:33:51 2017
Due to limited space in shared pool (need 6094848 bytes, have 4194112 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:33:51 2017
**********************************************************************
Mon Apr 10 23:33:51 2017
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)
Mon Apr 10 23:33:51 2017
Per process system memlock (soft) limit = UNLIMITED
Mon Apr 10 23:33:51 2017
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 802M
Mon Apr 10 23:33:51 2017
Available system pagesizes:
4K, 2048K
Mon Apr 10 23:33:51 2017
Supported system pagesize(s):
Mon Apr 10 23:33:51 2017
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
Mon Apr 10 23:33:51 2017
4K Configured 4 204804 NONE
Mon Apr 10 23:33:51 2017
2048K 0 401 0 NONE
Mon Apr 10 23:33:51 2017
RECOMMENDATION:
Mon Apr 10 23:33:51 2017
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
Mon Apr 10 23:33:51 2017
**********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 2
Number of processor cores in the system is 2
Number of processor sockets in the system is 1
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
[name='eth1:1', type=1, ip=169.254.178.54, mac=08-00-27-c5-68-87, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
[name='eth0', type=1, ip=192.168.78.52, mac=08-00-27-1e-02-8d, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:1' configured from GPnP for use as a public interface.
[name='eth0:1', type=1, ip=192.168.78.62, mac=08-00-27-1e-02-8d, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:2' configured from GPnP for use as a public interface.
[name='eth0:2', type=1, ip=192.168.78.253, mac=08-00-27-1e-02-8d, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is enabled
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Advanced Analytics
and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1
System name: Linux
Node name: collabn2.racattack
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Using parameter settings in server-side pfile /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initRAC2.ora
System parameters with non-default values:
processes = 300
sga_max_size = 800M
spfile = "+DATA/RAC/spfilerac.ora"
control_files = "+DATA/RAC/CONTROLFILE/current.282.931825275"
control_files = "+FRA/RAC/CONTROLFILE/current.256.931825275"
db_block_size = 8192
db_cache_size = 100M
compatible = "12.1.0.2.0"
log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
log_archive_dest_2 = "service="stby""
log_archive_dest_2 = "ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="stby" net_timeout=30"
log_archive_dest_2 = "valid_for=(online_logfile,all_roles)"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
log_archive_trace = 0
log_archive_config = "dg_config=(RAC,stby)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 4
archive_lag_target = 0
cluster_database = TRUE
db_create_file_dest = "+DATA"
db_recovery_file_dest = "+FRA"
db_recovery_file_dest_size= 4785M
standby_file_management = "MANUAL"
thread = 2
undo_tablespace = "UNDOTBS2"
instance_number = 2
remote_login_passwordfile= "EXCLUSIVE"
db_domain = "racattack"
dispatchers = "(PROTOCOL=TCP) (SERVICE=RACXDB)"
audit_file_dest = "/u01/app/oracle/admin/RAC/adump"
audit_trail = "DB"
db_name = "RAC"
open_cursors = 300
dg_broker_start = TRUE
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
diagnostic_dest = "/u01/app/oracle"
enable_pluggable_database= TRUE
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Mon Apr 10 23:33:55 2017
Cluster communication is configured to use the following interface(s) for this instance
169.254.178.54
cluster interconnect IPC version: Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0
Starting background process PMON
Mon Apr 10 23:33:56 2017
PMON started with pid=2, OS id=8257
Starting background process PSP0
Mon Apr 10 23:33:56 2017
PSP0 started with pid=3, OS id=8259
Starting background process VKTM
Mon Apr 10 23:33:57 2017
VKTM started with pid=4, OS id=8261 at elevated (RT) priority
Starting background process GEN0
Mon Apr 10 23:33:57 2017
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Starting background process MMAN
Mon Apr 10 23:33:57 2017
GEN0 started with pid=5, OS id=8265
Mon Apr 10 23:33:57 2017
MMAN started with pid=6, OS id=8267
Starting background process DIAG
Mon Apr 10 23:33:57 2017
Starting background process DBRM
DIAG started with pid=8, OS id=8271
Starting background process VKRM
Mon Apr 10 23:33:57 2017
DBRM started with pid=9, OS id=8273
Starting background process PING
Mon Apr 10 23:33:57 2017
VKRM started with pid=10, OS id=8275
Mon Apr 10 23:33:57 2017
PING started with pid=11, OS id=8277
Starting background process ACMS
Mon Apr 10 23:33:57 2017
Starting background process DIA0
ACMS started with pid=12, OS id=8279
Mon Apr 10 23:33:57 2017
Starting background process LMON
DIA0 started with pid=13, OS id=8281
Mon Apr 10 23:33:57 2017
Starting background process LMD0
LMON started with pid=14, OS id=8283
Mon Apr 10 23:33:57 2017
Starting background process LMS0
LMD0 started with pid=15, OS id=8285
Mon Apr 10 23:33:57 2017
* Load Monitor used for high load check
* New Low - High Load Threshold Range = [1920 - 2560]
Starting background process RMS0
Mon Apr 10 23:33:57 2017
LMS0 started with pid=16, OS id=8287 at elevated (RT) priority
Mon Apr 10 23:33:57 2017
Starting background process LMHB
RMS0 started with pid=17, OS id=8291
Mon Apr 10 23:33:57 2017
Starting background process LCK1
LMHB started with pid=18, OS id=8293
Mon Apr 10 23:33:57 2017
Starting background process DBW0
LCK1 started with pid=19, OS id=8295
Mon Apr 10 23:33:57 2017
DBW0 started with pid=20, OS id=8297
Starting background process LGWR
Mon Apr 10 23:33:57 2017
Starting background process CKPT
LGWR started with pid=21, OS id=8299
Mon Apr 10 23:33:57 2017
CKPT started with pid=22, OS id=8301
Starting background process SMON
Mon Apr 10 23:33:57 2017
SMON started with pid=24, OS id=8305
Starting background process RECO
Mon Apr 10 23:33:57 2017
RECO started with pid=26, OS id=8309
Starting background process LREG
Mon Apr 10 23:33:57 2017
Starting background process PXMN
LREG started with pid=27, OS id=8311
Mon Apr 10 23:33:57 2017
Starting background process RBAL
PXMN started with pid=7, OS id=8313
Starting background process ASMB
Mon Apr 10 23:33:57 2017
RBAL started with pid=28, OS id=8315
Mon Apr 10 23:33:57 2017
ASMB started with pid=29, OS id=8317
Starting background process MMON
Starting background process MMNL
Mon Apr 10 23:33:57 2017
MMON started with pid=30, OS id=8319
Mon Apr 10 23:33:57 2017
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Mon Apr 10 23:33:57 2017
MMNL started with pid=31, OS id=8323
Mon Apr 10 23:33:57 2017
NOTE: ASMB registering with ASM instance as Standard client 0xffffffffffffffff (reg:593534517) (new connection)
Mon Apr 10 23:33:57 2017
starting up 1 shared server(s) ...
Mon Apr 10 23:33:57 2017
NOTE: ASMB connected to ASM instance +ASM2 osid: 8321 (Flex mode; client id 0xffffffffffffffff)
NOTE: initiating MARK startup
Starting background process MARK
Mon Apr 10 23:33:57 2017
lmon registered with NM - instance number 2 (internal mem no 1)
Mon Apr 10 23:33:57 2017
MARK started with pid=34, OS id=8329
Mon Apr 10 23:33:57 2017
NOTE: MARK has subscribed
Mon Apr 10 23:33:59 2017
Reconfiguration started (old inc 0, new inc 4)
List of instances (total 2) :
1 2
My inst 2 (I'm a new instance)
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Mon Apr 10 23:33:59 2017
* domain 0 valid = 0 according to instance 1
Mon Apr 10 23:33:59 2017
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Apr 10 23:33:59 2017
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:33:59 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:34:00 2017
Reconfiguration complete (total time 0.8 secs)
Starting background process LCK0
Mon Apr 10 23:34:00 2017
LCK0 started with pid=38, OS id=8337
Mon Apr 10 23:34:00 2017
Using default pga_aggregate_limit of 2048 MB
Starting background process RSMN
Mon Apr 10 23:34:01 2017
RSMN started with pid=40, OS id=8351
Mon Apr 10 23:34:02 2017
Instance started by oraagent
Starting background process DMON
ORACLE_BASE from environment = /u01/app/oracle
Mon Apr 10 23:34:02 2017
DMON started with pid=41, OS id=8353
Mon Apr 10 23:34:03 2017
ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.62)(PORT=1521))' SCOPE=MEMORY SID='RAC2';
Mon Apr 10 23:34:03 2017
ALTER SYSTEM SET remote_listener=' collabn-cluster-scan.racattack:1521' SCOPE=MEMORY SID='RAC2';
ALTER DATABASE MOUNT /* db agent *//* {1:46642:1276} */
Mon Apr 10 23:34:04 2017
NOTE: ASMB mounting group 1 (DATA)
NOTE: Assigning number (1,0) to disk (/dev/asm-disk1)
NOTE: Assigning number (1,1) to disk (/dev/asm-disk2)
NOTE: Assigning number (1,2) to disk (/dev/asm-disk5)
SUCCESS: mounted group 1 (DATA)
NOTE: grp 1 disk 0: DATA_0000 path:/dev/asm-disk1
NOTE: grp 1 disk 1: DATA_0001 path:/dev/asm-disk2
NOTE: grp 1 disk 2: DATA_0002 path:/dev/asm-disk5
Mon Apr 10 23:34:04 2017
NOTE: dependency between database RAC and diskgroup resource ora.DATA.dg is established
Mon Apr 10 23:34:04 2017
NOTE: ASMB mounting group 2 (FRA)
NOTE: Assigning number (2,0) to disk (/dev/asm-disk3)
SUCCESS: mounted group 2 (FRA)
NOTE: grp 2 disk 0: FRA_0000 path:/dev/asm-disk3
Mon Apr 10 23:34:04 2017
NOTE: dependency between database RAC and diskgroup resource ora.FRA.dg is established
Mon Apr 10 23:34:14 2017
WARNING: STANDBY_FILE_MANAGEMENT initialization parameter is
not set to the value "AUTO".
This may cause recovery of the standby database to terminate
prior to applying all available redo data.
It may be necessary to use the ALTER DATABASE CREATE DATAFILE
command to add datafiles created on the primary database.
Starting background process TMON
Mon Apr 10 23:34:16 2017
ARCH: STARTING ARCH PROCESSES
TMON started with pid=45, OS id=8374
Starting background process ARC0
Mon Apr 10 23:34:16 2017
ARC0 started with pid=46, OS id=8376
ARC0: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:34:16 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC1
Starting background process ARC2
Mon Apr 10 23:34:16 2017
ARC1 started with pid=47, OS id=8378
Starting background process ARC3
Mon Apr 10 23:34:16 2017
ARC2 started with pid=48, OS id=8380
Mon Apr 10 23:34:16 2017
ARC3 started with pid=49, OS id=8382
ARC1: Archival started
ARC2: Archival started
Mon Apr 10 23:34:16 2017
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC1: Thread not mounted
Mon Apr 10 23:34:16 2017
ARC2: Becoming the heartbeat ARCH
Mon Apr 10 23:34:16 2017
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:34:16 2017
ARC0: Thread not mounted
Mon Apr 10 23:34:16 2017
ARC3: Thread not mounted
Mon Apr 10 23:34:16 2017
ARC2: Thread not mounted
Mon Apr 10 23:34:17 2017
Network Resource Management enabled for Process LGWR (pid 8299) for Exadata I/O
Successful mount of redo thread 2, with mount id 2528954836
Mon Apr 10 23:34:22 2017
Allocated 3981120 bytes in shared pool for flashback generation buffer
Starting background process RVWR
Mon Apr 10 23:34:22 2017
RVWR started with pid=50, OS id=8385
Physical Standby Database mounted.
Lost write protection disabled
Completed: ALTER DATABASE MOUNT /* db agent *//* {1:46642:1276} */
Mon Apr 10 23:34:35 2017
Starting Data Guard Broker (DMON)
Starting background process INSV
Mon Apr 10 23:34:36 2017
INSV started with pid=42, OS id=8424
Starting background process NSV1
Mon Apr 10 23:34:39 2017
NSV1 started with pid=51, OS id=8432
Starting background process RSM0
Mon Apr 10 23:34:47 2017
RSM0 started with pid=52, OS id=8439
Mon Apr 10 23:34:51 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:34:51 2017
ALTER SYSTEM SET log_archive_dest_2='' SCOPE=BOTH;
Mon Apr 10 23:34:51 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:34:52 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='RAC2';
Mon Apr 10 23:34:52 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='RAC2';
Mon Apr 10 23:41:28 2017
Managed Standby Recovery not using Real Time Apply
Mon Apr 10 23:41:30 2017
ARC2: Becoming the active heartbeat ARCH
ARC2: Becoming the active heartbeat ARCH
Mon Apr 10 23:41:37 2017
Shutting down instance (immediate)
Shutting down instance: further logons disabled
Stopping background process MMNL
Stopping background process MMON
License high water mark = 2
Mon Apr 10 23:41:40 2017
All dispatchers and shared servers shutdown
ALTER DATABASE CLOSE NORMAL
Stopping Emon pool
ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:40 2017
ARCH shutting down
ARC1: Archival stopped
Mon Apr 10 23:41:40 2017
ARCH shutting down
Mon Apr 10 23:41:40 2017
ARCH shutting down
Mon Apr 10 23:41:40 2017
ARC0: Archival stopped
Mon Apr 10 23:41:40 2017
ARC2: Relinquishing active heartbeat ARCH role
ARC2: Archival stopped
Mon Apr 10 23:41:40 2017
ARCH shutting down
ARC3: Archival stopped
Mon Apr 10 23:41:41 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 5
Mon Apr 10 23:41:41 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 15
Mon Apr 10 23:41:41 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 2
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:41:41 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:41:43 2017
NOTE: Deferred communication with ASM instance
Mon Apr 10 23:41:44 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:41:45 2017
NOTE: force a map free for map id 2
Mon Apr 10 23:41:45 2017
Stopping background process VKTM
Mon Apr 10 23:41:45 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:46 2017
NOTE: Shutting down MARK background process
Mon Apr 10 23:41:53 2017
NOTE: ASMB releasing group resources due to shutdown
NOTE: ASMB clearing idle groups before exit
Stopping background process RBAL
Mon Apr 10 23:41:55 2017
freeing rdom 0
Mon Apr 10 23:41:57 2017
Instance shutdown complete
Next, the original Standby (SingleInstance) node : the drcSTBY.log and alert_STBY.log
04/10/2017 23:31:03
Notifying Oracle Clusterware to prepare target standby database for switchover
04/10/2017 23:33:49
Deferring associated archivelog destinations of sites permanently disabled due to Switchover
Notifying Oracle Clusterware to buildup primary database after switchover
Switchover Complete
New Primary Site is named: stby
04/10/2017 23:41:22
Notifying DMON of db close
Notifying RSM0 of db close
04/10/2017 23:41:27
Data Guard Broker shutting down
RSM0 successfully terminated
2017-04-10 23:41:30.761 >> DMON Process Shutdown <<
04/10/2017 23:41:55
>> Starting Data Guard Broker bootstrap <<
Broker Configuration File Locations:
dg_broker_config_file1 = "/u01/app/oracle/product/12.1.0/dbhome_1/dbs/dr1STBY.dat"
dg_broker_config_file2 = "/u01/app/oracle/product/12.1.0/dbhome_1/dbs/dr2STBY.dat"
2017-04-10 23:41:55.856 DMON: Attach state object
2017-04-10 23:41:55.856 DMON: rfafoGetLocks reinitializing dubious PMYSHUT lock value block contents: sts=0, flags=0x0, spare1=0x0, spare2=0x0, cksm=0x0, rndm=0x0
2017-04-10 23:41:55.857 DMON: Broker state reconciled, version = 0, state = 00000000
2017-04-10 23:41:55.857 DMON: Broker State Initialized
2017-04-10 23:41:55.857 Version = 1
2017-04-10 23:41:55.857 State = 00000000
2017-04-10 23:41:55.857 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 2
2017-04-10 23:41:55.857 7fffffff 0 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 0
2017-04-10 23:41:58.902 7fffffff 0 DMON: start task execution: broker initialization
2017-04-10 23:41:58.902 DMON: Boot configuration (0.0.0), loading from "/u01/app/oracle/product/12.1.0/dbhome_1/dbs/dr1STBY.dat"
2017-04-10 23:41:58.903 DMON Registering service STBY_DGB with listener(s)
2017-04-10 23:41:58.903 DMON: Executing SQL [ALTER SYSTEM REGISTER]
2017-04-10 23:41:58.904 SQL [ALTER SYSTEM REGISTER] Executed successfully
04/10/2017 23:41:58
Broker Configuration: "rac"
Protection Mode: Maximum Performance
Fast-Start Failover (FSFO): Disabled, flags=0x0, version=0
Primary Database: stby (0x02010000)
04/10/2017 23:42:11
Failed to connect to remote database rac. Error is ORA-12528
Failed to send message to site rac. Error code is ORA-12528.
Version Check Results:
Database rac returned ORA-12528
Creating process RSM0
04/10/2017 23:42:35
Processing shutdown notification from database rac, instance 1
Broker operation
Setting SHUTDOWN status for database rac
04/10/2017 23:45:20
Notifying DMON of db close
Notifying RSM0 of db close
04/10/2017 23:45:28
Data Guard Broker shutting down
RSM0 successfully terminated
2017-04-10 23:45:31.850 >> DMON Process Shutdown <<
=======================================================================================
Mon Apr 10 23:30:59 2017
SWITCHOVER VERIFY BEGIN
SWITCHOVER VERIFY WARNING: no standby database is defined in LOG_ARCHIVE_DEST_n to protect this database if it is converted to a primary database
SWITCHOVER VERIFY COMPLETE
Mon Apr 10 23:31:00 2017
SWITCHOVER VERIFY BEGIN
SWITCHOVER VERIFY WARNING: no standby database is defined in LOG_ARCHIVE_DEST_n to protect this database if it is converted to a primary database
SWITCHOVER VERIFY COMPLETE
Mon Apr 10 23:31:08 2017
RFS[3]: Assigned to RFS process (PID:30383)
RFS[3]: Selected log 7 for thread 2 sequence 39 dbid 2519807290 branch 931825279
Mon Apr 10 23:31:08 2017
Archived Log entry 44 added for thread 2 sequence 39 ID 0x96312536 dest 1:
Mon Apr 10 23:31:08 2017
RFS[4]: Assigned to RFS process (PID:30386)
RFS[4]: Selected log 5 for thread 1 sequence 70 dbid 2519807290 branch 931825279
Mon Apr 10 23:31:09 2017
Archived Log entry 45 added for thread 1 sequence 70 ID 0x96312536 dest 1:
Mon Apr 10 23:31:09 2017
Resetting standby activation ID 2519803190 (0x96312536)
Mon Apr 10 23:31:09 2017
Media Recovery End-Of-Redo indicator encountered
Mon Apr 10 23:31:09 2017
Media Recovery Continuing
Media Recovery Waiting for thread 2 sequence 40
Mon Apr 10 23:31:10 2017
RFS[5]: Assigned to RFS process (PID:29959)
RFS[5]: Possible network disconnect with primary database
Mon Apr 10 23:31:15 2017
SWITCHOVER: received request 'ALTER DTABASE COMMIT TO SWITCHOVER TO PRIMARY' from primary database.
Mon Apr 10 23:31:15 2017
ALTER DATABASE SWITCHOVER TO PRIMARY (STBY)
Maximum wait for role transition is 15 minutes.
Switchover: Media recovery is still active
Role Change: Canceling MRP - no more redo to apply
Mon Apr 10 23:31:15 2017
MRP0: Background Media Recovery cancelled with status 16037
Mon Apr 10 23:31:15 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_pr00_29969.trc:
ORA-16037: user requested cancel of managed recovery operation
Managed Standby Recovery not using Real Time Apply
Recovery interrupted!
Mon Apr 10 23:31:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_pr00_29969.trc:
ORA-16037: user requested cancel of managed recovery operation
Mon Apr 10 23:31:16 2017
MRP0: Background Media Recovery process shutdown (STBY)
Mon Apr 10 23:31:17 2017
Role Change: Canceled MRP
Killing 1 processes (PIDS:30372) (all RFS) in order to disallow current and future RFS connections. Requested by OS process 30402
Backup controlfile written to trace file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_rmi_30402.trc
SwitchOver after complete recovery through change 3494835
Online logfile pre-clearing operation disabled by switchover
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_3.290.931826413: Thread 2 Group 3 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_3.259.931826417: Thread 2 Group 3 was previously cleared
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_4.291.931826417: Thread 2 Group 4 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_4.260.931826421: Thread 2 Group 4 was previously cleared
Standby became primary SCN: 3494833
Switchover: Complete - Database mounted as primary
SWITCHOVER: completed request from primary database.
Mon Apr 10 23:31:21 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:31:21 2017
ALTER SYSTEM SET log_archive_dest_2='service="rac1"','ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="rac" net_timeout=30','valid_for=(online_logfile,all_roles)' SCOPE=BOTH;
Mon Apr 10 23:31:21 2017
ARC0: Becoming the 'no SRL' ARCH
Mon Apr 10 23:31:21 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
ALTER DATABASE OPEN
Mon Apr 10 23:31:21 2017
Data Guard Broker initializing...
Ping without log force is disabled
.
Mon Apr 10 23:31:21 2017
Assigning activation ID 2528970530 (0x96bd0722)
Mon Apr 10 23:31:21 2017
Thread 1 advanced to log sequence 72 (thread open)
Thread 1 opened at log sequence 72
Current log# 2 seq# 72 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283
Current log# 2 seq# 72 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287
Successful open of redo thread 1
Mon Apr 10 23:31:21 2017
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Apr 10 23:31:21 2017
SMON: enabling cache recovery
Mon Apr 10 23:31:21 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************
Mon Apr 10 23:31:21 2017
Archived Log entry 46 added for thread 1 sequence 71 ID 0x96bd0722 dest 1:
Mon Apr 10 23:31:21 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_30433.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:21 2017
Error 16456 for archive log file 2 to 'rac1'
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc3_29915.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_30433.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_30433.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
FAL[server, ARC3]: Error 16456 creating remote archivelog file 'rac1'
ARC3: FAL archive failed with error 16456. See trace for details
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc3_29915.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Mon Apr 10 23:31:22 2017
ORACLE Instance STBY - Archival Error. Archiver continuing.
Mon Apr 10 23:31:24 2017
[29938] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:4159104 end:4161254 diff:2150 ms (2.2 seconds)
Dictionary check beginning
Dictionary check complete
Verifying minimum file header compatibility (11g) for tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
Mon Apr 10 23:31:26 2017
SMON: enabling tx recovery
Starting background process SMCO
Mon Apr 10 23:31:26 2017
SMCO started with pid=33, OS id=30448
Mon Apr 10 23:31:28 2017
Database Characterset is AL32UTF8
Mon Apr 10 23:31:29 2017
Redo thread 2 internally disabled at seq 40 (CKPT)
Mon Apr 10 23:31:30 2017
ARC2: Archiving disabled thread 2 sequence 40
Mon Apr 10 23:31:30 2017
Archived Log entry 47 added for thread 2 sequence 40 ID 0x0 dest 1:
Mon Apr 10 23:31:31 2017
No Resource Manager plan active
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process AQPC
Mon Apr 10 23:31:37 2017
AQPC started with pid=36, OS id=30543
LOGSTDBY: Validating controlfile with logical metadata
LOGSTDBY: Validation complete
Mon Apr 10 23:31:39 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC4
Mon Apr 10 23:31:39 2017
ARC4 started with pid=37, OS id=30553
ARC4: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:32:00 2017
Pluggable database PDB$SEED dictionary check beginning
Pluggable Database PDB$SEED Dictionary check complete
Database Characterset for PDB$SEED is AL32UTF8
Mon Apr 10 23:32:17 2017
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:32:36 2017
Opening pdb PDB$SEED (2) with no Resource Manager plan active
Shutting down archive processes
Mon Apr 10 23:32:40 2017
ARCH shutting down
ARC4: Archival stopped
Mon Apr 10 23:32:45 2017
Pluggable database PDB dictionary check beginning
Pluggable Database PDB Dictionary check complete
Database Characterset for PDB is AL32UTF8
ALTER SYSTEM: Flushing buffer cache inst=0 container=3 local
Mon Apr 10 23:32:58 2017
Cannot start service pdb, reason=-1
Starting background process CJQ0
Mon Apr 10 23:33:13 2017
CJQ0 started with pid=37, OS id=31055
Completed: ALTER DATABASE OPEN
ALTER PLUGGABLE DATABASE ALL OPEN
Mon Apr 10 23:33:15 2017
Database Characterset for PDB is AL32UTF8
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:33:36 2017
Shared IO Pool defaulting to 36MB. Trying to get it from Buffer Cache for process 29792.
Mon Apr 10 23:33:37 2017
Opening pdb PDB (3) with no Resource Manager plan active
Mon Apr 10 23:33:38 2017
===========================================================
Dumping current patch information
===========================================================
No patches have been applied
===========================================================
Mon Apr 10 23:33:38 2017
Pluggable database PDB opened read write
Completed: ALTER PLUGGABLE DATABASE ALL OPEN
Mon Apr 10 23:33:38 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='STBY';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='STBY';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET standby_file_management='MANUAL' SCOPE=BOTH SID='*';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET archive_lag_target=0 SCOPE=BOTH SID='*';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET log_archive_max_processes=4 SCOPE=BOTH SID='*';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET log_archive_min_succeed_dest=1 SCOPE=BOTH SID='*';
Mon Apr 10 23:33:40 2017
ALTER SYSTEM SET db_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY' SCOPE=SPFILE;
Mon Apr 10 23:33:40 2017
ALTER SYSTEM SET log_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY' SCOPE=SPFILE;
Mon Apr 10 23:33:40 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************
Mon Apr 10 23:33:41 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:33:41 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:33:41 2017
ALTER SYSTEM ARCHIVE LOG
Mon Apr 10 23:33:44 2017
Thread 1 advanced to log sequence 73 (LGWR switch)
Current log# 1 seq# 73 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279
Current log# 1 seq# 73 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281
Mon Apr 10 23:33:46 2017
db_recovery_file_dest_size of 16384 MB is 0.73% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Mon Apr 10 23:33:46 2017
TT00: Standby redo logfile selected for thread 1 sequence 72 for destination LOG_ARCHIVE_DEST_2
Mon Apr 10 23:33:46 2017
Archived Log entry 48 added for thread 1 sequence 72 ID 0x96bd0722 dest 1:
Mon Apr 10 23:33:59 2017
TT00: Standby redo logfile selected for thread 1 sequence 73 for destination LOG_ARCHIVE_DEST_2
Mon Apr 10 23:34:47 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:41:17 2017
Shutting down instance (immediate)
Stopping background process SMCO
Shutting down instance: further logons disabled
Stopping background process CJQ0
Stopping background process MMNL
Stopping background process MMON
License high water mark = 11
ALTER DATABASE CLOSE NORMAL
Mon Apr 10 23:41:21 2017
SMON: disabling tx recovery
Mon Apr 10 23:41:21 2017
Stopping Emon pool
Stopping Emon pool
Mon Apr 10 23:41:22 2017
SMON: disabling cache recovery
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:24 2017
ARCH shutting down
Mon Apr 10 23:41:24 2017
ARC2: Archival stopped
Mon Apr 10 23:41:24 2017
ARCH shutting down
Mon Apr 10 23:41:24 2017
ARC3: Archival stopped
Mon Apr 10 23:41:24 2017
ARCH shutting down
ARC0: Relinquishing active heartbeat ARCH role
ARC0: Archival stopped
Mon Apr 10 23:41:24 2017
ARCH shutting down
ARC1: Archival stopped
Mon Apr 10 23:41:25 2017
Thread 1 closed at log sequence 73
Successful close of redo thread 1
Completed: ALTER DATABASE CLOSE NORMAL
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:41:26 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:41:30 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:41:32 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:32 2017
Stopping background process VKTM
Mon Apr 10 23:41:35 2017
Instance shutdown complete
Mon Apr 10 23:41:44 2017
Starting ORACLE instance (normal) (OS id: 1281)
Mon Apr 10 23:41:44 2017
CLI notifier numLatches:3 maxDescs:354
Mon Apr 10 23:41:44 2017
Due to limited space in shared pool (need 6094848 bytes, have 4194112 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:41:44 2017
**********************************************************************
Mon Apr 10 23:41:44 2017
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)
Mon Apr 10 23:41:44 2017
Per process system memlock (soft) limit = 128G
Mon Apr 10 23:41:44 2017
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 802M
Mon Apr 10 23:41:44 2017
Available system pagesizes:
4K, 2048K
Mon Apr 10 23:41:44 2017
Supported system pagesize(s):
Mon Apr 10 23:41:44 2017
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
Mon Apr 10 23:41:44 2017
4K Configured 4 204804 NONE
Mon Apr 10 23:41:44 2017
2048K 0 401 0 NONE
Mon Apr 10 23:41:44 2017
RECOMMENDATION:
Mon Apr 10 23:41:44 2017
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
Mon Apr 10 23:41:44 2017
**********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 2
Number of processor cores in the system is 2
Number of processor sockets in the system is 1
_ksb_restart_policy_times={0,60,120,240}
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
IMODE=BR
ILAT =35
LICENSE_MAX_USERS = 0
SYS auditing is enabled
NOTE: remote asm mode is local (mode 0x1; from cluster type)
NOTE: Using default ASM root directory ASM
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1
System name: Linux
Node name: oem132.racattack
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Using parameter settings in server-side spfile /u01/app/oracle/product/12.1.0/dbhome_1/dbs/spfileSTBY.ora
System parameters with non-default values:
_ksb_restart_policy_times= "0"
_ksb_restart_policy_times= "60"
_ksb_restart_policy_times= "120"
_ksb_restart_policy_times= "240"
sga_target = 800M
control_files = "/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl"
control_files = "/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl"
db_file_name_convert = "+DATA/RAC"
db_file_name_convert = "/u01/app/oracle/oradata/STBY"
db_file_name_convert = "+FRA/RAC"
db_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"
log_file_name_convert = "+DATA/RAC"
log_file_name_convert = "/u01/app/oracle/oradata/STBY"
log_file_name_convert = "+FRA/RAC"
log_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"
compatible = "12.1.0.2.0"
log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
log_archive_dest_2 = "service="rac1""
log_archive_dest_2 = "ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="rac" net_timeout=30"
log_archive_dest_2 = "valid_for=(online_logfile,all_roles)"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
fal_server = "rac1"
log_archive_trace = 0
log_archive_config = "dg_config=(rac)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 4
log_buffer = 5064K
archive_lag_target = 0
db_create_file_dest = "/u01/app/oracle/oradata"
db_recovery_file_dest = "/u01/app/oracle/fast_recovery_area"
db_recovery_file_dest_size= 16G
standby_file_management = "MANUAL"
_compression_compatibility= "12.1.0.2.0"
remote_login_passwordfile= "EXCLUSIVE"
connection_brokers = "((TYPE=DEDICATED)(BROKERS=1))"
connection_brokers = "((TYPE=EMON)(BROKERS=1))"
plsql_warnings = "DISABLE:ALL"
result_cache_max_size = 4M
_gby_vector_aggregation_enabled= TRUE
_optimizer_vector_transformation= TRUE
core_dump_dest = "/u01/app/oracle/diag/rdbms/stby/STBY/cdump"
db_name = "RAC"
db_unique_name = "STBY"
optimizer_mode = "ALL_ROWS"
_optimizer_mode_force = TRUE
_always_anti_join = "CHOOSE"
_optimizer_null_aware_antijoin= TRUE
_optimizer_partial_join_eval= TRUE
_partition_view_enabled = TRUE
_b_tree_bitmap_plans = TRUE
_cpu_to_io = 0
_optimizer_extended_cursor_sharing= "UDO"
_optimizer_extended_cursor_sharing_rel= "SIMPLE"
_optimizer_adaptive_cursor_sharing= TRUE
_optimizer_cost_model = "CHOOSE"
_optimizer_undo_cost_change= "12.1.0.2"
_optimizer_system_stats_usage= TRUE
_new_sort_cost_estimate = TRUE
_complex_view_merging = TRUE
_unnest_subquery = TRUE
_optimizer_unnest_scalar_sq= TRUE
_eliminate_common_subexpr= TRUE
_pred_move_around = TRUE
_convert_set_to_join = FALSE
_px_ual_serial_input = TRUE
_px_minus_intersect = TRUE
_remove_aggr_subquery = TRUE
_optimizer_push_down_distinct= 0
_optimizer_cost_based_transformation= "LINEAR"
_optimizer_squ_bottomup = TRUE
_push_join_predicate = TRUE
_push_join_union_view = TRUE
_push_join_union_view2 = TRUE
_fast_full_scan_enabled = TRUE
_optimizer_skip_scan_enabled= TRUE
_optimizer_join_sel_sanity_check= TRUE
_optim_enhance_nnull_detection= TRUE
_parallel_broadcast_enabled= TRUE
_always_semi_join = "CHOOSE"
_ordered_nested_loop = TRUE
_optimizer_max_permutations= 2000
_nlj_batching_enabled = 1
query_rewrite_enabled = "TRUE"
_mmv_query_rewrite_enabled= TRUE
_local_communication_costing_enabled= TRUE
_left_nested_loops_random= TRUE
_improved_row_length_enabled= TRUE
_index_join_enabled = TRUE
_enable_type_dep_selectivity= TRUE
_improved_outerjoin_card = TRUE
_use_column_stats_for_function= TRUE
_subquery_pruning_enabled= TRUE
_subquery_pruning_mv_enabled= FALSE
_or_expand_nvl_predicate = TRUE
_table_scan_cost_plus_one= TRUE
_cost_equality_semi_join = TRUE
_new_initial_join_orders = TRUE
_optim_peek_user_binds = TRUE
_minimal_stats_aggregation= TRUE
_gs_anti_semi_join_allowed= TRUE
_optim_new_default_join_sel= TRUE
optimizer_dynamic_sampling= 2
_pre_rewrite_push_pred = TRUE
_optimizer_new_join_card_computation= TRUE
_union_rewrite_for_gs = "YES_GSET_MVS"
_generalized_pruning_enabled= TRUE
_optim_adjust_for_part_skews= TRUE
_optimizer_compute_index_stats= TRUE
_optimizer_push_pred_cost_based= TRUE
_optimizer_extend_jppd_view_types= TRUE
_optimizer_filter_pred_pullup= TRUE
_optimizer_connect_by_cost_based= TRUE
_optimizer_connect_by_combine_sw= TRUE
_optimizer_connect_by_elim_dups= TRUE
_connect_by_use_union_all= "TRUE"
_right_outer_hash_enable = TRUE
_replace_virtual_columns = TRUE
skip_unusable_indexes = TRUE
_optimizer_correct_sq_selectivity= TRUE
_optimizer_dim_subq_join_sel= TRUE
_query_rewrite_setopgrw_enable= TRUE
_optimizer_join_order_control= 3
_bloom_filter_enabled = TRUE
_bloom_folding_enabled = TRUE
_bloom_serial_filter = "ON"
_optimizer_join_elimination_enabled= TRUE
_gby_hash_aggregation_enabled= TRUE
_globalindex_pnum_filter_enabled= TRUE
_sql_model_unfold_forloops= "RUN_TIME"
_optimizer_cost_hjsmj_multimatch= TRUE
_optimizer_transitivity_retain= TRUE
_px_pwg_enabled = TRUE
_optimizer_cbqt_no_size_restriction= TRUE
_optimizer_enhanced_filter_push= TRUE
_optimizer_rownum_pred_based_fkr= TRUE
_optimizer_better_inlist_costing= "ALL"
_optimizer_or_expansion = "DEPTH"
_optimizer_outer_to_anti_enabled= TRUE
_optimizer_order_by_elimination_enabled= TRUE
_optimizer_star_tran_in_with_clause= TRUE
_optimizer_sortmerge_join_inequality= TRUE
_selfjoin_mv_duplicates = TRUE
_dimension_skip_null = TRUE
_optimizer_complex_pred_selectivity= TRUE
_bloom_pruning_enabled = TRUE
_first_k_rows_dynamic_proration= TRUE
_optimizer_distinct_elimination= TRUE
_optimizer_multi_level_push_pred= TRUE
_optimizer_group_by_placement= TRUE
_optimizer_distinct_placement= TRUE
_optimizer_coalesce_subqueries= TRUE
_optimizer_enable_density_improvements= TRUE
_optimizer_rownum_bind_default= 10
_optimizer_improve_selectivity= TRUE
_optimizer_native_full_outer_join= "FORCE"
_optimizer_ansi_join_lateral_enhance= TRUE
_optimizer_multi_table_outerjoin= TRUE
_optimizer_null_accepting_semijoin= TRUE
_optimizer_ansi_rearchitecture= TRUE
_optimizer_aggr_groupby_elim= TRUE
_optimizer_enable_extended_stats= TRUE
_pivot_implementation_method= "CHOOSE"
_optimizer_interleave_jppd= TRUE
_optimizer_fkr_index_cost_bias= 10
_optimizer_extended_stats_usage_control= 192
_optimizer_fast_pred_transitivity= TRUE
_optimizer_fast_access_pred_analysis= TRUE
_optimizer_unnest_disjunctive_subq= TRUE
_optimizer_unnest_corr_set_subq= TRUE
_optimizer_distinct_agg_transform= TRUE
_aggregation_optimization_settings= 0
_optimizer_eliminate_filtering_join= TRUE
_optimizer_join_factorization= TRUE
_optimizer_use_cbqt_star_transformation= TRUE
_optimizer_table_expansion= TRUE
_and_pruning_enabled = TRUE
_optimizer_use_feedback = TRUE
_optimizer_gather_feedback= TRUE
_optimizer_try_st_before_jppd= TRUE
_px_partition_scan_enabled= TRUE
_optimizer_false_filter_pred_pullup= TRUE
_optimizer_enable_table_lookup_by_nl= TRUE
_optimizer_cube_join_enabled= TRUE
_optimizer_outer_join_to_inner= TRUE
_optimizer_hybrid_fpwj_enabled= TRUE
_px_object_sampling_enabled= TRUE
_px_concurrent = TRUE
_px_replication_enabled = TRUE
_optimizer_full_outer_join_to_outer= TRUE
_px_filter_parallelized = TRUE
_px_filter_skew_handling = TRUE
_px_groupby_pushdown = "FORCE"
_px_parallelize_expression= TRUE
_optimizer_gather_stats_on_load= TRUE
_optimizer_batch_table_access_by_rowid= TRUE
_px_wif_dfo_declumping = "CHOOSE"
_px_wif_extend_distribution_keys= TRUE
_px_join_skew_handling = TRUE
_px_adaptive_dist_method = "CHOOSE"
_px_partial_rollup_pushdown= "ADAPTIVE"
_optimizer_dsdir_usage_control= 126
_px_cpu_autodop_enabled = TRUE
_px_single_server_enabled= TRUE
_optimizer_use_gtt_session_stats= TRUE
_optimizer_adaptive_plans= TRUE
_optimizer_strans_adaptive_pruning= TRUE
_optimizer_proc_rate_level= "BASIC"
_optimizer_use_histograms= TRUE
_adaptive_window_consolidator_enabled= TRUE
_optimizer_cluster_by_rowid= TRUE
_optimizer_cluster_by_rowid_control= 129
_distinct_agg_optimization_gsets= "CHOOSE"
_px_scalable_invdist = TRUE
_optimizer_reduce_groupby_key= TRUE
_optimizer_cluster_by_rowid_batched= TRUE
_optimizer_inmemory_table_expansion= TRUE
_optimizer_inmemory_gen_pushable_preds= TRUE
_optimizer_inmemory_autodop= TRUE
_optimizer_inmemory_access_path= TRUE
_px_external_table_default_stats= TRUE
_optimizer_nlj_hj_adaptive_join= TRUE
_optimizer_inmemory_bloom_filter= TRUE
_optimizer_inmemory_cluster_aware_dop= TRUE
_optimizer_inmemory_minmax_pruning= TRUE
dg_broker_start = TRUE
_diag_adr_trace_dest = "/u01/app/oracle/diag/rdbms/stby/STBY/trace"
enable_pluggable_database= TRUE
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Starting background process PMON
Mon Apr 10 23:41:46 2017
PMON started with pid=2, OS id=1315
Starting background process PSP0
Mon Apr 10 23:41:46 2017
PSP0 started with pid=3, OS id=1317
Starting background process VKTM
Starting background process GEN0
Mon Apr 10 23:41:47 2017
VKTM started with pid=4, OS id=1321 at elevated (RT) priority
Mon Apr 10 23:41:47 2017
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Mon Apr 10 23:41:47 2017
GEN0 started with pid=5, OS id=1327
Starting background process MMAN
Mon Apr 10 23:41:47 2017
MMAN started with pid=6, OS id=1329
Starting background process DIAG
Starting background process DBRM
Mon Apr 10 23:41:47 2017
DIAG started with pid=8, OS id=1333
Starting background process VKRM
Mon Apr 10 23:41:47 2017
DBRM started with pid=9, OS id=1335
Starting background process DIA0
Mon Apr 10 23:41:47 2017
VKRM started with pid=10, OS id=1337
Starting background process DBW0
Mon Apr 10 23:41:47 2017
DIA0 started with pid=11, OS id=1339
Starting background process LGWR
Mon Apr 10 23:41:47 2017
DBW0 started with pid=12, OS id=1342
Starting background process CKPT
Mon Apr 10 23:41:47 2017
LGWR started with pid=13, OS id=1344
Mon Apr 10 23:41:47 2017
CKPT started with pid=14, OS id=1347
Starting background process SMON
Mon Apr 10 23:41:47 2017
SMON started with pid=16, OS id=1355
Starting background process RECO
Starting background process LREG
Mon Apr 10 23:41:47 2017
RECO started with pid=18, OS id=1359
Starting background process PXMN
Mon Apr 10 23:41:47 2017
LREG started with pid=19, OS id=1361
Starting background process MMON
Mon Apr 10 23:41:47 2017
PXMN started with pid=20, OS id=1363
Starting background process MMNL
Mon Apr 10 23:41:47 2017
MMON started with pid=21, OS id=1365
Starting background process DMON
Mon Apr 10 23:41:47 2017
MMNL started with pid=22, OS id=1367
ORACLE_BASE from environment = /u01/app/oracle
Mon Apr 10 23:41:47 2017
DMON started with pid=23, OS id=1369
Mon Apr 10 23:41:47 2017
ALTER DATABASE MOUNT
Mon Apr 10 23:41:50 2017
Using default pga_aggregate_limit of 2048 MB
Mon Apr 10 23:41:52 2017
Successful mount of redo thread 1, with mount id 2528976638
Mon Apr 10 23:41:52 2017
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE MOUNT
Mon Apr 10 23:41:55 2017
ALTER DATABASE OPEN
Mon Apr 10 23:41:55 2017
Data Guard Broker initializing...
Mon Apr 10 23:41:55 2017
Starting Data Guard Broker (DMON)
Starting background process INSV
Mon Apr 10 23:41:55 2017
INSV started with pid=24, OS id=1402
Mon Apr 10 23:41:56 2017
Data Guard Broker initialization complete
Data Guard: verifying database primary role...
Starting background process NSV0
Mon Apr 10 23:41:58 2017
NSV0 started with pid=25, OS id=1410
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Starting background process RSM0
Mon Apr 10 23:42:11 2017
RSM0 started with pid=26, OS id=1452
Mon Apr 10 23:42:12 2017
Data Guard: version check completed
Data Guard: primary role verified
Ping without log force is disabled
.
Starting background process TMON
Mon Apr 10 23:42:12 2017
LGWR: STARTING ARCH PROCESSES
Starting background process ARC0
Mon Apr 10 23:42:12 2017
TMON started with pid=27, OS id=1456
Mon Apr 10 23:42:12 2017
ARC0 started with pid=28, OS id=1460
ARC0: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:42:12 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC1
Mon Apr 10 23:42:12 2017
ARC1 started with pid=29, OS id=1462
Starting background process ARC2
Starting background process ARC3
Mon Apr 10 23:42:13 2017
ARC2 started with pid=30, OS id=1464
Mon Apr 10 23:42:13 2017
ARC3 started with pid=31, OS id=1466
ARC1: Archival started
ARC2: Archival started
Mon Apr 10 23:42:13 2017
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
Mon Apr 10 23:42:13 2017
ARC2: Becoming the heartbeat ARCH
Mon Apr 10 23:42:13 2017
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:42:13 2017
Thread 1 advanced to log sequence 74 (thread open)
Thread 1 opened at log sequence 74
Current log# 2 seq# 74 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283
Current log# 2 seq# 74 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287
Successful open of redo thread 1
Mon Apr 10 23:42:13 2017
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Apr 10 23:42:13 2017
SMON: enabling cache recovery
Mon Apr 10 23:42:13 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Error 12528 received logging on to the standby
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Error 12528 for archive log file 2 to 'rac1'
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:13 2017
Archived Log entry 52 added for thread 1 sequence 73 ID 0x96bd0722 dest 1:
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Error 12528 received logging on to the standby
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc0_1460.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
FAL[server, ARC0]: Error 12528 creating remote archivelog file 'rac1'
ARC0: FAL archive failed with error 12528. See trace for details
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc0_1460.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Mon Apr 10 23:42:13 2017
ORACLE Instance STBY - Archival Error. Archiver continuing.
Mon Apr 10 23:42:14 2017
[1400] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:4809484 end:4810074 diff:590 ms (0.6 seconds)
Verifying minimum file header compatibility (11g) for tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
Mon Apr 10 23:42:14 2017
SMON: enabling tx recovery
Starting background process SMCO
Mon Apr 10 23:42:14 2017
SMCO started with pid=34, OS id=1474
Mon Apr 10 23:42:14 2017
Database Characterset is AL32UTF8
No Resource Manager plan active
Mon Apr 10 23:42:16 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:42:16 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Error 12528 received logging on to the standby
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Mon Apr 10 23:42:16 2017
Error 12528 received logging on to the standby
Mon Apr 10 23:42:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc2_1464.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
PING[ARC2]: Heartbeat failed to connect to standby 'rac1'. Error is 12528.
Mon Apr 10 23:42:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:16 2017
Error 12528 for archive log file 2 to 'rac1'
Mon Apr 10 23:42:16 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
***********************************************************************
Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Mon Apr 10 23:42:16 2017
Error 12528 received logging on to the standby
Mon Apr 10 23:42:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc2_1464.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
PING[ARC2]: Heartbeat failed to connect to standby 'rac1'. Error is 12528.
Mon Apr 10 23:42:17 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='STBY';
Mon Apr 10 23:42:17 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:17 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:17 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='STBY';
Mon Apr 10 23:42:17 2017
replication_dependency_tracking turned off (no async multimaster replication found)
Mon Apr 10 23:42:18 2017
Thread 1 advanced to log sequence 75 (LGWR switch)
Current log# 1 seq# 75 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279
Current log# 1 seq# 75 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281
Mon Apr 10 23:42:18 2017
Archived Log entry 53 added for thread 1 sequence 74 ID 0x96bd0722 dest 1:
Starting background process AQPC
Mon Apr 10 23:42:18 2017
AQPC started with pid=37, OS id=1489
Mon Apr 10 23:42:21 2017
Database Characterset for PDB$SEED is AL32UTF8
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Opening pdb PDB$SEED (2) with no Resource Manager plan active
Mon Apr 10 23:42:35 2017
ALTER SYSTEM SET log_archive_dest_state_2='RESET' SCOPE=BOTH;
Starting background process CJQ0
Completed: ALTER DATABASE OPEN
Mon Apr 10 23:42:39 2017
CJQ0 started with pid=50, OS id=1904
Mon Apr 10 23:42:58 2017
Shared IO Pool defaulting to 36MB. Trying to get it from Buffer Cache for process 1365.
===========================================================
Dumping current patch information
===========================================================
No patches have been applied
===========================================================
Mon Apr 10 23:42:59 2017
db_recovery_file_dest_size of 16384 MB is 0.84% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
alter pluggable database pdb open
Mon Apr 10 23:43:00 2017
Database Characterset for PDB is AL32UTF8
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:43:15 2017
Opening pdb PDB (3) with no Resource Manager plan active
Pluggable database PDB opened read write
Completed: alter pluggable database pdb open
Mon Apr 10 23:45:07 2017
alter pluggable database pdb close
ALTER SYSTEM: Flushing buffer cache inst=0 container=3 local
Mon Apr 10 23:45:12 2017
Pluggable database PDB closed
Completed: alter pluggable database pdb close
Shutting down instance (immediate)
Stopping background process SMCO
Shutting down instance: further logons disabled
Stopping background process CJQ0
Killed process oracle@oem132.racattack (Q002) with pid is 48, OS pid 1537
Stopping background process MMNL
Stopping background process MMON
License high water mark = 9
ALTER DATABASE CLOSE NORMAL
Mon Apr 10 23:45:20 2017
SMON: disabling tx recovery
Mon Apr 10 23:45:20 2017
Stopping Emon pool
Stopping Emon pool
Mon Apr 10 23:45:20 2017
SMON: disabling cache recovery
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:45:24 2017
ARCH shutting down
Mon Apr 10 23:45:24 2017
ARCH shutting down
ARC2: Archival stopped
Mon Apr 10 23:45:24 2017
ARC3: Archival stopped
Mon Apr 10 23:45:24 2017
ARCH shutting down
Mon Apr 10 23:45:24 2017
Mon Apr 10 23:45:24 2017
ARC1: Archival stoppedARCH shutting down
Mon Apr 10 23:45:24 2017
ARC0: Archival stopped
Mon Apr 10 23:45:25 2017
Thread 1 closed at log sequence 75
Successful close of redo thread 1
Completed: ALTER DATABASE CLOSE NORMAL
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:45:27 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:45:31 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:45:33 2017
Mon Apr 10 23:45:33 2017
ARCH: Archival disabled due to shutdown: 1089
Stopping background process VKTM
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:45:41 2017
Instance shutdown complete
Ignore the Fatal NI and TNS errrors. I plan to address them later.
.
.
.
1 comment:
Post a Comment