03 September, 2011

Understanding Obsolescence of RMAN Backups

Questions about the the LIST OBSOLETE command keep appearing in RMAN forums.

Given this configuration :
RMAN> show all;

RMAN configuration parameters for database with db_unique_name ORCL are:
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 15 DAYS;
CONFIGURE BACKUP OPTIMIZATION OFF;
CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
CONFIGURE CONTROLFILE AUTOBACKUP ON;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
CONFIGURE DEVICE TYPE DISK PARALLELISM 2 BACKUP TYPE TO BACKUPSET;
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
CONFIGURE COMPRESSION ALGORITHM 'BASIC' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD TRUE ; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/home/oracle/app/oracle/product/11.2.0/dbhome_1/dbs/snapcf_orcl.f'; # default

RMAN>

which indicates that my Recovery Window is set to 15 days.

And given this listing of backups :
RMAN> list backup summary;


List of Backups
===============
Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag
------- -- -- - ----------- --------------- ------- ------- ---------- ---
130 B F A DISK 31-MAY-11 1 1 YES TAG20110531T224447
132 B A A DISK 31-MAY-11 1 1 YES TAG20110531T224945
133 B F A DISK 31-MAY-11 1 1 NO TAG20110531T224947
134 B F A DISK 05-JUL-11 1 1 YES TAG20110705T230230
135 B F A DISK 05-JUL-11 1 1 YES TAG20110705T230230
136 B F A DISK 05-JUL-11 1 1 NO TAG20110705T230640
137 B F A DISK 13-JUL-11 1 1 NO TAG20110713T224918
138 B F A DISK 03-SEP-11 1 1 NO TAG20110903T232648
139 B F A DISK 03-SEP-11 1 1 YES TAG20110903T234602
140 B F A DISK 03-SEP-11 1 1 YES TAG20110903T234602
141 B F A DISK 03-SEP-11 1 1 NO TAG20110903T235631

RMAN>


which backups do you expect to be OBSOLETE ?

.
.
.
.














.
.
.
Why are these reported as OBSOLETE ?
RMAN> report obsolete;

RMAN retention policy will be applied to the command
RMAN retention policy is set to recovery window of 15 days
Report of obsolete backups and copies
Type Key Completion Time Filename/Handle
-------------------- ------ ------------------ --------------------
Backup Set 130 31-MAY-11
Backup Piece 130 31-MAY-11 /addtl/oracle/flash_recovery_area/ORCL/backupset/2011_05_31/o1_mf_nnndf_TAG20110531T224447_6y9zq1fr_.bkp
Backup Set 132 31-MAY-11
Backup Piece 132 31-MAY-11 /addtl/oracle/flash_recovery_area/ORCL/backupset/2011_05_31/o1_mf_annnn_TAG20110531T224945_6yb009xg_.bkp
Backup Set 133 31-MAY-11
Backup Piece 133 31-MAY-11 /addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_05_31/o1_mf_s_752626187_6yb00d1h_.bkp
Backup Set 136 05-JUL-11
Backup Piece 136 05-JUL-11 /addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_07_05/o1_mf_s_755737600_716b41kb_.bkp

RMAN>

Why are the Backups in Key 134 and 135 (both on 05-Jul-11) not OBSOLETE ?
(and yet, Key 136 is OBSOLETE)
Why is the Backup of 13-Jul-11 not OBSOLETE ?

.
.
.

UPDATE : 05-Sep-11 :

Here is some more information :
sh-3.2$ pwd
/addtl/oracle/flash_recovery_area/ORCL
sh-3.2$ du -sk */*
4 archivelog/2011_08_30
4 archivelog/2011_09_01
4 archivelog/2011_09_02
4 archivelog/2011_09_03
28884 archivelog/2011_09_04
11156 autobackup/2011_04_22
11156 autobackup/2011_05_31
11156 autobackup/2011_07_05
11156 autobackup/2011_07_13
22308 autobackup/2011_09_03
11156 autobackup/2011_09_04
1276 backupset/2011_05_31
677584 backupset/2011_07_05
2951688 backupset/2011_09_03
6400 backupset/2011_09_04
sh-3.2$
sh-3.2$ rman target /

Recovery Manager: Release 11.2.0.1.0 - Production on Mon Sep 5 22:32:18 2011

Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.

connected to target database: ORCL (DBID=1229390655)

RMAN> crosscheck backup;

using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=38 device type=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: SID=44 device type=DISK
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_05_31/o1_mf_nnndf_TAG20110531T224447_6y9zq1fr_.bkp RECID=130 STAMP=752625889
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_05_31/o1_mf_annnn_TAG20110531T224945_6yb009xg_.bkp RECID=132 STAMP=752626185
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_05_31/o1_mf_s_752626187_6yb00d1h_.bkp RECID=133 STAMP=752626188
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_07_05/o1_mf_nnndf_TAG20110705T230230_7169w9t3_.bkp RECID=134 STAMP=755737353
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_07_05/o1_mf_nnndf_TAG20110705T230230_7169w8s7_.bkp RECID=135 STAMP=755737352
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_07_05/o1_mf_s_755737600_716b41kb_.bkp RECID=136 STAMP=755737601
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_07_13/o1_mf_s_756427758_71vd3h8n_.bkp RECID=137 STAMP=756427759
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_09_03/o1_mf_s_760922808_764kssdh_.bkp RECID=138 STAMP=760922809
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_09_03/o1_mf_nnndf_TAG20110903T234602_764lxwdb_.bkp RECID=139 STAMP=760923964
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_09_03/o1_mf_nnndf_TAG20110903T234602_764lxvd6_.bkp RECID=140 STAMP=760923963
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_09_03/o1_mf_s_760924591_764mkhww_.bkp RECID=141 STAMP=760924591
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_09_04/o1_mf_annnn_TAG20110904T001454_764nmzy4_.bkp RECID=142 STAMP=760925695
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_09_04/o1_mf_annnn_TAG20110904T001454_764nmzf8_.bkp RECID=143 STAMP=760925695
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_09_04/o1_mf_s_760925699_764nn3xy_.bkp RECID=144 STAMP=760925699
Crosschecked 14 objects


RMAN>


The Backups of 31-May-11 and 05-July-11 are present on disk and found to be AVAILABLE when I run CROSSCHECK.
Also : I have not marked any Backup for KEEPing.


You will also note that RMAN is listing backups that are much older than the control_file_record_keep_time which is 7 days.
SQL> show parameter control_file_record

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
control_file_record_keep_time integer 7
SQL>


UPDATE 07-Sep-11 : Answers

BackupSet 136 is OBSOLETEd because it is a Controlfile AutoBackup. (The listing of LIST BACKUP SUMMARY shows that it is not a Compressed Backup-- controfile autobackups are not compressed -- and the REPORT OBSOLETE shows that it is an autobackup piece).
BackupSets 134 and 135 are *Database* (i.e. Datafile) backups. These are not obsolete because they comprise the two BackupSets of the most recent BACKUP DATABASE run just before the Retention Policy obsolescence date. As of 03-Sep, the Retention Policy of 15 days sets a date of 19-Aug. There was no database backup on 18-Aug or 17-Aug or 16-Aug ... so Oracle works backwards and finds the database backup of 05-July in the BackupSets of 134 and 135.

Why is it two BackupSets ? Because I have configured PARALLELISM 2. Every BACKUP DATABASE command creates two BackupSets. So the run on 05-July created two BackupSets.

What about the BackupSet (137) of 13-July ? It, too, is a controlfile autobackup. This is the most recent controlfile autobackup that I have to match the obsolescence date. So, this canot be OBSOLETEd.

Also note that backups much older than 7 days (i.e. exceeding the control_file_record_keep_time) are still "maintained" by RMAN -- some of them (e.g. those of 31-May and the controlfile autobackup of 05-July) are OBSOLETE and yet still listed. They will continue to be listed until I DELETE them.

Even now, on 07-Sep, the backups are still present :

[oracle@localhost ORCL]$ pwd
/addtl/oracle/flash_recovery_area/ORCL
[oracle@localhost ORCL]$ du -sk */*
4 archivelog/2011_08_30
4 archivelog/2011_09_01
4 archivelog/2011_09_02
4 archivelog/2011_09_03
28884 archivelog/2011_09_04
26268 archivelog/2011_09_07
11156 autobackup/2011_04_22
11156 autobackup/2011_05_31
11156 autobackup/2011_07_05
11156 autobackup/2011_07_13
22308 autobackup/2011_09_03
11156 autobackup/2011_09_04
1276 backupset/2011_05_31
677584 backupset/2011_07_05
2951688 backupset/2011_09_03
6400 backupset/2011_09_04
[oracle@localhost ORCL]$

[oracle@localhost ORCL]$ rman target /

Recovery Manager: Release 11.2.0.1.0 - Production on Wed Sep 7 22:33:33 2011

Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.

connected to target database: ORCL (DBID=1229390655)

RMAN> crosscheck backup ;

using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=46 device type=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: SID=47 device type=DISK
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_05_31/o1_mf_nnndf_TAG20110531T224447_6y9zq1fr_.bkp RECID=130 STAMP=752625889
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_05_31/o1_mf_annnn_TAG20110531T224945_6yb009xg_.bkp RECID=132 STAMP=752626185
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_05_31/o1_mf_s_752626187_6yb00d1h_.bkp RECID=133 STAMP=752626188
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_07_05/o1_mf_nnndf_TAG20110705T230230_7169w9t3_.bkp RECID=134 STAMP=755737353
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_07_05/o1_mf_nnndf_TAG20110705T230230_7169w8s7_.bkp RECID=135 STAMP=755737352
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_07_05/o1_mf_s_755737600_716b41kb_.bkp RECID=136 STAMP=755737601
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_07_13/o1_mf_s_756427758_71vd3h8n_.bkp RECID=137 STAMP=756427759
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_09_03/o1_mf_s_760922808_764kssdh_.bkp RECID=138 STAMP=760922809
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_09_03/o1_mf_nnndf_TAG20110903T234602_764lxwdb_.bkp RECID=139 STAMP=760923964
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_09_03/o1_mf_nnndf_TAG20110903T234602_764lxvd6_.bkp RECID=140 STAMP=760923963
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_09_03/o1_mf_s_760924591_764mkhww_.bkp RECID=141 STAMP=760924591
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_09_04/o1_mf_annnn_TAG20110904T001454_764nmzy4_.bkp RECID=142 STAMP=760925695
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/backupset/2011_09_04/o1_mf_annnn_TAG20110904T001454_764nmzf8_.bkp RECID=143 STAMP=760925695
crosschecked backup piece: found to be 'AVAILABLE'
backup piece handle=/addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_09_04/o1_mf_s_760925699_764nn3xy_.bkp RECID=144 STAMP=760925699
Crosschecked 14 objects


RMAN>

RMAN> list backup summary;


List of Backups
===============
Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag
------- -- -- - ----------- --------------- ------- ------- ---------- ---
130 B F A DISK 31-MAY-11 1 1 YES TAG20110531T224447
132 B A A DISK 31-MAY-11 1 1 YES TAG20110531T224945
133 B F A DISK 31-MAY-11 1 1 NO TAG20110531T224947
134 B F A DISK 05-JUL-11 1 1 YES TAG20110705T230230
135 B F A DISK 05-JUL-11 1 1 YES TAG20110705T230230
136 B F A DISK 05-JUL-11 1 1 NO TAG20110705T230640
137 B F A DISK 13-JUL-11 1 1 NO TAG20110713T224918
138 B F A DISK 03-SEP-11 1 1 NO TAG20110903T232648
139 B F A DISK 03-SEP-11 1 1 YES TAG20110903T234602
140 B F A DISK 03-SEP-11 1 1 YES TAG20110903T234602
141 B F A DISK 03-SEP-11 1 1 NO TAG20110903T235631
142 B A A DISK 04-SEP-11 1 1 YES TAG20110904T001454
143 B A A DISK 04-SEP-11 1 1 YES TAG20110904T001454
144 B F A DISK 04-SEP-11 1 1 NO TAG20110904T001459

RMAN> report obsolete;

RMAN retention policy will be applied to the command
RMAN retention policy is set to recovery window of 15 days
Report of obsolete backups and copies
Type Key Completion Time Filename/Handle
-------------------- ------ ------------------ --------------------
Backup Set 130 31-MAY-11
Backup Piece 130 31-MAY-11 /addtl/oracle/flash_recovery_area/ORCL/backupset/2011_05_31/o1_mf_nnndf_TAG20110531T224447_6y9zq1fr_.bkp
Backup Set 132 31-MAY-11
Backup Piece 132 31-MAY-11 /addtl/oracle/flash_recovery_area/ORCL/backupset/2011_05_31/o1_mf_annnn_TAG20110531T224945_6yb009xg_.bkp
Backup Set 133 31-MAY-11
Backup Piece 133 31-MAY-11 /addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_05_31/o1_mf_s_752626187_6yb00d1h_.bkp
Backup Set 136 05-JUL-11
Backup Piece 136 05-JUL-11 /addtl/oracle/flash_recovery_area/ORCL/autobackup/2011_07_05/o1_mf_s_755737600_716b41kb_.bkp

RMAN>



So there you have it :

1. CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 15 DAYS does not necessarily mean that every backup more than 16 days old is obosleted. Every backup older than the most recent backup before 15 days ago is obsoleted. If the most recent backup before 15 days from now is 62 days ago (05-July) so be it. It cannot be OBSOLETEd.

2. control_file_record_keep_time of 7 days does not necessarily mean that backups older than 7 days are "purged" from the controlfile or "unknown" to RMAN. I show some backups (of 31-May and 05-July) that are Obsoleted and yet maintained in the controlfile.

.
.
.

11 comments:

Naresh said...

Firstly

CROSSCHECK YOUR backup.

crosscheck backup,
crosscheck archivelog

Narendra said...

Is it because those backups are marked to not become obsolete (i.e. KEEP option) ?

Hemant K Chitale said...

Narendra,
See my updates.
All the backups are AVAILABLE.
None are marked for KEEPing.
The backups shown are much older than the control_file_record_keep_time.

Hemant

Robert said...

Hi
I understand your explanation about retention window and how RMAN could keep an extra day to cover its retention window.
I have a situation where my retention window is 3 days but still it keeps backupsets one month back. When I do crosscheck backupset all my backups are available. I would have expected obsolete and expired so I can’t understand why it needs to keep all of them.
Do you have any ideas why my environment is not following its retentin window.
Its running on 11g.

Narendra said...

I might be shooting in the dark here but is it possible that backups 134 and 135 are level 0 backups and hence are needed for recovery (and hence are not reported as obsolete)?

Pierre Forstmann said...

FRA autobackup directory contains control files and SPFILE backups.
I think that 15-JUL control files autobackup is kept because recovery window is set to 15 days, your control files has only a 7 days retention and this control file auto backup is the more recent backup to make control file data available for the 15 day recovery window.

Hemant K Chitale said...

Narendra,
Yes you are correct. BackupSets 134 and 135 are BackupSets of a Full Backup (done with BACKUP AS COMPRESSED BACKUPSET DATABASE).

Hemant

Hemant K Chitale said...

Pierre,
Yes, you are correct. The BackupSet of 13-July is a controlfile autobackup and is the most relevant one to be preserved.

Hemant

Hemant K Chitale said...

Robert,
When you do a CROSSCHECK, backups are EXPIRED *if* they are not found to be present. e.g. if the default DEVICE is DISK, the CROSSCHECK searches for the backups on disk, in the expected location.
If you have not DELETED them (either using an OS command or using RMAN DELETE) and they are still present, they are not marked as EXPIRED.

As for why they are not OBSOLETE ... it could be that some files are required for recovery -- e.g. datafiles/tablespaces being taken offline/online or database in BEGIN BACKUP mode -- or that some BackupSets are marked for KEEPing.

Hemant

Ahmed AANGOUR said...

Hi Hemant,

Thank you for this very interesting article.
I'm facing an issue concerning Obsolescence of my RMAN backups.
I wrote a post on OTN forum yesterday where I describe my problem.
Could you please have a look into it? I would appreciate to get your opinion.

Ahmed AANGOUR said...

I forgot to post the OTN link:
https://community.oracle.com/message/12447875#12447875