07 May, 2019

Partitioning -- 16 : Hybrid Partitioning

Oracle 19c introduces Hybrid Partitioning whereby you can have external and internal Partitions co-existing.  External Partitions are on storage (filesystem) outside the database.

Let's say we have a List Partitioned table for the widgets that we manufacture. The table is Partitioned by WIDGET_CLASS_ID, based on an ISO standard.  So all companies that manufacture widgets adopt the same WIDGET_CLASS_ID:

SQL> desc widgets_list
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 WIDGET_CLASS_ID                                    VARCHAR2(5)
 WIDGET_ID                                          VARCHAR2(32)
 WIDGET_NAME                                        VARCHAR2(32)
 WIDGET_DESCRIPTION                                 VARCHAR2(128)

SQL>
SQL> l
  1  select table_name, partitioning_type, partition_count
  2  from user_part_tables
  3* where table_name = 'WIDGETS_LIST'
SQL> /

TABLE_NAME                       PARTITION PARTITION_COUNT
-------------------------------- --------- ---------------
WIDGETS_LIST                     LIST                    3

SQL>
SQL> l
  1  select partition_name,high_value, num_rows
  2  from user_tab_partitions
  3* where table_name = 'WIDGETS_LIST'
SQL> /

PARTITION_NAME   HIGH_VALUE         NUM_ROWS
---------------- ---------------- ----------
P_A              'A'                    1520
P_B              'B'                     520
P_C              'C'                     119

SQL>


Later, another widget manufacturer that manufactures widgets of CLASS_ID 'X' is acquired.  The WIDGETS_LIST table is in a non-Oracle database and is received as a CSV file.  We accept the CSV file onto a filesystem location :

sh-4.2$ pwd
/home/oracle/ACQUIRED_COMPANY
sh-4.2$ cat AC_Widgets_List.CSV
'X','ABCXX2','The1','cddfdaxx'
'X','XXD2','The2','dda3'
'X','XRC34','The3','ff33355312'
sh-4.2$


So, we have a CSV file "AC_Widgets_List.CSV" listing the widgets manufactured by this company. We want to add it to our WIDGETS_LIST table.

Enter user-name: / as sysdba

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0

SQL> alter session set container=ORCLPDB1;

Session altered.

SQL> create directory acquired_company as '/home/oracle/ACQUIRED_COMPANY';

Directory created.

SQL> grant read, write on directory acquired_company to hemant;

Grant succeeded.

SQL>
SQL> connect hemant/hemant@ORCLPDB1
Connected.
SQL>
SQL> l
  1  alter table widgets_list
  2  add external partition attributes (
  3  type oracle_loader
  4  default directory acquired_company
  5  access parameters (
  6  fields terminated by ','
  7  (widget_class_id, widget_id, widget_name, widget_description)
  8  )
  9* )
SQL> /

Table altered.

SQL>
SQL> l
  1  alter table widgets_list
  2  add partition P_ACQ_CO values ('X')
  3* external location ('AC_Widgets_List.CSV')
SQL> /

Table altered.

SQL>
SQL> exec dbms_stats.gather_table_stats('','WIDGETS_LIST');

PL/SQL procedure successfully completed.

SQL>
SQL> l
  1  select partition_name, high_value, num_rows
  2  from user_tab_partitions
  3  where table_name = 'WIDGETS_LIST'
  4* order by partition_position
SQL> /

PARTITION_NAME                   HIGH_VALUE         NUM_ROWS
-------------------------------- ---------------- ----------
P_A                              'A'                    1520
P_B                              'B'                     520
P_C                              'C'                     119
P_ACQ_CO                         'X'                       3

SQL>
SQL> l
  1* select * from widgets_list partition (P_ACQ_CO)
SQL> /

WIDGET_CLASS WIDGET_ID                        WIDGET_NAME
------------ -------------------------------- --------------------------------
WIDGET_DESCRIPTION
--------------------------------------------------------------------------------
'X'          'ABCXX2'                         'The1'
'cddfdaxx'

'X'          'XXD2'                           'The2'
'dda3'

'X'          'XRC34'                          'The3'
'ff33355312'


SQL>


The rows in the "AC_Widgets_List.CSV" file are now visible as rows in a *Partition* in our Oracle Table WIDGETS_LIST.
Of course, these being external, cannot be modified by INSERT/UPDATE/DELETE DML.

The External Attribute Type that I used is ORACLE_LOADER to use the SQL Loader libraries on a filesystem file.  Oracle 19c also supports ORACLE_DATAPUMP, ORACLE_HDFS and ORACLE_HIVE to reference files stored in other types of storage.

Hybrid Partitions are supported with single-level Range and List partitioning methods.  ALTER TABLE to ADD, DROP and RENAME Partitions is supported.

An External Partition can be Exchanged with an External Non-Partitioned Table only.
.
.
UPDATE :  Later, if I update the CSV file (using an external editor) to remove the quotation mark :

sh-4.2$ cat AC_Widgets_List.CSV
X,ABCXX2,The1,cddfdaxx
X,XXD2,The2,dda3
X,XRC34,The3,ff33355312
sh-4.2$

SQL> l
  1* select * from widgets_list partition (P_ACQ_CO)
SQL> /

WIDGET_CLASS_ID  WIDGET_ID                        WIDGET_NAME
---------------- -------------------------------- --------------------------------
WIDGET_DESCRIPTION
------------------------------------------------------------------------------------
X                ABCXX2                           The1
cddfdaxx

X                XXD2                             The2
dda3

X                XRC34                            The3
ff33355312


SQL>


So, it is possible to edit the External Partition using other methods (here I used "vi" on Linux)
.
.
.

08 April, 2019

Partitioning -- 15 : Online Modification of Partitioning Type (Strategy)

Oracle 18c introduces the ability to convert a Partitioned Table from one Type to another -- e.g. from Hash Partitioning to Range Partitioning.  This is effectively a change of the Partitioning strategy for a table without actually having to manually rebuild the table.

I start with a Hash Partitioned Table.

SQL> create table customers(customer_id number, customer_name varchar2(200), customer_city_code number)
  2  partition by hash (customer_id) partitions 4;

Table created.

SQL> select partitioning_type from user_part_tables
  2  where table_name = 'CUSTOMERS'
  3  /

PARTITION
---------
HASH

SQL> select partition_name from user_tab_partitions
  2  where table_name = 'CUSTOMERS'
  3  /

PARTITION_NAME
--------------------------------------------------------------------------------
SYS_P221
SYS_P222
SYS_P223
SYS_P224

SQL>
SQL> insert into  customers
  2  select dbms_random.value(1,1000001), dbms_random.string('X',25), mod(rownum,5)
  3  from dual
  4  connect by level < 1000001
  5  /

1000000 rows created.

SQL> commit;

Commit complete.

SQL> exec dbms_stats.gather_table_stats('','CUSTOMERS');

PL/SQL procedure successfully completed.

SQL> select partition_name, num_rows
  2  from user_tab_partitions
  3  where table_name = 'CUSTOMERS'
  4  /

PARTITION_NAME     NUM_ROWS
---------------- ----------
SYS_P221             250090
SYS_P222             249563
SYS_P223             250018
SYS_P224             250329

SQL>


I now want to convert this Hash Partitioned Table to a Range Partitioned Table online.

SQL> alter table customers
  2  modify
  3  partition by range (customer_id)
  4  (partition P_100K values less than (100001),
  5   partition P_200K values less than (200001),
  6   partition P_300K values less than (300001),
  7   partition P_400K values less than (400001),
  8   partition P_500K values less than (500001),
  9   partition P_600K values less than (600001),
 10   partition P_700K values less than (700001),
 11   partition P_800K values less than (800001),
 12   partition P_900K values less than (900001),
 13   partition P_1MIL values less than (1000001),
 14   partition P_2MIL values less than (2000001),
 15   partition P_MAXVALUE values less than (MAXVALUE))
 16  online;

Table altered.

SQL>
SQL> select partitioning_type
  2  from user_part_tables
  3  where table_name = 'CUSTOMERS'
  4  /

PARTITION
---------
RANGE

SQL> exec dbms_stats.gather_table_stats('','CUSTOMERS');

PL/SQL procedure successfully completed.

SQL> col high_value format a12
SQL> select partition_name, high_value, num_rows
  2  from user_tab_partitions
  3  where table_name = 'CUSTOMERS'
  4  order by partition_position
  5  /

PARTITION_NAME   HIGH_VALUE     NUM_ROWS
---------------- ------------ ----------
P_100K           100001           100116
P_200K           200001            99604
P_300K           300001            99941
P_400K           400001           100048
P_500K           500001            99841
P_600K           600001            99920
P_700K           700001           100081
P_800K           800001           100024
P_900K           900001           100123
P_1MIL           1000001          100302
P_2MIL           2000001               0
P_MAXVALUE       MAXVALUE              0

12 rows selected.

SQL>


The Hash Partitioned Table is now converted to a Range Partitioned Table.  The number of Partitions has been changed.  And the operation was performed online with the ONLINE keyword added to the ALTER TABLE ... statement.  The UPDATE INDEXES clauses can also be used to update existing Indexes on the Table.





04 April, 2019

Everyone should read this

An excellent article that anyone promising, developing, maintaining or using any system that is non-trivial should read :

https://embeddedartistry.com/blog/2019/4/1/what-can-software-organizations-learn-from-the-boeing-737-max-saga




25 March, 2019

Partitioning -- 14 : Converting a non-Partitioned Table to a Partitioned Table

Pre-12cRelease2, there were only three methods to convert a non-Partitioned Table to a Partitioned Table

(a) Create a new, empty, Partitioned Table and copy (using INSERT .... AS SELECT ... ) all the data from the non-Partitioned Table to the new, Partitioned Table (and subsequently rename the new Partitioned Table after renaming or dropping the old non-Partitioned Table)

(b) Create a new, empty, Partitioned Table and use EXCHANGE PARTITION to switch the non-Partitioned Table into the Partitioned Table (and then run subsequent SPLIT PARTITION or ADD PARTITION commands as needed to create the additional Partitions)

(c) Create an interim Partitioned Table and use DBMS_REDEFINITION to do an online copy of the data to the interim Partitioned Table and automatically switch the name at the end


12.2 introduced the ability to use ALTER TABLE  ... MODIFY PARTITION ... to convert a non-Partitioned Table to a Partitioned Table

I start with a non-Partitioned Table :

SQL> select table_name, partitioned
  2  from user_tables
  3  where table_name = 'SALES_DATA_NONPARTITIONED'
  4  /

TABLE_NAME                     PAR
------------------------------ ---
SALES_DATA_NONPARTITIONED      NO

SQL> select index_name, uniqueness, partitioned
  2  from user_indexes
  3  where table_name = 'SALES_DATA_NONPARTITIONED'
  4  /

INDEX_NAME                     UNIQUENES PAR
------------------------------ --------- ---
SALES_DATA_UK                  UNIQUE  NO

SQL> 


I then convert it to a Range-Partitioned Table.

SQL> alter table sales_data_nonpartitioned
  2  modify 
  3  partition by range (sale_date)
  4  (
  5  partition P_2015 values less than (to_date('01-JAN-2016','DD-MON-YYYY')),
  6  partition P_2016 values less than (to_date('01-JAN-2017','DD-MON-YYYY')),
  7  partition P_2017 values less than (to_date('01-JAN-2018','DD-MON-YYYY')),
  8  partition P_2018 values less than (to_date('01-JAN-2019','DD-MON-YYYY')),
  9  partition P_2019 values less than (to_date('01-JAN-2020','DD-MON-YYYY')),
 10  partition p_MAXVALUE values less than (MAXVALUE) 
 11  )
 12  online 
 13  update indexes
 14  /

Table altered.

SQL> 
SQL> alter table sales_data_nonpartitioned rename to sales_data;

Table altered.

SQL>
SQL> select partition_name, high_value
  2  from user_tab_partitions
  3  where table_name = 'SALES_DATA'
  4  order by partition_position
  5  /

PARTITION_NAME   HIGH_VALUE
---------------- --------------------------
P_2015           TO_DATE(' 2016-01-01 00:00
P_2016           TO_DATE(' 2017-01-01 00:00
P_2017           TO_DATE(' 2018-01-01 00:00
P_2018           TO_DATE(' 2019-01-01 00:00
P_2019           TO_DATE(' 2020-01-01 00:00
P_MAXVALUE       MAXVALUE

6 rows selected.

SQL> 
SQL> select index_name, partitioned, uniqueness, status
  2  from user_indexes
  3  where table_name = 'SALES_DATA'
  4  /

INDEX_NAME                     PAR UNIQUENES STATUS
------------------------------ --- --------- --------
SALES_DATA_UK                  NO  UNIQUE    VALID

SQL> 


The SALES_DATA_NONPARTITIONED was converted to a Range Partitioned Table.  If I didn't have to rename the table (e.g. if the table name was actually, properly SALES_DATA only), then there would be no need to lock the table as the RENAME command does.



20 March, 2019

Partitioning -- 13d : TRUNCATE and DROP Partitions and Global Indexes

A TRUNCATE or DROP Partition makes Global Indexes on a Partitioned Table UNUSABLE.

You may be lucky if the target partition was empty, resulting in Oracle maintaining Global Indexes as valid.  However, the accepted rule is that you either (a) use the UPDATE INDEXES clause [resulting in the TRUNCATE or DROP taking longer to run, effectively locking the table partitions] OR  (b) do a REBUILD of the Indexes that become UNUSABLE after the TRUNCATE or DROP.

12c has introduced what it calls Asynchronous Global Index Maintenance.  With this feature present, the TRUNCATE or DROP runs much faster as a DDL without actually removing the target rows from the Global Indexes [but still requires the UPDATE INDEXES clause to be specified]

So, now in my 12.2 database I have these two Indexes on SALES_DATA :

SQL> select index_name, partitioned, status
  2  from user_indexes
  3  where table_name = 'SALES_DATA'
  4  order by 2,1
  5  /

INDEX_NAME                     PAR STATUS
------------------------------ --- --------
SALES_DATA_PK                  NO  VALID
SALES_DATA_LCL_NDX_1           YES N/A

SQL> 


I then TRUNCATE a non-empty Partition and check the Indexes

SQL> alter table sales_data truncate partition P_2015 update indexes;

Table truncated.

SQL>
SQL> select index_name, partitioned, status, orphaned_entries
  2  from user_indexes
  3  where table_name = 'SALES_DATA'
  4  order by 2,1
  5  /

INDEX_NAME                     PAR STATUS   ORP
------------------------------ --- -------- ---
SALES_DATA_PK                  NO  VALID    YES
SALES_DATA_LCL_NDX_1           YES N/A      NO

SQL> 


The ORPHANED_ENTRIES column indicates that SALES_DATA_PK is subject to Asynchronous Index Maintenance.

This is the job that will do the Index Maintenance at 2am  :

SQL> l
  1  select owner, job_name, last_start_date, next_run_Date
  2  from dba_scheduler_jobs
  3* where job_name = 'PMO_DEFERRED_GIDX_MAINT_JOB'
SQL> /

OWNER
---------------------------------------------------------------------------
JOB_NAME
---------------------------------------------------------------------------
LAST_START_DATE
---------------------------------------------------------------------------
NEXT_RUN_DATE
---------------------------------------------------------------------------
SYS
PMO_DEFERRED_GIDX_MAINT_JOB
20-MAR-19 10.18.51.215433 AM UTC
21-MAR-19 02.00.00.223589 AM UTC


SQL> !date
Wed Mar 20 20:05:24 SGT 2019

SQL> 


So, I could
(1) wait for the next run of the job OR
(2) manually trigger the job (which will scan the entire database for all indexes that require such maintenance) OR
(3) Execute  DBMS_PART.CLEANUP_GIDX  to initiate the maintenance for the specific index OR
(4) Execute an ALTER INDEX REBUILD to make the Index USABLE again.

SQL> execute dbms_part.cleanup_gidx('HEMANT','SALES_DATA');

PL/SQL procedure successfully completed.

SQL> select index_name, partitioned, status, orphaned_entries
  2  from user_indexes
  3  where table_name = 'SALES_DATA'
  4  order by 2,1
  5  /

INDEX_NAME                     PAR STATUS   ORP
------------------------------ --- -------- ---
SALES_DATA_PK                  NO  VALID    NO
SALES_DATA_LCL_NDX_1           YES N/A      NO

SQL> 


Note that the argument to CLEANUP_GIDX is the *Table Name*, not an Index Name.


Here I have demonstrated a TRUNCATE Partition, but the same method would be usable for a DROP Partition.




12 March, 2019

Partitioning -- 13c : Merging Partitions

The reverse of SPLITting a Partition is to MERGE two adjacent partitions.

I reverse the SPLIT that I did in the previous blog post.

SQL> l
  1  select partition_name, tablespace_name, high_value
  2  from user_tab_partitions
  3  where table_name = 'SALES_DATA'
  4* order by partition_position
SQL> /

PARTITION_NAME                 TABLESPACE_NAME                HIGH_VALUE
------------------------------ ------------------------------ --------------------------
P_2016                         ARCHIVE_SALES_DATA             TO_DATE(' 2017-01-01 00:00
P_2017                         TBS_YEAR_2017                  TO_DATE(' 2018-01-01 00:00
P_2018                         TBS_YEAR_2018                  TO_DATE(' 2019-01-01 00:00
P_2019_H1                      TBS_YEAR_2019                  TO_DATE(' 2019-07-01 00:00
P_2019_H2                      TBS_YEAR_2019                  TO_DATE(' 2020-01-01 00:00
P_2020                         TBS_YEAR_2020                  TO_DATE(' 2021-01-01 00:00
P_MAXVALUE                     USERS                          MAXVALUE

7 rows selected.

SQL> 
SQL> alter table sales_data                                                  
  2  merge partitions P_2019_H1, P_2019_H2
  3  into partition P_2019
  4  update indexes
  5  /

Table altered.

SQL> 
SQL> select partition_name, tablespace_name, high_value
  2  from user_tab_partitions
  3  where table_name = 'SALES_DATA'
  4  order by partition_position
  5  /

PARTITION_NAME                 TABLESPACE_NAME                HIGH_VALUE
------------------------------ ------------------------------ --------------------------
P_2016                         ARCHIVE_SALES_DATA             TO_DATE(' 2017-01-01 00:00
P_2017                         TBS_YEAR_2017                  TO_DATE(' 2018-01-01 00:00
P_2018                         TBS_YEAR_2018                  TO_DATE(' 2019-01-01 00:00
P_2019                         HEMANT                         TO_DATE(' 2020-01-01 00:00
P_2020                         TBS_YEAR_2020                  TO_DATE(' 2021-01-01 00:00
P_MAXVALUE                     USERS                          MAXVALUE

6 rows selected.

SQL> 


But, we find that the new Partition was created in the default "HEMANT"  tablespace !  So, we have to be careful about specifying target tablespace(s).

Let me reverse the action and try again.

SQL> alter table sales_data
  2  split partition P_2019 at (to_date('01-JUL-2019','DD-MON-YYYY'))
  3  into (partition P_2019_H1 tablespace TBS_YEAR_2019, partition P_2019_H2 tablespace TBS_YEAR_2019)
  4  update indexes
  5  /

Table altered.

SQL> 
SQL> select partition_name, tablespace_name, high_value
  2  from user_tab_partitions
  3  where table_name = 'SALES_DATA'
  4  order by partition_position
  5  /

PARTITION_NAME                 TABLESPACE_NAME                HIGH_VALUE
------------------------------ ------------------------------ --------------------------
P_2016                         ARCHIVE_SALES_DATA             TO_DATE(' 2017-01-01 00:00
P_2017                         TBS_YEAR_2017                  TO_DATE(' 2018-01-01 00:00
P_2018                         TBS_YEAR_2018                  TO_DATE(' 2019-01-01 00:00
P_2019_H1                      TBS_YEAR_2019                  TO_DATE(' 2019-07-01 00:00
P_2019_H2                      TBS_YEAR_2019                  TO_DATE(' 2020-01-01 00:00
P_2020                         TBS_YEAR_2020                  TO_DATE(' 2021-01-01 00:00
P_MAXVALUE                     USERS                          MAXVALUE

7 rows selected.

SQL> alter table sales_data
  2  merge partitions P_2019_H1, P_2019_H2
  3  into partition P_2019 tablespace TBS_YEAR_2019
  4  update indexes
  5  /

Table altered.

SQL> select partition_name, tablespace_name, high_value
  2  from user_tab_partitions
  3  where table_name = 'SALES_DATA'
  4  order by partition_position
  5  /

PARTITION_NAME                 TABLESPACE_NAME                HIGH_VALUE
------------------------------ ------------------------------ --------------------------
P_2016                         ARCHIVE_SALES_DATA             TO_DATE(' 2017-01-01 00:00
P_2017                         TBS_YEAR_2017                  TO_DATE(' 2018-01-01 00:00
P_2018                         TBS_YEAR_2018                  TO_DATE(' 2019-01-01 00:00
P_2019                         TBS_YEAR_2019                  TO_DATE(' 2020-01-01 00:00
P_2020                         TBS_YEAR_2020                  TO_DATE(' 2021-01-01 00:00
P_MAXVALUE                     USERS                          MAXVALUE

6 rows selected.

SQL> 


So, when running Table Partition DDL, be careful about the intended and actual target Tablespace(s).  (What about Index Partitions ?  The UPDATE INDEXES clause can specify target tablespaces for each Index Partition of each Index as well ... something like : (this is from the documentation on the ALTER TABLE command)
UPDATE INDEXES (cost_ix (PARTITION c_p1 tablespace tbs_02, 
                           PARTITION c_p2 tablespace tbs_03))

13 January, 2019

Partitioning -- 13b : Splitting a Partition

Let's say the business anticipates growing sales volume in 2019 and new reporting requirements.  IT analyses the requirements and decides that the SALES_DATA Table that is currently Partitioned by YEAR, needs to be Partitioned by HALF-YEAR from 2019 onwards.

SQL> select partition_name, tablespace_name
  2  from user_tab_partitions
  3  where table_name = 'SALES_DATA'
  4  order by partition_position
  5  /

PARTITION_NAME                 TABLESPACE_NAME
------------------------------ ------------------------------
P_2016                         ARCHIVE_SALES_DATA
P_2017                         TBS_YEAR_2017
P_2018                         TBS_YEAR_2018
P_2019                         TBS_YEAR_2019
P_2020                         TBS_YEAR_2020
P_MAXVALUE                     USERS

6 rows selected.

SQL> 
SQL> alter table sales_data
  2  split partition P_2019 at (to_date('01-JUL-2019','DD-MON-YYYY'))
  3  into (partition P_2019_H1, partition P_2019_H2)
  4  update indexes
  5  /

Table altered.

SQL> 
SQL> col high_value format a26 trunc
SQL> select partition_name, tablespace_name, high_value
  2  from user_tab_partitions
  3  where table_name  = 'SALES_DATA'
  4  order by partition_position
  5  /

PARTITION_NAME                 TABLESPACE_NAME                HIGH_VALUE
------------------------------ ------------------------------ --------------------------
P_2016                         ARCHIVE_SALES_DATA             TO_DATE(' 2017-01-01 00:00
P_2017                         TBS_YEAR_2017                  TO_DATE(' 2018-01-01 00:00
P_2018                         TBS_YEAR_2018                  TO_DATE(' 2019-01-01 00:00
P_2019_H1                      TBS_YEAR_2019                  TO_DATE(' 2019-07-01 00:00
P_2019_H2                      TBS_YEAR_2019                  TO_DATE(' 2020-01-01 00:00
P_2020                         TBS_YEAR_2020                  TO_DATE(' 2021-01-01 00:00
P_MAXVALUE                     USERS                          MAXVALUE

7 rows selected.

SQL> 


I used the UPDATE INDEXES clause to ensure that all (specifically Global) Indexes affected by the SPLIT are updated so that they don't go into an UNUSABLE state.

I could have optionally used a TABLESPACE clause for each of the two new Partitions P_2019_H1 and P_2019_H2


(Also see a previous BlogPost on using SPLIT PARTITION to add a new Partition at the "end" of the table by splitting the last Partition)



Preview Popular Posts