Automatic Data Archiving urf Information Lifecycle Management

Automatic Data Archiving urf Information Lifecycle Management (ILM)

Many organizations are floating with so much data due to digitalization. With the digital transformation at every stage in industry, data volumes are increasing like anything. Data is generated from a variety of applications and a variety of data in different forms of data i.e. relational, non relational, documents, files, videos etc…

Now Challenge for every organization is how to manage this data. Because data is growing exponentially, day by day…

Businesses capture day by day data in amounts that are sometimes overwhelming, mostly when you think of the Internet of Things (IoT) that provides countless data points from its machines, sensors, cameras, and other access points.

According to IDC, more than 41.6 billion connected IoT devices, or things, will exist in 2025, generating 79.4 Zettabytes (ZB). One ZB is circa a billion Terabytes or a trillion Gigabytes.

Disks are becoming cheaper but they are not free. Increasing storage will create other problems like, increasing infrastructure, reducing performance, higher maintenance like tuning and archiving. Organizations value different parts of their data differently, and they would like to associate different costs with storing that data. The objective is to apply the right storage optimizations to the right data at the right time.

Oracle Database provides the ideal environment for implementing your ILM solution, because it offers a cost-effective solution that’s secure, transparent to the application and achieves all of this without compromising performance, and often improving performance.

Oracle's ILM solution is based on a combination of two components, 

1. Oracle Partitioning and 
2. Oracle Advanced Compression

Many people called ILM as Data Lifecycle Management and Many called it as Automatic Data Archiving. 

Three steps to achieve, Automatic Data Archiving or Information Lifecycle Management as below,

"Oracle Partitioning addresses these challenges by limiting the amount of data to be scanned and improving performance. Also, it partitions data as and when it comes into data. Hence partitioned data significantly improving performance and manageability beyond what is possible with a non-partitioned data set. It fully complements Oracle Database performance features, and is used in conjunction with any indexing technique, join technique, or parallel access method. Plus, partitioning is implemented at the database level and doesn’t require any changes to application code or query statements in order to easily take advantage of performance benefits.

Oracle Advance compression provides a comprehensive set of compression capabilities to help improve database performance and reduce storage costs. It allows Banks to reduce their overall database storage footprint by enabling compression for all types of data: relational (table), unstructured (file), index, network, Data Guard Redo and backup data. While query performance, storage cost savings and data optimization are often seen as the most tangible benefits, additional innovative technologies can help reduce CapEx and OpEx costs for all components of an IT infrastructure, including memory and network bandwidth as well as heating, cooling and floor-space." Oracle Docs,

With these two components you can achieve Automatics Data Archiving urf ILM. Oracle advanced compression's Heat map and ADO helps to achieve this.

Heat Map: Collects data usage information at the block and segment levels.
Automatic Data Optimization: Enables you to create policies that implement compression and storage tiering automatically. 

To implement your Automatic Data Archiving urf ILM strategy, you can use Automatic Data Optimization (ADO) to automate the compression and movement of data between different tiers of storage within the database. The functionality includes the ability to create policies that specify different compression levels for each tier, and to control when the data movement takes place.

To use Automatic Data Optimization, you must enable Heat Map at the system level. You enable this functionality with the HEAT_MAP initialization parameter.

It helps you to achieve,

• Automated Data Lifecycle Management to control and govern data compression and storage movement based on the actual usage of that data. 
• Oracle Heat Map tracks the actual usage of tables and partitions down to the row-level. 
• Oracle Automatic Data Optimization automatically detects less active data/indexes, compresses and/or moves to lower cost storage tier. 
• Transparently compress and move data online, no application downtime or administrator burden.
• Business policy driven compression tiering and storage tiering.

Many Customer Do this Manual. They manual identify older data -> Copy data from original table and move to archive/history table -> and Delete data from original table.

Why don't you Adopt Automation?
Identify older data automatically -> Compress the data -> Move data to another tablespace.

Benefit

• One time policy definition.
• No further manual work.
• Automatically identify data, compress it and move it.

In next article I will explain with example and commands how to achieve Automatic Data Archiving.

Thanks & Regards,

Chandan Tanwani

Oracle Performance Tuning Certified Expert