Most ERP databases running for years constitute significant data footprints that are unused and add to the cost, stress to the IT resources, and decreased productivity in ERP management. IT leaders & business data owners need to decide on data usage and growth by understanding the database management challenges and implementing effective data governance strategies.
The sheer amount of aged data and rapidly increasing new data from a variety of sources stored both within on-premises environments and in the cloud creates severe governance, security, and management challenges. The antiquated thought process of just adding more storage or compute is not the answer, as it is inefficient and costly, especially over time. It’s important for a better understanding of Sensitive, Outdated, Redundant, and Trivial (SORT) data available in the ERP Database to decide on whether to retain, archive, and purge.
4 Database Management Challenges
Lack of strategic Intent
Organizations are complacent in not managing or disposing of data due to inexplicit direction, lack of interest, and basic neglect over years of inconsistent retention policy enforcement. Information governance involves categorizing data, gaining insight into its business value, and managing it systematically based on such policies as the frequency of access and retention date. Organizations need to understand and balance what has to be kept versus what data is increasing stress on IT infrastructure, while neither providing value nor placing the organization under obligation to retain.
Many parts of the organization have different responsibilities and requirements for data, which complicates IT efforts around the overall enforcement of the retention, archive, and purge policies. Policy management is a key tool for establishing a baseline for compliance.
Lack of Communication
Between business data owners / legal / compliance departments leading to over-collection of data. Established data storage processes, methods, and technologies may not be sufficient. Given that most organizations’ efforts around retention policies are unenforced or immature, there is often an excess of content to search through. In order to guarantee data identification and quality, IT tends to be overly inclusive in many data preservation efforts that search across multiple hybrid data repositories.
In addition, organizations that lack the full awareness of regulatory requirements to which they must adhere can over-collect owing to excessive “keep everything forever” retention policies propelling corporate data glut.
Lack of New Tools or Processes
Unsure of relevant methodologies, tools, and best practices to help identify unused data in the ERP environment. Sometimes the process to identify, execute, and time-consuming, considering environments which have multiple databases and complex environments and may require expert skillsets to execute.
The process would require an intimation to the business stakeholders, Identification of data, assessment of change impact, execution of archive and purge, and finally post-execution validation and testing of accuracy systems and reports. Once completed, data governance practices such as archive and purge and database compression must be regularly practiced depending on the quantum of data growth and impact on the IT resources. One of the methods obviously is to buy a new archive and purge tool, however, it comes with additional cost and setting up new processes and training.
Migrating to New Environment
Poor data management makes implementing new solutions or migrating to a new environment (like cloud or data center) extremely difficult, forcing organizations to continue business legacy systems. The increasing demand to store more data manifests itself as a data management and complex problem. Initiatives involving data policies often arise from new investments in upgrades and migration projects.
These events should be viewed as opportunities to both 1) manage information strategically according to policy and 2) archive and purge unused data to maximize the efficiencies to be derived from the new investments.
As legacy environments and methodologies are retired and replaced with systems that support improved third-party integration and accessibility, SORT data can cause difficulties that may hinder the achievement of these goals. In the process of moving files from one location to another, many organizations take the opportunity to create rules that allow data to be identified, classified, and assessed for ongoing retention or deletion. Organizations should identify data that adds value, include it in selective migrations, and thereby reduce the overall data footprint through defensible deletion of leftover data.
Once current initiatives have been addressed through the management of existing data, the same plans can be implemented for ongoing needs to manage newly created data decreasing data management costs and reducing legal and regulatory risks.
4 Database Management Strategies to Follow
Requirements Gathering
Engage business data owners along with legal/compliance, with executive sponsorship, to determine corporate compliance and industry-specific data retention requirements across all geographical areas. Regulatory compliance is generally the most compelling reason organizations implement a data retention policy. Laws and mandates define the minimum length for which data must be retained in the context of industries and vary by legal jurisdiction and location. This common ground will help set the direction for enterprise data to be retained, archived or purged
Data Classification
By analyzing a combination of all the data and weighing corporate goals, regulatory requirements, and viable usage of data, working groups can help set realistic policies for data classification, storage, analysis, and disposal. Create a content map of the data that provides the necessary information for legal, compliance, and business user representatives in making decisions regarding data to be retained or deleted. Pulling together different entities within an organization is vital for the success of this exercise.
Engaging a Certified Expert
A cost-effective method is to engage a certified ERP expert for advisory/assessment of the current environment to identify unused data and classify data according to your organization’s Data management policies. The ERP experts have developed tried and tested custom proprietary scripts for archive and purge which can help save considerable Cost, Time, and risk of execution of such a project. If you have been running your ERP for several years and have not executed an Archive and Purge strategy to date, it is advisable to get a free assessment which will help you get an idea of the quantum of resources that will be free and save considerable cost.
Data Cleansing
Migrating inefficiencies of the existing environment to the new environment will lead to more complexities and increase cost and legal risks. Archive and purge data before a change to the new environment and maximize the value of the new investment. The new environment may bring in newly created or revised retention policies and data archive and purge should be a continuous data governance practice.
Database Cleansing Methods
Data Compression
- Data Compression is more Technology-driven and would not have any involvement with other business data stakeholders.
- Comes with additional cost in a tool for advanced compression along with stress on current compute, network, and storage of the unused data.
- Retaining old data beyond regulatory limits, and those needed for ongoing business operations can pose a risk in case of litigation.
Data ‘Archive & Purge’
- Data ‘Archive & Purge’ requires interdependencies between the IT and business data owner.
- Deleting obsolete data during the archive and purge process helps maximize the utilization of existing resources and minimizes time to run maintenance activities only on the relevant data.
- It can be executed through tools like Oracle ILM / Application or through ITC developed proprietary custom scripts to save the cost of recurring clean-up.