- Creative District Ahmad Yani No. 456 Medan
- Phone : +800-3374-4676
- Email : hola@dominantsite.com
Recently media reports have reported an introduction of technological marvel that is expected to generate whopping 1.4 TB of data on daily basis. While such cases are not norm, we could not have imagined such a growth just a few years ago and we would see more such cases as AI becomes a mainstream tool. The revolution augments human capability multifold to invent more quickly and identify diseases faster. Besides science and technology industry, business applications also noticed huge increase in the growth of data and documents (unstructured data). Information overload can not only require high capital (i.e. sunk cost) but importantly it also skews the business analytics. Traditional archiving and technologies often may fall short to control the growth in its existing form and risk organizational operations. Secondly, the storage and backup budget also grow unreasonably high if the database/infrastructure is not designed properly ahead of time. Here are some of the challenges of aggressive data/information growth in SAP area:
Here is our distilled approach of Hybrid Archiving for big-data/fast data systems:
First task is to understand the characteristics of the ingested data- Fast-data, Big data, business data, health/pharma data, compliance data, research/science data, analytics/denormalized data, entertainment data, etc. Based on the expected data volume / data type, and its consumption model (reporting), find the most suitable software/database and hardware for now and foreseeable future needs. It is important to add future scaling factor in consideration as well. This will avoid unexpected high storage bill and sudden drop in user experience/capabilities when data growth aggravates. Recent cloud storage options provides a balanced combination between cost and storage/retrieval speeds.
Reporting of historical data is vital for business decision making, and legal/regulatory obligations. A clear distinction is required between reporting of live/recent data Vs reporting of historical data. Once the distinction is clearly understood, the reporting capabilities must be planned in such a way that it balances with the storage cost. A clear list of reporting requirements can aid to aggressive archiving of data and therefore control the cost without introduction of any business risk. A special care should also be taken to avoid using one size fits all rule – i.e. avoid using one residence period for all types of data. Every data/information type is unique in terms of its requirements so build a specific retention category for each of them for optimized success of the project. Continued controlled access of online/archived information is also important for complex data structure and Axcellent uses this aspect exclusively in the analysis.
The value of recent data is generally higher than relatively older data. However, the general trend is that all data is stored on the same media or database which is cost incursive. Some corporations have been holding up technical/logs data into primary or secondary storage media that is older than 20 years. Our philosophy is to first purge the data if it is not required for business/legal reasons, or to move these data to a frozen state with “known” retrieval delay. In addition to the implementation of age-sensitive storage strategy, An internal/external communication should also be required to inform the stakeholders of any delay of accessing historical data form the frozen assets (service level agreement). Axcellent can help review such situation and craft a Hybrid strategy for age-sensitive storage media.
DVM tools can only be effective if used correctly and right-sized. Often, classical archiving, ILM (Information Lifecycle Management), NSE (Native Storage Extension), In-memory computing are not used in right context or settings that do not yield any noticeable benefits. At times, we have even noticed ineffective archive jobs results with insignificant archiving ratio. Deep knowledge/understanding of these tools and their capabilities are important for their optimal & perpetual application. Archiving/eDVM should not be a one-time success but it pays back when the database growth curve gradually flattens out (gradient). The periodical review of database may reveal either adjustment of archiving strategy or addition of new objects for the success of the initiative. Axcellent will review the data growth and match it to one or more suitable tools as a part of Hybrid archiving strategy.
Each case of Hybrid archiving is unique and presents different challenges – there is no silver bullet. The good news is that a lot of novel tools & field experience are now available at our disposition to control the database size. Axcellent reviews each case separately based on above approach and creates a suitable strategy that includes combination of one or multiple approaches to resolve the issue. Please contact us to review your situation and how can we help!
Contact our round-the-clock service centers for a tailored data archiving blueprint.
Recently media reports has have reported an introduction of technological marvel that is expected to generate whopping 1.4 TB of data on daily basis. While such cases are not norm, we could not have imagined such a growth just a few years ago and we would see more such cases as AI becomes a mainstream tool. The revolution augments human capability multifold to invent more quickly and identify diseases faster. Besides science and technology industry, business applications also noticed huge increase in the growth of data and documents (unstructured data). Information overload can not only require high capital (i.e. sunk cost) but importantly it also skews the business analytics. Traditional archiving and technologies often may fall short to control the growth in its existing form and risk organizational operations. Secondly, the storage and backup budget also grow unreasonably high if the database/infrastructure is not designed properly ahead of time. Here are some of the challenges of aggressive data/information growth in SAP area:
While industry-specific law such as, Privacy Act of 1974, HIPAA, etc. existed between 1970-1990, the PII regulations came into limelight after introduction of GDPR, and CCPA in late 2010 and early 2020. Axcellent solution has partnered with a law firm to closely interpret the PII regulations for SAP ERP system. Based on that and our deep understanding of SAP ERP data set, we implement SAP ILM to conform to PII regulations based on your industry while continuing with archiving/DVM initiatives. The solution entails multiple disciplinary areas such as activation of business functions, configuration of rules/policies, discovery of record usage (transactional/master data level), and review of the results followed by selective destruction or blocking of the data. This sensitive process requires balances talent of knowledge of business, technology, and regulatory framework.
Proactive analysis of data volume (Planning):
First task is to understand the characteristics of the ingested data- Fast-data, Big data, business data, health/pharma data, compliance data, research/science data, analytics/denormalized data, entertainment data, etc. Based on the expected data volume / data type, and its consumption model (reporting), find the most suitable software/database and hardware for now and foreseeable future needs. It is important to add future scaling factor in consideration as well. This will avoid unexpected high storage bill and sudden drop in user experience/capabilities when data growth aggravates. Recent cloud storage options provides a balanced combination between cost and storage/retrieval speeds.
Regulatory reporting requirements:
Reporting of historical data is vital for business decision making, and legal/regulatory obligations. A clear distinction is required between reporting of live/recent data Vs reporting of historical data. Once the distinction is clearly understood, the reporting capabilities must be planned in such a way that it balances with the storage cost. A clear list of reporting requirements can aid to aggressive archiving of data and therefore control the cost without introduction of any business risk. A special care should also be taken to avoid using one size fits all rule – i.e. avoid using one residence period for all types of data. Every data/information type is unique in terms of its requirements so build a specific retention category for each of them for optimized success of the project. Continued controlled access of online/archived information is also important for complex data structure and Axcellent uses this aspect exclusively in the analysis.
Introduction of age-sensitive storage media:
The value of recent data is generally higher than relatively older data. However, the general trend is that all data is stored on the same media or database which is cost incursive. Some corporations have been holding up technical/logs data into primary or secondary storage media that is older than 20 years. Our philosophy is to first purge the data if it is not required for business/legal reasons, or to move these data to a frozen state with “known” retrieval delay. In addition to the implementation of age-sensitive storage strategy, a an internal/external communication should also be required to inform the stakeholders of any delay of accessing historical data form the frozen assets (service level agreement). Axcellent can help review such situation and craft a Hybrid strategy for age-sensitive storage media.
Proper application of Traditional DVM features such as Native Storage Extension (Courtsey: SAP):
DVM tools can only be effective if used correctly and right-sized. Often, classical archiving, ILM (Information Lifecycle Management), NSE (Native Storage Extension), In-memory computing are not used in right context or settings that do not yield any noticeable benefits. At times, we have even noticed ineffective archive jobs results with insignificant archiving ratio. Deep knowledge/understanding of these tools and their capabilities are important for their optimal & perpetual application. Archiving/eDVM should not be a one-time success but it pays back when the database growth curve gradually flattens out (gradient). The periodical review of database may reveal either adjustment of archiving strategy or addition of new objects for the success of the initiative. Axcellent will review the data growth and match it to one or more suitable tools as a part of Hybrid archiving strategy.
Each case of Hybrid archiving is unique and presents different challenges – there is no silver bullet. The good news is that a lot of novel tools & field experience are now available at our disposition to control the database size. Axcellent reviews each case separately based on above approach and creates a suitable strategy that includes combination of one or multiple approaches to resolve the issue. Please contact us to review your situation and how can we help
Contact our round-the-clock service centers for a tailored data archiving blueprint.
