Businesses are reluctant to migrate their data warehouse to the cloud due to several reasons, including time taken for migration, perceived security threats and lack of skilled resources to enable these migrations. 31% of organizations identify migration cost as one of the top 3 barriers for Cloud migration. Lack of knowledge of new platform, lack of migration governance, lack of insights into performance on new platform as per expectations are some of the contributing factors for the reluctance.
A data migration solution needs to address the above concerns, at the same time it has to be quick and easy. The speed of delivery in the present day is like driving on the Autobahn. There is no speed limit but you need to ensure you drive safely and efficiently. Speed and agility is the name of the game with reduced time to market, high level of automation, minimal human intervention, high data accuracy, high level of security and improved Total Cost of Ownership.
Some of the key modules needed in any data migration tools are:
- Source analyzer module: The key first step to any migration is to understand the source data model. It needs to be reviewed for accuracy, enable users to define rules and produce reports. It should also help generate data model including table names, column names, indexes, datatypes etc., The extraction SQL should have flexibility to modification.
- Performance Optimizer module: The data movement has to be optimized for performance. This can be achieved by splitting the data files into multi-parts and engaging all the available nodes in the target database. Data files need to be compressed for faster data movement onto the cloud landing zone.
- Data Migration module: The data loading should leverage all available resources for maximum parallelism
- Post Load Validation module: Finally, upon completion of data loading, the data between the source and target at individual table level needs to be validated for accuracy. It can be achieved either by comparing row counts or checksums for each table in source and target.
- Governance module: The migration tool should have end to end governance.
Today we require an end to end self-service platform-agnostic and cloud-agonistic solution that accelerates migration of data-warehouse from on premise to cloud. The comprehensive solution with above mentioned modules will be able to provide below features for users
- Analyses existing data warehouse prior to migration
- Creates knowledge repository for target optimization and scalability
- Provides Data remediation and migration capabilities
- Provides Data validation
- Enables Data security through encryption
- Provides Intuitive user interface that can be used by business users
- Provide end-to-end data migration governance
Such cloud migration tools could help us achieve below benefits
- Enables 60% improvement in average productivity by automating migration from on-premise data warehouse to cloud data warehouse
- 50 to 60% productivity improvement by automating end-to-end data movement
- 40 to 50% improved Total Cost of Ownership (TCO) through optimized license, storage, support etc.,
- 40 t0 55% reduction in time-to-market through rapid provisioning of new data warehouse platform
Wipro research survey 2014: https://www.wipro.com/cloud/business-runs-on-the-cloud/