The above images portray the graphical representation of System of Engagement, System of Records and IT layers enabling multi-channel customer journeys. A closer look at these states that both systems need to interact in order to deliver a great user experience. And that is precisely why the point at which they intersect is of importance to data managers - and requires new architectural capabilities (such as Lambda and Data Lakes) to augment existing enterprise data warehouses (EDW).
With the trend of migrating applications and data to cloud and to improve analytical capabilities, reduce OPEX and TCO, the approach to integrate the two systems acquires new dimensions. The data integration approach now requires:
- Data synchronization between application databases in cloud
- Data migration from non-relational databases to NOSQL databases (Document and Column Family Databases)
- API driven DI between SaaS apps
- Data migration and synchronization between on-premise and cloud applications and data platforms
- Continuous data extraction from cloud applications into enterprise data lakes
- DI from cloud applications into analytical data warehouses
- Web scraping and mining to continuously monitor websites, global trading information, etc.
These new workloads are not just batch, but consist of a mix of batch, real-time and streaming data. They require IT to handle Fast Data - which is central to a real-time enterprise and demands that the Enterprise Data Warehouse (EDW) be able to ingest the data as a stream or complex events to make it available for real-time analysis.
The data, as can be imagined, doesn't come in rows and columns anymore. Log Files, JSON, XML, images, unstructured text, etc., stress the existing IM capabilities and demand polyglot storage capabilities. Today, a hypothetical digital app may store its application data in more than one database like customer information on a relational database, catalogue information on document database, click-stream and mobile session information data in Cassandra, etc. In addition, the data needs to interact in order to produce insights. From an information management perspective the data supply chain has to be re-configured with new capabilities put in place to manage this integration. The solutions to resolve these challenges are growing in number as digital is opening more possibilities with each passing day.
So, does this make things easier or more difficult for information management professionals? The answer really depends on the underlying information management architectures to acquire, store, process and manage the data that they opt for.