A recent trend that we see emerging in the data warehousing marketplace is the continued integration of data warehouses into the enterprise IT infrastructure. The need for integrated store of detailed data available - for both tactical and strategic business decision making - is finding wider acceptance across industries today.
A traditional data warehouse is predefined with rigid load cycles and ETL Windows. In a traditional context, source data for making critical decisions might appear at unpredictable times and this results in the need for real time data access. Real Time Integration Architectures like Representational State Transfer (REST) Framework or Simple Object Access Protocol (SOAP) offer an advantage over the traditional set-up because the data can be continuously loaded and accessed, usually with 24x7 availability.
As the data appliance industry continues to grow tremendously with high speed data processing and storage capabilities, a real time access to its data that too with easily deployable API’s, makes REST Framework more popular against other real time services like SOAP etc. in the industry.
Why Real Time Data Access for Data Appliance?
- Real time data transformation, integration and storage in a centralized place helps IT organization to process and move data across the enterprise faster and on demand
- REST also addresses the need for processed data needs of applications across the IT
- External real time data feeds for real time analytics is possible with REST without any security lapses
- It also facilitates in building data pipe line for real time data flow and high availability of treated data for further usage among downstream systems
- REST helps in continuous integration and extraction of data for analytics
- REST facilitates integration with data generation chips embedded in Smart devices is easier
- Real time movement of big data from machines can be handled easily with REST integration
Why REST is best suited in today's context?
- Being a public API, REST API is very easy to adopt and develop
- REST API for Teradata provides driver less connectivity to read and write data into Teradata Database
- Similarly, REST API for HDFS makes it easier to work with Hadoop Clusters
- REST API helps in work load balancing. Enterprises don’t have to depend more on ETL tool or ESB (Enterprise Service Bus) for Real Time Integration.
- REST works on top of HTTP. A browser is all what we need.
- Unlike SOAP or WSDL, it does need a binder design for communication.
Data Appliance Support for REST
Data Appliance market leaders like Teradata has already provided Teradata compatible REST API’s. This helps in providing driverless connectivity from client machines, all that using simple commands like GET and PUT. The Web HDFS Feature which is being offered by Apache or Hortonworks could be used for Real time data ingestion. In addition, the REST API can easily communicate to hadoop clusters. It uses the full bandwidth of the hadoop cluster for streaming data. The ‘file read’ and ‘file write’ calls are redirected to the corresponding data nodes.
With major data appliances rendering support for REST, it has become easier to build services around data appliances on real time. Both applications and machine-data now can be integrated with other available data in the warehouse to build insights immediately.