Even as the Big Data technology and services market gears up for exponential growth in the coming years, most enterprises that plan to adopt advanced analytics tools are faced with one key challenge - lack of Big Data specialists. So, while certain estimates suggest that the global Big Data market will grow to a staggering $16.9 billion by 2015, a Mckinsey's report says that by 2018 the US alone could face a shortage of 140,000-190,000 "deep analytical talent" and 1.5 million people capable of analyzing data in ways that enable business decisions. This shortage may be felt across all key global markets.
Big Data skill is not just about managing and mining huge data volumes but it is also about addressing a large volume of unstructured formats like video, graphics, weblog data and documents. At the present, there are not many educational institutions that train people in these areas, resulting in a shortfall in the Big Data people. As such, data science is predicated to three key areas - technology (IT, systems, hardware and software), quantitative (statistics, math, modeling, algorithms) and business. Building a workforce that is adept at all these three areas is a big challenge for global enterprises.
Big Data analytics is a not a monolithic profession. There are specialists operating at different levels. Big Data scientists - with strong background in artificial intelligence, natural language processing or data management - are perhaps the most scarce resources. Then, there are the data architects, data visualizers, data change agents, data engineers/operators, data stewards and data virtualization/cloud specialists.
Each of these skills has a unique function in Big Data analytics. For instance, data architects need the creativity to harness disparate types of data in new ways to create fresh insights. Likewise, a data visualizer translates analytics into information a business can use, whereas data engineers/operators develop the architecture that helps analyze and supply data in the way the business needs.
Many universities are now introducing advanced degree programs in analytics to manage Big Data. At the same time, several technology majors have begun working with universities across the globe to help them establish programs that will help fulfill the need for Big Data scientists, data analytics professionals, and data-savvy managers.
Emerging markets like India are quick off the block in building a strong knowledge base around Big Data, and are fast becoming the Big Data analytics delivery hubs. That being the case, even as all major markets step up the efforts to build up their Big Data workforce, with greater adoption of Big Data analytics a significant part of this is likely to be outsourced to the global analytics delivery hubs.