What does a Big Data Analyst do?

The advent of modern computer networks and the democratisation of information has exponentially multiplied the amount of digital data being generated. According to a Cisco estimate global IP Traffic will reach 2 zettabytes by 2019. (1 zettabyte = 1 billion terabytes).

This unprecedented increase in data has come about not only due to employment of existing methods, but from new horizons that have seen massive growth in the number of connected devices. Portable devices are at the center of data growth. Plus, additional volumes from M2M (Machine to Machine) data through AI and IoT technologies are responsible for constant data accumulation.

The term Big Data has come into daily parlance owing to a shift in analysing structured relational databases toward working with massive quantities of unstructured databases generated through a variety of sources. There developed an urgent need to build tools that analyse and interpret such unstructured datasets.

Rapid emergence of massive data on the Internet
Always connected portable devices have expanded the opportunities for inputs to the internet massively. Data, today, is being referred to as the lifeblood of digital organisations and the currency of the new digital economy, suggesting a level of significance and value distinctly higher than in previous times. While the price of computing continues to fall, data driven business models are becoming attractively affordable.

Research firm, Markets and Markets, tells us that Enterprise Data Management including software and services for migrating, warehousing, integrating and analysing all forms of data will approach USD 105 billion by 2020.

Skill shortage to handle Big Data
However, one of the major challenges to handling Big Data is a paucity of skills required in the workforce. Usage in industry is moving away from SQL (Sequential Query Language) to manage structured relational database sets to management and analysis of unstructured data. In order to handle Big Data, we need to anticipate the three phases of data analytics, viz., collection and storage; processing and organising and analysis and visualisation.

What does a Big Data Analyst do?
Since data analytics and management are relatively older areas in the tech world as compared to Artificial Intelligence, cloud systems and the Internet of Things, employers are somewhat aware of the skills that would be required. Nevertheless, the requirements of today and the future will depend more on the analytics part.

The most important question that we can then ask is: How can we intelligently use huge volumes of data for business transformation covering incremental revenue, trimming costs, enhanced and informed decision making and target marketing. Therefore, much of the skill gaps around data-handling now rest on real time analytics, predictive modeling, data security, distributed storage and data mining.

Current industry skill set requirements for Big Data Analytics
Based on current industry trends and nature of business needs, Big Data professionals ought to have the following competencies:

• Data Interpretation and Visualisation encompasses job functions such as data analytics and business intelligence. In this domain, the competency requirement will emphasise upgrading the instruments of data analytics and warehousing, for example, Netezza, MicroStrategy, SAS and Tableau.
• Data Processing and Management encompasses job functions like database administration, creation, data integration and data lifecycle management.
• Workers in this profession must work on database software and platforms like Microsoft, Oracle, Cloudera and Hadoop.
• Data Infrastructure is the platform upon which data management lies and job functions covering data storage, data centre management, business continuity and data security are emerging.
• In order to gain entry into this interesting and much-in-demand field one would need to go through a comprehensive program designed and delivered by leading experts in the field. Different roles require different focus competencies and one must choose accordingly.

- Diwakar Chittora