Understanding and Auditing Big Data

Big data is a popular term used to describe the exponential growth and availability of data created by people, applications, and smart machines. The term is also used to describe large, complex data sets that are beyond the capabilities of traditional data processing applications. The proliferation of structured and unstructured data, combined with technical advances in storage, processing power, and analytic tools, has enabled big data to become a competitive advantage for leading organizations that use it to gain insights into business opportunities and drive business strategies. However, the challenges and risks associated with big data must also be considered.

Increased demand, immature frameworks, and emerging risks and opportunities that are not widely understood or systematically managed by organizations have created a need for more guidance in this area. Internal auditors, in particular, must develop new skill sets and obtain knowledge of big data principles to effectively provide assurance that risks are addressed and benefits are realized.

Risks associated with big data include poor data quality, inadequate technology, insufficient security, and immature data governance practices. Internal auditors working with big data should engage with the organization’s chief information officer (CIO) and other key leaders to better understand the risks in terms of data collection, storage, analysis, security, and privacy.

This guidance provides an overview of big data: its value, components, strategies, implementation considerations, data governance, consumption, and reporting, as well as some of the risks and challenges these may present. This guide also explains internal auditors’ roles and responsibilities when performing assurance or advisory procedures related to big data efforts.