Shine a light on your Dark Data

In our first blog of the new year, Jahanzaib Malik explains how dark data can pose a challenge for GDPR compliance and provides a possible solution.

What happens when we don’t use data? When organisations collect data but only use some, letting most of it float down to the floor to collect dust?  

We use the term “dark data” to refer to the unstructured data lying deep down and hidden, not used for any purpose, with most dark data being generated by IoT devices. Gartner defines dark data as the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes (for example analytics, business relationships and direct monetizing).  

Gartner estimates that almost 80% of enterprise data is unstructured. As the world is becoming more connected it is only natural that a gigantic amount of data is generated every minute in which just 20% is structured. The rest is the intimidatingly huge underwater portion of the iceberg.  

That’s where GDPR comes in. When the General Data Protection Regulation came into effect on 25th May 2018 it was not only applicable to structured data but all forms of data. It follows that dark data can pose a problem in complying with GDPR, as the huge amount of unstructured data brings a serious challenge to organisations across the world – especially those who are in the EU or work with organisations in the EU – in making sure that their dark data complies with GDPR to avoid the reputational and financial consequences of non-compliance. 

When it comes to unstructured data most businesses have very little insight into their data stores. The extraction and management of dark data can prove to be a big turning point for an organisation and it can help in better decision-making and efficient customer service. According to one survey, if the companies process and analyse their dark data in time, they are more likely to get 60% more business than those who don’t shine a light on their dark data. Analytics can transform dark data into valuable strategic insights, and the great thing about dark data is that it’s already in your reach; all you need to do is extract it in the right way and make it safe and useful.  

“The huge amount of unstructured data brings a serious challenge to organisations across the world – especially those who are in the EU or work with organisations in the EU – in making sure that their dark data complies with GDPR to avoid the reputational and financial consequences of non-compliance” 

There’s no way around it – in order to comply with GDPR regulations, we need new practices surrounding SaaS products in order to ensure data integrity. This can be a challenge for modern applications which are typically not concerned with data safety, but the enterprise should nevertheless make sure that they have the right procedures in place to detect, report and investigate a data breach.  

So, what can you do? Documentation is the best way to report the personal data we hold, where it came from, and who we’re sharing it with. Budget allocation will be the other best strategy to tackle GDPR compliance of unstructured data. An organisation needs to allocate budget to modern tools such as AI and machine learning, as well as the training of employees, to process the data in an effective way.  

Despite the name, dark data isn’t something to be feared. The other term used to describe it is “dusty data”, which perfectly illuminates the way forward – all you need to do is blow the dust off.  

Jahanzaib Malik

Jahanzaib Malik

Jahanzaib Malik is a third year Computing student at Dublin Business School. He is a Software Test Support Engineer with Odyssey VC and Compliant Cloud.

Share on

Share on twitter
Share on linkedin
Share on facebook
Notify of