Harnessing structured & unstructured data: A new challenge means new thinking

For anyone interested in getting places faster, the invention of the internal combustion engine was great news. In fact, better than great; it transformed how we get around.

But no transformational development comes without challenges. No sooner did the car arrive, than society was dealing with the issue of how to allow them to get places. That was a massive problem that called for new thinking followed by a lot of hard work.

Society is facing a similar situation with the explosion of data, both structured and unstructured. It’s exciting, and it makes possible things that none of us foresaw. But now that it’s here, we’re also discovering how inadequate existing infrastructure is to make the most of it.

How do we find the value in unstructured data?

In broad terms, we went from having small volumes of data that was already structured, to having massive volumes of unstructured data that needed to be assessed or analysed in ways that had not been thought of.

We’ve made headway, that much is sure. New workloads like NoSQL, HDP and SAP HANA are allowing organisations to derive powerful insights from data gathered and collated from multiple sources.

But we’re still only scratching the surface. Data that’s available but not necessarily very usable yet includes emails, social media, blogs, documents, images and videos. It’s generally agreed that these sources account for about 80% of the world’s data. What’s more, some of it is invaluable. Want to know what your customers are saying about you to their friends? Tune in to social media.

So what’s the problem with mining this information? Unlike your typical database, it’s unstructured. Conventional systems for mining data aren’t up to the task when the information isn’t contained within a spreadsheet of some sort.

So we have access to a lot of data (we have cars that go faster than horses) but we don’t have the means of mining all of it (we only have roads within our town, but not between towns).

Entering the cognitive era

But rather than simply building more roads, so to speak, the solution here is a new approach called cognitive technology.

Cognitive technology can harness unstructured data by mimicking a human thinker. It can analyse unstructured data, interpret it to create insights, evaluate decisions based on evidence, and come to a conclusion with a certain level of confidence.

That’s different from your typical computer, which uses a lot of processing power to crunch information and arrive at a definitive answer. That’s powerful, but not very subtle. Humans, on the other hand, can be subtle but are not so good at processing lots of data. That said, we also need more power.

Roughly 2.5 billion GBs of data are created every day – a figure that may well be out of date soon after this blog is published. About 90% of the world’s data was created in the last two years alone. That’s one heck of a mountain – and if you’re moving a mountain, you need a lot of monster diggers and trucks.

The beauty of cognitive computing is that it marries brute processing power with insightful thinking. It allows the discovery of patterns within data at a ridiculously fast pace, which in turn allows you to make better decisions sooner. Think product purchases, product lines, investment decisions and more.

But think beyond the familiar decision-making areas as well. As insights deepen, unexpected opportunities start to show up.

How do you ensure you’re keeping pace with the possibilities opened up by the explosion of data? Watch our webinar that explains the new world in clear terms, and shows how running IBM Power systems can help you make the most of it.

Watch the webinar now.

Share this:

Comments are closed.