Qualitative and Quantitative Data Are Like Apples and Oranges
The midsize business world operates in a sea of data. Much of it - the sort that most people think of when they hear the term "data" - is quantitative. It is numerical, or at least it can easily be represented in a numerical way. Apples can be counted, and their dimensions can be measured. With appropriate sensors, even their redness (or yellowness, for varieties such as Golden Delicious) can be measured and quantified.
Data, however, can be either (or both) qualitative and quantitative. And qualitative data is commonly regarded trickier to deal with, at least in the IT world, and even in the broader midsize business world. How does a system distinguish between apples and oranges?
Businesses tend to love quantitative data. It is crisp and clear. It lends itself to all sorts of manipulations that can generate real insights. As Kerry Bodine of Forrester Research tweeted at the IBM Big Data Hub, "To play off a common phrase, 'No one ever got fired for using spreadsheets.'"
Quantitative reasoning, however, can stumble when confronted with the difference between apples and oranges. And a case can be made that many of the most critical business decisions really involve apples-or-oranges questions.
Great products are rarely just quantitative improvements on their more ordinary rivals. They are, in some ways, essentially - qualitatively - different. (On the flip side, disastrous business missteps typically fail because they missed something fundamental, and again, qualitative.)
Security is one critical area where an understanding of qualitative differences is essential. An attempt to breach and compromise a system may be quantitatively similar to thousands or millions of normal, legitimate accesses to the system. A request is made for software to be uploaded or for data to be downloaded, modified, or perhaps deleted.
What, then, distinguishes attacks on a system from legitimate uses of the system? Ultimately it is a matter of motive - a qualitative factor that does not flag itself and indeed seeks to conceal itself.
Pattern recognition technology, however, can bridge the gap between qualitative and quantitative information. An orange is not just essentially different from an apple: It also differs in a huge number of details of shape, texture, color, and so forth. If a system measures apples thoroughly enough, oranges will stand out as anomalous. If enough information is available, an anomaly-handling subroutine can match the pattern with the characteristics of an orange.
Big Data analytics thus provides a powerful tool set for bringing qualitative information into the scope of quantitative understanding. These tools are now being applied not only to security, but to a host of other qualitative business issues that may previously have been too elusive to pin down.
Moreover, this business analytics capability is now affordable by midsize firms - precisely the agile companies that are positioned to benefit the most from faster and more flexible market information. This ability to quickly distinguish apples and oranges is thus very good news for IT workers at midsize firms.
This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise, and solutions they need to become engines of a smarter planet. Like us on Facebook. Follow us on Twitter.