As businesses continue to deal with more data on a daily basis, how many organisations are actually focusing on true data quality?
In an effort to make data more “useful” companies needs to examine implementing a data quality “fit for purpose” approach, says PBT Group.
Data quality management can be defined as the various tools and/or processes that are implemented within a business, which results in the development of data that is accurate, complete and valid. data quality is about having the right (correct) information at the right time, in the right place and for the right people. This data, then in return, is used to make accurate decisions – to allow the business to serve its customers effectively and achieve company goals.
Says Janine Sutherland, a Business Intelligence Consultant at PBT Group; “The above sounds simple and standard, right? However, the reality is that multiple departments within a business look at the same piece of data, yet the information value derived from that data is actually vastly different.”
“For example; let’s look at a customer account record, which contains the customer name, surname, account/product type and account balance. The data found here is consumed by the marketing, finance and collections department, however, each department finds different value in this data provided.”
For the marketing department, the customer detail is directly related to the value it places on customer behaviour and based on this, possible cross sell opportunities, to name a few. For the finance department, the customer itself only plays a minor role in budget reporting. However, this data and the related details plays a very critical role in collections department – especially for debt collection and recovery.
Continues Sutherland; “For the marketing and collections department, having the correct account/product type and up to date contact details and data would be of utmost importance. Incorrect data could lead to potential business revenue loss (where doubtful debt could not be collected) or the marketing department could miss potential cross sell opportunities. In contrast, the finance department would only require the contact details data for the purposes of invoicing.”
The problem, however, comes in, when, say for example, the finance department notices that the contact detail data is incorrect. “Should this happen, the finance department flags the data with a record as poor quality data. However, this data would still be most suitable for marketing purposes as the account type and product have been verified as correct. Irrespective of which data is incorrect, the whole record should not be deemed as being of ‘poor data quality’, but rather should be looked at in context, to define its fit and purpose for use.”
Quality of the data can be measured by implementing various data quality algorithms associated to each data quality dimension – these being accuracy, completeness, consistency, currency, precision, privacy, reasonable, referential integrity, timeliness, uniqueness and validity.
Based on the business area where the data is used and the business value add or importance, weightings can be assigned to each of the key measures and their respective evaluations would then assist in disseminating and highlighting any problem areas. Additionally, you can determine an overall data quality index by summing up the overall scores for each data quality dimension and you could apply weighting factors according to importance.
“In the scenario above, the objective of ‘fit for purpose’ data quality could have been achieved by a lower referential integrity score, but with a much higher currency and accuracy weighting for the finance department. The inverse would apply to the marketing department when it comes to their referential integrity, currency and accuracy weighting scores” adds Sutherland.
By aligning the data quality dimensions and by assigning weightings and business rules to the information value that business derives from data, ‘bad data’ can no longer be a general quality statement, as it is now expressed in terms of being ‘fit for purpose’.
Concludes Sutherland; “This ‘fit for purpose’ approach, if implemented within a business would be more pleasing to the consumer of the data – in other words, the individual who needs to use the data for decision-making. As such, data quality is in fact in the Eye of the Beholder.”
