Introduction

Once an organization has decided to institute a data quality scorecard, what types of metrics should be used for data quality performance management? Too often, data governance teams rely on existing measurements as the metrics used to populate a data quality scorecard. But without a defined understanding of the relationship between specific measurement scores and the business’s success criteria, it is difficult to determine how to react to emergent data quality issues - and determine whether their fixing these problems has any measurable business value. When it comes to data governance, differentiating between “so-what” measurements and relevant metrics becomes a success factor in managing business use expectations for data quality.

This paper explores ways to qualify data control and measures to support the governance program. We will also examine how data management practitioners can define metrics that are relevant to the achievement of business objectives. For this, organizations must look at the characteristics of relevant data quality metrics, and then provide a process for characterizing business impacts in association with specific data quality issues. The next step is providing a framework for defining measurement processes in a way that reflects the business value of high quality data in a quantifiable manner.

Processes for computing raw data quality scores for base-level metrics can then feed different hierarchies of complex metrics, with different views addressing the scorecard needs of different constituencies across the organization. Ultimately, this drives the description, definition and management of base-level and complex data quality metrics such that:

• Scorecards reflecting business relevance are driven by a hierarchical rollup of metrics
• The definition of metrics are separated from their contextual use, thereby allowing the same measurement to be used in different contexts with different acceptability thresholds and weights
• The appropriate level of presentation can be generated based on the level of detail expected for the data consumer’s specific data governance role and accountability

Useful vs. “So-What” Metrics

The famous physicist and inventor Lord Kelvin’s quote about measurement – “If you cannot measure it, then you cannot improve it” – is a rallying cry for the data quality community. The need for measurement has driven quality analysts to evaluate ways to define metrics and their corresponding processes for measurement. Unfortunately, in their zeal to identify and exploit different kinds of metrics, many people have inadvertently flipped the concept around in a number of ways, thinking that “If you can measure it, you can improve it,” or even less helpfully, “If it is measured, you can report it.”