DataFlux

One of the dirty little secrets of corporate America — not to mention doomed IT projects — is lack of clear job roles. People pile all their psychic baggage onto the flimsy foundation of their own semantic interpretations. Just because what’s-her-name is a project manager doesn’t mean she can boss everyone around and not keep track of task completion metrics. Just because the guy in the corner cube is an architect doesn’t mean he gets to sequester himself in Visio-land and not make a contribution. And you’ve heard me loud and clear on the data steward conundrum. - Read the rest of this entry »

At the company where I work, I asked the CEO and the President, “What are your top two concerns about data quality? You know, the ones that keep you up at night?” I was hoping for data quality indicators that we could track over time. Of course, I didn’t get just two concerns from each of them; there were many concerns. In my next few blogs, I will explain their concerns, and how we implemented the solution as part of our quality program. - Read the rest of this entry »

So I’m at the Teradata Partners conference last week in Vegas and, after the requisite conference theatrics — I’ve come to expect hydraulics at any vendor event with over 3000 attendees — down comes Mike Koehler, he having recently taken the helm of the newly-independent Teradata, talking about his company’s plans for the coming year. Then there’s the inevitable dry ice sublimation — I’ve come to expect dry ice at any vendor event with over 3000 attendees — through which appears Dr. Jim Goodnight, the founder and leader of the venerable SAS. What gives? - Read the rest of this entry »

At the recent Teradata Partner’s conference (October of 2007 in Las Vegas, NV), Teradata and SAS announced a new partnership between the two companies. I really hope this means we can bring the goodness of analytics, quality, and profiling tools to the existing (and future) Teradata communities. I can’t think of anything more pleasant than profiling billions of rows of data. Yahoo, what fun for me! - Read the rest of this entry »

A significant component of the traditional data quality tool space is based on cleansing sets of records by identifying unique entities within each set and resolving variant instances to those unique entities. More simply, the process looks for records that represent duplicated data within each set, then attempt to match records across the sets. - Read the rest of this entry »

Gosh oh Golly Gee Whiz – who would have thought that the acquisition programmer put in empty strings instead of null? I am positive when we created the data model that we decided what fields should be null, and what fields should be NOT null. - Read the rest of this entry »