Data quality is a constant issue in identity solutions.  Usually, this is what happens:  we tell the client that they need to clean up their identity data before we start doing any integration work.  They assure us that this will happen.  We have been burned by this in the past, so we write it into the contract language to try and protect ourselves.  Doesn’t matter: the data cleansing project is so unsexy that it never comes to the top of the priority pile and so it doesn’t get done.  The identity integration, the pipework, is complete before the data, the water that’s supposed to flow through the pipes, is clean.  The client sponsor is furious that the identity work isn’t complete, but it’s a data quality issue and so there’s a delay at the end of the project while data cleansing happens.

Almost every organization I’ve ever looked at has bad identity data.  Even if you go through a data cleansing process, over time the quality will decay.  Based on anecdotal evidence, I think it’s around three or four years to get to the point where you have to do another data cleansing exercise.  For an organization with a relatively static employee base, it might be at the longer (four year) end of that range; for a dynamic business with lots of hires/fires, acquisitions/divestitures, and the like, probably less than three years.

For example, we recently did a simple integration of a file directory with an HR database.  We were able to automatically match with confidence 60% of the accounts in the file directory with actual people.  (Actually, it was 56%; 4% were service accounts that don’t map to people.)  The remaining 40% required some level of manual intervention: 12% were possible matches that needed to be reviewed manually.  Another 8% simply didn’t have enough information to make a judgement on.  And fully 20% were unmatched; we have good information on them but they don’t match people in the HR system.  At some point, after a manual review, those accounts will be closed down and we’ll listen for the complaints to figure out who they belong to.