Usually when a company\u00a0enrolls into a data clean-up exercise the focus is mainly on the in-depth analysis and profiling of the data.<\/p>\n
Sometimes it’s worthwhile to step back from the in-depth look at data cleaning to get some perspective.<\/p>\n
You should start asking questions like “Why is data dirty in the first place?”, \u00a0“Are any of the clean-up exercise that we are currently doing reversible ?” or “Does this really need to become an ongoing process ?”<\/p>\n
In most of the cases the root cause falls into two main categories :<\/p>\n
Often, someone such as a salesperson, field adjuster, or courier\u00a0is stuck with the job of entering critical data. That person may have an awkward system with no built-in support for data integrity.<\/p>\n
For example, a courier may simply have an \u201cAddress field\u201d field on an hand-held device in which he is to enter the delivery address for the package he just picked up. Often you’ll end up with 20 variations of “Saint Katharine’s Way, London, UK”.<\/p>\n
At the same time, the person responsible for data entry may not be motivated to drive data quality to the 100-percent level.<\/p>\n
If your job is to drop data down a pipe and no one ever tells you that your efforts are worthwhile, then you aren\u2019t going to do a perfect job for very long. You probably won\u2019t even know what a perfect job entails.<\/p>\n
The answer to the problem of data-entry quality is a kind of business re-engineering, which requires multiple steps such as:<\/p>\n
Data warehouse and data-cleaning tools play a unique role in defining the need for this type of business re-engineering as the data warehouse is the perfect place to see the value of good data.<\/p>\n
Sometimes, paradoxically, the data warehouse must make imperfect data available in order to show the organization how valuable perfect data would be.<\/p>\n
Data-cleaning tools alert IS to the exact issues involved in in achieving clean data. The final architecture for data delivery will ideally be balanced between data originally entered as accurately as possible and powerful data-cleaning systems operating downstream in the data extract process to bring data up to the shining goal of being 100-percent correct.<\/p>\n
What processes do you follow to ensure a high data quality & integrity ?<\/p>\n
Until next time,<\/em><\/strong><\/p>\n \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0keep learning, keep searching, keep\u00a0succeeding…<\/em><\/strong><\/p>\nShare this:<\/h3>