{"id":974,"date":"2015-07-09T07:22:50","date_gmt":"2015-07-09T06:22:50","guid":{"rendered":"http:\/\/dotnetsql.info\/net-sql\/?p=974"},"modified":"2015-07-31T16:17:40","modified_gmt":"2015-07-31T15:17:40","slug":"why-do-we-end-up-with-dirty-data-platform-agnostic","status":"publish","type":"post","link":"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/","title":{"rendered":"Why do we end up with Dirty Data [platform agnostic] ?"},"content":{"rendered":"<p>One of the biggest\u00a0themes in operational databases and data warehouses alike that is universally recognized but far too often ignored is the cleanliness of the data.<\/p>\n<p>From hundreds of meetings with data processing and IS staff, I have identified three consistent themes.<\/p>\n<p>Although these three themes stand out dramatically as the biggest problems in corporate data access, the same data processing and IS staffs that identify them are usually attacking only the first two of them.<!--more--><\/p>\n<p>The three themes can be expressed by the following comments:<\/p>\n<ul>\n<li>The data access issue: \u201cWe have one of the world\u2019s largest sets of data but we can\u2019t get access to it.\u201d<\/li>\n<li>The query tool issue: \u201cI want the system to show me what is important and then I want to ask why.\u201d<\/li>\n<li>The data integrity issue: \u201cWe know that some of our data isn\u2019t very good. For instance, we don\u2019t have a single, centrally maintained customer list.\u201d or &#8220;We know that the source doesn&#8217;t hold that info but it&#8217;s critical for us to compute it and report on it.&#8221;<\/li>\n<\/ul>\n<p>Given the universality of these comments, it is strange that entire industries are being organized around the first two issues but the third issue seems to be something that we don\u2019t want to talk about.<\/p>\n<p>The database marketplace has responded to the need for data access with client\/server architectures, dedicated data warehouse hardware and software, and whole families of communications schemes to connect users to their data.<\/p>\n<p>The query tool marketplace is an embarrassment of riches. There are dozens of ad hoc query tools, report writers, and application development environments. We are well into the magnificent new\u00a0generation of powerful tools for data warehousing end-user applications with dimensional OLAP\/ ROLAP tools and the hot new data mining tools.<\/p>\n<p>Yet the third issue, data integrity, languishes in the backwater of operational data &amp; data migration &amp; data warehousing. It is talked about briefly and then completely ignored.<\/p>\n<p>There is a definite avoidance of the topic and very few plans in place to address data integrity at the same level as those plans that address data access or query tools.<\/p>\n<p>Enterprises across the globe face these issues mostly due to:<\/p>\n<ul>\n<li>Excessive unstructured growth of the IT &amp; Data architecture<\/li>\n<li>Mergers &amp; Acquisitions<\/li>\n<li>Faulty Enterprise Integration<\/li>\n<li>Slim or Missing Master Data Management<\/li>\n<\/ul>\n<p>All of the issues above can be mitigated with proper access to expert resources that can deliver and maintain an enterprise data architecture and the business value gained through a data clean up exercise in the long term is priceless<\/p>\n<p><strong>Thanks \u00a0for reading \u00a0this \u00a0article,<\/strong><\/p>\n<p><strong>Next steps :<\/strong><\/p>\n<p style=\"margin-left: 18pt;\"><strong>1.\u00a0 Share this with your colleagues because Sharing\u00a0 is Learning<\/strong><\/p>\n<p><strong>\u00a0 \u00a0 \u00a0 \u00a0 2. Comment below if you need any assistance<\/strong><\/p>\n<p style=\"text-align: right;\"><em><strong>Powered by <a href=\"http:\/\/www.codereview.co\">CodeReview &#8211; Let&#8217;s make it Better!<\/a><\/strong><\/em><\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-twitter\"><a rel=\"nofollow\" data-shared=\"sharing-twitter-974\" class=\"share-twitter sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to share on Twitter (Opens in new window)<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow\" data-shared=\"sharing-linkedin-974\" class=\"share-linkedin sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to share on LinkedIn (Opens in new window)<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow\" data-shared=\"sharing-facebook-974\" class=\"share-facebook sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=facebook\" target=\"_blank\" title=\"Share on Facebook\"><span><\/span><span class=\"sharing-screen-reader-text\">Share on Facebook (Opens in new window)<\/span><\/a><\/li><li class=\"share-google-plus-1\"><a rel=\"nofollow\" data-shared=\"sharing-google-974\" class=\"share-google-plus-1 sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=google-plus-1\" target=\"_blank\" title=\"Click to share on Google+\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to share on Google+ (Opens in new window)<\/span><\/a><\/li><li class=\"share-pocket\"><a rel=\"nofollow\" data-shared=\"\" class=\"share-pocket sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=pocket\" target=\"_blank\" title=\"Click to share on Pocket\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to share on Pocket (Opens in new window)<\/span><\/a><\/li><li class=\"share-email\"><a rel=\"nofollow\" data-shared=\"\" class=\"share-email sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=email\" target=\"_blank\" title=\"Click to email this to a friend\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to email this to a friend (Opens in new window)<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div><div class='sharedaddy sd-block sd-like jetpack-likes-widget-wrapper jetpack-likes-widget-unloaded' id='like-post-wrapper-76243027-974-69e28e85b26b0' data-src='\/\/widgets.wp.com\/likes\/#blog_id=76243027&amp;post_id=974&amp;origin=dotnetsql.info&amp;obj_id=76243027-974-69e28e85b26b0' data-name='like-post-frame-76243027-974-69e28e85b26b0'><h3 class='sd-title'>Like this:<\/h3><div class='likes-widget-placeholder post-likes-widget-placeholder' style='height:55px'><span class='button'><span>Like<\/span><\/span> <span class=\"loading\">Loading...<\/span><\/div><span class='sd-text-color'><\/span><a class='sd-link-color'><\/a><\/div>","protected":false},"excerpt":{"rendered":"<p>One of the biggest\u00a0themes in operational databases and data warehouses alike that is universally recognized but far too often ignored is the cleanliness of the data. From hundreds of meetings with data processing and IS staff, I have identified three consistent themes. Although these three themes stand out dramatically as the biggest problems in corporate &hellip; <a href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Why do we end up with Dirty Data [platform agnostic] ?<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-twitter\"><a rel=\"nofollow\" data-shared=\"sharing-twitter-974\" class=\"share-twitter sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to share on Twitter (Opens in new window)<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow\" data-shared=\"sharing-linkedin-974\" class=\"share-linkedin sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to share on LinkedIn (Opens in new window)<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow\" data-shared=\"sharing-facebook-974\" class=\"share-facebook sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=facebook\" target=\"_blank\" title=\"Share on Facebook\"><span><\/span><span class=\"sharing-screen-reader-text\">Share on Facebook (Opens in new window)<\/span><\/a><\/li><li class=\"share-google-plus-1\"><a rel=\"nofollow\" data-shared=\"sharing-google-974\" class=\"share-google-plus-1 sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=google-plus-1\" target=\"_blank\" title=\"Click to share on Google+\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to share on Google+ (Opens in new window)<\/span><\/a><\/li><li class=\"share-pocket\"><a rel=\"nofollow\" data-shared=\"\" class=\"share-pocket sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=pocket\" target=\"_blank\" title=\"Click to share on Pocket\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to share on Pocket (Opens in new window)<\/span><\/a><\/li><li class=\"share-email\"><a rel=\"nofollow\" data-shared=\"\" class=\"share-email sd-button share-icon no-text\" href=\"http:\/\/dotnetsql.info\/net-sql\/index.php\/why-do-we-end-up-with-dirty-data-platform-agnostic\/?share=email\" target=\"_blank\" title=\"Click to email this to a friend\"><span><\/span><span class=\"sharing-screen-reader-text\">Click to email this to a friend (Opens in new window)<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div><div class='sharedaddy sd-block sd-like jetpack-likes-widget-wrapper jetpack-likes-widget-unloaded' id='like-post-wrapper-76243027-974-69e28e85b2bbd' data-src='\/\/widgets.wp.com\/likes\/#blog_id=76243027&amp;post_id=974&amp;origin=dotnetsql.info&amp;obj_id=76243027-974-69e28e85b2bbd' data-name='like-post-frame-76243027-974-69e28e85b2bbd'><h3 class='sd-title'>Like this:<\/h3><div class='likes-widget-placeholder post-likes-widget-placeholder' style='height:55px'><span class='button'><span>Like<\/span><\/span> <span class=\"loading\">Loading...<\/span><\/div><span class='sd-text-color'><\/span><a class='sd-link-color'><\/a><\/div>","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[77,290,289,79,5,6,54,82,25],"tags":[291,299],"aioseo_notices":[],"_links":{"self":[{"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/posts\/974"}],"collection":[{"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/comments?post=974"}],"version-history":[{"count":4,"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/posts\/974\/revisions"}],"predecessor-version":[{"id":978,"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/posts\/974\/revisions\/978"}],"wp:attachment":[{"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/media?parent=974"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/categories?post=974"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/dotnetsql.info\/net-sql\/index.php\/wp-json\/wp\/v2\/tags?post=974"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}