BI Stands for Business Integrity

One of the key aspects of working within the field of Business Intelligence is the need for an absolute commitment to Integrity. In my career I have run across a number of situations that have put this to the test.

Data Integrity Issues

No matter the maturity of your data governance program, we have all run across situations where a data quality issue is impacting the quality of our Business Intelligence information or highlights an operational issue. Quite often these issues are highly sensitive within the organization. We have a responsibility as Business Intelligence professionals to communicate these issues to our stakeholders in a sensitive way that is focused on resolving the issue.

Truth Hurts

The facts don't lie. The beauty of Business Intelligence is that once you take a look at the data, there really is no defending the various theories that people use to operate their business in the absence of facts. Again, we need to engage our stakeholders in the process of moving to fact based decision making by having them part of the solution so they can adjust their business practices to the true reality.

Oops

Nobody is perfect. The reality is that we will make mistakes in our BI implementations by inaccurate meta data, ETL coding errors, report logic and the hundreds of other places we can make mistakes in our BI implementations. We need to raise these issues and the impact this has had to business operations and decision making. This isn't about quickly fixing and hoping no one notices, we need to be up front and accept responsibility for our mistakes.

Communicating these issues is not about sending a blanket email coldly highlighting the issues, the best way is to get out of your chair or pick up the phone and talk to people. This is not the fun part of the job. We always need to do the right thing, especially when it is hard.

 

TDWI Conference – Day 2 Highlights

Another great day at the TDWI BI Executive Summit, covered a lot more ground today.  Here is my list of highlights and “a ha” moments:

• Many Data Warehouses only get leveraged for standard reporting, it takes a leap to start leveraging this investment for true analytics.
• An assessment of who does reporting / BI and what systems they leverage is an essential step to understand the requirements for BI at your organization.
• Just having the latest tools doesn’t ensure success in your BI program, you need to engage your business closely to get true value.
• Think hard about what will amaze and delight your end users when it comes to the customer service you provide through your BI team.
• Text Analytics is critical to gaining insight from the vast amount of unstructured data your organization has.  Lots of great applications of how to understand and take action on the feedback from your customers.
• Idea of having people in your BI team be adept at all parts of the technology stack from Database, ETL, and BI reporting (and requirements).  This helps ensure that you implement business logic in the right part of the stack.  For example, a report developer will put the logic in the report, because that is what they know.
• Business Intelligence is a process of discovery, it is worthless to document requirements at the beginning of an iteration because these requirements will and should change.  Document what you have to once you have what the customer wants.
• User happiness is the key to success, the goal of meeting the requirements as documented at the beginning of the project is meaningless.
• Everyone on your BI team needs exposure to the end customer to truly understand their needs, including your ETL developers.  Strikes me that your ETL developers probably know your organization’s data the best, and can bring this knowledge to the conversation with your customer.
• Lots of discussion on data governance / MDM in the afternoon:
• Data quality needs to be part of everything you do, it is not just about a governance model
• Idea of putting the name of the business owner on the report, they should be the ones answering questions about data quality
• Pick your battles, only focus on data quality issues that have significant impact
• You need to link data quality / governance to business process management, they are very tightly linked
• It is not really about garbage data, it is about bringing together multiple, valid formats so they can be integrated and linked.

Tomorrow wraps up the BI Executive Summit with a half day of content focused on future trends in the BI space.  Should be interesting!  Stay tuned for tomorrow’s update.

Mark

I Dream of Data

For those of you that aren't certified "data geeks", you would probably not understand dreaming of data.  For some of us this love affair is new, and for some of us this has been a long standing obsession.

I clearly remember the first time I got the data bug.  It was an entry level database programming course and the lab assignment was to build and execute SQL scripts (inserts, deletes, updates, selects) against a provided data model using an Oracle (v5 I think) database.  It just seemed so natural to examine the data model and the problem and just pick what you wanted and join tables to build "grids" of data.  The career path in IT was set from the 2nd year in university!

Throughout my career, I have always been fascinated by manipulating data to gain insight into what is happening in the business of my organizations.  This has fostered a deep appreciation for the value of Business Intelligence, but on a larger scale an appreciation for how overall data management is critical to the success of an organization.  The Business Intelligence discipline must be supported by other components of an Enterprise Information Management (EIM) strategy such as Master Data Management (MDM), Data Governance, Data Architecture, and Data Quality Management to provide maximum value to an organization.  To forge ahead without engaging the rest of your IT department in treating data as an asset is a recipe for frustrated teams and ineffective solutions.

So when I fall asleep, I dream of a perfect environment where all pieces of an Enterprise Information Management strategy are in place.  Sometimes I feel like I am drawn to a big light at the end of a long corridor filled with data.  When I finally get to the end I see beautiful visualizations of information presented by my local Business Intelligence portal, harnessing all the inherent value in my beautifully managed data sources.  This is a big dream, but what dream worth pursuing is easy?

Good night, sleep tight!

Foundational Concept for Performance Management and Data Quality

I was going through my reading, and ran across this article on the B-Eye-Network that talks about the foundational concepts that need to ground a Data Quality program. Part of this article discussed the Deming Cycle (PDSA Cycle). The PDSA acronym stands for the following:

• Plan - Plan for the future desired state
• Do - Execute actions to get to future state
• Study - Check the results against desired state
• Act - Act to correct towards desired state

This is a foundational concept that we can apply to both Data Quality programs and the use of Key Performance Indicators in a performance management system. In the data quality program we would apply the PDSA model as follows:

• Plan - Define data quality issues in a given data source
• Do - Put in place monitoring tools to analyze data quality
• Study - Analyze the results to understand issues and identify root causes
• Act - Act on root causes (typically process issues) and fine tune monitoring tools

Similarily, when we define and use Key Performance Indicators we might approach it this way:

• Plan - Define a set of KPIs that are thought to be critical to business success
• Do - Collect data on actual performance against the KPIs
• Study - Analyze the results to validate KPI effectiveness and to search for other supporting factors of success
• Act - Fine tune KPIs from learnings to improve overall performance management

If we look at this type of process as a framework, we can use it to build more effective processes that are rooted in an iterative approach to improving quality and value. With this type of execution you can climb any hill by taking small, measured steps. A word of caution however, you may suddenly realize you are climbing the wrong hill...8)

Good night,

Mark