Keeping your BI environment clean

Over the past year or so the "green" revolution has been sweeping IT as a way for organizations to reduce their environmental impact and to cut costs at the same time.  This has driven the need for robust measurement systems to manage the transition to being a green organization.  A number of the big BI vendors have come out with pre-packaged solutions to drive more "green"  for organizations by encapsulating best practices.

It is great for BI teams to provide these systems to the organization, but we would be remiss to not apply these techniques within our BI environments.

What can you do?  One thing I recommend you do is to perform a complete review of your Business Intelligence data stores and reporting at least once a year.  The overall goals of this review are to remove obsolete information and to address any deficiencies.  There are a number of activities you could incorporate into this review:

• Validate with your business stakeholders that the information has continuing value.  Solicit feedback on the information that is valuable, but is somehow incomplete.
• Review usage statistics for data marts and reports to determine what content is accessed regularly and by whom
• Validate that the report or data store reflects current business logic
• Gather metrics on batch load processes, response time, and storage usage to ensure your environment is stable and is scaling as volumes grow.  I've never heard of a Data Warehouse that shrinks in size over time.

The majority of these activities do not cost you a dime other than people's time.  This is especially attractive in a financial environment like today, when capital spending is being closely scrutinized.  Putting in place a regular program such as this will enable your organization to realize the following benefits:

• Reduce maintenance costs for your BI environment.  Less data marts and reports to support as your environment grows and changes.  You can quickly find yourself drowning in too many items to support.
• Lower the related data centre costs to support your infrastructure.  Believe it or not, all your ETL processes, database servers and storage solutions consume significant amounts of power and cooling.
• Identify duplicated information that is redundant.  Root out conflicting business definitions.
• Reduce the opportunity for information overload.  Having focused information will allow your users to find what they are looking for quicker.

Spare the environment and lower your costs by supporting less while extracting maximum value from your existing investments.  You can't beat this type of ROI!

CIA: Collaboration in Action

One of the greatest examples of effective collaboration is the partnership between 007 and Q in the James Bond movies.  Q would design and build gadgets that seemed to be uniquely suited to the next round of challenges and daring escapes James would encounter on his upcoming mission.  During each movie it appears that James just drops in to Q's lab after receiving his assignment to find everything he needs ready and waiting for him.

Being this ready for anything doesn't happen magically, but is the result of an agile development methodology.  Delivering information solutions is a constantly changing landscape that can benefit from applying the design principles used in Q's lab.

• Rapid Prototyping - Every time James would drop by, the "latest" version of a new tool is ready to be tried.  Take every opportunity to show your users the look and feel of your solution, this will unleash their creativity and you will benefit from their increased engagement and suggestions.
• Fail and Fail Fast - These gadgets would often have unintended side effects, or would explode spectacularly.  There was never any other reaction than cool, collected analysis to fix the problem.  Create an atmosphere where the ability to quickly fail, learn from the mistake, and try again happens as quickly as possible.
• Open Environments - Tests were not undertaken in closed laboratories, but were instead conducted in environments that were open to other researchers and their devilishly handsome customers.  Have a physical environment that encourages collaboration with everyone that is core to a project.  Virtual environments are great, but nothing replaces in person communication.
• Continuous Improvement - Every time 007 returns his shot up, smoking, destroyed vehicles, he provides feedback directly to the designer on what did and didn't work properly.  Find different channels to receive feedback from your users, and make a point of acting on it.

Business Intelligence is not something that is done well in isolation.  Encourage your team to use the above concepts, and look for every opportunity to engage the right people in your initiatives no matter what area of the organization they are from.

"Now do please remember to take care of all this equipment 007...DON'T TOUCH THAT!!...That's my lunch!" - Q

BI for BI

One of the core disciplines within a Business Intelligence team or program is to develop internal metrics and performance management practices. The idea of "BI for BI" is to exploit your internal strengths and live the fact-based decision mantra.

Business Intelligence is a core component of corporate performance management systems. Conversely, Business Intelligence can only deliver maximum value when it is coupled by a strong performance management culture. In order to communicate the value of spend on BI / performance management initiatives, we need to talk about both the BI technology and the culture required to support it.

If we have the management practices in place to constantly act on and refine our internal program / departmental metrics, we can then use this as a working example to the rest of the organization. Not only will you be able to demo the metrics produced for your team, but you will be able to relate success stories that you had acting on this information. Not only show a dashboard trending average bugs per release, but also tell the story of how your team saw a negative trend 6 months ago, implemented an improvement to the code review process, and reduced overall bugs by 50%. How powerful is that!

Here are just a couple examples of metrics you could put in place to measure your BI program:

• ETL Job Completion Times
• Uptime by Infrastructure Component (Presentation, ETL, Database Layers)
• Volume of Requests for Information
• Average Frequency of Visits to BI Portal (i.e. 3.5 times per week)
• Volume of Data Quality Issues
• Resolution Time on Service Tickets
• Production Incidents by Severity
• Changes to Report / Asset Catalogue

A couple hints on how to put this in place:

• Make people accountable for their metrics. They own them and report on them.
• Start small and grow. Start with measuring only a handful of metrics, and add additional metrics over time. Look for patterns of key performance indicators, and their relation to other metrics.
• Review and publish your metrics. Have a regular time set for metric review. Make metrics available to anyone that interacts with you, including your customers. Open the Kimono!

It is a basic professional requirement for an organization to produce regular metrics on their performance. Leverage your team's expertise in the discipline of Business Intelligence to improve your team's performance, and show the rest of the organization how it is done!

The Agile Organization: Powered by Data

One of the core characteristics of an analytical competitor is the ability to rapidly leverage the organization's data to make decisions faster than their competitors. Being first to market with a new idea, product or service will ensure success when the profit margins are high before the big new thing becomes a commodity.

Here is an example of a business using analytics to gain market share:

We need to retain our high end customers with a loyalty program. Who are our most profitable customers?

Our most profitable customers must be the ones that drive highest revenues, lets go and see what this customer base looks like.

Thus begins a cycle of analysis to answer this question.

• Data Discovery - What data is available on revenue? What systems contain this data? What is the quality and integrity of the data?
• Data Acquisition - Know that I know what data is available, how do we get it? Does it have to be brought into the Data Warehouse? Do I need to get an extract from the ERP system?
• Analysis - Looks like our highest revenue customers are large call centres...
• Act - Call these clients as the pilot group to offer them 5% off new equipment installs.
• Measure - Gather feedback from commercial account reps

Feedback comes in stating that many customers offered this discount said they would prefer more free support calls instead. What type of support call volume do we have anyway?

So begins another cycle of analysis to look at support call volume. If this data does not exist in a readily available source such as a Data Warehouse, then we have to go through the whole cycle starting with Data Discovery.

If we go through this again, we will likely lose at least days if not weeks until we are able to determine the next course of action on our journey to retain our most valuable customers. If a company is able to cut out the Data Discovery and Data Acquisition phases, they will be that much farther ahead of the competition.

Having high quality data that is available to decisions makers when they need it is essential to becoming an analytical competitor. Focus on building out a robust information architecture that contains data that is relevant to your business today. With this approach you will literally be two steps ahead of the competition.