Selecting Effective Metrics and Dashboards for Process Improvement Efforts
Add bookmark
Metrics and dashboards play an integral and critical part of process improvement efforts. Metrics complement the Six Sigma methodology by providing vital signs of the effectiveness and the efficiency of the process improvement efforts.
Metrics help the organization focus on the most critical business factors (Key Performance Indicators, KPI) that have a profound impact on sustaining business success. Metrics also help with the understanding of the current status of the process with decision making and making reliable prediction. Metrics are useful in planning (for cost, schedule, quality, risk management, etc.), in process analysis and improvement in monitoring and controlling Six Sigma projects. Dashboards are used to enhance process and Six Sigma project performance using PDSA cycles of the DMAIC model.
Organizations that do not use objective metrics but rather subjective assessments of progress or measure process improvements using incorrect metrics commonly exhibit wide variability among staff and processes, low morale, lack of clear goals and poor performance among other symptoms.
Commonly used metrics are key process objectives (key process indicators, KPI), customer objectives, staff related objectives, financial objectives, safety, satisfaction, quality and morale. Common Six Sigma metrics include defects per million opportunity (DPMO), Process Capability (Cpk), cost of poor quality (COPQ) and sigma level.
Metrics could be at the strategic or enterprise level or at the tactical or operational level. At an enterprise level metrics could be shareholder value and brand value. At the operational level metrics could include quality, cost, staff satisfaction, productivity, cycle time, etc. When designed and implemented correctly, metrics can align the organization in achieving mission critical objectives.
The Benefits of Using Metrics and Dashboards in Healthcare
Healthcare organizations that use metrics and dashboards have reported several benefits. Dashboards promote accountability, clarify and bring focus to objectives, promote systems thinking and problem solving (statistical analysis, special cause variation vs. common cause variation), and enable project selection, prioritization and resource allocation (continuous improvement). Metrics serve as an early warning system and facilitate decision making, assess business performance, guide performance management, identify what to target for process improvement and change an organization’s problem solving approach from reactive to pro-active.
The common steps involved in using metrics consist of the following: collecting data, analyzing data, making decisions based on data, evaluating results and repeating cycle (PDSA) to improve accuracy of predictions.
Some Helpful Hints for Metrics Selection
Ask the question: "How does the proposed metric relate to what the customer values?" Verify that you are using the correct data and present it in an easy to comprehend format (for the intended audience), at the right time to the right person. Use only those decisions that are critical to the customers of the process.
Good metrics being tracked by the organization should provide both an internal and external perspective (strengths and opportunities for improvements should be included). It is equally important that one has a good understanding of what is being measured, why it is being measured, who utilizes the data and how this is utilized to determine if the metric is relevant to the business. The emphasis of any metric should be on timely collection of data, analysis and dissemination of the analysis to the key constituents.
Some pitfalls to avoid: Watch out for the desire to collect perfect data in order to avoid analysis paralysis. It is important to have laser like focus on a few key metrics rather than to track a multitude of metrics. Remember, you manage only what you measure.
Managing without metrics is like flying blind. While you cannot manage what you do not measure, remember that measurement costs money; if measurement is done incorrectly, this can result in your organization focusing on the wrong areas for process improvement and could result in wrong decisions.
Wrong measurements could be due to the misunderstanding of what is being measured, incorrect data gathering and analysis resulting in wrong conclusions. Using metrics in the wrong context could do more harm than good. For example, utilizing hospital bed utilization as a proxy for profitability may not work in all cases, especially if the hospital is losing revenue due to under-insured or un-insured patients. This could lead to suboptimal results for profitability. Watch out for aggregated metrics, especially if the separate measurements are strongly correlated (as it provides no additional value).
Another pitfall with the aggregate metric is that it could hide inefficiencies or show wide variation when none exists, hence the need to understand what is being measured. Also watch out for delays in metrics updates (time lag between data collection and analysis), measuring parts of the process and not looking at the big picture and not having accountability (or plans when off course.)
Summing Up Metrics and Dashboards
In summary, metrics are be used as headlights (pro-active to see what is ahead) rather than as a "rear view mirror" (review of historical performance). Metrics and dashboards should translate the organization’s vision into focused strategies and key performance indicators, align the organization's strategic goals with the operational metrics and provide feedback at each level of the organization so that the organization can make decisions and meet business objectives and goal.
Metrics help the organization focus on the most critical business factors (Key Performance Indicators, KPI) that have a profound impact on sustaining business success. Metrics also help with the understanding of the current status of the process with decision making and making reliable prediction. Metrics are useful in planning (for cost, schedule, quality, risk management, etc.), in process analysis and improvement in monitoring and controlling Six Sigma projects. Dashboards are used to enhance process and Six Sigma project performance using PDSA cycles of the DMAIC model.
Organizations that do not use objective metrics but rather subjective assessments of progress or measure process improvements using incorrect metrics commonly exhibit wide variability among staff and processes, low morale, lack of clear goals and poor performance among other symptoms.
Commonly used metrics are key process objectives (key process indicators, KPI), customer objectives, staff related objectives, financial objectives, safety, satisfaction, quality and morale. Common Six Sigma metrics include defects per million opportunity (DPMO), Process Capability (Cpk), cost of poor quality (COPQ) and sigma level.
Metrics could be at the strategic or enterprise level or at the tactical or operational level. At an enterprise level metrics could be shareholder value and brand value. At the operational level metrics could include quality, cost, staff satisfaction, productivity, cycle time, etc. When designed and implemented correctly, metrics can align the organization in achieving mission critical objectives.
The Benefits of Using Metrics and Dashboards in Healthcare
Healthcare organizations that use metrics and dashboards have reported several benefits. Dashboards promote accountability, clarify and bring focus to objectives, promote systems thinking and problem solving (statistical analysis, special cause variation vs. common cause variation), and enable project selection, prioritization and resource allocation (continuous improvement). Metrics serve as an early warning system and facilitate decision making, assess business performance, guide performance management, identify what to target for process improvement and change an organization’s problem solving approach from reactive to pro-active.
The common steps involved in using metrics consist of the following: collecting data, analyzing data, making decisions based on data, evaluating results and repeating cycle (PDSA) to improve accuracy of predictions.
Some Helpful Hints for Metrics Selection
Ask the question: "How does the proposed metric relate to what the customer values?" Verify that you are using the correct data and present it in an easy to comprehend format (for the intended audience), at the right time to the right person. Use only those decisions that are critical to the customers of the process.
Good metrics being tracked by the organization should provide both an internal and external perspective (strengths and opportunities for improvements should be included). It is equally important that one has a good understanding of what is being measured, why it is being measured, who utilizes the data and how this is utilized to determine if the metric is relevant to the business. The emphasis of any metric should be on timely collection of data, analysis and dissemination of the analysis to the key constituents.
Some pitfalls to avoid: Watch out for the desire to collect perfect data in order to avoid analysis paralysis. It is important to have laser like focus on a few key metrics rather than to track a multitude of metrics. Remember, you manage only what you measure.
Managing without metrics is like flying blind. While you cannot manage what you do not measure, remember that measurement costs money; if measurement is done incorrectly, this can result in your organization focusing on the wrong areas for process improvement and could result in wrong decisions.
Wrong measurements could be due to the misunderstanding of what is being measured, incorrect data gathering and analysis resulting in wrong conclusions. Using metrics in the wrong context could do more harm than good. For example, utilizing hospital bed utilization as a proxy for profitability may not work in all cases, especially if the hospital is losing revenue due to under-insured or un-insured patients. This could lead to suboptimal results for profitability. Watch out for aggregated metrics, especially if the separate measurements are strongly correlated (as it provides no additional value).
Another pitfall with the aggregate metric is that it could hide inefficiencies or show wide variation when none exists, hence the need to understand what is being measured. Also watch out for delays in metrics updates (time lag between data collection and analysis), measuring parts of the process and not looking at the big picture and not having accountability (or plans when off course.)
Summing Up Metrics and Dashboards
In summary, metrics are be used as headlights (pro-active to see what is ahead) rather than as a "rear view mirror" (review of historical performance). Metrics and dashboards should translate the organization’s vision into focused strategies and key performance indicators, align the organization's strategic goals with the operational metrics and provide feedback at each level of the organization so that the organization can make decisions and meet business objectives and goal.
