You are on page 1of 81

Measure Phase

Introduction to Measure

ASU Lean Six Sigma Green Belt DMAIC

Introduction to Measure
Define Opportunities Measure Performance Analyze Opportunity Improve Performance Control Performance Has fact-based decision making been consistently applied? Are the teams assumptions continuously validated? Is business risk being actively managed? Is leadership informed & on board with the teams findings & conclusions? Can the likelihood of success be improved by revisiting previous conclusions or analysis? 1.0 Define Opportunities 2.0 Measure Performance 3.0 Analyze Opportunity 4.0 Improve Performance 5.0 Control Performance

Staying on track ------------------------------------ Maintaining Perspective

Is the team operating within the context of the project charter? project scope goal statement project plan

ASU Lean Six Sigma Green Belt DMAIC

Changing Our Perspective


Outside-In Focus Drives DMAIC Success

Select the Y that the customer uses to judge your performance

Start with the customer


Measure the same as the customer does Understand the variation in the output (what your process produces that is of value to the customer) Use data to find the process keys that drive the variation

What Is A Measure?
A Measure Is A Description Of One Characteristic Of An Object Or Activity

Measurement can be applied to any type of process or product to assess performance

For a process: Time required to complete process steps Timely execution of process Number of errors in different process steps Percent yield of a process For a product: Errors in a product Product arrives/shipped on time Number of products shipped per month

Uses Of Data And Measurement


Relation To The DMAIC Methodology

Define Measure Analyze

Collect facts about a problem or opportunity and voice of the customer information Establish a baseline performance for Project Y to understand how well we meet customer expectations Identify the root cause of a problem and find the key to solving the problem Evaluate competing solutions based on their impact on performance; degree and direction of change; to compare process performance before and after the solution is implemented

Improve

Control

Quantify the change in process performance to ensure improvement gains are sustained

Process Context For Measurement


CTQ CTQ

Supplier

Input

Process

Output

C O C

Start

Stop

Customer

CTQs

Suppliers

Inputs

Process

Outputs Customers

Input and process measures quantify some aspect of an input to the process or the performance at one or more process steps. They are often referred to as independent variables.

Output measures typically quantify product or service characteristics or process outcomes. They are often referred to as dependent variables.

Output Measures Should Reflect What The Customer Feels

The Relationship Between Process Ys And Xs


Subprocess Ys May Be High-Level Xs
Core Process L1

Cash Delivery Process


Total Cycle Time = Y

Subprocess L2

x = cycle time

Y Credit

x = cycle time Y Documentation

x = cycle time Y Funding

Y = Cycle Time

X Application

X Decision

X
Notification

Quality Measurement
Two Aspects Of Measuring Performance

Effectiveness:

The degree to which customer CTQs are met and exceeded


Some examples: Percent defective Response time Efficiency: The amount of resources allocated in meeting and exceeding customer CTQs Some examples: Cost per transaction Turnaround time

Billing accuracy

Time per activity Amount of rework

Why Is Type Of Data Important?


Choice of data display and analysis tools Amount of data required: continuous data often requires a smaller sample size than discrete data Information about current and historical process performance

Use Continuous Data Whenever Possible

Types Of Data
Data Type Is An Important Consideration

Discrete Data Binary (Yes/No, Defect/No Defect) Ordered categories (1-5)

Continuous Data Can be broken down into increments Infinite number of possible values

Counts

Examples
Number of incomplete applications Percent of responding with a 5 on survey Number of Green Belts trained

Examples
Cycle time (measured in days, hours, minutes, etc.) Weight (measured in tons, pounds, etc.)

Discrete
Ordered Categories (Limited options, i.e., 1-15)

Technically Discrete, but can often be analyzed as Continuous Ordered Categories (Many options, i.e., 1-100) Count Data (Limited possibilities) Count Data (Many possibilities)

Continuous
Cycle Time

Binary (Y/N)

Importance Of Data Type


Sometimes we have choices. When we do, we should choose continuous data
Project Y
Time to process Delivery time Customer satisfaction

Discrete Y Measure
% within specifications Number late Yes/no questions

Continuous Y Measure
Actual times for each unit Actual time deviated from target Rating 1-100

Policies lost due to price

Number lost

Difference from competition

The More Continuous We Can Make The Data, The More It Will Tell Us About Our Process

Plan For Data Collection


1 Establish Data Collection Goals 2 Develop Operational Definitions And Procedures 3 Ensure Data Consistency And Stability 4 Collect Data And Monitor Consistency

Clarify purpose of data collection Identify what data to collect

Write and pilot operational definitions Develop and pilot data collection forms and procedures Establish a sampling plan

Test and validate measurement systems

Train data collectors Pilot process and make adjustments Collect data Monitor data accuracy and consistency

Data Collection Is The First Step To Understanding The Variation The Customer Feels

What To Collect?
Operational Definitions & Procedures
Start:

Measure Data Type Units


Cycle Time C Days

Stop: Instructions

Data Collection Form


Date # Start Stop

MSA / Gage R & R

Sampling Plan How Many When Who Where

Measure MSA% Error Improvement Sources Plan

Training Plan For Collectors


M easurement

Analysis Plan
Frequency

Needs Form Use

Ti me

Measurement

Audit Plan

Establish Data Collection Goals

In order to establish your data collection goals you must:


State the purpose of the data collection Identify what data is required

Asking these questions may help you clarify your goals:

What do I need to know about my process?

What data do I need?


What is the plan for analysis once the data is collected? What data is already available?

Preparing for the Analyze Phase


Segmentation
Collect output data (Y)

To identify patterns and performance trends

Collect segmentation factor data

To be used for later analysis

Segmentation Helps Us Understand Variation In Project Y

Step 1: Establish Data Collection Goals

Segmenting by external factors will help us identify the drivers of variation in the process

Possible categories:

Product Customer Market Time Geography

Common Factors Used For Segmentation


Other Categories

Factor
What type When

Example
Complaints, defects, problems Year, month, week, day

Where
Who

Country, region, city, work site


Business, department, individual, customer type, market segment

Tip: Begin with factors outside the process box - often these are factors that were not considered when the process was first designed

How To Collect Data For Segmentation


Identify the factors for segmentation before you start collecting data Make sure the segmentation factors can be measured reliably Record the segmentation factors for each Y data point collected Segmentation factors are typically easy to collect, so collect more segmentation factors rather than less

Step 1: Establish Data Collection Goals (5 Minutes)


Table Team Exercise

For your team project: Brainstorm a list of segmentation factors for your project Remember to also segment on unlikely parameters Report out to group

Develop Operational Definition And Procedures


Establish Data Collection Goals

Clearly specify variables to be collected: Operational definitions for all metrics Specific descriptions of how to take the measurement Specify the details of the data collection process: How to collect the data How to record the data The period of time for data collection The sampling plan to be followed

Operational Definitions
Defining The Measure
Definition

An operational definition is a clear, concise description of a measurement and the process by which it is to be collected.
To remove ambiguity Everyone has a consistent understanding To provide a clear way to measure the characteristic Identifies what to measure Identifies how to measure it Makes sure that no matter who does the measuring, the results are consistent
Always Pilot Your Operational Definitions

Purpose

Operational Definitions For Output Measures


Features

What: Must have specific and concrete criteria How: Must have a method to measure criteria Must be useful to both you and the customer (the wing-to-wing concept)

Example: Loan Application Cycle Time What: Loan application cycle time is the number of hours from receipt of a loan application, to successful notification of decision for the loan application

How:

The clock starts when the computer attaches the time of application receipt at data entry
The clock stops when the phone caller notes time completed application decision notification in desk log

of

Operational Definitions Scale Of Scrutiny


Choosing The Level Of Measurement

Measure one scale or level smaller than what your customer measures For Example: If your customer measures cycle time in days, your scale of scrutiny would be hours If your customer measures cycle time in hours, your scale of scrutiny would be in minutes Scale of scrutiny may expose larger true variation

Operational Definition Partner Exercise (25 Minutes)


What How

Who All

Time

Desired Outcomes Practice applying operational definitions Collect data on the number of defects in a package of M&Ms

Partner Preparation

Find a partner for the exercise. Determine timing for each activity below. Read the background information. Develop an operational definition for one of the defect types found in an M&M, either, 1) chips and cracks, or 2) unclear and illegible M. The definition should include: What How Importance to customer

Develop An Operational Definition

Partners

Measure And Record Data

With your partner apply your operational definition to your package of M&Ms. Use the form on the following page to record the total number of M&Ms you inspect and the number of defective M&Ms . Note: If an M&M has one or more chips/cracks, classify the M&Ms as defective. Brainstorm the challenges of developing an operational definition for this exercise, and how these challenges may impact your own project work. Choose a spokesperson to report out on your operational definition, the challenges you experienced, and how these may impact your project work in the future.

Partners

Close Exercise

Partners

Operational Definition Partner Exercise (continued)


Background
Customers of M&Ms candy have various needs related to the consumption of the candy. Because the candy should melt in your mouth, not in your hands, one of the Project Y CTQs is for the candy to have no chips or cracks. Part of the internal process for making the candy is printing the letter M on the candy. While not a high priority for external customers it is important to internal customers for marketing and product branding.
Customer Need
No chips or cracks

Business Need
Clear and legible M on the M&Ms

Core Process M&Ms Production

Project Y
M M

Subprocesses 1 2 k

A defective M&M is . . . 1. Any M&M with a chip or crack. 2. Any M&M with an unclear or illegible M. The two defect types should be measured separately.

Operational Definition Partner Exercise (continued)


Data Collection Check Sheet
Date: Location:

Operational Definition:

Data Collectors Name

# Of Pieces Inspected

# Of Pieces Chipped Or Cracked

# Of Pieces With Unclear Or Illegible M

Data Summary Sheet


Data Collectors Name # Of Pieces Inspected % Of Pieces Chipped Or Cracked % Of Pieces With Unclear Or Illegible M % Of Pieces Defective

Operational Definition Table Team (5 Minutes)


Define Your Project Y
Write an operational definition for your Project Y

Write the definition on a flip chart


Report out

Sampling
Objectives

Understand the purpose and advantages of sampling


Understand the application of different sampling techniques to ensure accurate process representation Gain experience in asking appropriate questions to ensure a robust sampling plan is implemented effectively and efficiently Understand guidelines and formulas used to determine sample size

Sampling
Basic Definitions And Symbols

Population (N): The entire set of objects or activities for a process : the mean (arithmetic average) calculated for a population : the standard deviation calculated for a population Sample (n): a group that is a part or subset of a population x: the mean (arithmetic average) of a sample s: the standard deviation of a sample

Sampling
Sampling is the process of: Collecting only a portion of the data that is available or could be available, and drawing conclusions about the total population (statistical inference)

Population
x x x x x x x x x x x x x
N = 5,000 What is the average resolution time?

Sample
x x x x x x
n = 100 From the sample, we infer that the average resolution time (x) is 1.2 days

When To Sample?
When to sample Collecting all the data is impractical or too costly Data collection can be a destructive process When measuring a high-volume process When not to sample A subset of data may not accurately depict the process, leading to a wrong conclusion (every unit is unique e.g., structured deals)

Statistically Sound Conclusions Can Often Be Drawn From A Subset Of The Total Available Data

Goal Of A Useful Sample


Representative Samples

Representative Sample: All parts of the target population are represented (i.e., selected for measurement) equally The customers view is captured How to guarantee a representative sample: Designing the sampling strategy Understand special characteristics of the population before sampling

Important Sampling Concepts


Bias

Bias occurs when systematic differences are introduced into the sample as a result of the selection process Not representative of the population Will lead to incorrect conclusions about the population

Important Sampling Concepts


How Is Bias Introduced?

Strategic Level Developing The Sampling Plan

Selection Bias

Convenience sampling: the ones I can reach Systematic sampling: at noon every day

Environmental Bias

Outdated sample: 1996 external survey results

Important Sampling Concepts


How Is Bias Introduced?

Tactical Level Carrying Out Sampling Plan


Measurement Bias Gage R&R Issues

Inconsistent operational definitions


Inconsistent collectors or procedures (assess using Measurement Systems Analysis)

Non-Response Bias

Initiated by respondents: only a subset of the population responds to survey (typically the 1s and 5s)

Determine Your Sampling Strategy


Where Are You Standing?

Process Data
Population Data

Population Approach Make probability statements about the population from the sample I have 95% confidence that the mean of the population is between1.5 and 2.5 seconds Use sample size formula Process Approach Assess the stability of the population over time Are the shifts, trends, or cycles occurring? Do I take a special or common cause variation approach to process improvement? Use rational subgrouping

Determine Your Sampling Strategy


Where Are You Standing? Process Data Population Data Think about these two questions: 1. What should you do if you are standing at process but wish to use a population approach? 2. What should you do if you are standing at population data but wish to use a process approach?

Sampling Strategy: Random Sampling


Population Study

Population N

Sample n

Description

XXX XXXXX XXXXX XXX

X X X

Each unit (X) has an equal probability of being selected in a sample

Sampling Strategy: Stratified Random Sampling


Population Study

Population
Segments Large Medium Units LLLLL

Sample

Description

MMM MMM MMM MMM SSSS SSSS SS

L MMMM SS

Small

Randomly sample within a stratified category or group Sample sizes for each group are generally proportional to the relative size of the group

Sampling Strategy: Systematic Sampling


Process Study

Process
X X X X X X X X X

Sample

Description

X X

Sample every nth one (e.g., 4th one)

Must select sampling frequency

Sampling Strategy: Rational Subgrouping


Process Sampling

Hour 1

Process Hour 2

Hour 3

Sample
Subgroup of samples

X X X X X X X X X X X X X X X X X X X X X X X X

Description Sample at point A in the process every Xth hour

Sampling Strategy: Process Study


Determine Sampling Frequency

Key points to consider Monitor process frequently enough to catch it going from good to bad Better to collect several small samples over different times then one large sample at a single point in time Unstable process more frequently Stable process less frequently Rapid cycle process time more frequently Long cycle process time less frequently

Sampling Situations Table Exercise (continued)


A

You are interested in estimating the proportion of callers (within 1%) who experience first call resolution. A customer survey will be used to gather the data. A random sample will be used to select potential survey respondents. 1. Are there any potential problems with the approach described? 2. What other approaches might be used in this sampling situation? 3. What other information is required? You are interested in improving billing accuracy and have decided to collect a subgroup sample of 30 bills processed from 4 to 5 p.m., every day for the next 4 days. 1. What sampling scheme(s) is planned? 2. Is there a potential to introduce bias using the plan described? If yes, how would the bias be introduced? 3. What other approaches might be used in this sampling situation?

Sampling Situations Table Exercise (continued)


C A business wants to estimate the total cycle time for deals. There are three types of deals (large, medium and small) and four regional offices (Atlanta, New York, Chicago and Los Angeles). The business randomly sampled deals from the Atlanta regional office who had data readily available.

1.
2. 3. D

Are there any potential problems with the approach described?


What other approaches might be used in this sampling situation? What other information is required?

An improvement team is interested in improving billing accuracy. They decided to sample and pull every 20th bill processed over the next 30 days 1. What sampling scheme(s) is planned?

2. Is there a potential to introduce bias using the plan described? If yes, how
would the bias be introduced? 3. What other approaches might be used in this sampling situation? .

Sample Size For Continuous Data


How Do I Determine Sample Size?

Sample size (n) depends on three things Level of confidence required for the result, How confident I am that the result represents the true population Level of confidence increases as sample size increases Precision or accuracy () required in the result, The error bars or uncertainty in my result Precision increases as sample size increases Standard deviation of the population (s), How much variation is in the total data population? As standard deviation increases, a larger sample size is needed to obtain reliable results
n=

(1.96s )2

In this equation, 1.96 represents a 95% confidence level

Sample Size For Continuous Data


What Is The Performance For Delivery Time?
Population
Y = Delivery Time (Days)

n=

(1.96s )2

Calculate sample size (n) based on: Precision () 95% confidence Level (1.96) Standard deviation (s)

Sample
N Values

X-
X

X+

Calculate average (X)

Conclusion: I know with 95% confidence that the population mean is X +

Finite Population Correction


If You Have A Finite Population

1. Calculate sample size (n) 2. If n/N > .05 OR If n > N 3. Calculate n finite nfinite= n/1+ n/N

Where n = sample size; N = population size

Sample Size For Discrete Data


How Do I Determine Sample Size?
Sample size (n) depends on three things: Level of confidence required for the result, How confident I am that the result represents the true population Level of confidence increases as sample size increases Precision or accuracy () required in the result, The error bars or uncertainty in my result Precision increases as sample size increases Estimated proportion defective of the population (P) Sample size is maximized at P = 0.5 In this equation, 1.96 represents a 95% confidence interval

n = 1.96

( )2 P(1-P)

Sample Size For Discrete Data


What Is The Defect Rate (P) Of A Process?
Population
Y = Proportion Defective

n=

(1.96

P (1-P)

Calculate sample size (n) based on: Precision () 95% confidence Level (2) Estimated proportion defective (P)

Sample
Conclusion: n Values I know with 95% confidence that the population proportion defective is P +

Calculate Proportion Defective (P)

Recalculate n* based on the calculated P. If the new required sample size (n*) is more than the number of samples taken, take (n*-n) samples and recalculate P base on the full sample size. If it is not practical to take more samples, then use the actual n and P to recalculate the actual precision ()

Sample Size Considerations


Beyond The Formulas . . .

The formulas give an approximate sample size Dont forget these important factors! Is the population homogeneous? If not, you will need to segment before sampling What is the opportunity segment for bias? Plan ahead to make sure your data is representative of the true population

What Is The Impact On The Customer If Your Sample Size Is Not Representative Of The Process?

Variation In Measurement Systems


Observed Variation In Data

Actual Process Variation

Measurement System Variation Variation In The Measurement Tool

Gage R&R Issues

Long Term

Short Term

Variation In The Act Of Measuring


Repeatability

Random Bias

Reproducibility
Accuracy Stability Linearity

Address Measurement System Variation Before Collecting Data To Analyze Process Variation

Two Cases Of Measurement Error


+ =

True Process

Random Variation Due To Measurement

Total Process Variation Observed

True Process

Random Variation Due To Bias

Total Process Variation Observed

Measurement System Analysis (MSA)

MSA is a set of methods for estimating the current amount of variation in the measurement process

Plan For Consistency And Stability


Why MSA Is Important

Data is only as good as the process that measures it Identifies how much variation is present in the measurement process Understanding measurement variation is necessary for identifying true process variation and maximizing true Y improvements Without MSA, you run the risk of making decisions based on an inaccurate picture of your process MSA helps direct efforts aimed at decreasing measurement variation Excessive measurement variation distorts our understanding of what the customer feels

Measurement Systems Analysis


Key 1. Questions And Procedures To Answer Them

How much variation is caused by the measurement system? Determine which MSA is appropriate based on the type of data collected Determine which aspects of measurement are most relevant for the MSA study (accuracy, repeatability, reproducibility, stability, linearity) Measure units repeatedly. How items are measured depends on the aspect being quantified 2. Quantify the measurement process variation How much error or uncertainty is allowable for this data? 3. Determine if the measurement process must be improved

What are the sources of measurement error? 4. Determine how the measurement process will be improved
How can the error sources be eliminated or minimized Determine how the measurement process will be improved Audit the measure process to ensure accurate and consistent measurement

Measurement System Analysis


(MSA)
Dont Confuse Measurement Error With Process Variation When Planning A Measurement Study
Call Taking Process Variation: Measurement Error: Application Processing Process Variation: Measurement Error: Deal Approval Process Variation: Measurement Error: Process Variation: Measurement Error: Y = Time to Respond Time to answer question varies for different operators, locations Errors in capturing the time measurement Y = Time to Decision Different types of applications vary in time to complete. Associates vary in their productivity Errors in measuring time time stamp inconsistencies, application arrival times not recorded accurately Y1 = Time to Complete Differing cycle times clients are treated differently Inconsistent start/stop definitions Y2 = Agreeable Terms & Errors in recording net income Conditions Deal value variation (net income generated)

Measurement System Analysis


Step 1: Determine Which MSA Tool Is Appropriate

The type of Measurement System Analysis conducted depends on the type of data: When using continuous data, MSA is conducted through a Gage R&R (repeatability and reproducibility) study When using discrete data (discrete, count or ordered categories), MSA is conducted through a DDA (discrete data analysis) study

Measurement System Analysis


Step 2: Determine Which Aspects Of Measurement Error Are Relevant For Your MSA Study

Accuracy the differences between observed average measurement and a standard Repeatability variation when one person repeatedly measures the same unit with the same measuring equipment Reproducibility variation when two or more people measure the same unit with the same measuring equipment Stability variation obtained when the same person measures the same unit with the same equipment over an extended period of time Linearity the consistency of the measurement system across the entire range of the measurement system

Measurement System Analysis: Accuracy

The difference between observed average measurement and a master or standard


Accuracy

Standard Value

Observed Average

Continuous Difference between observed and standard in measurement units


Discrete Number of instances where the wrong answer was observed

Measurement System Analysis: Accuracy (continued)


Validating accuracy involves repeated measurement of something with a known value (master/standard) The difference between the average of repeated measurements of the same master/standard and the true value of the master/standard represents the amount of inaccuracy or bias in the measurement system Service application: Validating the judgement of the person making the measurement against an agreedupon master/standard

Measurement System Analysis: Accuracy (continued)


Step 3: Measure Units Repeatedly
Discrete Data Example A marketer wants to understand the accuracy of his measurement process to measure the number of misspelled words in the first draft of a marketing brochure. A spell checker is given a brochure and asked to identify the words that are spelled incorrectly. Any difference between the words the spell checker identifies as misspelled, that are in reality correctly spelled (as defined by the standard dictionary), represents the accuracy of the measurement process.

Data Summary

Word Committee Battallion Asbestos Seperate Flamboyant Abbacus Catagory Lieutenant Occassionally Liquefy

Spell Checker 1 C C C I I I I C I C

Standard Dictionary C I C I C I I C I C

C = Spell Checker identified the word as correctly spelled I = Spell Checker identified the word as incorrectly spelled = Differences between Spell Checker and Standard Dictionary
Step 4: Quantify The Measurement Process Variation
Spell Checker 1
Correct Incorrect 1 Correct 4

Standard Dictionary

5 5 5 1 0

Incorrect

Measurement System Accuracy = (4 + 4)/(4 + 1 + 1 + 4) x 100 = 80%

Measurement System Analysis: Accuracy (continued)


Step 3: Measure Units Repeatedly
Continuous Data Example A deal business is measuring the cycle time on deals. A reference set of 20 deals (standard) is collected and a panel of experts determines the true cycle time for each deal. The average cycle time for the 20 deals, as measured by the panel of experts is 5.5 days. The normal measurement process then measures the cycle time for the 20 deals (see chart below).

Measure of Deal Cycle Time


(20 "Standard Deals") 9 3.0SL=8.526 8

Cycle Time (in days)

7 6 5 4 X=5.063

Step 4: Quantify The Measurement Process Variation 3


2 1 0 10 20 -3.0SL=1.599

Standard Deal #
The difference between the average of the 20 deals measured by the panel of experts (the standard) and the average of the normal measurement process represents the bias, or inaccuracy, of the measurement process. Thus bias is Xstandard Xnormal , or 5.5 5.1 = 0.4. This means the measurement process has a bias or inaccuracy of 0.4 of a day in measuring cycle time.

Measurement System Analysis: Repeatability

The variation when one person repeatedly measures the same unit with the same measuring equipment
Repeatabilit y

(Minimum Variation)

Continuous Discrete

Calculate variation in terms of measurement units (standard deviation, span, etc.) Count number of times the same result is achieved for a given point (% correct)

Measurement System Analysis: Repeatability (continued)

Validating repeatability involves repeated measurement of the same item by one person with the same measurement device The difference between the first time an item is measured and the second time represents the error of the measurement process Therefore repeatability is the ability of the person or measurement device to consistently repeat measurements for the same items

Measurement System Analysis: Repeatability (continued)


Discrete Data Example
A marketer wants to understand the repeatability of his measurement process to measure the number of misspelled words in the first draft of a marketing brochure. A spell checker is given a brochure and asked to identify the words that are spelled incorrectly. After a short period, the spell checker is given the brochure and asked to repeat the process. The difference between the first and second spell check represents the spell checkers repeatability.
Data Summary
Word Committee Battallion Asbestos Seperate Flamboyant Abbacus Catagory Lieutenant Occassionally Liquefy First Check C I C I C I I C I C Second Check C I C I C I I C C C

C = Spell Checker identified the word as correctly spelled I = Spell Checker identified the word as incorrectly spelled = Differences between Spell Checks

Measurement System Analysis: Repeatability (continued)


Continuous Data Example
A deal business is measuring the cycle time on deals. A reference set of 20 deals is collected and measured by one person. The same person then re-measures the 20 deals. The difference between the first set of measurements and the second represents the repeatability of the measurement process.

1st 2nd 5.4 5.5 4.7 4.6 5.5 5.5 6.3 6.3 3.9 3.9 4.7 4.5 4.8 4.8 5.5 5.5 4.6 4.6 4.7 4.7 6.2 6.2 5.0 4.8 3.9 3.9 6.9 6.8 4.4 4.5 4.0 4.0 4.8 4.8 7.1 7.2 3.7 3.7 5.2 5.4

Source Total Gage R&R Repeatability Part-to-Part Total Variation

% Contribution 0.43 0.43 99.57 100.00

Measurement System Variation

Actual Deal

Variation

Total Observed Variation

Using either Minitab or Excel we can calculate the percent of the Total Variation contributed by the measurement process in terms of repeatability (see partial printout above). In this case we see that the repeatability of the measurement process is 99.57%, which means that nearly all of the observed variation is coming from the process, not the measurement system.

Measurement System Analysis: Reproducibility

The variation when two or more people measure the same unit with the same measuring equipment Data Collector 1

Data Collector 2 Reproducibility

Continuous
Discrete

Calculate the difference between two people in terms of measurement units


Calculate the difference in number of times each person achieved a given result (% difference)

Measurement System Analysis: Reproducibility (continued)

Validating reproducibility involves repeated measurement of the same item by two people using the same measurement device The difference between the two measures represents the ability of the measurement process to be reproducible Therefore reproducibility is the ability of the measurement process to consistently reproduce measurements for the same items across people

Measurement System Analysis: Reproducibility (continued)


Discrete Data Example
A marketer wants to understand the reproducibility of his measurement process to measure the number of misspelled words in the first draft of a brochure. Two people called spell checkers are given a list of words and asked to identify the words that are sp elled incorrectly. Any difference between the words the two spell checkers identify as misspelled represents the reproducibility of the measurement process.

Word Committee Battallion Asbestos Seperate Flamboyant Abbacus Catagory Lieutenant Occassionally Liquefy
C I

Spell Checker 1 C C C I I I I C I C

Spell Checker 2 C I C I C I I C C C

= Spell Checker identified the word as correctly spelled = Spell Checker identified the word as incorrectly spelled

= Differences between Spell Checkers


Spell Checker 1

Correct Correct
Spell Checker 2 5 4

Incorrect
12 6

Incorrect

0 1 5

43 5

10

Measurement System Reproducibility = [(5 + 4)/(5 + 1 + 0 + 4)] x 100 = 90%

Measurement System Analysis: Reproducibility (continued)


Continuous Data Example
A deal business is measuring the cycle time on deals. A reference set of 5 deals is collected and measured by two people, and then one of the same people re-measures the 5 deals. The difference between people represents the reproducibility, while the difference between the first and second measure by the same person represents repeatability.
Cycle Time 5.4 5.4 5.5 5.5 3.9 3.9 4.8 4.7 4.7 4.7 5.3 5.3 5.5 5.5 3.8 3.9 4.7 4.7 4.5 4.5 Deal # 1 1 2 2 3 3 4 4 5 5 1 1 2 2 3 3 4 4 5 5 Measurer # 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2

Using either Minitab or Excel we can calculate the percent of the Total Variation contributed by the measurement process in terms of reproducibility and repeatability (see partial printout below). In this case we see that the reproducibility and repeatability of the measurement process is 98.52%, which means that nearly all of the observed variation is coming from the process, not the measurement system.

Source Total Gage R&R Repeatability Reproducibility Part-To-Part Total Variation

% Contribution 1.48 0.23 1.25 98.52 100.00 Variation due to measurement system

Actual deal variation

Measurement System Analysis: Stability

The variation obtained when the same person measures the same unit with the same equipment over an extended period of time
Stability Time 1 Time 2

Continuous Discrete

Calculate variation change in measurement over time Calculate change over time (% correct or consistent)

Measurement System Analysis: Linearity

Linearity is the consistency of the measurement system across the entire range of the measurement scale

Continuous:

The endpoints of a pressure gage are typically not as accurate as the center of the gages range Assessing items for defects is easy in very obvious cases, but can be very difficult in borderline or less clear cases. Operators may have very consistent judgement performance at the beginning of a shift, but poor consistency before breaks or near the end of a work shift

Discrete:

Conducting An MSA
Step 5: Determine If The Measurement System Must Be Improved

Examine context of business environment, process, and customer How critical is the measurement? What are the risks of making an error? Review results of MSA study Typical Gage R&R specs: % < 30% of total process variation <10% of specification window

What Is The Impact Of Error On The Measurement?

Conducting An MSA
Step 6: Determine How The Measurement Process Will Be Improved

Identify factors that could cause measurement process variation (measurement error) Reduce the impact of those factors

Conducting An MSA
Step 7: Audit The Measure Process To Ensure Accurate And Consistent Measurements

Determine how often (frequency of audits) Determine what to audit Procedures/documentation up-to-date Procedures/documentation used Quantify MSA

Measurement Systems
Summary

Measurement error is always present in your total observed variation Minimize the measurement process variation Use MSA to identify the amount of process variation Measurement error is always a bigger deal than you think Understand how measurement error impacts your customer

Make Sure Your MSA Is Examining The Actual Measurement System Itself

Measurement Systems Analysis Breakout (35 Minutes)


What Team Preparation

How Choose facilitator, timekeeper, scribe

Who All

Time 1 min

Part 1: MSA For Your Project

Develop an MSA for your project Y data 1. Determine which MSA is appropriate for your data 2. Determine which aspect of measurement are relevant (accuracy, repeatability, etc.) 3. Develop the plan for how you will collect the data 4. List factors that might cause the measurement of an item to vary and how you would reduce the impact of those factors OR

All 1 min 5 min

15 min 13 min

Part 2: If You Do Not Have A Project

How would you run a MSA on the following conditions: 1. Collecting dates from insurance files 2. Errors on billing statements 3. The number of customers contacted/converted by a broker 4. Call-center: categorizing call types

All

34 min

Establish Data Collection Goals

Develop Operational Definitions And Procedures

Ensure Data Consistency And Stability

Collect Data And Monitor Consistency

Collect Data And Monitor Consistency


Communicate the what and why to the data collectors and process participants Train everyone who will be collecting data Pilot the data collection process and adjust as needed Confirm understanding of operational definitions Make data collection procedures error-proof Be there in the beginning to oversee data collection

Collect Data And Monitor Consistency

Check to make sure the measurement system is stable Check to make sure the measurement procedures remain consistent (over time, and from data collector to data collector) Check to see if the data look reasonable

Summary of Measure Performance

2.1 Determine what to measure Understand the role that data plays in process improvement Understand the cause and effect relationships that occur inside the team's process Determine the indicators needed to evaluate current process performance 2.2 Manage measurement Understand different types of data and how each type can provide the team with different insights and knowledge of a process Develop operational definitions and data collection plans that build validity and consistency in the data which the team gathers 2.3 Understand variation Understand the concept of variation and how a process can be evaluated by assessing its variation over time Plot and calculate the variation of the team's business process Gain hands-on experience with the use of the statistical software package MINITAB
ASU Lean Six Sigma Green Belt DMAIC

Summary of Measure Performance (contd)


2.4 Determine Sigma performance

Understand the various calculations associated with determining process sigma Calculate the sigma performance of the team's process Understand the difference between First Pass Yield and Rolled Throughput Yield

2.5 Managing the measurement system


Understand the different uses of our measurement systems Understand the language of measurement Understand how to conduct a measurement system analysis Understand how to interpret the results of a MSA study Understand MSA in administrative processes

ASU Lean Six Sigma Green Belt DMAIC

You might also like