This post was
inspired by a question from a colleague in the medical device industry. To
summarize the question:
How can you find
previously unknown critical to quality variables (CTQs) to drive preventative
action instead of reaction?
I think there are
5 general steps to approaching this question.
- Define critical to quality (CTQ)
- Setup a methodology to find unknown CTQs
- Engage your team and focus your method on a single product line, code, feature, or process step depending on the level of product/process complexity
- Test the hypothesis of an unknown CTQ with extreme variations to the current nominal and evaluate the impact.
- If the impact is low, then it's not likely a CTQ.
- If the impact is high, then more detailed analysis is warranted to confirm a CTQ.
- Repeat
Step
1: What is critical to quality (CTQ)?
- What is critical to quality (CTQ)? Information.
- I know that seems simplistic, but it leads me to this question: How many of the issues you have worked on were a result of a lack of information? I would guess all of them since you performed an investigation.
Ever heard these
phrases during those investigations?
- The design is fine, there must be something else
- We didn't think that would have any impact (so we didn't characterize it)
- Nothing's changed from when we ran the qualification
- Each one of those phrases is a result of lack of information. Whether it's considerations that weren't made during the design process, variables that were overlook during a qualification effort, or drifts in tribal knowledge, at the core they are all a lack of information.
- I think any organization struggles with defining what truly is critical to quality and what is not. I think the biggest reason for the struggle is because of the wide variety of products and processes across the medical device industry. Everyone's looking for the end answer when usually every product or process requires a deep dive to truly understand what is critical to quality for that particular product or process.
Step
2: How can you find CTQs?
- Start with a simple question: Where do we have a lack of information?
While that once
again may seem simplistic, it can help your team focus discussions on what's
important when searching for CTQs. Some of the questions below can help
guide a deeper dive and discussion to determine if there is not enough
information to answer the question. If there isn't enough information to
answer the question, then there could be an unknown CTQ variable lurking
around.
Good
places to start:
- Manufacturing procedures
- Ask the question: If you handed the procedure to a co-op with all the necessary components and tooling, would they be able to properly make the end product?
- I use the example of a co-op because often asking someone this question who has less knowledge about a product or process will help guide your search. The questions they ask when trying to follow a manufacturing procedure will let you know where there is room for interpretation and drift in process steps.
- If the team thinks that step could be a CTQ, I would talk with each operator performing that action and determine if they have a particular method they all use that isn't clearly defined in the procedure then it could be a CTQ.
- How many manual steps does the procedure require?
- How long are operators typically trained?
- How much experience does the average operator for this product have?
- Lots of experience means lots of tribal knowledge that may or may not have been captured.
- Raw material specs and incoming inspection records
- How large is the allowable band on the various material components?
- What's the range and standard deviation of the components analyzed on materials certs?
- There are two situations that come to mind:
- High variation in raw material composition
- Have you had historical issues with specific or concurrent lots of product that don't have? A CTQ may be found when correlating those lots to raw material compositions.
- Have you had no historical issues with the product? Probably a lower chance of a CTQ.
- Low to no variation in raw material composition
- Little variation of material composition is indicative of a potential CTQ. If you have never received a lot of raw material outside of a narrow band within the allowable spec, then you don't know how your process will behave if your vendor's process shifts (they want to improve their quality too, but could shift their composition if they change their standard process window)
- Qualification documentation
- What variables were measured during the qualification for both process and product?
- Could more have been measured that may have seemed non-critical at the time?
- How long ago was the qualification run?
- Are the people who performed the original qualification still around? Has tribal knowledge about that product drifted?
- Manufacturing environment
- Are any components or raw materials stored in an uncontrolled environment prior to final assembly?
- Do raw material and component vendors have uncontrolled storage environments?
- Are any of the product components or materials sensitive for humidity or temperature changes?
- Adhesives, Nylon, rubbers, etc.
- Are there any operations that would be affected by static changes?
- Adhesives, inks, coatings, etc.
- Product/Process design
- Has anyone ever voiced concerns about an aspect of the product design?
- Is this a legacy product with little change over the long term?
- Has the product or process had many changes over time or very few? An older product with little to no changes over time likely has a lot of CTQs that are not well defined.
- How extensive was the process design and development?
- Are there new manufacturing methods or technology that would improve the process that didn't exist when launched?
- How new was the design or process when it was implemented? Has it been being characterized since that time in detail or only at a high level?
- Project Questions
- How well was the original project run? Were there many delays? Many delays is a sign of challenges that can result in a rushed aspects of the project. Rushed actions make it more likely that something was overlooked.
- Did the team work well together? If they did not, then they would be more likely to overlook something.
- Was the product rushed to market? Could an important variable have been overlooked?
- How much VOC was performed prior to the product being designed?
- What about when there is a lot of information?
- Throughout the analysis there could be situations in which there is a lot of information. That could mean that a CTQ is already well known and characterized. It could also mean that because there was so much information, that an important variable was missed.
- In these cases, take a step back and ask the original question. Where is there a lack of information? It can allow you to take a step back and evaluate a large amount of data from a top-level view to see what's missing instead of being overwhelmed by a mountain of data.
Step
3: Engage your team and focus your method on a single product line, code,
feature, or process step depending on the level of product/process complexity
- Building the right team and engaging them in the right way will greatly improve the efforts of your CTQ search. The focus should be able collaborative problem solving.
- Team Composition
- People with a good bit of knowledge and experience
- Manufacturing engineer
- Line supervisor
- Operator
- Design engineer
- Quality engineer
- People with less knowledge and experience
- At least 2, to provide fresh ideas and unbiased perspective on information
- Ensure they are encourages to ask the team any questions they come up with
- Break the analysis into smaller chunks. With any CTQ search there will be a lot of information and complexity to work through, so it will need to be guided through smaller chunks by whoever is facilitating the search.
Step
4: Test the hypothesis of an unknown CTQ with extreme variations to the current
nominal and evaluate the impact.
- Test extreme variations of the CTQ in a simple environment first
- Is it a readily changed process variable (time, temperature, pressure, or speed)? Make a few runs outside of the current process window in a small scale and compare the results.
- Evaluate how to proceed
- If the impact is low, then it's not likely a CTQ.
- If the impact is high, then more detailed analysis is likely warranted to confirm a CTQ.
Step
5: Repeat
In Summary
- Define CTQ as information.
- Continually ask: Where do we lack information?
- Evaluate smaller pieces of the product/process to ensure focused analysis
- Test the hypothesis of a CTQ by testing extreme variations quickly and iteratively
- Repeat
I hope this has
help spark some thought around what a CTQ analysis method could look like and
how to approach the various items that can be analyzed to help predict if an
unknown CTQ is likely or not. I would appreciate any feedback.