Product design reviews
Product design reviews are formal, documented, systematic evaluations of product designs that are performed at various stages of product development. The purpose of these reviews is to identify any problem areas that may exist in the design or design process and to initiate corrective action to address these areas. Design reviews are important control measures for ensuring that designs meet performance requirements as well as identify potential safety hazards.
Product safety reviews can identify potential hazards that may exist in a product or process; ensure that safeguards have been provided and are adequate; and establish controls for producing the product according to specifications. In litigation, they can also be used as evidence that a company acted reasonably.
Once a product is in production and out in commerce, it becomes substantially more expensive to make changes. One author cites a product safety incident where the cost of correcting a product already in the marketplace was over 16,000 times the cost of addressing the problem at an early stage in product development.[6]
This report provides an overview of the product design review process and issues that should be considered when developing a review process. For information on legal issues associated with design defects, see Design Defects on Hanover’s Risk Solutions website.
Safety reviews
Representatives from various functions in the company, such as design, production, quality control, sales, legal, etc., should participate in the formal review of product designs. This team approach to evaluating product safety will bring diverse perspectives to the process. These reviews should be in addition to any informal safety reviews that are performed by the design team during the development of the design.
Objectives of review
A design safety review has several key objectives. These include:
- To identify and correct hazards to prevent injury and illness
- To ensure that the product complies with applicable product safety regulations
- To ensure that the product conforms to any relevant, voluntary product safety standards
- To resolve outstanding safety- and health-related issues
- To contain project cost by reducing redesign and rework
Timing of design reviews
Design reviews may be performed at several different stages during the design process. The timing of these reviews may be based upon arbitrary points in the development process (e.g., every two weeks, at set milestones in product development, or when design specifications have been developed).
Typical milestones where formal design reviews may occur include:
- When target specifications have been developed
- When a conceptual design has been created
- When the product architecture has been developed
- When detailed design specifications have been completed
- Prior to production ramp-up
Design reviews performed early in the concept phase of the design, before specifications are developed, will involve reviews of information on similar products produced by organizations or competitors and identifying potential product hazards involved with a design concept. As the design progresses, more information will become available about the proposed product and the review will focus more on the particular characteristics and specifications of the proposed product and its manufacturing process. At the end of the design process, reviewers will have a physical prototype or first-made product to evaluate, as well as initial results of product safety tests.
Scope of the review
In general, the scope of a design safety review should be to identify and address any product safety concerns that arise from a proposed product design. The specific scope of an individual review meeting will vary depending upon the design culture of the organization and the timing of the review.[1]
Performing design reviews
The following section provides general recommendations for performing design reviews.
Process
The organization should develop a formalized policy and procedure for performing design reviews. The policy should identify the purpose of the review and why the organization is performing the design review. A senior company official should sign off on the policy to demonstrate management commitment to the project.
The procedures should include specific process measures for holding responsible persons accountable for completing the review. Measurement tools should be aligned with other project measures, such as scheduling and cost goals. This joint accountability is important to ensure that all the participants of the product development team work together.
Participants
A multidisciplinary team should perform the design review. The people for the review should represent the key functional areas in the company. The people should have differing backgrounds and experience so that the product may be looked at from multiple viewpoints. Also, they should not have participated in the development of the product. That way, the product can be reviewed with a new and unbiased viewpoint. Not all participants may be required for every review. If the design is being performed for a client, a representative from the client should be invited to participate.
The head of the team should be a senior official in the organization. Senior management involvement is essential for demonstrating that the proceedings have high priority and for assuring that a rigorous review is performed. Also, the senior manager may be used to reduce interdepartmental conflicts or defensive reactions to opinions.
At least one participant should have specific training and knowledge in safety engineering principles. Organizations should not assume that engineers or designers routinely possess these skills. A survey of mechanical engineers performed in the early 1990s identified that, although many design engineers recognize the value of safety on good product design, many engineers are either unaware of or have not been trained in, formal safety analysis techniques. [1] The majority of the respondents had not taken a single course in safety in college or attended any lectures or seminars specifically addressing product safety.
Guidance document
A guidance document should be developed that identifies the procedures the participants should follow at each stage of the design review process and instructions for performing the procedures. The document may also include any standardized analysis tools that the reviewers will use when performing the review.
Checklists are often recommended for performing reviews. Such tools are helpful for expediting design reviews of repetitive design tasks or where variations in design are very small. They are also useful for making sure that key considerations have not been overlooked. Also, the creation of a checklist for a new product requires the checklist designer to complete a hazard analysis in order to create the checklist.
Checklists do have limitations and should not be used as a replacement for a formal safety analysis. The most important limitations are their completeness, currency and applicability. If the checklist is not complete, the user may miss a potentially significant hazard. If the checklist is not current, the user may not be evaluating the product against the current state-of-the-art bar. If there are too many, or too significant, differences between the product being reviewed and the product it was designed for, the checklist may not provide an accurate review of the hazards present.[2]
Corrective action
All possible product hazards that are identified during the review process should be categorized according to the type and degree of hazard. The hazards should then be corrected to the degree of risk acceptable to the company and the decisions should be documented.
Hazard categorization
Hazards should be categorized according to their expected frequency of occurrence and the likelihood of severity. The criteria used may be qualitative (e.g., high probability, medium probability, or low probability of failure) or quantitative (e.g., one in one million, one in one thousand, or one in ten). These criteria can be used to rank the priority of identified hazards and be used when identifying possible corrective action measures.
Hazard elimination and control
Product safety practitioners have established a preferred hierarchy for corrective actions using system safety principles. In general, this hierarchy consists of four basic options. In order of preference these are:
- Eliminate the hazard through design
- Protect or guard against the hazard
- Warn users about the hazard
- Train users to avoid the hazard
In cases where the identified hazard is so extreme, too expensive to fix, or beyond the risk tolerance of the organization, the organization may choose to end product development.
Factors affecting decisions
Corrective action decisions may be affected by internal and external factors. These include making trade-offs between the risk posed by the hazard and the utility or cost. The decisions made by a company will depend upon the risk tolerance of the company. Decisions made regarding trade-offs in corrective action should be documented to provide a record of the organization’s rationale for the decision.[6]
Third-party reviews
There are several nationally recognized testing laboratories (NRTL) that provide product safety assessment services. These services can range from initial design reviews, to the testing of prototypes to verify that a design meets a particular safety standard. The organization may even certify that a design meets a particular safety standard.
Product manufacturers should consider using a third party reviewer as an external check on the soundness of the design. The reviewer can provide a more impartial assessment of the product since there is no emotional attachment to the product design. Certification of the design can provide a marketing advantage. Also, a third party certification may be useful should the design later be challenged.
The review should be documented and any hazards identified by the review addressed in a timely manner.
Documentation
The product design review process should be documented to provide a record that the examination occurred and the issues covered by the review. The documentation should include information on the follow-up actions that occur.
Recordkeeping serves several different functions. It is important for litigation defense because it provides a factual record that the review did occur, indicates the thoroughness of the review, and states the reason for design choices. This is important to show that the manufacturer acted in a reasonably prudent manner and to help defend against arguments that the product could have been made safer with a reasonable alternative design. (See Design Defects on Hanover’s Risk Solutions website for additional information on the legal aspects of product defects.) Also, it provides design organizations with a history as to why certain designs were approved or rejected and the concerns posed by the design in case the product is revisited in the future.
Specific documentation that should be maintained includes:
- Design drawings and specifications
- Copies of voluntary standards and regulations applicable to the design
- Minutes of formal design review meetings
- Hazard analyses and corrective actions
- Test results
- Assessments of alternative designs
- Correspondence with clients and third party reviewers
In most cases, not all documentation will be positive. However, negative documentation should not simply be discarded; otherwise, it has the potential to become a “smoking gun.” Instead, the negative comments should be addressed in writing by someone in authority and included in the product file.
Evaluating design review effectiveness
Design review is a demanding and complex task. The following are general questions that may be used to evaluate a design review.[3]
- Are design safety reviews required for all new product designs?
- Does the design organization have a clearly assigned responsibility for performing product safety design reviews?
- Are design safety goals incorporated in the performance objectives of designers and engineers?
- How much training is provided annually to participants on the safety-related aspects of product design?
- Has the organization established an independent group representing functions in addition to design for conducting product safety reviews?
- Does a senior executive chair the review group?
- Is technical guidance available to reviewers?
- Are records kept of all proposed design review changes?
- Are records kept which explain why a design change was or was not accepted?
Summary
There are three levels of formal design reviews: preliminary, intermediate, and final design review. Each phase offers benefits to the manufacturer and reduces the chance that they will manufacture and sell an unsafe product. The process of design reviews has additional benefits to an organization including cost improvements, increased communication with customers, and education of staff so that future designs have fewer changes required. [1]References
- CCH Incorporated. Managing for Product Liability Avoidance, 3rd edition. Chicago, IL: Stacey Caywood, 2004.
- Case, T. and Michael Keating. Designing an Effective Products Liability Compliance Program, 2013-2014 edition. Eagan, MN: Thomson Reuters Westlaw, 2013.
- Goodden, R. Product Liability Prevention: A Strategic Guide. Milwaukee, WI: Quality Press, 2000.
- Hammer, W. Product Safety Management and Engineering Second Edition. Des Plaines, IL: American Society of Safety Engineers, 1993.
- National Safety Council (NSC). Product Management Guidelines, Second Edition. Chicago, IL: NSC, 1997.
- U.S. Consumer Product Safety Commission (CPSC). Handbook for Manufacturing Safer Consumer Products. Washington, DC: CPSC, 2006. https://www.cpsc.gov/s3fs-public/pdfs/blk_pdf_handbookenglishaug05.pdf.
Appendix A: design review activities at various stages of product development
The following lists provide descriptions of activities that may occur at various stages of new product development. They are adapted from Figure 12-2 of Product Safety Management and Engineering, Second Edition. Consult the reference directly for a more comprehensive listing of activities.
Concept phase activities
Concept phase review activities include ensuring that the product developer has:
- Reviewed similar products for safety concerns
- Performed a preliminary hazard analysis on the proposed and accepted design concept
- Reviewed and taken into account any applicable safety standards and regulations when developing the preliminary design specification
- Established additional requirements for safety concerns not addressed by standards and regulations
- Determined basic criteria for evaluating components and external suppliers
- Initiated a means for monitoring safety concerns during product design
Design phase activities
Design phase review activities include ensuring that the product developer has:
- Conducted safety tests on all materials, components, and devices to be used in the product
- Evaluated test results to determine feasibility of design
- Identified safety-critical materials, components, and procedures
- Established procedures for coordinating activities between groups responsible for various design activities
- Established procedures for reporting design problems
- Reviewed the effect proposed design changes have on product safety
- Documented analyses, test results, and corrective actions taken to address each safety concern
- Conducted all activities required by mandatory regulation and standard
Pre-production activities
Pre-production review activities include ensuring that the product developer has:
- Developed and tested a prototype of the product
- Evaluated the prototype test results for safety concerns
- Addressed any safety concerns raised by prototype testing
- Documented any changes made to the design
- Ensured that changes are distributed to appropriate people
- Performed a safety analysis of the final product design
- Documented the results of this analysis
- Reviewed all advertising materials, dealer training materials, warnings, and instructions for safety content
- Trained field service and dealer personnel on safety-related subjects
- Made production and quality personnel aware of safety-critical items
To learn more about Hanover Risk Solutions, visit hanoverrisksolutions.com
Copyright © 2019, ISO Services, Inc.
The recommendation(s), advice and contents of this material are provided for informational purposes only and do not purport to address every possible legal obligation, hazard, code violation, loss potential or exception to good practice. The Hanover Insurance Company and its affiliates and subsidiaries ("The Hanover") specifically disclaim any warranty or representation that acceptance of any recommendations or advice contained herein will make any premises, property or operation safe or in compliance with any law or regulation. Under no circumstances should this material or your acceptance of any recommendations or advice contained herein be construed as establishing the existence or availability of any insurance coverage with The Hanover. By providing this information to you, The Hanover does not assume (and specifically disclaims) any duty, undertaking or responsibility to you. The decision to accept or implement any recommendation(s) or advice contained in this material must be made by you.
LC NOV 2018-346
171-1746 (6/19)