Search for the term design life cycle in Google Images and you will find a plethora of circular flowing images, creating the impression that a design life cycle is an abstract concept, one that guides the flow rather than provides any detailed structure or definition to the design process.
While an abstract guide may be useful, it has little place in a regulatory or legal context. Regulations aren't about good guidance such as writing clean code; regulations and standards need to provide requirements which at least in principle can be both interpreted consistently and where compliance can be verified.
Fortunately, a close look at IEC 62304 finds that a design life cycle is in fact well defined and verifiable requirement. A life cycle consists of defined phases, each with specific inputs and outputs (deliverables). These inputs and outputs form the tangible elements which can be checked for plausible implementation.
For example, a realistic life cycle might start with an "Draft documentation" phase, which has no formal inputs, but outputs documents such as the draft project plan, system specifications, initial risk analysis. The next phase may be "Stage 1 Approval" which reviews and approves those documents (as inputs) and and creates a review report, and formally approved plan, specifications, risk analysis (as outputs). A later stage might be "Final Testing" which uses the test plan, starting code, prototype, with the output being the final code, configuration report, test reports, summary report, completed risk traceability matrix, design change records and so on.
As such, a design life cycle starts when the first document is approved (typically the plan), and continues to control the activities in the design process. It is closely related to a design plan: a plan includes the life cycle and adds supporting information such as requirements for document control, configuration management, change control, definitions for responsibility and authority, linkage to system specifications and so on.
Life cycle timing
Given that a regulatory life cycle starts when the first document is approved (and not at the brainstorming meeting or when the designer has a 3am eureka moment), it is an important question to ask: when should this formal life cycle start?
Experts are likely to say "as soon as possible", or "whenever any significant design activity occurs". These answers, while said with all good intentions are both wrong and often dangerous.
The correct answer is: whenever the manufacturer likes. There is no legal obligation to keep records of the design process as it happens. A manufacturer can spend 10 years in R&D, legally shred all the records covering the messy development period, and then start a 6 month period creating all the documents required by ISO 13485, IEC 14971, IEC 62304, IEC 62366 and other management standards standards, and be perfectly in compliance with all standards and regulations.
And in fact, this approach is far more safe than it may appear.
Why do experts assume earlier is better?
One of the reasons why "earlier is better" persists is the common mistake of regarding prototypes as medical devices, which in turn makes us think that regulations naturally apply over the whole development period.
Another reason is the assumption that early design controls and in particular early risk management yields better outcomes.
Both of these are wrong, and the good news is that IEC 62304 appears to have (for the most part) been written by people who have real world experience, and the standard itself does not load designers with any unrealistic expectations. The main problem consists of people who have not read the standard, have no experience in design, and overlay their well intentioned but ultimately misguided interpretations on the standard.
Prototypes are not medical devices
Only the device actually placed on the market is a medical device. When the word "medical device" appears in management system standards such as IEC 62304, we tend to naturally extend this to pre-market versions and prototypes. But legally speaking, a manufacturer's responsibility is limited to demonstrating that the marketed device meets the requirements. How they do this is up to them: the history of the product, which is often complicated, messy, includes a mix of new ideas, old ideas, borrowed designs, software bought from a bankrupt company with a great product but no records; all of this is not under the scope of regulations or standards.
You don't need to pretend the process was smooth and controlled: it rarely is. It is only necessary that records exist to support the final marketed device.
In seminars, this concept is often difficult to get across. Consider the following example for the software history:
- Version 1.0.1.2 is released to marked, properly verified according to IEC 62304
- Version 1.0.1.3 adds new Feature A, is tested but not released, as requests for new feature B came in the mean time
- Version 1.0.1.4 is released to market with new Features A & B
A typical exchange in a seminar might go along the following lines:
[attendee] "So what level of records do I need to keep for V1.0.1.3?"
[speaker] "Well, if you start at V1.0.1.4 ... "
"Sorry to interrupt, but I am talking about V1.0.1.3"
"Yes, but we need to start at V1.0.1.4 ... "
"SIR I AM TALKING ABOUT V1.0.1.3"
"I DON'T CARE ABOUT V1.0.1.3" [deep breathing, calming down]. "It never went to market. It is not a medical device. You do not need to keep any records for it. Now, we need to look at the records for the real medical device, V1.0.1.4. We might find that tests on V1.0.1.3 were used to to support compliance, or we might find V1.0.1.4 was 100% re-tested, so we can ignore all data from V1.0.1.3."
"Really? I'm still not sure I can just throw out the design test data."
[Inaudible sigh]
To understand the regulatory perspective, always start with the actual marketed device, and work back to the required evidence. Once the evidence is found, you can stop looking.
The "prototype is a medical device" thinking is not only confusing, it can be dangerous as designers often forget about their obligation to make sure data is representative of the marketed device. In the above example, a court of law would find any testing on V1.0.1.3 for Feature A does not itself form any legal evidence for the marketed version 1.0.1.4. This is one area where IEC 62304 is weakly structured: if a serious incident occurred and the cause found that Feature B interfered with Feature A, there is no record required by the standard which clearly identifies the responsible person that decided not to re-test Feature A again in V1.0.1.4. Amendment 1 improved the text in 6.3.1, but there are no required records pertaining to the decision not to retest. As design changes accumulate, the relationship between the tests on the older versions and the final marketed device gets progressively weaker.
This may seem a digression from the design life cycle, but understanding the need to be representative of the marketed device is an important factor in deciding when to start the formal, regulatory life cycle.
Early designs change more than you think
Those outside of the design process might think that design is simply about writing specifications, writing some code and checking that it works as expected, ironing out the occasional bug that occurs along the way.
In reality, designs are multi-dimensional problems that our brains are simply not smart enough to solve in a logical way. This means that trial and error forms a huge part of design, with as much as 90% of trials ending in failure (hence the phrase trial and error). Some failures are detected quickly, others advance for months or years before realising it is unworkable. Some failures are quickly corrected, others require major surgery.
This is important to understand for two reasons: first, formal design controls can be more of a hindrance than a help for designs that are still young, unstable and working through this trial and error phase. Having to update specifications, architecture every step of the way can be so onerous as to grind the process to a halt. To survive, designers often opt for a simplified "lite mode" approach to design controls, keeping everything at a surface level, avoiding detail and not particularly accurate with respect to the real design.
The problem is this "lite mode" continues even to the point of market release. Although designers often have good intentions to document more detail later, the "lite" verses "detail" distinction is not formally identified in the plan, so there is no specific point to switch from "lite" to "detail" and no time or resources allocated for the "detail mode". Couple this with schedule overruns in years and the pressure from management to get the product to market typically means designers quietly stick to the "lite mode" all the way to product release.
Regulators often look for the structure of the documents, but fail to check if the detail is accurate and complete for the final marketed device. If they did, for example, take several random samples from the features in the real medical device, and auditors checked that the architecture, specifications and tests covered these features accurately, they are likely to find at least one or more features where the specifications no longer fit, are lacking reasonable detail, the testing is out of date with the actual design or even the feature is missing entirely from the formal records.
Thus, it is actually safer to allow designers a free hand to develop the product until the design is stable, and then apply detailed, formal controls with an emphasis of working back from the finished design to ensure the details are accurate according to the marketed product.
The second is good understanding that system specifications are not black box derived - in other words, you can't sit down and write good specifications without knowing how the design will be implemented. In the early stages of design, solutions will not yet be found, hence reasonable specifications cannot be created early in the cycle. This problem is one of the issues addressed by modern techniques such as agile software development, which understand that rather than preceding detailed design, specifications will emerge and evolve from the design.
Equally, until the design is stable it won't be clear exactly what the risks are and the best way to handle them. Risk is again not black box driven - the final architecture, structure of the system, platforms and software will yield significantly different risks and risk controls. For example, these days wireless patient monitoring applications via smart phone and the cloud are the flavour of the month. Each company will choose different solutions and in particular vary greatly in delinting the software functions handled by each device in the chain (device attached the patient / smartphone / cloud). Decision on the amount of data storage in each unit, where time stamps are recorded, degree of pre-processing, controls and feedback available to the patient can all have a big influence on any risk evaluation and the associated risk controls.
This again points to the need to delay formal specifications and risk evaluation until after the design is stable.
None of this prohibits the manufacturer (designers) from keeping informal documentation: draft plans, specification and risk evaluation, test results which are undoubtedly useful and may even be necessary to keep some control of the design. Nevertheless, it remains a benefit to distinguish this draft stage, which may be deliberately light on detail, not always up to date, and may lack approvals from the formal regulatory stage, which is required to be accurate, approved and complete for the final medical device.
The best time to start
Deciding the best time to start formal controls is in reality an efficiency decision, and will depend on the individual product.
For simple designs, it can often be best to leave the formal controls until the design is thought to be 100% stable. This usually means informal testing and debugging against draft specifications until all issues are cleared, followed by the formal life cycle: approval of the specifications, risk management; actual testing; summary report. If during the formal testing issues are still found, the manufacturer can opt to apply configuration management and regression testing after fixing the issues; or re-start the process again from scratch.
For bigger systems, it can be more efficient to break the system into blocks. The blocks can then be treated the same as above (no formal controls until stable); with formal controls applied when the blocks are stable and ready to be integrated.
The main aspect to consider is the probability that specifications, risk management and test data will be representative of the final medical device. This probability improves as the design becomes more stable. On the other side, there are costs associated with repeating tests with every design change. Having design controls in place allows you to use data on earlier configurations as long as it is representative, which can save considerable time and cost especially for larger systems. An efficient point occurs when the costs of formal design controls balance against the savings from being able to use test data on earlier configurations.
What if risk evaluation is left too late?
The worry that risk evaluation left late is a valid concern, but the emphasis on early design is a misdirection, at least in the context of standards and regulations. Irrespective of the final solution, the risk must be deemed acceptable by the manufacturer. If the solution is not acceptable, it is not acceptable. Legally, the timing of the decision cannot influence the decision on acceptability. If it does it suggests a deeper problem with the risk management process, such as a lack of control for conflicts of interest. If an unreasonable solution is deemed acceptable just because of the high cost associated with late stage implementation ... then something smells bad in the process that supports that decisions. It is more important to address the inadequacies of the analysis than blame the timing of the analysis.
Again, none of the above suggests the manufacturer should not be thinking about risks, risk controls and drafting documents prior to the formal design life cycle. But above all this sits the simple approach that the documentation on the final, marketed medical device should be complete and appropriate. The history is irrelevant - the main point is that the marketed medical device should be safe.
What about agile software development?
There has been significant discussion about whether agile and similar practices meet the requirements of IEC 62304 and FDA software guides. AAMI TIR45:2012 has been written to address this space, and the FDA has been supportive of the practices given the superior results over waterfall based methods.
However, much of the guidance continues to use the "prototype is a medical device" philosophy, hence requiring that agile practices, while lighter and focusing on iterations, still need to be documented at every iteration.
Instead, this article suggests agile practices should be considered part of the informal phase of design, where regulatory documentation is not retained. The iterative, collaborative design process eventually outputs a stable design and draft specifications/risk management. Those outputs then forms an input to the formal regulatory stage in which the focus switches to ensuring the documentation is complete and reasonable for the final marketed, medical device.
For example, a surgical laser may have had a internal software controlled start up self test of the overpower protection systems added at the 7th iteration, which while implemented was largely forgotten by the 10th iteration as the focus turned to user interface and high level performance of the system. Left to agile practice alone, the start up test could be easily overlooked in final stage verification tests. This overlooking of internal functions is a frequent issue found in independent testing of safety systems, not only missing specifications and verification, but actual logic errors and software bugs in the final implementation.
The view of this article is that regardless of the history, approach, development model used, the manufacturer needs to be confident that the such a start up self test has been verified for the configuration released to market. Reasonable confidence can only be derived by ignoring the development history and working back from the final, stable device.