Agile for Software as a Medical Device

Editor's Notes

• #3: (As Ken said, I’m Bernhard Kappe, CEO of Orthogonal, formerly Pathfinder Software.)
• #4: Orthogonal is a medical software development company, we build connected care systems for medical devices. We build everything except the devices and the firmware. Connectivity, mobile, web, cloud, integration to midleware for clinical systems. Most falls under medical device regulations, Class I, MDDS, Class II, some class 3.
• #5: We work primarlly with medical device companies, both startups and large companies. We started out as an agile software development shop, working in a number of industries, and ended up doing work in medical devices, and dealing with the challenges of making Agile work. We learned a lot along the way. At some point I started writing a book about it, and talked to quite a few folks as part of that, including folks at the FDA and folks who had pioneered the use of agile, We picked up a number of things out of that and incorporated them into what we do.
• #6: So today I’ll start with a quick intro/recap of agile, then talk about why it makes sense in a medical device context, get down to brass tacks on how to be agile in a QMS, talk about automation, agile for non developers, highlight some common implementation mistakes, and then give you a couple of resources.
• #7: OK, Agile in 5 minutes. How many of you have done agile? Are familiar with it?
• #8: Look at traditional waterfall development process – how we used to do everything, including software. This works great when problems and solutions are well known. But the problem is that when you’re doing new product development, with new and changing underlying technologies – that’s not the case.
• #9: That’s where you run into the problems with waterfall – the cost of change curve. If you have change, you have to go back to the beginning. There’s no mechanism for dealing with change.
• #10: This is not unique to software development. That’s where iterative methods have evolved, that use fast feedback loops to deal with uncertainty and change. All based on a build-measure-learn loop and making adjustments. From military Observe, Orient, Decide and Act. – if your OODA loop is faster, you increase your chances of winning. To the Toyota production system and all the lean methodologies which came from that.
• #11: Agile comes out of the same area, and is applied to software development. Lot’s of flavors, scrum, crystal, etc. Agile does not attempt to lock down all requirements up front. It assumes that the requirements and design will evolve and change as more is learned and discovered during the process. It works by breaking projects down into user stories - little bits of user functionality that create value for the user - prioritizing them, continuously delivering them in short one to three week cycles called iterations, and getting feedback on the working software to incorporate back into the development process.
• #12: So another key concept here is behavior driven development which fits in well with agile. Stories – requirements. Acceptance Criteria: Specification. Acceptance Tests: Verification Protocols
• #13: So let’s walk through an agile iteration.
• #14: OK, so let’s look at the typical agile cadence. Review – planning as required. Activities in one iteration to prepare for the next, testing automation afterwards.
• #15: So the effect of these methods on the cost of change curve – they uncover a lot of issues earlier and allow you to adjust.
• #16: The effect is a higher up front cost to set up, but then a much more gradual cost of change curve.
• #17: So this has been accepted in a lot of other industries – why do we need it in this industry, and if we need it, why has it taken so long?
• #18: Software has gotten more and more important in medical devices, and now with connected care, even more so. New product development, etc.
• #19: So in an environment of change and uncertainty, you need business value and real world outcomes, you want process efficiency, both of which agile can deliver on, but you also want patient safety and regulatory compliance.
• #21: So let’s start with the FDA. The FDA does not dictate process. It describes but does not prescribe.
• #22: A number of folks pioneered the use of agile in medical devices, both at small companies and at large companies. But in a safety conscious industry, we like standards. So some of these folks took a crack at that through AAMI TIR:45, the guidance document on applying agile practices. I can highly recommend this document. It was recognized as a standard by the FDA in 2013.
• #23: OK, so let’s dig into this a little more
• #24: What AAMI TIR:45 does is focus on the software development process in IEC 62304, the generally accepted standard for medical device software development. For Agile to be successful, you need to address some of the other processes, risk management, configuration management, and problem resolution, but we’ll get to those later
• #25: This is the key diagram from TIR:45, slightly modified to include risk management. The activities are mapped at different levels – project level software development planning, requirements analysis and architecture - At the release level – any planning changes, integration and integration testing, system and regression testing, and release The next level – increment – related group of stories typically – the same set and then the meat of it is at the user story level. Requirements, emergent architectural design, detailed design (software implementation, acceptance criteria, unit testing, integration and system testing. Risk management is a separate process, which we’ll get to later, but the yellow boxes represent inputs and outputs of those processes.
• #26: One of the biggest things that trips people up is this old graph from the 1970’s that’s still in the FDA guidance. Conceptually it’s correct, but people look at it as a waterfall type process map.
• #27: The v model is a better representation. You trace down from user requirements (at system and story level) through specifications (architecture, story acceptance criteria) and then up from unit testing through system and integration testing to user acceptance testing. The key thing here is that the relationship between artifacts are time independent. It’s not start to finish parallelism, but finish to finish parallelism. You can’t finish the outputs before the inputs, but you can start them. That gives you flexibility in Agile.
• #28: So if you start with test driven development (write the unit test first, then the unit) and extend it to bdd – story first, story acceptance criteria – you have the start of verification driven agile - verification protocols composed mainly of story and system level acceptance tests. Stories – requirements. Acceptance Criteria: Specification. Acceptance Tests: Verification Protocols BDD drives validation. Lose sight of intended use.
• #29: Let’s talk about risk management – design controls and risk management the core of what you need to do.
• #30: Here’s what ISO 14971 says about risk management -
• #31: Start with preliminary hazard analysis. Risk management core to the process Responsive to inputs Safer!
• #32: Another iterative process is human factors – doing formative and summative human factors is part of the FDA’s guidance. Use errors are discovered through this, fed into risk management, risk mitigations can be tested, designs and usability. This feeds into development (faster to test and check in prototype. That’s a whole ‘nother talk.
• #33: So if you stop there and don’t look at automation – Agile is going to be pretty hard. Automation is the key to maintaining quality at speed, especially for long, complex projects.
• #34: If you don’t do this, bug propagation, full manual retesting every time.
• #35: Conceptually, when you do this – combine tdd and bdd with testing automation – you shorten the bug cycle and prevent propagation.
• #36: Another thing that is a great benefit is Tooling to manage the process, lightweight, manage agile requirements, traceability, hazard analysis, and documentation. We’ve found that the right tools let you set up worflows and approvals, dependencies between entities, include risk management – output documentation.
• #37: OK, let’s talk through agile in a QMS for non developers
• #38: The story of pigs and chickens is famous in agile. The pigs run the process – they’re committed. They need to manage the chickens, educate the chickens, work with the chickens. If the chickens run it, or you end up waiting on the chickens, slow down the process, gum up the works. As a chicken, you need to understand that as well.
• #39: I’d actually go a little further – a lot of agile development is wasted if none of the related processes can iteratively provide and receive feedback. You can still deal with technology change and architectural issues, missed requirements, but a lot of the benefits are there if you can actually use that input.
• #40: Ideally you have a series of iterative processes feeding into each other.
• #41: If you put it all together – you get beter usabilty, Faster development, streamlined maintenance, etc.
• #42: So fi we have time, I’d lover to cover some of the implementation mistakes.
• #43: We’ve seen some of the same mistakes over and over again.
• #53: pathfindersoftware.com/agile-in-an-fda-regulated-environment/
• #56: Think of it also as iteration activities and documentation activities. We can skip this slide.