no tools exist to audit medical devices
Closed this issue · 3 comments
I cannot abide it. I need to know how much insulin I've been given. I need logs of my therapy.
Over the fall or winter of 2008, I had freshly given up on CGM. I was frustrated because I knew how important CGM was supposed to be in type 1 diabetes therapy. When I was diagnosed, doctors explained to me how a CGM combined with an insulin pump could be automated in such a way that I would no longer need to inject myself, calculate insulin, prick my fingers, and constantly guess carbohydrate and dozens of other numbers throughout the day. The automation with CGM and pump promised to liberate me from burdens I feel too weak to bear, and I'd do anything to achieve the promised liberty. It was painful, and worst of all, there was no way to interact with the data in a meaningful way. People were using their smartphones to take pictures of the displays. Then they would make a document of the pictures and print it out for their doctors. Doctors would literally hold up several printouts against the lights in the ceiling, attempting to divine therapy recommendations. I swore never to use CGM again until I had access to the data. I thought this should be a simple thing to ask for. In December 2009, I was at home with my parents over the holidays working on https://github.com/bewest/insulaudit researching protocols for my meter and carelink usb stick used to communicate with my insulin pump. The earliest commit is from September of 2010, because I joined github after joining Meraki at the same time. Previously, I had used mercurial with bitbucket. I had managed to get a python prototype for the meter protocol working and realized that each device would probably need it's own repo, especially if "reverse engineering" was going to be one of the project's first tasks. The vast complexity of diabetes technology dawned on me over this time. I realized that technical questions like "how to communicate with a device" needed to be hosted in a technical forum that could collaborate specifically on those questions in an easy way, and that separately, documentation and community support would be needed for an ecosystems of applications. I started separating out repositories for each type of device shortly hereafter. At this time, while Scott Hanselman had a zip file for some Palm software, no one had any software published to talk with commonly used meters or insulin pumps or CGMs. Meanwhile, their were mature SDKs for lots of devices, for the web, and the number of places this type of access started appearing proliferated.
It started to dawn on me that access to the data was going to be a critical issue so I created decoding-* as a placeholder for "well-framed" questions for each device I knew about in the diabetes ecosystem.
I also managed to download and run through a decompiler the Medtronic web applet called carelink. At the time, the repository was private out of concerns regarding DMCA, as well as other liability issues and only converted this to pubic status recently, in 2016. I discovered that the serial protocol used to control communications with the insulin pump was implemented in the web applet.
Between 2009 and 2011, I learned a lot from my transition between Linden Lab and experience at Meraki. It turn out I was not the most talented at the task of translating the java code into workable python infrastructure. I found a number of bad coding practices in the java code, sloppy copy & paste issues, and what I soon learned were serious security vulnerabilities. At the time, I found it surprising that there was no way to "read" data without executing commands and found it exceedingly surprising that their code included a "temp basal for 0 hours and 0 units" in a place meant to check on the status, specifically whether or not older models were currently bolusing, before attempting to read any data. That was kind of spooky, so armed with my reproducible repos and well framed questions, I began asking security researchers and other folks familiar with reverse engineering efforts for help. To my surprise, my work was considered extremely relevant. Several people used this as a basis for their own security work or for their own initial R&D efforts (some of them are now running their own closed loop firms). Unfortunately, my concerns regarding liability were widely shared to the extent that sometimes I did not receive as much technical feedback as I would have preferred.
In 2011, I wrote a vision of diabetes data bus highlighting interoperability and connectivity between diabetes devices.
In 2012, I opened this issue, and re-organized the github issues. I started reaching out and working on regulation and policy in addition to technology.
In 2012, at the encouragement of policy advocates, I reached out to the FDA https://github.com/bewest/insulaudit/blob/master/questions/fda.markdown and started researching federal regulations on quality systems, insulin pumps, and guidance on diabetes technology. The FDA told me that device vendors are required to submit documentation regarding communication and network activity of devices, but that these information are considered proprietary and protected by DMCA, and therefore cannot be exposed by FDA. In addition, it might be possible that my aspirational transfer of my therapeutic data to my hypothetical data management software might require an IDE, or change in "intended use" from what the vendor originally approved.
In other words, what I originally thought was a technical problem had become a policy, legal, and clinical problem as well.
These kinds of system problems require systems thinking. Is real-time access to data the same as retrospective access? What kinds of decisions would you make in real time if you knew this information about what this machine is doing to your body? If your software makes a change, and something unexpected happens, who is responsible? How do we know it's safe?
"How do we know?" is an epistemological question. Appeal to authority is only authentic when independent certainty can also be empirically verified. Independent verification, or "access to data" is a requirement for epistemological certainty and trustworthiness: "how do we know it's safe."
In February 2013, I came up with a quick infographic to help think about the kind of ecosystem which would provide high-fidelity therapy in a timely fashion through the FDA by supporting the same kind of "crowd-sourced" innovation we see in other technology infrastructure domains, like the web or mobile phones. Howard Look had a similar vision and Tidepool was launched soon after; I was very fortunate to join as employee number 2.
If safety relies on independent access to data, what is the impact on innovation? As lead users prove which features are valuable to a market, industries typically adapt to meet evolving market requirements. Type 1 diabetes patients don't have a choice to forego therapy, which shapes the ecosystem along two political forces that Thucydides explains well.
"Right, as the world goes, is only in question between equals in power, while the strong do what they can and the weak suffer what they must." http://en.wikipedia.org/wiki/Thucydides
Over the years, this has forced patients into awkward positions. However, market forces are inevitable. Over 2012 - 2013, decoding-carelink and other projects received lots support and features. By 2013, decoding-carelink could capture most history records form most Medtronic insulin pumps reliably, and could bolus, temp basal, and do a variety of other things.
In April of 2014, I posted a video showing temp basal and some possible use cases.
I had been living with Brandon, who was raving over the new Dexcom G4 CGM, and found access to https://github.com/compbrain/dexcom_reader. Brandon's experience of CGM was very different from my prior experience. For the first time, I had first hand access to the recipe for high fidelity therapy: access to CGM data and access to pump data.
"The future is already here — it's just not very evenly distributed." However, it's not good enough just to have access to the bare materials. No one wants to be a patient on an orphan-treatment, as the idea is to distribute and bear less burden as individuals. Less work on diabetes, more liberty in life. What was missing is an ecosystem of applications, a fabric for features to adapt to users lives, a data bus to communicate the data, and people and projects to drive progress in what Eric von Hippel and Andrew Torrence have described as "the innovation wetlands" and a process to deliver those innovations into the market in a safe and effective way. In April 2014, I left Tidepool and began working on the vision of being liberated from diabetes through technology, solo. Emboldened by my progress on the insulin pumps, I knew that with an algorithm and CGM connectivity, we could automate diabetes therapy. The notion of an "artificial pancreas" was very popular, which shaped the video concept. We now typically avoid the term in favor of Hybrid Closed Loop (HCL) or simply "looping." However, what I wanted was an entire ecosystem of applications with a framework for clinicians and academics to examine what happens and a process for industry to delivery value to customers. As I used the money I earned from Cisco's acquisition of Meraki from 2014 - 2016, the balance and doubt between what is good for the ecosystem, and app improvements I can deliver to myself weighed on me night and day. During much of this time, I went without health insurance.
In April, Brandon came home with a new cell phone plan, an android phone, and in May 2014 forwarded me a long email chain with attachments, links to google drive documents, and links to android developer debugging posts. It was bits and pieces of what John Costik had worked on with what Lane had presented at the previous DData Exchange. There was an Android java application that used "USB OTG" to read data from the Dexcom receiver over a usb cable. There was a node js server and static web page to view the data. There was also a really neat pebble watch application in C and js that used the connection to your smartphone to get CGM readings from the server and put it on your wrist. There was a nascent Facebook group called "CGM in the Cloud" with dozens of people wanting to get set up, trading documents, and links and attachments. Even technically savvy folks were having a difficult time getting started, and the code had itself had to be changed in order for it to work.
The vision of an ecosystem and a universal diabetes data bus from the prior years for the first time seemed possible in unique and unexpected ways. The first thing I knew the community needed was a centralized way to accommodate development. I created https://github.com/nightscout, moved a version of the docs to a webpage backed by git, and recruited help. Jason and Ross and I agreed to collaborate using several modern development practices. We started sharing our work and branches and updated the documentation to suite. We also started recruiting community members and within a few weeks made several critical changes to Nightscout. We modified the code so that no one would need to modify the code to make it work for them. This meant we could now share the exact same code with everybody, and the only thing different between them would be a configuration file or parameter. I worked with every developer to ensure that almost all new diabetes projects started doing this early on. Then I started taking people through the new set up process. James Wedding was one of the first, and after several weeks almost everyone began referring to a shared version maintained by a dozen people.
We also ensured that the software was as easy to get running as possible. There are several "free" developer accounts that are needed, so signing up for these accounts takes some work. In addition, there are several "magic strings" that need to be copy and pasted into several configuration screens to make the whole thing work. Most people think of yarn when the docs mention strings, so this remains a severe setback for us.
Having spent several months adapting modern development practices, researching quality system regulations in response to concerns over liability, in October I had a Nightscout meeting with the FDA.
Over the months, we add substantial features to Nightscout, many to the API, in order to support a wide range of interoperable devices for retrospective and real-time use cases. In order to support an ecosystem, we ensured that we offered features to enhance privacy and security, to provide traceable debuggable information, and an interface to adapt to many unforeseen use cases.
Much of 2014 was spent getting to know Dana and Scott and helping to mitigate their liability concerns, as well as adding features to Nightscout.
Once I finally had access to a full-featured Nightscout, decoding-carelink, and Dana and Scott's algorithm, I carefully assessed the gap between 2015 and the vision from 2012. I created https://github.com/openaps specifically to address the need for a toolkit to make interoperability between dosing algorithms and data management systems easy. Dana and Scott work on openaps/DIYPS system features as well as oref0 as a reference design available for the ecosystem.
During 2015, I attended conferences, business workshops and advocated for policy change. I advocated for increased control over privacy as well as freedom to repair and innovate in the 2015 DMCA 1201 hearings.
Meanwhile, openaps was serving it's intended purpose, allowing Nate, Pete, Chris, Mark and others to advance status quo, exactly as I had hoped.
Fast forward to 2018, and Tidepool is now planning to make Loop available in the app store.
Closing with issue a #wearenotwaiting benediction:
Let us all consider how we can unify our demands for trustworthy, high fidelity therapy that in adapting to our lifestyles allows us to return to the lives we want to pursue with the people and activities we love. Let us make the benefits we discover accessible to as many as possible.
Thanks! Missing most connections with others, so very incomplete from a narrative perspective. When people ask me "why?" all this happened or why I worked on decocare/FDA/Nightscout/openaps/1201 hearings, I'd like to firm up my story linking the desire for an ecosystem, not just a "killer app/feature" to the decisions I made and the results of those decisions today. For example, Eric Raymond's Cathedral and the Bazaar, and Homesteading the Noosphere informed the decisions I made. Paul Graham was right and wrong: you can take two years off and make something nifty. Taking that nifty thing and making it matter requires so many people and sometimes regulatory and cultural change before industry can make it accessible to more people. Another version or narrative is needed to kind of weave everything and everyone together.