Selma is a multimodal CUI designed to provide accessible and inclusive access to healthcare self-management tools like medication trackers, mood and symptom trackers, theraputic journals, time, activity and exercise trackers, personal planners, reliable knowledge bases on health conditions and diseases, and similar tools used in the management of chronic physical and mental diseases and disorders and conditions like ADHD or chronic pain where self-management skills for life activities are critical.
Selma follows in the tradition of 'therapy bots' like ELIZA but updated with powerful ML-trained NLU models for interacting with users in real-time using both typed text and speech. Existing self-management apps like journal, activity-, and symptom-tracking apps all use GUIs or touch UIs and assume users are sighted and dexterous. The reliance on a visual medium and complex interface for entering and reviewing daily self-management data is a significant barrier to adoption of these apps by people with disabilities and chronic conditions, who form a majority of a self-management app's user base.
Selma eschews complex GUI forms and visual widgets like scales and calendars and instead uses a simple line-oriented conversational user interface that uses automatic speech recognition and natural language understanding models for transcribing and extracting symptom descriptions, journal entries, and other user input that traditionally requires navigating and completing data entry forms. Patients interact with Selma using simple natural language commands or questions and enter their journal or medication or symptom tracking entry using speech or text. The captured audio and text is analyzed using NLU models trained to extract relevant details spoken by the patient on their medication intake, mood, activities, symptoms and other self-management details, which are then added to the user’s self-management journals.
The Selma CUI is an accessible user interface that produces text output easily read by screen readers, braille displays and other assistive technology. Users interact in a conversational style with Selma which gathers information in specific areas and guides the user through specific tasks like daily medication and mood tracking and filling out periodic journal entries and evaluations. Users can ask questions (“Did I take my meds today?”, “Did I go out this week?”) and bots can answer intelligently based on information previously captured and analyzed. With the user's consent the information gathered can be automatically sent to the patient’s health providers, reducing the time needed for administering these routine tasks and allowing face-to-face communication and direct supervision with a practitioner to be conserved and more effectively use. The information can also be analyzed for possibly warning symptoms or threats of acute events that may require intervention.
Selma uses Facebook's Wit.ai NLU service and the expert.ai deep text analysis NLU service.
Self-management can be defined as the methods, skills, and strategies by which individuals effectively direct their own activities toward the achievement of specific objectives. It usually includes goal-setting, planning, scheduling, task tracking, self-evaluation, self-intervention, and self-development. In healthcare, self-management typically refers to the training, skill acquisition, and interventions through which patients who suffer from a disease or a chronic condition may take care of themselves and manage their illnesses.
- From Self-Management of Depression A Manual for Mental Health and Primary Care Professionals by Albert Yeung et.al
A self-management program teaches patients to see treatment as a collaborative process with the patient taking responsibility for self-monitoring and tracking their symptoms as well as medication intake and other vital information, together with a commitment to using evidence-based, self-administered structured therapy and intervention as adjuncts to professionally delivered interventions for managing their illness.