Observe User Behaviour of the Application

[philippzagar]

Problem

Over the past few months, LLMonFHIR has effectively showcased the capabilities of LLMs within health literacy. As we transition towards a usability study for evaluation purposes in the upcoming months, the study conductors must be able to objectively assess the user's perception and interaction with the application.

Solution

For the usability study, we need an app version that observes the user behavior of LLMonFHIR. While we haven't finalized the specifics of the data to be collected, initial considerations include standard metrics such as screen durations (highlighting specific screens), app launch/close frequencies, error occurrences (particularly related to LLM prompts), and typical text-based user interactions paired with the respective LLM responses.
Both researchers and participants should have access to this data. Likely storage solutions include SpeziFirebase and SpeziStorage, including proper visualization within the application.

Tools that might be helpful for observing the behavior:

• Swift Log: https://github.com/apple/swift-log -> For Errors, ...
• Swift Metrics: https://github.com/apple/swift-metrics -> For numeric information, ...

Additional context

In the long run, it might be an option to consider developing a distinct module within the Spezi ecosystem dedicated to observing user interactions within digital health applications. This is in line with our future plans to conduct studies on various Spezi applications.

See: StanfordSpezi/.github/issues/38

Code of Conduct

• I agree to follow this project's Code of Conduct and Contributing Guidelines