/mad-gui

A python package for interaction with time series data.

Primary LanguagePythonMIT LicenseMIT

Publication

You can use this to cite our work, however, the actual information on how to use this package is contained in this readme and not in the mansucript.

Ollenschläger, M.; Küderle, A.; Mehringer, W.; Seifer, A.-K.; Winkler, J.; Gaßner, H.; Kluge, F.; Eskofier, B.M.
MaD GUI: An Open-Source Python Package for Annotation and Analysis of Time-Series Data. Sensors 2022, 22, 5849. https://doi.org/10.3390/s22155849

MaD GUI

Machine Learning and Data Analytics Graphical User Interface

Codestyle black Test and Lint Documentation Status PyPI version shields.io PyPI stats PyPI stats

MaD GUI is a basis for graphical annotation and computational analysis of time series data. It gives easy access to plotted data / annotations and handles interaction with annotations for you in the background. Developers must create plugins and inject them into the GUI to read data of their format or use their algorithm.

⚠️ WARNING ⚠️
This is still an early version. Things might not work as expected but we are happy to help!
Experiencing issues? Report a bug here!

click to enlarge the image

click images below to show videos on YouTube
(open video descriptions on YouTube for chapters)

Quickstart for developers

In a python environment, execute the following commands or use the section Development installation:

pip install mad_gui
mad-gui

You can download our example data to test our built-in exemplary importer, exemplary algorithms and exemplary label. To see how to open our example data within the GUI, please refer to our section about the User Interface.

Contents of this readme

General Information Development Additional Information
  • Why and what?
  • Download / Installation
  • Example data
  • User Interface (Videos & Shortcuts)
  • Load / display data of any format
  • Use a custom algorithm
  • Terminology
  • Development installation
  • Developing Plugins
  • Communicating with the user
  • Adjusting Constants
  • Changing the theme


  • Support & Contributing
  • Background





  • Why and what?

    Existing graphical user interfaces (GUIs), that deal with time series data, usually do not allow support of annotation by algorithms, although such algorithms exist. If they do, the algorithm and the GUI are tightly coupled, such that reusing it becomes hard.

    Furthermore, algorithms that were developed to analyze time series data, can usually not be used by researchers, who do not have programming experience, although the algorithms were developed for their research area. This is for example the case in the area of gait analysis, where algorithms are provided by computer scientists and need to be used by clinical researchers.

    The MaD GUI can be used as a framework to incorporate algorithms for supporting time series annotation and for using algorithms for analyzing time series data even without having programming experience. You can read about this in more detail in our Background section.

    Download / Installation

    Plugin Developers

    Please see the section Development installation.

    GUI user: Standalone executable

    You do not need to install anything. Simply download the file from the table below for your regarding operating system. Afterwards you can start the GUI as described in the rightmost column of the table.

    When downloading a file from the table, your browser may warn you that this is a potentially dangerous file. You will only be able to use our GUI by selecting "Keep anyway / download anyway / ...". In the case of Microsoft Edge, this possibility is hidden, but you can select it after downloading as explained here.

    Operating system File to download What to do
    Windows Windows (64 bit) Download the file on the left. Then open the downloaded file and read about example data in the section below this table.

    Note: If prompted with a dialog Windows protected your PC, click More info and then select Run anyway.
    Ubuntu Ubuntu (64 bit) Download the file on the left. Then, in your terminal, navigate to the file loaction and then: chmod +x ./mad_gui_ubuntu and then ./mad_gui_ubuntu.

    Note: you might need to install some additional packages. You can use this script to do so. Just right click the link, save it on your machine and execute it.
    Mac OS Mac OS (64 bit)
  • Download the file on the left and extract it, for example to a folder called mad_gui.
  • Right click on the folder mad_gui and choose "New Terminal at Folder"
  • In the terminal, type sudo -xvf mad_gui.tgz and press return.
  • Close the terminal.
  • Right click on the file mad_gui_mac, select "Open", and confirm that you want to open the file.
  • Wait about half a minute until the GUI opens
  • You can read about example data to load in the GUI in the section below this table.


  • Note: If your Mac does not allow you to open this file, perform the actions for "If you want to open an app that hasn’t been notarized or is from an unidentified developer" on the Apple Support Page. Afterwards, try to open it again as described above.
    other Supplied upon request Contact us

    Example Data

    After clicking this link, you can download our example data by either using Ctrl+S on your keyboard or click the right mouse button and then select Save as... to save it on your machine. Make sure to remember, where you save this file. After starting MaD GUI, you can open the previously downloaded example data as shown in User Interface.

    You want to load / display data of a specific format/system or want to use a specific algorithm? In this case please refer to Load and display data of a certain data type

    You want to load / display data of a specific format/system or want to use a specific algorithm? In this case please refer to Load and display data of a certain data type

    Terminology

    Word Meaning
    Annotation An annotation has a start and a stop value, e.g. it goes from second 1 to second 2 of the time series data.
    Label An annotation is always of a certain type, for example "Activity". These types are called labels. A Label keeps the possible descriptions of an annotation. For example, the label "Activity" can have the descriptions "walk" or "run". Furthermore, it keeps information about where the annotation should be plotted, e.g. lowest 20% of the plot.
    User Someone who performs actions in the GUI using the mouse and keyboard.
    Developer Someone who creates plugins for the GUI, for example to load a specific data format or to use a specific algorithm.

    User Interface

    You can get information on how to use the GUI from the keywords below or from the video referenced in each of the sections. You may also want to take a look at ACQ4 documentation to learn about mouse interaction with the plots (e.g. zooming).

    Load data

    Text Exemplary Video
  • Click Load data button on the left
  • Select recording system in dropdown
  • Select data file to specify the sensor data to be loaded
  • Optionally: select video to be displayed
  • Optionally: select annotations previously calculated using another program
    (do not confuse with Use Algorithm, see below.)
  • Click Start
  • Use algorithm

    Passes the currently displayed data to an algorithm, which then calculates new annotations or features for existing annotations.

    Text Exemplary Video
  • Click Use algorithm button on the left
  • Select algorithm
  • Click Start processing
  • Synchronize sensor data and video

    Please watch the video:

    Export data

    Exports displayed annotations into an excel file. If you want to export displayed sensor data, either use the Save displayed data button. If you want to customize exports, please create an Exporter.

    Shortcuts

    Please watch the videos linked above, if you want to learn more about the different actions.

    Shortcut Mode Action
    a, e, r, s, Esc all Switch between modes Add label, Edit label, Remove label, Synchronize data
    Left Mouse Click Add label Set start/end of a label
    Space Add label Can be used instead of Left Mouse Click
    1, 2, 3,... TAB Add label Navigate in the pop-up window
    Shift + Left Mouse Click Add label Start a new label directly when setting the end of a label
    Ctrl + Left Mouse Click Add label Add a single event

    Load and display data of any format

    This will need someone who is familiar with Python, who has to develop a plugin of the type Importer. For more information on how to create plugins, please refer to Developing Plugins. You do not have experience with python but still want to load data from a specific system? Contact us!

    Use a custom algorithm

    This will need someone who is familiar with Python, who has to develop a plugin of the type Algorithm. For more information on how to create plugins, please refer to Developing Plugins. You do not have experience with python but still want to load data from a specific system? Contact us!

    Development

    In case you experience issues, please try to find a solution in our documentation regarding Troubleshooting.

    Development installation

    Info: We recommend to use pip install mad_gui in a clean python 3.8 conda environment, or if you know what you are doing in a python venv. .

    pip install mad_gui
    

    Then, from your command line simply call:

    mad-gui
    

    Alternatively, within a python script use our start_gui function:

    from mad_gui import start_gui
    start_gui()
    

    If you want to see an example, download our example data. Afterwards, you can open the previously downloaded example data as shown in User Interface. You want to load data of a specific format/system or want to use a specific algorithm? In this case please refer to the sections Load and display data of a certain data type and Use a custom algorithm.

    Developing plugins

    The MaD GUI package can be extended with different kinds of plugins and labels. The user can access your plugins within the GUI using dropdowns, after clicking "Load data" or "Use algorithm". However, this is only possible if your plugin was passed to the GUI at startup, as we show it for an importer in this GIF, or as shown in this code snippet:

    from mad_gui import start_gui
    from my_importer import MyImporter
    from my_algorithm import MyAlgorithm
    from my_labels import MyFirstLabel, MySecondLabel
    
    # only passing plugins
    start_gui(plugins=[MyImporter, MyAlgorithm])
    
    # passing plugins and labels
    start_gui(plugis=[MyImporter], labels=[MyFirstLabel, MySecondLabel])

    In the sections in the following list we describe how you can develop your own plugins and labels, which must inherit one of our base plugins or BaseRegionLabel. Each of these sections keeps working example to get you started as quick as possible:

    Communicating with the user

    Pop-up

    If - at any point - you want to send a message to the user of the GUI, you create a message box with an OK button like this:

    from mad_gui.components.dialogs.user_information import UserInformation
    
    UserInformation.inform_user("Your message")
    
    # will return PySide2.QtWidgets.QMessageBox.Yes or PySide2.QtWidgets.QMessageBox.No
    yes_no = UserInformation().ask_user("Yes or No?") 

    Tooltip of labels

    You can display results of your algorithms for example by putting the results into the label's description as shown in our section about custom algorithms.

    Adjusting Constants

    You can create your own settings by creating a class, which inherits from our BaseSettings. The following example makes use of the BaseSettings and simply overrides some properties:

    from mad_gui.config import BaseSettings
    from mad_gui import start_gui
       
    class MySettings(BaseSettings):
        # decide which axes/channels should be plotted by default
        # Note that this will result in an error if the passed data does not have those axes
        CHANNELS_TO_PLOT = ["acc_x", "acc_z"]
        
        # used if a label has `snap_to_min = True` or `snap_to_max = True`
        SNAP_AXIS = "acc_x"
        SNAP_RANGE_S = 0.2
        
        # in all your labels you can add an event by using `Ctrl` as modifier when in `Add label` mode
        # when adding an event the user will be prompted to select one of these two strings as a
        # `description` for the event
        EVENTS = ["important event", "other type of important event"]
        
        # Set the width of IMU plot to this, when hitting the play button for the video.
        PLOT_WIDTH_PLAYING_VIDEO = 20  # in seconds
        
        # If plotting large datasets, this speeds up plotting, however might result in inaccurate
        # representation of the data
        AUTO_DOWNSAMPLE = True
    
    start_gui(
    settings=MySettings,
    )

    Changing the theme

    You can easily change the two dominating colors by passing your own theme to the GUI.

       from mad_gui import start_gui
       from mad_gui.config import BaseTheme
       from PySide2.QtGui import QColor
    
       class MyTheme(BaseTheme):
          COLOR_DARK = QColor(0, 255, 100)
          COLOR_LIGHT = QColor(255, 255, 255)
    
       start_gui(
        theme=MyTheme,
       )

    Support & Contributing

    You can find some help in our section about troubleshooting. In case you can not solve an issue using that section, please get in touch.

    Background

    User perspective

    The challenge

    New algorithms for analyzing time series data are published frequently. However, the developed algorithms can hardly be used if the area of application is outside of computer science. For example in clinical research, the user likely will lack necessary programming expertise.

    Best available solutions

    Many graphical user interfaces exist, that allow interaction with time series data without programming expertise. Usually they can be used to annotate sequences of a signal, resulting in information like "from timestamp x to timestamp y, z happend" or "at timestamp x, z happened". In case they need a more detailed analysis using a specific kind of algorithm, usually the data has to be sent to the developer of the specific algorithm, who has to process it and send back the results.

    Limitations of available solutions

    If available graphical user interfaces contain algorithms, these tools are often not open-source and/or require the user to install software on their machine. The latter is not always easily possible due to restrictions of an institution's security policy. In case an algorithm is not incorporated into a graphical user interface, the user interested in using this algorithm would have to send data to developers of the specific algorithm. This may be undesirable for several reasons. Among others, data protection laws and the created overhead are important aspects.

    As a result, many algorithms are developed to support research fields outside of computer science, as for example clinical research, but can hardly be used.

    Our solution

    The MaD GUI offers an easy way of accessing algorithms for the analysis of time series data. This enables researchers to make use of state-of-the-art-algorithms for time series analysis in their respective research area even if they do not have programming expertise.

    Developer perspective

    The challenge

    Researching algorithms for time series data in many cases requires annotation of data. To accomplish the task of annotation, developers usually need some kind of graphical user interface.

    Best available solutions

    While many graphical user interface exist for annotating time series data, they usually do not incorporate algorithms, which could support the developer with labelling the data - and they do not offer an interface to add algorithms. However, many algorithms exist for such kind of support.

    Limitations of avaialable solutions

    In case a graphical user interface incorporates algorithms to support labelling, these are often very specific to the use-case. This results in each developer building their own graphical user interface, which usually is not developed with the aim of being modular or to be used in other projects. Therefore, there is a lot of work done over and over again.

    Our solution

    The MaD GUI offers a basis for such scenarios. It can be extended by developers to load their specific data format. Furthermore, developers can easily inject their algorithms to support labelling or analyze time series data as a plugin. In addition, developers can specify the labels to be used - it could be one layer (= one label from top to bottom) or an arbitrary amount of layers, (e.g. use one kind of label in the upper part of the plot and another kind of label in the lower part of the plot). During annotation, each label can easily be assigned an additional description to add details to the specific annotation.