/patient-chat-hybrid-webrtc

Primary LanguageTypeScript

Local Video Chat

How it works

Frontend

The frontend is coded in Vue through the Quasar framework.

Quasar allows to define multiple destination platforms: Single Page Applications, Progressive Web Apps, Android or IOS through Apache's Cordova or Ionic's Capacitor, OSX or Windows though Electron...

This project focuses on the use of Cordova to deliver an Android app, but also generates an SPA for developement and testing purposes.

Backend

The Android app embeds a NodeJS environment. It then runs an HTTP Express server, that includes a PeerJS server that acts as a WebRTC STUN server.

This backend server is then published as a Bonjour/Zeroconf/mDNS service.

By default, the Android user connects to its own PeerJS server and is listening to any call.

Any client can then get the list of available peers though DNS Service Discovery.

The user picks one server from this list, checks if there is already someone connected to it, then connects with the PeerJS client and call the other person though WebRTC.

When the call ends, the client disconnects from the remote server and connects again to its own local server, so it waits for calls.

Why not a single Android "server" app + SPA "client" ?

We could have used the embedded Node Express server to serve a static SPA app, so other users would not have needed to install an Android app. However:

  • https is required to get getMediaDevice() working, and an SSL certificate can't be obtained without internet (e.g. Let's Encrypt). Only self-signed certificates are then possible, but will raise an alert on the client that the site is not secure, and will possibly still flag context as unsecure and therefore block video.
  • Service Discovery through Bonjour/Zeroconf would not have been available. The client would have needed to enter the server path to start the application. It could have been mitigated by setting a QR-code / SMS sending system to get the link, plus in installing the client as a PWA
  • If the user running the initial server leaves, everyone in the network is disconnected. While it is not a problem in the case of a two-users-only network, this would become messy if more people use the same network.

The main caveats for using an Android app everywhere is the need of the internet to install the app from the Google Playstore or any other place. This can be mitigated in serving the apk on the Express server for download (e.g. through a QR-code or an SMS), or finding other ways to share the file though a local network. The link could be available on the client side through a QR code.

Development

Prerequisites

You need Node to be installed.

For Android developement, you will need the Android SDK, the Android NDK, etc.

TODO: complete this

Clone

git clone https://github.com/plmercereau/patient-chat-hybrid-webrtc
cd patient-chat-hybrid-webrtc

Install dependencies

yarn
cd server
npm install
cd ..

Android

TODO: Put the following tasks into Corvova hooks. See:Steps to patch the cordova-nodejs-module

https://medium.com/@ivancse.58/how-to-resolve-no-toolchains-found-in-the-ndk-toolchains-folder-for-abi-with-prefix-b37086380193 https://stackoverflow.com/questions/32601116/not-able-to-test-cordova-android-app-in-device-attached-via-usb

cd src-cordova
npm install --save nodejs-mobile-cordova
cp setup/plugin.xml node_modules/nodejs-mobile-cordova/
cordova add platform android
cd www
ln -sFf ../../server nodejs-project
cd nodejs-project
npm install
cd ../..
echo "ext.cdvMinSdkVersion = 21" > platforms/android/build-extras.gradle
cd ..
quasar build -m android
chmod +x platforms/android/gradlew
  • Check src-cordova/plugins/android.json. It should have:
 "installed_plugins": {
    ...
    "nodejs-mobile-cordova": {
      "PACKAGE_NAME": "plus.platy.chat.app"
    }
  }

Android 5

Update Android System Webview. See: https://developer.chrome.com/multidevice/webview/overview

Start the app in 'web' mode

You have to run three services in parallel:

  1. The Express server
cd server
npm run dev
  1. Its mDNS publication
dns-sd -R <service_name> _http._tcp. local. 3000 name=patientchat
  1. The Web client (served over WebPack)
quasar dev

Build the app for production

Build web version

quasar build

Build android version

TODO

Prototype

  • Pick the right permissions
  • Prompt Android permissions
  • hide own local server from the list of servers
  • build a "stable" apk to test between devices and share with the team
  • Double check if server works correclty in background. Seems ok.

Strengthening

  • fix this damn latency/echo thing!!!
  • improve disconnection UX: send a 'end call' event, instead of waiting for the connection to be lost
  • change the server port from 3000 to something else
  • "calling" spinner
  • simpler navigation bar
  • custom video controls
  • better local video presentation, e.g. bottom-right of the remote video
  • put the service discovery / service publishing / service information (e.g. online, busy...) on the server side

Later

  • adjust video/audio quality to the existing bandwidth, latency, etc.
  • "calling" prompt / put the app in the foreground when running in the background
  • let the Android app set a Wifi hotspot
  • let the Android app select a Wifi hostpot?
  • Share the apk. Options:
    • send the server url by SMS
    • print the server url on a QR code
  • publish to the app store
  • build and publish with GH Actions or another CI tool

Nice to try

  • iOS app
  • Electron app

Not applicable anymore