/M5Stack-HID-Input-Framework-for-xR

Primary LanguageC++GNU General Public License v3.0GPL-3.0

M5Stack HID Input Framework for xR

This project is a firmware for M5Stack developed by a team of instructors for the participants of NEWVIEW SCHOOL 2021 TAIPEI, a workshop on xR (VR/AR/MR), to enable them to create DIY controllers.

The M5Stack is a toolkit that allows you to easily create various IoT projects by simply connecting sensors and other units with cables. The controller functions as a Bluetooth keyboard, sending keyboard events to the device in response to input from sensors, buttons, etc.

Instead of just accepting a pre-made controller as a constraint and using it as it is, by thinking about what a suitable controller for your work should be and realizing it, you may be able to create a work that expands the concept of VR and AR.

該項目是由講師團隊為參加NEWVIEW SCHOOL 2021 台北 xR (VR/AR/MR)研討會的參與者開發的M5Stack固件，使他們能夠創建DIY控制器。

M5Stack是一個工具包，讓您只需用模組連接線連接感測器和其他單元，即可輕鬆創建各種物聯網項目。控制器用作藍牙鍵盤，將鍵盤事件發送到設備以響應來自感測器、按鈕等的輸入。

不再受限於既有的控制器，讓你可以創造不同體驗的控制器去實現你對VR與AR的概念及創意

このプロジェクトは、xR（VR/AR/MR）に関するワークショップ「NEWVIEW SCHOOL 2021 TAIPEI」参加者のために講師チームが開発した、DIYでコントローラを制作できるようにするためのM5Stack用ファームウェアです。

M5Stackは、センサなどのユニットをケーブルで接続するだけで簡単に多彩なIoTプロジェクトを実現できるツールキットです。M5Stackとこのファームウェアを組み合わせることにより、電子工作やプログラミングの経験がない人でも、短時間でDIYコントローラを制作できるでしょう。コントローラはBluetoothキーボードとして機能し、センサやボタンなどの入力に応じたキーボードイベントをデバイスに送信します。

あらかじめ用意されたコントローラを制約条件として受け入れてそのまま用いるのでなく、自分の作品に適したコントローラはどうあるべきかから考えて実現することにより、VRやARの概念を拡張するような作品を制作できるかもしれません。

Status

Supported Units

Supported platforms

  • Oculus Quest 2 - STYLY app (from App Lab)
  • Windows - STYLY VR app
  • Windows - Chrome
  • macOS - Chrome
  • iOS/iPadOS - STYLY app
  • Android - STYLY app

Supported combinations (maximum)

The connected Unit(s) input is transmitted on three different channels: analog, joystick, and buttons. The analog and joystick channels can handle up to one sensor Unit input for each at once, and the buttons channel can handle up to six button inputs simultaneously.

Unit Port Channel Pattern A Pattern B Pattern C Pattern D Pattern E Pattern F
Dual Button B Buttons 💡 💡 💡 💡
Light Sensor B Analog 💡 💡
Joystick A Joystick 💡 💡 💡 💡
Gas Sensor A Analog 💡 💡
Ranging Sensor A Analog 💡 💡
Touch Sensor A Buttons 💡 💡 💡
RFID Reader A Buttons 💡 💡 💡

Notes

  • If you want to connect multiple Units to port A, please connect via a 1 to 3 HUB Expansion Unit.
  • When a Ranging Sensor Unit is connected, you can't use a Joystick Unit nor Gas Sensor Unit.
  • To change the connection, double-click the power button on the left side of the M5Stack controller to turn off the power (you cannot turn it off while it is connected to a USB port).

How to try

Download

  1. Download M5Burner for your platform from the official website
  2. Extract and launch the M5Burner (move to your Applications folder before launching on macOS)
  3. Choose the serial port in the COM menu on the top-left corner
  4. Choose "HID_Input_Framework_for_xR" from the projects
  5. Click on the Download button of the project
  6. Once finished downloading, the Download button becomes the Burn button
  7. Click on the Burn button to burn the firmware

Test

  1. Choose a pattern from the table above and connect Unit(s) to your M5Stack controller (e.g., M5Stack FIRE)
  2. Power on your M5Stack controller
  3. If you want to use Dual Button, Light Sensor, or RFID reader, please refer to the "How to setup" section and setup
  4. Connect the controller as a Bluetooth device to your device (please follow standard instruction for the device)
  5. Open the Test_Gamepad scene in a browser and bring the browser frontmost
  6. Press the Send (C) button to start sending
  7. Control the joystick, sensor, etc.
  8. Once confirmed, press the Stop (C) button again to stop sending

STYLY_marker

Setup

PORT B

  1. Press the Setup (A) button to enter the preferences screen
  2. Press the Next (C) button (if needed) to choose the PORT B: NONE line
  3. Press the Go (B) button
  4. Press the - (A) or + (C) button to be matched to the device connected to the Port B (i.e., DUAL BUTTON or LIGHT)
  5. Press the Done (B) button
  6. Press the Exit (A) button to back to the main screen

RFID Tags

  1. Press the Setup (A) button to enter the preferences screen
  2. Press the Next (C) button (if needed) to choose the RFID 1: **:**:**:** line
  3. Press the Go (B) button
  4. Press the Reset (C) button
  5. Put an RFID Tag on the RFID Unit, then remove the Tag
  6. Press the Next (C) button to choose the next line
  7. Repeat from step 3 for 3 times to register the remaining 3 RFID Tags
  8. Press the Exit (A) button to back to the main screen

Troubleshooting

  • "alt. controller" is shown as connected but no input to the STYLY scene. → Please try to unpair the controller in the Bluetooth preference and pair it again.
  • Bluetooth connection status on my controller keeps switching between Connected and Disconnected when not connected. → The controller might have been paired with an old host (i.e., a PC or smartphone). If you no longer use the controller with the host, please remove the device from the host.

Protocol

keyboard-layout

  • Keys in green: used in STYLY Web Player
  • Keys in yellow: used for the Analog channel
  • Keys in orange: used for the Joystick channel
  • Keys in red: used for the Buttons channel

Analog channel

Analog value Key
0 `
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 0

Joystick channel

Joystick Key
Left-Up y
Up u
Right-Up i
Left h
Center j
Right k
Left-Down n
Down m
Right-Down ,

Buttons channel

No. Name Key Assigned input
1 Fire1 v Dual Button - Red
2 Fire2 b Dual Button - Blue
3 Fire3 f Touch Sensor - CH0
4 Jump g Touch Sensor - CH1
5 r Touch Sensor - CH2
6 t Touch Sensor - CH3

Requirements (for developers)

Preparing the development environment

Libraries

How to install

The ESP32 BLE Keyboard library

  1. Download the ESP32 BLE Keyboard library (don’t have to extract after downloading)
  2. In the Arduino IDE, navigate to Sketch → Include Library → Add .ZIP Library...
  3. Select the file you just downloaded

Other libraries

  1. In the Arduino IDE, navigate to Tools → Manage Libraries...
  2. Type in a part of each library (i.e., VL53L0X, Adafruit SGP30, and Adafruit MPR121) in the text field in the top right corner, choose the right library and press the install button
  3. Repeat the second step for all required libraries

VL53L0X

Adafruit SGP30

Adafruit MPR121

Note

  • Regarding the VL53L0X library, please install the one by Pololu, not by Adafruit.
  • If you want to install the latest stable ESP32 SDK, please put the Stable release link described in the Arduino core for the ESP32 project's document and install the esp32 board package instead of the M5Stack board package. Please don't confuse this with the M5Stack library; we definitely need the library.

Credits