OpenKVM

DIY KVM device for remote-controlling a computer. KVM stands for Keyboard & Video & Mouse.

A side system on a rackmount server, IPMI in Dell Server for example.

Unlike an IP-KVM, this project respects VNC protocol.

Just like the MIT license says, no warranty or guarantee.

And do NOT use for any illegal purposes.

TODO

Installation script
- Register as a system service
- Start on boot
VNC authentication
- DES encryption in Golang can NOT directly apply to VNC Authentication
More effective to calculate the difference between frames
- Balance between the power of SBC and the network efficiency
- Or achieve more support for noVNC, beyond rfc6143
Device filter for USB devices: for multiple device controlling
OTG as keyboard and mouse, see Linux USB Gadget API

Using a single command to get the frame for Video, like

v4l2-ctl --device=/dev/video0 --stream-mmap --stream-count=1 --stream-to=- --set-fmt-video="width=640,height=480,pixelformat=MJPG"

Dev Environment

Hardware

NONE of them is sponsored, use them at your own risk!

Essential hardware are:

A computer that can run Golang
- An old computer
- SBC
  - RaspberryPi
  - OrangePi
  - BTT-Pi
  - etc...
- Maybe an Android phone?

HDMI Recorder

The common seen HDMI Recorder is a USB 3.0 device.
There is only blank data from the HDMI Recorder on a USB 2.0 port.
It seems that this is the problem of the HDMI Recorder itself,
the HDMI Recorder I bought can NOT work on Linux (Debian ARM or Ubuntu 24.04) even with a USB 3.0 port.
Then, on the selection of SBC, beware of the support of USB 3.0.

This should be the problem with HDMI recorder itself of the driver in Linux,
I am lack of the knowledge of the driver in Linux, but I might find a solution for it:

# Display all USB device, make sure the HDMI recorder is in 480M or above
lsusb -t
# /:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
#     |__ Port 1: Dev 3, If 0, Class=Video, Driver=uvcvideo, 480M
#     |__ Port 1: Dev 3, If 1, Class=Video, Driver=uvcvideo, 480M
#     |__ Port 1: Dev 3, If 2, Class=Audio, Driver=snd-usb-audio, 480M
#     |__ Port 1: Dev 3, If 3, Class=Audio, Driver=snd-usb-audio, 480M

# Get the id of the USB device
lsusb
# Bus 001 Device 003: ID 1de1:f105 Actions Microelectronics Co. Hagibis

# Reset the USB device
# Replace this with the id or the name of your USB device. 
# For me, this is `Hagibis` or `1de1:f105`
usbreset Hagibis

# Capture one frame from device
v4l2-ctl --verbose \
--device=/dev/video0 \
--stream-mmap \
--stream-count=1 \
--stream-to=frame.jpg \
--set-fmt-video="width=640,height=480,pixelformat=MJPG"

# BUT! But, in size of 1920x1080, frame will be corrupted

# Without resetting the USB device, there will be some error message in `dmesg` command
# Non-zero status (-71) in video completion handler.

# Here are some helpful commands
# List all video devices
v4l2-ctl --list-devices
# List all supported formats for a video device
v4l2-ctl --list-formats -d [device name or path or index] # v4l2-ctl --list-formats -d 0
# List all supported frame sizes
v4l2-ctl --list-framesizes [pixel format] -d [device] # v4l2-ctl --list-framesizes MJPEG -d 0

Or a webcam with an always-on monitor.
- I know...this is a stupid way -- pointing a camera to a screen.

Keyboard & Mouse Emulator
- ESP32-S3
- ESP32-S2
- Arduino?
- Some other device that supports USB HID output.
  - HID over BLE is not recommended, because it may not work in BIOS.
Relay and/or delayed relay

My Gears

SBC: OrangePi 3 LTS
- ¥241 RMB ≈ $34 USD
HDMI Recorder: hagibis UHC07
- ¥69 RMB ≈ $10 USD
- ⚠️ This device ONLY supports 1920x1080 and 3840x2160.
Keyboard & Mouse: ESP32-S3
- I bought a third-party one of WeAct ESP32 S3 (A) DevKitC 1.
- ¥52 RMB ≈ $7.5 USD
5V relay * 2, 5V delayed relay * 1
- ¥15 RMB ≈ $2 USD

Others

Test Device:
- RaspberryPi 2B
- A Windows computer
- Ubuntu 24.04 x86_64
Some SD cards.
Some USB Type-C2C/C2A cables.
A HDMI cable.
Some power supplies.

It costs about ¥300 RMB ≈ 40 USD.

Price is for reference only, the actual price may vary.

Software

Diagram

飞书文档, FeiShu Doc

Run

Debian ARM64

Install GO dev kit

sudo apt-get update
sudo apt-get install -y wget curl ffmpeg v4l-utils
GO_ZIP="go1.22.3.linux-arm64.tar.gz"
wget "https://go.dev/dl/$GO_ZIP"
sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf "$GO_ZIP"

Pull this repo

sudo apt-get update
sudo apt-get install -y git
git clone https://github.com/allape/openkvm.git

Install OpenCV

Method 1: Install from APT

sudo apt-get update
sudo apt-get install -y build-essential g++ cmake pkg-config unzip libopencv-dev

# But in BTT-Pi, it is only up tp v4.5.1
# Then, I have to change the version of `gocv.io/x/gocv` to corresponding version
sed -i 's/gocv.io\/x\/gocv v0.36.1/gocv.io\/x\/gocv opencv-4.5.1/g' openkvm/go.mod

Method 2: Build OpenCV from source
This may cost over 10 hours to build on BTT-Pi, I crashed at 96% :(
https://docs.opencv.org/4.x/d7/d9f/tutorial_linux_install.html

mkdir opencv4 && cd opencv4
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.x.zip
unzip opencv.zip
mkdir -p build && cd build
cmake ../opencv-4.x
cmake --build .

Get noNVC

git clone --depth 1 https://github.com/novnc/noVNC.git

# [Optional] You can run noVNC separately
python3 -m http.server --directory noVNC/ 8081

Flash ESP32-S3
- ~~PIO: Espressif has dropped the support for PIO in ESP-IDF v5.x. See git log to retrieve the code.~~
  - Open folder km/esp32s3 of this project in VSCode
  - After installing PlatformIO extension
    - Go to PlatformIO Tab
    - PROJECT TASKS -> Default -> General -> Upload
    - Or open command palette and type PlatformIO: Upload
      - ⌘ + shift + p on macOS to open command palette
      - ctrl + shift + p on Windows or Ubuntu
- Arduino
  - Open Perferences -> Additional Board Manager URLs -> Add https://espressif.github.io/arduino-esp32/package_esp32_dev_index.json
    - Click HERE for more details
  - Open km/esp32s3-arduino/main/main.ino with Arduino IDE
  - Select board ESP32S3 Dev Module and corresponding port
  - Click Upload

Run or build repo

cd openkvm
cp kvm.toml.tpl kvm.toml

# Find out serial port
dmesg | grep tty

# Edit this file to apply your settings
vim kvm.toml 

# Should run with super user privilege
sudo go run .

Open browser and go to http://ip:8080/vnc.html, then click Connect
- Hostname and port may vary depending on your settings

allape/openkvm