/desk-commander

A build for a RPi-based desktop control touchscreen

Primary LanguageJavaScriptGNU General Public License v3.0GPL-3.0

desk-commander

A build for a RPi-based desktop control touchscreen.

Idea

I want a unified system to control various desktop devices. My criteria are:

  • Touchscreen with status feedback (which device is selected, etc.)
  • Avoid IP control as much as feasible
  • Be faster than controls on various devices
    • Monitor input switching is a far reach
    • My Orei UKM-404 USB matrix switch is 1 button per device, so each device I want to move from output 2 to output 1 is 3 presses, and there is a delay between them.
  • Be able to interact with my Home Assistant for room control
  • Allow for expansion in the future
  • Document this and make it flexible enough for other people to build
  • Add some cyberdeck styling :)

Parts

Total BOM is looking to be < $250, currently around $212.

Software

  • Raspbian OS
  • Electron
    • Uses Python backend launch model from Python Electron proof of concept, but using an HTTP backend instead
  • Flask
  • Device-specific libraries and debug tools listed with External Device types

External Devices to Control

Monitors

This is a complex chain, but actually easy to use:

  • Both HDMI and Displayport generally support the the VESA Display Data Channel Command Interface standard (DDC/CI) (a copy of the spec can be found here). Many monitors support this functionality, but some do not or may have odd issues (flashes, etc).
  • On this, you can issue commands in the Monitor Control Command Set (MCCS) (a copy of the spec can be found here), which standarizes sending Virtual Control Panel (VCP) commands that equate to typical monitor physical control functions.
  • VCP 60h is input selection, and the values are generally standardized:
    • 01h - 02h: Analog video (R/G/B) (typically VGA) inputs (2x)
    • 03h - 04h: Digital video (TMDS) over DVI inputs (2x)
    • 05h - 06h: Composite video inputs (2x)
    • 07h - 08h: S-video inputs (2x)
    • 09h - 0Bh: Tuners inputs (3x)
    • 0Ch - 0Eh Component video (YPbPr / YCbCr) inputs (3x)
    • 0Fh - 10h DisplayPort inputs (2x)
    • 11h - 12h Digital Video (TMDS) over HDMI inputs (2x)
    • It's unclear how monitors with more inputs than covered (such as the Acer X32 FP with 4 HDMI inputs) or non-covered inputs (USB-C comes to mind, but it's likely treated as a DisplayPort)
  • Linux and Windows both have solid DDC/CI support. DDCutil on Linux is the easiest way to test this, and there are bindings for a bunch of languages.

Tooling

  • Linux Debug tool: DDCUtil - CLI tool, great for at least testing
  • Python library: None - just use subprocess to call ddcutil.

USB Switch/etc. via RS232 on a USB-TTL adapter

As an example, the following matrix switches support RS-232:

Notes:

  • These simple RS-232 devices cannot handle typing-speed commands - commands must be sent all at once.
  • By default, the OREI is 9600 baud, while the GoFanco is 115200 baud. Both are 8N1.
  • The OREI includes a premade 3-pin to serial adapter, and the GoFanco includes 3-pin connectors (since it uses them for audio too). Additional connectors are available on Amazon.

Tooling

  • Linux Debug tool: cu or minicom
  • Python library: PySerial - Python serial library

Hubspace

Hubspace is Home Depot's Afero-based smart home product line, and I have a ceiling fan with it. I could do this through HomeAssistant, but that integration is a little messy. I know a guy who wrote a clean Python backend, and may eventually make a better HA integration, but in the mean time, why not use the Python backend directly? Nosce te ipsum.

Tooling

Home Assistant

It'd be nice to be able to provide some Home Assistant controls. I don't want to use HA to control the monitors/etc., but I'd love to be able to control other home-automation devices.

Tooling

Design Decisions

Why separate the video and USB switches?

Especially with multiple monitors, flexibility is useful. I eventually want to find a good Matrix switch.

Why not use a DisplayPort switch?

They are very rare outside of KVMs, and they tend to be feature limited.

Why not just use an external HDMI switch alone?

My monitors (and many others) only have HDMI 2.0, which can only do 2K@144Hz in SDR. For 4K, higher refresh rates, and HDR, you need HDMI 2.1 (with up to 48Gbps bandwidth) or DP 1.4+.

Why not use a GPIO serial HAT?

I think USB is slightly simpler here. I'd rather not tie up GPIO pins with something that is easily handled via USB, and I may need a number of RS232 serial connections.

Why use that case and that touchscreen?

I want a DSI touchscreen because I want both HDMI ports to control 2 monitors. There are very few DSI touchscreens, and the Pi one - while not the biggest or best - is a de-facto standard, so I can get a pre-made case for it. I also want to minimize fidgetiness - some other touchscreens require non-standard kernel params, etc.

Why avoid IP/network control?

It's unnecessary, may add complexity, and brings a host of security issues. Devices with it baked in often have bad implementations; custom implementations might mean a bunch of small edge devices.

Why not use X for the UI renderer?

Why not use X for the Web UI widgets?

  • Arwes - Awesome looking, but not wholly ready for prime time - might use for background and frame still?
    • Lack of documentation
    • Many elements that should be part of the framework (like Button) are instead part of the demo site app and would need re-implemented
  • Augmented UI - Only covers frame-like elements, not really the aesthetic I want
  • Codepens

Why not just use node.js the whole way through?

  • I already had my Hubspace-ng library and some prototype code.
  • I wasn't originally doing an Electron GUI.
  • JS is just a bit messier than I want for the backend.

Other Refs

Install

  1. Clone repo
  2. Run pip install -e . from the root project folder
  3. Run npm install from the electron-react-app folder