NDI/VISCA PTZ Controller for Raspberry Pi This code is a complete, but basic console-based pan-tilt-zoom controller for the Raspberry Pi 4. (In other words, this does not run in X11; it literally writes directly to the framebuffer.) --------- Features: --------- * Requires the Pimoroni I/O Expander for input purposes. * Supports three-axis analog joysticks. * Supports five pushbuttons for loading stored positions, and a set button for storing new positions. * Supports lights for showing which position was most recently loaded (reset when you move the camera) and for the set button (toggle) state. * Supports remote tally light polling to show whether the camera is in preview mode, program mode, inactive, or unresponsive (e.g. a network failure). (Requires a VISCA-compatible camera.) * Supports zooming at variable speed. (Requires a VISCA-compatible camera.) ------------------- A Note About VISCA: ------------------- VISCA is a protocol for controlling PTZ cameras over IP. Every NDI camera that I have tested so far supports VISCA, though some devices may require you to use a different port. This code has been tested against cameras by AVKANS, NewTek, and Marshall. Your mileage may vary. The reason VISCA is required for full functionality is that the NDI protocol doesn't provide some required features. I've filed bugs with the NDI SDK team about some of these issues, and will continue to try to find ways to eliminate the need for VISCA, because it's kind of a bloated, unreliable mess (and this code is a particularly hackish implementation that I wrote very quickly to work around those limitations). Specifically, the following functionality does not work when working with a pure NDI camera: * Tally light support. The NDI SDK lets you set the tally light, but not obtain its current state. If you're using OBS, it would theoretically be possible to add OBS websocket support to this app, which would allow you to find out whether OBS has turned on the tally light or not. However, all of my cameras support VISCA, so it wasn't worth the effort to do that when I already had a mostly working VISCA implementation. * Zoom speed support. This should work; the NDI SDK lets you pass a zoom speed value from -1.0 to 1.0. However, in real-world testing, exactly none of the cameras I tested were able to zoom at multiple speeds when driven through that function call (from either the Linux/Raspbery Pi SDK or the macOS SDK). Presumably, this will get fixed at some point on the camera side, at which point VISCA will no longer be required for this purpose. * Exposure adjustment and manual exposure. The NDI SDK lacks exposure compensation APIs, though it does provide manual exposure settings. But because all my cameras support VISCA, it wasn't worth the extra effort to hook up the manual exposure calls via NDI. (It's would take only about two lines of code, though, so if you need it, it should be an easy change.) This PTZ controller software looks up the camera using Avahi, then connects to the VISCA UDP port on that same device. This may or may not be the best approach, but it works (with the caveat that you have to provide the VISCA port). -------------- Prerequisites: -------------- * Download my C translation of the Pimoroni I/O Expander library: https://github.com/dgatwood/ioe-c and symlink the files constants.h and ioexpander.c into the source folder. * Configure the console to use 32 bpp. In config.txt, add/change: disable_overscan=0 framebuffer_depth=32 display_rotate=2 dtparam=i2c_arm=on dtparam=spi=on * For HDMI panels, add/change: max_usb_current=1 hdmi_force_hotplug=1 hdmi_group=1 hdmi_mode=16 hdmi_drive=2 * For DSI panels, add/change: video=DSI-1:1920x1080@60 framebuffer_width=1920 framebuffer_height=1080 * Disable the videocore 3D driver by commenting out the following line: # dtoverlay=vc4-fkms-v3d * Disable the blinking cursor on the virtual console that you plan to use. In cmdline.txt, change the console to: console=tty3 And add: vt.global_cursor_default=0 * Disable X11 in raspi-config. * Install the NDI SDK from NewTek (available separately). * Install Avahi (client and common libraries). * Install pigpiod and enable the daemon. * Build the custom hardware. * Configure dhcpcd on your device to use a static IP fallback in the same IP range as your cameras (typically 192.168.100.x). For example: profile static_eth0 static ip_address=192.168.100.23/24 static routers=192.168.100.1 static domain_name_servers=192.168.100.1 interface eth0 fallback static_eth0 * Add NDI to ld.so configuration if it is not there already: Create a file named /etc/ld.so.conf.d/ndi.conf and paste in the following: /usr/local/NDISDK/lib/arm-rpi3-linux-gnueabihf or wherever you installed the SDK. ------------------- Command-line flags: ------------------- To see a list of available cameras, type ./cameracontroller and press return. This currently loops forever, because cameras may not appear instantly. Press Control-\ to terminate the app. We should probably fix this at some point. Usage: ./cameracontroller [flags] <camera_name> where <camera_name> is typically something like "NDIHX-PTZUHD (chan 1, 192.168.100.168)". Note that this tool searches for a camera with a matching name and IP address first, but then falls back to a name match at a different IP. Be certain that each camera has its own distinct name, because there is no guarantee that this software will correctly detect the camera in the initial scan before falling back and matching against identically named cameras at different IP addresses. General flags: -f / --fast -- Enables preview mode, in which a lower quality (typically 720p) stream is requested. This can be helpful for cameras that produce higher bitrate streams. (You can play NDI-HX streams from iOS devices without this flag.) -D / --duty_cycle -- Sets the duty cycle that should be used for all LEDs (range 0 to 255). VISCA command-line flags: -V / --enable_visca -- Enables VISCA for camera control. -u / --visca_use_udp -- Enables UDP mode for VISCA. At this point, TCP support is untested, so you should always pass this flag. -p / --visca-port -- Sets the VISCA port. Most cameras use port 52381 UDP. However, this code defaults to the PTZOptics port, 1259 UDP. Note that PTZOptics cameras are entirely untested. -O / --onscreenlights -- Configures the code to use on-screen boxes instead of physical status LEDs. (Note that some status features are available only with VISCA.) VISCA-only features: -e / --exposure_compensation -- Sets exposure compensation amount (-5 to 5) -a / --auto_exposure -- Enables automatic exposure (disables -i/-ig/-s). -i / --iris -- Sets manual exposure with the specified iris (range 0 to 20). The meaning of these values is camera-dependent. -g / --gain -- Sets manual exposure with the specified gain (range 0 to 15). The meaning of these values is camera-dependent. -s / --shutter -- Sets manual exposure with the specified shutter (range 0 to 21). The meaning of these values is camera-dependent. Debugging: -d / --debug -- Enables some basic debugging -B / --buttondebug -- Enables button-specific debugging -P / --ptzdebug -- Enables PTZ (joystick) debugging -v / --verbose -- Enables more detailed debugging -------------------- Other Configuration: -------------------- The file LEDConfiguration.h lets you change the brightness of each LED individually. ---------------- Common Mistakes: ---------------- If the tool can see your camera but cannot receive video: 1. Reboot the camera. Some cameras have known bugs. 2. Make sure your IPv4 address is working. Most cameras support IPv6 for discovery, but do NOT support streaming over IPv6. 3. In particular, make sure that you aren't sharing an IP address with another camera or PTZ controller! 4. Make sure you aren't running any weird firewalls that might interfere. ---------------------- Building the Hardware: ---------------------- The custom hardware construction (for actual PTZ control) is documented here: https://docs.google.com/document/d/1Ss_mx8vgRw_ys8ocJK7LZhjXRocf9EI3qovyQXoxxJw/edit?usp=sharing or in the hardware directory. Enjoy!
glikely/ndi_camera_control
An NDI-based and VISCA-based pan-tilt-zoom controller for Raspberry Pi (with limited support for debugging on macOS).
HTML