Multi-Plexer, Formely Jetson_ffmpeg_trancode_cluster

This project aims to bring together a few things together.

The inciting goal was to: lower my power consumptions learn aarch64 (AKA 64 bit arm) learn about clustering implement platform agnostic hardware accelerated transcoding for ffmpeg

I've also been tracking some ups and downs on the Level1Techs forum in this thread! https://forum.level1techs.com/t/building-a-10-100w-distributed-arm-based-transcode-cluster-i-missed-devember-jobless-january-floundering-february/167831/40

Components

Hardware:

Host server (either dedicated or worskstation PC)
Networking switch (poe?)
Multiple Nvidia Jetson's (ideally NX (agx is the dream)) but probably 2gb nano's or, depending on launch dates of Nvidia roadmap, Jetson Nano Next or Jetson Orin

Other potential option:

Turing Machines are launching a V2 of their Turring Pi cluster board, and it will have 4 260 pin slot's, initially deisgned for the Rpi CM4. I've reached out to their questions email adress asking about potential support, but have yet to hear back. Would alow for any mix of Nano (2 GB or 4GB) and Xavier NX + any other Jetson SOM that Nvidia may release.

Software:

Host
OS for server- debian based
Unicorn Transcoder or Kuber Plex
NFS share (Data itself is accessed over the network on my nas, ZFS and so on)
Custom capture scrip to modify arguments sent from PMS to transcoder
Plex Media Server
Load balancer
Jetson (aka transcoder node)
Jetpack (4.3?)
client side of UT or KP from 1.2.
custom ffmpeg build
Personal cuda patch (should upstream to newest branch once I'm done) for tonemapping. Currently Reinhard and Hable, but will change to eotf BT.2390 Soon TM
Jcover90 ffmpeg patch to enable the use of the transcode blocks
Nvidia build of ffmpeg to enable decoding and vf_scale_cuda
Client side of load balancer from 1.6.
(things I forgot will go here)

How do I plan to architect this?

Update August 2021

Been busy and burned out of late with other projects, and hoping to come pack to this one enventually. here's a layout for what needs to be done in a clear outline that isnt overwelming and can be takled in small bites.

Step 1: have ffmpeg tonemap using a filter

Step 2: compile plex with custom filter built in

step 3: have local plex call custom filter

step 4: have local plex call external plex, without custom filtering

step 5: have local plex call external plex, with custom filtering

step 6: have local plex turn off unneeded nodes and turn them back on as needed.

step 7: integration testing