CONCEPT REPO! This is an idea I have that I might try out someday.
You can add blocks and connect different portions of the outputs. Blocks can be things like convolution, pooling, sparse. Blocks have a section for different activation functions. Inputs can be remapped, etc.
Once the design is done, you can export the model code using different libraries, such as TensorFlow, PyTorch, and Mojo.