Hi-Dolly

Exploratorium Mini Science Lab 471

Status

I've decided to make a mini Exploratorium science lab for the street. With a mini science lab, there can be a lot of neighborhood magnet and rocks sharing, which is great, plus small electronic science and robotic experiments. First round will be sms and web speech to text to image generator. Thanks to @nucleardreamer for all the great ideas and for working on this with me!

Highlights

Chrome Speech to Text Web Api: talk at your internet browser!
Twilio Node Api: SMS to Nodejs to SQLite, save your texts for eternity on a pi
Motion Sensor Magic Wave hand in the air and stuff happens, magically
SQGAN-CLIP Art: Generated Images

What I have working so far:

twilio-block sms /web speech to text -> node server to sqlite on pi zero fleet. (Could this be a balena block?)
SQGAN-CLIP dockerized and generating images on the NUC
motion sensor trigger to turn on an off web microphone
a box for the mini lab
randomized display for images

Todo:

Fix the fastapi server on the SQGAN docker(had issues with volumes in 2.1 docker so paths are wonky)
add a micro usb pi microphone
setup new 1.5" TFT instead of big 9" TFT
dig a post hole and mount the lab outside
Steel sheet roof for mini lab box
gather documentation from notes, git wiki, turn into blog post

Nice to haves:

solar panel on the mini roof for power, otherwise someone's going to be charging batteries constantly.
GPU renderfarm type situation with the image generation so make it faster
MQQT block for intercommunication

Setup and configuration

Create a fleet of zeros. Nice to have one as primary and one as backup sms server
Create a fleet of GPUs ideally ( I am trying on the NUC which is slow )
clone the code git clone git@github.com:zoobot/twilio-block.git git clone git@github.com:zoobot/vqgan-clip-docker.git
cd to each repo
push code to the fleets (Nuc for vqgan-clip-docker and pizero for twilio) balena push username\fleetname OR local dev balena push UUID.local balena push internal-ip-address
Setup Twilio account and get a phone number and setup balena public urls on the twiml and phone api messaging at twilio

Deploying

Need to figure out multiarchitecture/multifleet deployment! Currently just run balena push with the twilio-block and sqgan-clip-docker separtely. Both repos have separate docker-compose.yml.

Documentation

Check the wiki for not quite organized yet docs.

Motivation

I love the science experiements at the Exploratorium and it would be great to have a neighborhood crowd sourced mini version of it. DALLE-2 is a fascinating evolution of human knowledge and has inspired so much interesting art generation.

Getting Help

If you're having any problem, please raise an issue on GitHub and we will be happy to help.

Contributing

Have at it!

zoobot/hidolly