/baby-AI

A learning baby-AI, brewing from recipes

MIT LicenseMIT

Notes from a dream

How to build an AI

  • Google clips: important moments
  • Motion track, object recognition
  • Movie audio -> ASR + review or subtitled dialogs: train language model for good moment
  • Knowledge graph: link pic+words
  • Reinforce learning: update network with verbol interaction

Provisional ideas

  • memory/knowledge/concept generation

  • some intuition to neural network

    • spiking neural network: add memory/time to neuron
    • knowledge graph/model checkpoint as output
    • output ensembled from a 'random' selection of sub neural nets
    • kernel output with a distribution (upscaling kernel): adding degenerecy to neighbor neuron $y_{ij}^l = \sum_{a=0}^{m-1} \sum_{b=0}^{m-1} \omega_{ab} \sigma(x_{(i+a)(j+b)}^l)$ unnecessary, equivilent to add additional convolutional layer
      • non-linear distribution added on activated out?

  • mind-blasting

    • random firing: all necessary weights can be random initialized, just need to activate the 'right' neuron and reinforce during the training

  • attention heat-map

  • ...

What I can prototype on

  • YI Dome camera for vision
    • motion track
  • Respeaker HAT for sound
    • direction of orientation
  • Raspberry Pi

Refs

Video Feed using Yi Dome 1080p

Status

  • ssh for dev: dropbear
  • managing recording: ftp
  • RTSP server for real-time feed: conflict with cloud service
  • Yi Dome to Pi
    • no dclient to scp
    • rsync from Yi Dome, not packed
    • scp from Yi Dome, dclient not packed
    • ftp/scp from Pi + crontab
HOST='192.168.1.24'

cd ~/Documents
check=`cat latest`
latest=`ssh root@$HOST " cd record ; ls -tr */*.mp4 | tail -1"`
if [ "$check" != "$latest" ]
then
	file=`echo $latest | sed "s/\///g"`
	scp root@$HOST:/tmp/sd/record/$latest Vision/$file
	echo $latest > latest
fi

REFs

Notes

hardware specs

$ cat /proc/version
Linux version 3.4.35 (zhangbao@ANTS-SH-SV02) (gcc version 4.8.3 20131202 (prerelease) (Hisilicon_v300) ) #33 Tue Nov 1 17:49:35 CST 2016

$ cat /proc/cpuinfo
Processor	: ARM926EJ-S rev 5 (v5l)
BogoMIPS	: 269.10
Features	: swp half fastmult edsp java
CPU implementer	: 0x41
CPU architecture: 5TEJ
CPU variant	: 0x0
CPU part	: 0x926
CPU revision	: 5

Hardware	: hi3518ev200

extra hardware specs

Smart Moment

  • tweet AIY Vision
  • implement my way
    • video to frame
    • mobilenet, object recognition
    • motion detect flag

REFs