Chain of models

Use multiple ACT models to solve more complex robotics tasks.

For example, in the lamp testing demo, we combined 3 models:

For getting the lamp from a random position
For precise insertion into the tester
For sorting working/not working bulbs

Intro

Intro_SA_web.mp4

Hardware

Follow SO-100 to build your arm.

Buy assembled in US:

There are additional costs of assembling

BOM price: $241
Shipping: $30-$80 depending on speed (5-20 days)
Printing: 30-50 hours, $10 plastic
Assembly and calibration: 1-2 days

We can offer properly assembled and calibrated robot in the US for $400

DM me if interested:

https://x.com/ihorbeaver

Installation

If you didn't install Lerobot, install it:

git clone -b user/rcadene/2024_09_04_feetech https://github.com/huggingface/lerobot.git
cd lerobot
pip install -e .

2 Clone Simple Automation scripts to another folder

git clone https://github.com/1g0rrr/SimpleAutomation.git
cd SimpleAutomation

3 Setup ports for your robot in "core/configs/robot/so100.yaml".

Run

Run evaluation

Change config file for using your models "core/configs/chains/lamp_testing.yaml"
While evaluating press "right" key to move to the next model

python core/models_chain.py evaluate \
  --robot-path core/configs/robot/so100.yaml \
  --chain-path core/configs/chains/lamp_testing.yaml

Run recording

The difference from Lerobot's recording is added teleoperation between episodes. This is usefull to be able to switch between models in not "resting" position.

python core/models_chain.py record \
  --robot-path core/configs/robot/so100.yaml \
  --fps 30 \
  --root data \
  --repo-id 1g0rrr/koch_test21 \
  --tags tutorial \
  --warmup-time-s 5 \
  --episode-time-s 5 \
  --reset-time-s 5 \
  --num-episodes 2

Run teleoperation:

Use it for testing if all is working.

python core/models_chain.py teleoperate \
 --robot-path core/configs/robot/so100.yaml \
 --robot-overrides '~cameras'

Tips

Make sure you have all inintial positions in the following model to prevent robot from sudden movements.
"Pick and place" task is hard for the model and gripper can grab object not precisely at the center. To solve this re-grab object at the beginning of next model.

Training

Train model in Google Colab:

You can model in Google Colab to save time. https://colab.research.google.com/github/1g0rrr/SimpleAutomation/blob/main/colab/SimpleAutomationTrainModel.ipynb

It will take about 2.5 hours and $1.5 to train typical 80K steps.
Choose A100 as fastest GPU.
Don't disconnect colab and don't close browser as all data will be deleted.