This library is meant to be a lightweight bridge between the VEX V5 or Cortex systems and the Jetson (Orin) Nano. Note that different versions of Jetson and VEX systems have different library support and behaviors, and unfortunately that means that this guide will be quite involved. However, once you are able to set up the bridge, creating your robot code should be straightforward.
Follow the Jetson Orin Nano guide or the Jetson Nano Guide. Remember to set "log in automatically" instead of "require my password to log in". Set power usage to 15W (Jetson Orin Nano) or MAXN (Jetson Nano). Expand storage to max size as shown (ie. 59300).
Once you are logged in, open a Terminal, and type in the update cmds below. Try to keep all default (N) options as it prompts you for a choice. If you pick (Y), don't worry - as long as it boots you are good to go. You don't have to restart docker daemon.
sudo apt-get update && sudo apt-get upgrade
sudo reboot
After reboot, open a new Terminal and type of the following:
sudo apt-get install python3-pip
sudo pip3 install --upgrade pip
sudo pip3 install pyserial numpy scipy websockets requests
# only for Jetson Orin Nano, as the older Jetson Nano fails:
sudo pip3 install opencv-python pyrealsense2 qoi
Clone this repo to the Jetson, navigate to the jetson_nano/
folder and run the installer:
git clone https://github.com/timrobot/CortexNanoBridge.git
cd CortexNanoBridge/jetson_nano
sudo bash ./install.sh
sudo reboot
On Ubuntu 22.04 (the operating system that Jetpack6 uses), CH340 drivers do not work all that well. So, a couple of steps may need to be followed to get it to work:
sudo apt-get uninstall brltty
git clone https://github.com/juliagoda/CH341SER
cd CH341SER
sudo make clean
sudo make
sudo make load
Pyrealsense2 does not exist on python3.6, and neither does qoi. So, after downloading and unzipping the pyrealsense library from this package, install both:
sudo apt-get install python3-opencv
cd /path/to/pyrealsense
sudo bash ./install.sh
cd /path/to/CortexNanoBridge/jetson_nano
sudo cp -r qoi /usr/local/lib/python3.6/dist-packages
In addition to installing the library on the Jetson Nano, we will need to push comms code to the robot controller. On your laptop or desktop, clone this repo so that we can upload the comms code. You will only have to do this once.
- Install the VEX Extension for VSCode. Connect a microusb cable to the V5 Brain, and update firmware.
- Create a new VEX Project > V5 > C/C++ > Clawbot Template (Motors). Copy
CortexNanoBridge/vex_v5/src/main.cpp
to the new project'ssrc/
folder. Build and download the code to the V5 Brain, and disconnect the microusb cable. - Plug in the RS485 cables on ports 18 and 20, and plug in their USBs into the Jetson. Run the application from the V5 Brain screen.
- Install the RobotC GUI. Connect a usb cable to the Cortex, and update firmware. Remember to select (USB only) mode from the communication style menu.
- Open
CortexNanoBridge/vex_cortex/main_app.c
. Compile and download the code to the Cortex, and disconnect the usb cable. - Wire the UART Cable to the UART1 pins on the Cortex, and plug in the USB into the Jetson.
Congratulations, you have finished setting everything up for the bridge! 👏
You have two options to control your robot. If you have the Jetson Nano and a beefy desktop or laptop GPU, it is highly recommended that you do option 2.
Remember to plug in your Realsense camera into the robot if you wish to get the color and depth frames.
from cortano import VexV5, RealsenseCamera
if __name__ == "__main__":
realsense = RealsenseCamera()
robot = VexV5()
while robot.running():
color, depth = realsense.read()
sensors, battery = robot.sensors()
robot.motor[0] = 0 # you can set this to any value from -100 to 100
robot.motor[9] = 0
Num | Action | Control Min | Control Max | Unit |
---|---|---|---|---|
0 | Angular velocity target of the left motor | -100 | 100 | percent |
1 | ❌ | |||
2 | Angular velocity target of the claw | -100 | 100 | percent |
3 | ❌ | |||
4 | ❌ | |||
5 | ❌ | |||
6 | ❌ | |||
7 | Angular velocity target of the arm | -100 | 100 | percent |
8 | ❌ | |||
9 | Angular velocity target of the right motor | -100 | 100 | percent |
Num | Action | Min | Max | Unit |
---|---|---|---|---|
0 | Left Motor ang position | -inf | inf | position (degrees) |
1 | Left Motor ang velocity | -inf | inf | velocity (degrees/second) |
2 | Left Motor torque | -inf | inf | Nm * 1e3 |
3 | Right Motor ang position | -inf | inf | position (degrees) |
4 | Right Motor ang velocity | -inf | inf | velocity (degrees/se5ond) |
5 | Right Motor torque | -inf | inf | Nm * 1e3 |
6 | Arm ang position | -inf | inf | position (degrees) |
7 | Arm ang velocity | -inf | inf | velocity (degrees/second) |
8 | Arm torque | -inf | inf | Nm * 1e3 |
9 | Claw ang position | -inf | inf | position (degrees) |
10 | Claw ang velocity | -inf | inf | velocity (degrees/second) |
11 | Claw torque | -inf | inf | Nm * 1e3 |
On the Jetson, run either of the following to enable to automagical websocket communication stack.
# Vex V5 Brain
cd CortexNanoBridge/jetson_nano/scripts
bash ./enable-v5-autostart.sh
# Vex Cortex
cd CortexNanoBridge/jetson_nano/scripts
bash ./enable-cortex-autostart.sh
Then once you have obtained the ip address for the Jetson, proceed with installing the remote API onto your laptop or desktop to connect. The service should be running every time the robot powers on.