Vehicle Path Tracking Extension

1. About

Omniverse Vehicle Path tracking extension allows a physics-enabled vehicle created with a PhysX Vehicle extension (omni.physx.vehicle) to move and automatically track a user-defined path. User-defined path is represented by an instance of USD BasisCurves, and a path tracking algorithm is inspired by a classic Pure Pursuit algorithm [3].

Figure 1. Preview of Vehicle Path Tracking Extension

System Requirements:

Code 2022.1.3+ or Create 2022.1.5+ (support for Create 2022.3.0 is in progress)
Pyhton 3.7+, numpy (this requirement should be satisfied when using Omniverse Kit's embedded CPython 3.7)

Limitations

For the moment, the extension is simple and a number of shortcuts have been taken and a few simplifications applied, including the following:

Pure Pursuit Tracking algorithm is kinematics-based and therefore several physics vehicle dynamics properties are not considered while computing wheel steering angle, such as tire slipping etc.
A vehicle might go off the track if proposed an input path of a physically "impossible" trajectory, or at high-speed turn.
Limited unit test coverage; occasional bugs might exist.

Future Work

Implement automatic computation of vehicle path which satisfies certain constraints (waypoints, collision free path etc.).
Add support for different vehicle controller algorithms, including more sophisticated ones (e.g., PID controller).
Getting rid of limitations, bugfix.

2. Installing Extension

Add a path to a local clone to Omniverse extension search path

git clone -b main $PATH_TO_DIR
Window -> Extension Manager -> ⚙️ Gear Icon -> Extension Search Path
Add a path to just cloned extension as an extension search path: $PATH_TO_DIR/exts

Omniverse Community Tab

Extension is also available in the community tab in the Extension Manager: just search for path.tracking in the search field.

Activate extension

When extension search path configuration is done, start the extension:

Window -> Extension Manager
Find Vehicle path tracking extension in the list and enable it (Figure 2)

Figure 2. Activating path tracking extension in extension manager.

3. Getting Started

3.1. Evaluate vehicle path tracking on a preset configuration

The fastest way to evaluate how vehicle path tracking extension works is to use a preset vehicle and curve (could be considered as HelloWorld before importing your own physx-vehicle and custom paths). To get started with the preset configuration please proceed as follows (Figure 3):

Click Load a preset scene button
Click Start scenario button

Figure 3. Getting started with a preset scene.

The extension also allows a quick way to load a ground plane, a sample physics vehicle, and a sample basis curve. See Figure 4.

Figure 4. Other extension controls.

3.2. Create your custom vehicle-to-curve attachment setup

Extension supports path tracking for any Omniverse PhysX Vehicle. One could load a template vehicle using the extension ui or using a conventional method via Create->Physics->Vehicle. It is also straightforward to add a custom mesh and materials to a physics vehicle [2].

You can create a curve for vehicle path tracking using either of the following methods (Figure 5):

Create->BasisCurves->From Bezier
Create->BasisCurves->From Pencil

|
Figure 5. Create a custom path to track via USD BasisCurves.

Once a physics vehicle and a path to be tracked defined by USD BasisCurves is created, select the WizardVehicle and the BasisCruves prims in the stage (via Ctrl-click) and click Attach Selected button. Note that is very important to select specifically WizardVehicle prim in the scene, not WizardVehicle/Vehicle for instance. Please see Figure 6 for the illustration.

Figure 6. Attachment of a path (USD BasisCurves) to a physics-enabled vehicle.

In case if vehicle-to-curve attachment was successful it should be reflected on the extension UI (Figure 7).

Figure 7. Successful vehicle-to-curve attachment is shown on the right side.

When vehicle-to-curve attachment(s) is created, proceed by clicking Start Scenario button.

If you want to get rid of all already existing vehicle-to-curve attachments, please click Clear All Attachments (Figure 8). It is very important to clear vehicle-to-curve attachments, when changing vehicles and corresponding tracked paths.

Figure 8. Removing existing vehicle-to-curve attachments.

3.3. Multiple Vehicles

The extension supports multiple vehicle-to-curve attachments. Note, that for attachment to work, a pair of WizardVehicle and BasisCurve objects should be selected and attached consequently. Results of path tracking with multiple vehicles is shown in Figure 9.

Figure 9. Support of multiple vehicle-to-curve attachments.

Troubleshooting

Note that extension is in Beta. The following items might be of help if any issues:

It always takes a few seconds between clicking 'Start scenario' button and actual start of the simulation, so please be patient.
On a fresh install some physx warnings/errors might be occasionally reported to the console log, they should not prevent the extension from producing expected results though.
If path tracking is not working on a custom vehicle and path, please verify that exactly WizardVehicle1 from omni.physx.vehicle is selected (not a child prim 'WizardVehicle1/Vehicle' or some parent prim) along with a prim of type BasisCurves (which is to be tracked) before clicking 'Attach Selected’.
Use 'Clear All Attachments` if there are some issues.

4. Results

[youtube video] Vehicle Path Tracking Extension Overview
[youtube video] Vehicle Dynamics and Vehicle Path Tracking: Forklift Usecase
[youtube video] OmniPhysX & Vehicle Dynamics Showcase

5. References

[Omniverse Developer Contest] https://www.nvidia.com/en-us/omniverse/apps/code/developer-contest/
[Omniverse Vehicle Dynamics] https://docs.omniverse.nvidia.com/app_create/prod_extensions/ext_vehicle-dynamics.html
[Coutler 1992] Coulter, R. Craig. Implementation of the pure pursuit path tracking algorithm. Carnegie-Mellon UNIV Pittsburgh PA Robotics INST, 1992. (https://www.ri.cmu.edu/pub_files/pub3/coulter_r_craig_1992_1/coulter_r_craig_1992_1.pdf)
Credits for a forklift model model: https://sketchfab.com/3d-models/forklift-73d21c990e634589b0c130777751be28 (license: Creative Commons Attribution)
Credits for a Dodge Challenger car model: https://sketchfab.com/3d-models/dodge-challenger-ef40662c84eb4beb85acdfce5ac4f40e (license: Creative Commons Attribution NonCommercial)
Credits for a monster truck (used in the result video): https://sketchfab.com/3d-models/hcr2-monster-truck-811bd567566b497a8cbbb06fd5a267b6 (license: Creative Commons Attribution)
Credits for a race track model (used in the result video): https://sketchfab.com/3d-models/track-5f5e9454fd59436e8d0dd38df9ec83c4 (license: Creative Commons Attribution NonCommercial)