Unity Connection time out!
simonjeger opened this issue · 5 comments
I'm using flightmare on branch "dev/version_22" (2.5.2022) on Ubuntu 18.04 with Unity 2020.3.36f1.
I can train a policy with "python3 run_control_demo.py --train 1" without any problems.
Now I want to test and visualize it. I press play on unity, type "python3 run_control_demo.py --train 0 --render 1" and the following things happen:
- The python script tries to connect to unity but doesn't succeed, eventually saying "Unity Connection time out!".
- On Unity, the splash screen gets deleted, but other than that everything stays exactly the same (Camera moving around as before). It also throws an error "NullReferenceException: Object reference not set to an instance of an object" (presumably because the splash screen is not there anymore).
I managed to run all of this successfully your master branch, so I think I'm using the right commands.
In the "dev/version_22" branch different type of objects (dynamic and static objects) can be passed to flightmare_unity (see flightmare/flightlib/src/envs/vision_env/vision_env.cpp, line 67-82) and that's not yet implemented on the flightmare_unity "master" branch.
So I propose to use flightmare_unity "dev/agile_flight" branch instead. Now if you don't have any static objects in a .csv file, you'll still get an error. I therefore adapted the following lines to catch that case (not sure if that's a good fix but it worked for me):
In flightmare_unity/Asset/Flightmare/Script/CameraController.cs replace the following (line 706 - 722)
void instantiateObjects()
{
config_object.ReadCSVFile(internal_state, settings.object_csv, ListToVector3(settings.render_offset));
//
// Initialize additional objects
foreach (var obj_state in settings.static_objects)
{
GameObject prefab = Resources.Load(obj_state.prefabID) as GameObject;
// Debug.Log("obj_state id : " + obj_state.ID);
// GameObject obj = internal_state.getGameobject(obj_state.ID, gate_template);
GameObject obj = internal_state.getGameobject(obj_state.ID, prefab);
obj.transform.localScale = ListToVector3(obj_state.size);
Vector3 obj_position = ListToVector3(obj_state.position) + ListToVector3(settings.render_offset);
obj.transform.SetPositionAndRotation(obj_position, ListToQuaternion(obj_state.rotation));
obj.transform.localScale = ListToVector3(obj_state.size);
// obj.layer = 9;
}
with
void instantiateObjects()
{
try
{
config_object.ReadCSVFile(internal_state, settings.object_csv, ListToVector3(settings.render_offset));
//
// Initialize additional objects
foreach (var obj_state in settings.static_objects)
{
GameObject prefab = Resources.Load(obj_state.prefabID) as GameObject;
// Debug.Log("obj_state id : " + obj_state.ID);
// GameObject obj = internal_state.getGameobject(obj_state.ID, gate_template);
GameObject obj = internal_state.getGameobject(obj_state.ID, prefab);
obj.transform.localScale = ListToVector3(obj_state.size);
Vector3 obj_position = ListToVector3(obj_state.position) + ListToVector3(settings.render_offset);
obj.transform.SetPositionAndRotation(obj_position, ListToQuaternion(obj_state.rotation));
obj.transform.localScale = ListToVector3(obj_state.size);
// obj.layer = 9;
}
}
catch (Exception)
{
print("No static objects found");
}
}
Hi
Which directory did you put the "flightmare_unity" repo?
I don't think it matters where you put the flightmare_unity folder. You just need to click play when you run flightmare.
I am also in the dev/version_22 branch to run the RL example.
I have trained the RL model, and wanted to visualise it user --render 1.
Flightmare has this in their documentation:
"
To use unity rendering, you need first download the binary from Releases and extract it into the flightrender folder. To enable unity for visualization, double click the extracted executable file RPG_Flightmare.x84-64 and then test a pre-trained controller
"
I double click the RPG_Flightmare.x84-64 in the directory: flightmare/flightrender/RPG_Flightmare/RPG_Flightmare.x84-64. I get this:
After that, i go to flightpy/flightrl and run python3 run_control_demo.py --train 0 --render 1
It just says unity connection timed out.
Could you please tell me what might be wrong here? You pointed out that it is something to do with flightmare_unity, but where are we even using it?
Thank you
Hey, I cloned flightmare_unity, built everything from scratch from https://github.com/uzh-rpg/flightmare/blob/master/docs/source/building_flightmare_binary/standalone.rst
And then followed steps as outlined by you. I can see the drone on unity now.
Thanks!