Oculus Quest 2 to ROS connection
rbriki opened this issue · 18 comments
Hi everyone,
I am very new to
I'm really sorry to open a new issue even though there is a closed issue that I would like to refer but I can't manage to reopen it.
The problem is the same as for issue #353, I am trying to play the pick and place demo on my Oculus Quest 2. The build goes fine and no errors are prompted but when the app is running ROS does not receive any connection.
I followed the steps explained in issue #353 by Niccolò but as I press Build and Run and put my Oculus Quest 2 on the application is running but nothing happens on the ROS server as no connection is detected. (for how to run the application on the oculus directly from unity I am referring to a you tube video: https://www.youtube.com/watch?v=YwFXQeBmxZ4)
Moreover, I could not see the publish button but I think this is more a design problem since it is my first time developing VR application and I'm not familiar with the process. Anyway, if anyone has a quick solution even to this problem I would be grateful.
I'm running Unity on Windows 10 and ROS on the Docker Engine suggested at Pick and Place tutorial Part 0.
I followed the Tutorial Up to Part 3 Step by Step many times before reopening the issue.
For what regard the VR implementation I followed the guide linked below:
https://learn.unity.com/tutorial/create-a-vr-starter-project-from-scratch#620f8f56edbc2a705d1c3578
which should be the same proposed by Niccolò.
Still, I followed it step by step the same number of times I followed the tutorial of this Repo. Only thing that differs from the unity VR tutorial is that I could not manage to change the Scriptable Render Pipeline Settings as the Pick_and_Place Project does not include the URP as far as I understood.
Last thing is that I also added the Oculus plug-in from the XR Plug-in Management.
OS: Windows 10, for Unity
ROS running on docker as for Part 0 instruction
Packet Manager in Unity: 1) Ros Tpc Connector 2) URDF Importer
Laptop model: Asus Rog GL752VW, that means gtx 960m so no Oculus link possibility in case that matters.
I am writing everything as I am not familiar with none of the plication involved (ROS, Unity, Oculus Quest) and I want to be as complete as possible.
Again, sorry for opening a new issue but me and my project made are running out of time and ideas, if I missed something important for your understanding of the problem or done something wrong let me know let me know.
I am waiting for your help,
thank you in advance
RBR
Ciao,
So let's start simple (which is not because at least my oculus had problems also connecting to a simple wi fi):
-
Did you check that the oculus and the device where ROS run are under the same network? To check it try to ping your oculus from your device. If something returns they are both connected under the same network.
-
the ip that is under the launch file of ROS tcp is the same as the one in unity-ros?
-
this could be strange but it happened to me: are you using the university network? It could be that your oculus "refuses" to connect to it without telling you (in my case because of some keys I needed to download in the oculus to let it connect to the uni wi fi). You will need then to use a access point to connect the oculus to the pc.
If everything above is ok let's try something else.
The main problem is that a successful build does not mean that the Unity packages (ROS message generator and URDF importer) are correctly build for the right platform (in my case android app, but I suppose also for you).
Before trying to actually move the robot with the button I would check that the connection between unity and ROS is actually established. If this happens, it should appear that a device has connected in the terminal where you launched ROS tcp connector. The connection happens once you put on the oculus.
If this does not happens check again that you are correctly building for the correct platform (android) both in the .asdef (or something like this) for ROS tcp generator in unit and you project. You should find this file both in the script folder of you project and under the library folder of you project under CachePackages and the ROS connection package. Open them and leave only Editor and android as target platforms to build on.
It took a week of my life this problem and it was a big pain in the ass to solve. I hope this solves your problems.
Niccolo
Thank you so much for your fast response.
I spent some time today verifying all the stuff you suggested, so:
1-I can ping the oculus, after struggling to find out the IP address.
2- regarding this second point, sincerely, I do not know. Is it the fact that in play mode the application runs for this point to be satisfied? If the fact that in play mode everything works is not a sufficient indicator, if you could please tell me how to check if the launch file of ROS tcp is the same as the one in unity-ros I would do it asap and let you know if it is not the case.
3-Network connection is not the problem in THIS case, but as I will present the project at Polimi and I would have used the university network this is good to know as I can manage to use a different network, maybe the smartphone hotspot (?)
Regarding the .asdef files I was using a version of this repo where the principal .asdef file, the one under scripts folder, refers not to three but to five other .asdef files. I tried changing all of them to "Android" and "Editor". Did not work.
So I managed to obtain an older version of the repo in what i do not think is the best way to do it: downloading the zip. There the referenced scripts by the principal one were just three, chenged them. Did not work.
The terminal were I launched ROS stay silent.
If it could be usefull, what i use to louch ROS is simply:
docker run -it --rm -p 10000:10000 unity-robotics:pick-and-place /bin/bash
roslaunch niryo_moveit part_3.launch
Am I missing something obvious maybe?
below are the screens of what the ping gave as a response and the five .asdef files (Ps: use GUIDs is not ticked, should it be?)
Mate, I am also from politecnico. I can send you a meet and discuss about your problem. If not you can come to my office at DEIB and we can discuss about it
Wow that's great to know. I'm relatively free this week and I am in Milan until saturday (I'll be back next week anyway). Let me know how/when you prefer to meet us and if you need my mail or stuff.
Send me a mail: niccolo.lucci@polimi.it
Hi all,
I have been having the same problem lately with connecting to ROS from my Unity app built onto my Oculus Quest 1. Here are the environment details:
OS: Windows 10 Pro (with Unity 2021.2.7f1)
ROS Distro: ROS Noetic (running on the same Windows PC)
VR Device: Oculus Quest 1
ROS-TCP-Connector version: 0.7.0-Preview (as listed in Unity's package manager)
I am using the ROS-TCP-Connector package (Unity side) and ros-tcp endpoint package (ros side), and have set up a client in Unity to call a service spun by a ros node that is running on my PC. I am able to successfully connect to the service and get the expected response from the ros service while running the app through the editor and wearing the headset. However after building and running the app on my oculus quest, I am no longer able to connect to my ros service.
A per the suggestions above, I have done the following:
- Pinged the Quest headset from my windows machine, and all requests are successful. So both the PC and Quest should be able to see each other
- Only checked "Editor" and "Android" in the ROSTCPConnector.asmdef, MessageGeneration.asmdef and Messages.asmdef files. Tried rebuilding and rerunning but still no luck
I would expect that there should be no difference between connecting the Quest 1 with ROS vs the Quest 2. My post is based off of this tracked issue as well as suggested tips that were provided in #353 and #169.
If there is someone that could help me out, that would be greatly appreciated!
Hi mate,
If I remember correctly also the .asdef of the URDF package should be set to "android" and "Editor" only.
Another stupid mistake could be that you did not set the right ip in ROS CONNECTOR (it should be the one of the machine where ROS run, so your personal computer) or the opposite of the VR (the one inside the launch of the TCP package, I always set it to 0.0.0.0).
Pay attention that the VR connects to you pc only if you wear it!
Other than that I don't know.
Niccolo
Thank you for your prompt reply!
I don't have a URDF package in my Unity project (that might be applicable only to the Pick And Place demo, I'm only using the ROS-TCP-Connector package just to integrate ROS with Unity). Regarding the IP, I have the same IP set under Robotics->ROS Settings and in my launch file for the TCP package which is the IP of the machine running my ros service. This should be the correct setup as I used the same IP when I ran my app through the editor.
Did you manage to find a solution to rbriki's problem? If not I guess I could try setting up the Pick and Place demo but doubt it would help much.
Mmm no I don't think that the pick and place demo will help you.
The IP you set is the one coming out from the command "hostname -I"?.
Are the port the same? Default should be 10000
In unity did you set the correct ROS version in the TCP connector?
Are the Oculus and your pc connected to the same network? (The oculus sometimes seems connected but when you open an internet browser complains about no-connection)
Other than that I honestly don't know. It seems a connection error but if you correctly set up the .asdef files you should be ok.
Did you check the guide I wrote some time ago? It could have some hints to help you
Niccolo
Hi Niccolo,
Sorry for my late response. I ran "ipconfig /all", the equivalent command for windows PC since I am running my ros nodes on a windows machine and that's where I got my IP from. Its the same IP that I am using in the TCP endpoint launch file and that is set under ROS Settings in Unity. My app works with this IP in the editor but I have issues connecting to ROS in the built app. The port is also set to 10000 and the ROS version that I am using is ROS 1 (Noetic). Both my PC and Oculus are connected to the same network as I can ping my Quest from my windows machine. I am also able to browse online in the Quest without issues.
Lastly I did check your guide, those were the first things I tried when I initially ran into this issue.
I am connecting to my home internet via ethernet while my Quest is connected wirelessly but I highly doubt that means anything. Also, I do have other asmdef files in my project (i.e. default ones from the Oculus integration package) but I doubt those need to have their build platforms set to Editor and Android?
Ok then.
This is my last shot before sending you all the screenshots of my config for oculus.
Is the Oculus in developer mode? Apparently you cannot transfer files with an external device if it is not active:
"Developer mode allows Oculus you as a user to unlock more functions of your Oculus headset. For instance, it lets you transfer files between your computer and VR device"
Let me know otherwise I will make screenshot about my VR configuration in Unity.
Niccolo
Hi Niccolo,
Thanks for your patience, I just got back to addressing this issue.
Developer mode is enabled for me, I just disable my computer from being able to access files on my quest device every time it prompts me to do so. I did also try adding a firewall rule on my windows PC for port 10000 and the IP of my oculus quest and still no luck.
Hi Niccolo, do you know if there is anything else I could try to fix this issue with the Oculus Quest?
Thanks for your help in advance!
Hi @Stan850. I am sorry I ran out of ideas...
Niccolò
Hello, I am trying to compile it to android and it appears to have this issue:
Exception: Cannot include plugin 'Packages/com.unity.robotics.urdf-importer/Runtime/UnityMeshImporter/Plugins/AssimpNet/Native/win/x86/assimp.dll' (Key: assimp.dll;None) since plugin with the same name and architecture was already added
It seems that AssimpNet is not able to be ported into Android and was looking for a solution for this. Any ideas?
(I am trying to deploy it in the oculus rather than running it from the laptop)
Hello, I am trying to compile it to android and it appears to have this issue:
Exception: Cannot include plugin 'Packages/com.unity.robotics.urdf-importer/Runtime/UnityMeshImporter/Plugins/AssimpNet/Native/win/x86/assimp.dll' (Key: assimp.dll;None) since plugin with the same name and architecture was already added
It seems that AssimpNet is not able to be ported into Android and was looking for a solution for this. Any ideas? (I am trying to deploy it in the oculus rather than running it from the laptop)
I encountered the exactly same problem also. Here is the whole error message.
Exception: Cannot include plugin 'Packages/com.unity.robotics.urdf-importer/Runtime/UnityMeshImporter/Plugins/AssimpNet/Native/win/x86/assimp.dll' (Key: assimp.dll;None) since plugin with the same name and architecture was already added ('Packages/com.unity.robotics.urdf-importer/Runtime/UnityMeshImporter/Plugins/AssimpNet/Native/win/x86_64/assimp.dll (Architecture: None)').
UnityEditor.Android.AndroidBuildPostprocessor.CollectPlugin (System.String path, Unity.Android.Types.AndroidArchitecture architecture, System.Collections.Generic.Dictionary`2[TKey,TValue] pluginContainer) (at <7cdb938ba9f04cedb4bcf6a470508adf>:0)
UnityEditor.Android.AndroidBuildPostprocessor.PluginsDataFor (UnityEditor.Modules.BuildPostProcessArgs args) (at <7cdb938ba9f04cedb4bcf6a470508adf>:0)
UnityEditor.Modules.BeeBuildPostprocessor+d__38.MoveNext () (at /home/bokken/buildslave/unity/build/Editor/Mono/Modules/BeeBuildPostprocessor.cs:402)
UnityEditor.Android.AndroidBuildPostprocessor+d__21.MoveNext () (at <7cdb938ba9f04cedb4bcf6a470508adf>:0)
UnityEditor.Modules.BeeBuildPostprocessor.SetupBeeDriver (UnityEditor.Modules.BuildPostProcessArgs args) (at /home/bokken/buildslave/unity/build/Editor/Mono/Modules/BeeBuildPostprocessor.cs:413)
UnityEditor.Modules.BeeBuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args) (at /home/bokken/buildslave/unity/build/Editor/Mono/Modules/BeeBuildPostprocessor.cs:544)
Rethrow as BuildFailedException: Exception of type 'UnityEditor.Build.BuildFailedException' was thrown.
UnityEditor.Modules.BeeBuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args) (at /home/bokken/buildslave/unity/build/Editor/Mono/Modules/BeeBuildPostprocessor.cs:605)
UnityEditor.Modules.DefaultBuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args, UnityEditor.BuildProperties& outProperties) (at /home/bokken/buildslave/unity/build/Editor/Mono/Modules/DefaultBuildPostprocessor.cs:28)
UnityEditor.Android.AndroidBuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args, UnityEditor.BuildProperties& outProperties) (at <7cdb938ba9f04cedb4bcf6a470508adf>:0)
UnityEditor.PostprocessBuildPlayer.Postprocess (UnityEditor.BuildTargetGroup targetGroup, UnityEditor.BuildTarget target, System.Int32 subtarget, System.String installPath, System.String companyName, System.String productName, System.Int32 width, System.Int32 height, UnityEditor.BuildOptions options, UnityEditor.RuntimeClassRegistry usedClassRegistry, UnityEditor.Build.Reporting.BuildReport report) (at /home/bokken/buildslave/unity/build/Editor/Mono/BuildPipeline/PostprocessBuildPlayer.cs:370)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr, Boolean&) (at /home/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)
Can anybody help us?
I just got the same error.
Just deleting the folder containing that file worked for me.
Since you are building for Android you SHOULD not need that directory as far as I understood (which is not a lot)
I just got the same error. Just deleting the folder containing that file worked for me. Since you are building for Android you SHOULD not need that directory as far as I understood (which is not a lot)
Beside deleting the containing folder, I also commented out the line below otherwise not found error will pop up.
https://github.com/Unity-Technologies/URDF-Importer/blob/main/com.unity.robotics.urdf-importer/Runtime/UnityMeshImporter/MeshImporter.cs#L25