Integrate ARCore Geospatial API into AR Toolkit
kamszyc opened this issue · 2 comments
On I/O 2022 Google introduced ARCore Geospatial API
https://developers.googleblog.com/2022/05/Make-the-world-your-canvas-ARCore-Geospatial-API.html
https://developers.google.com/ar/develop/geospatial
The ARCore Geospatial API utilizes data from Google Earth 3D models and Street View image data from Google Maps to enable your app for immersive, global-scale, location-based augmented reality experiences.
Integrating with it in AR Toolkit should improve ARSceneView positioning precision in urbanized areas.
However, it relies on availability of Street View data, so it should probably be an optional feature.
What do you think, is it a good idea to integrate it? Do you intend to do it?
I made some progress on a prototype. I have it in a branch: https://github.com/Esri/arcgis-toolkit-dotnet/tree/ncastle/geospatial-ar. In my limited testing, performance was very good under ideal conditions (outdoors, urban center, full daylight, strong cellular connection).
If you want to try it out, you would need to supply an API key (per instructions at https://developers.google.com/ar/develop/java/geospatial/enable#api_key_authorization) and an ARCore package for 1.33.0. There is a PR open for 1.31.0 version of Xamarin ARCore bindings, and I suspect building with that version would also work. xamarin/XamarinComponents#1377
At least according to the documentation, the Geospatial API depends on Google Play location services. I'm referencing an older version (118.0) of that package because of dependency conflicts with Xamarin.Forms and the newer Play location services packages.
This comment is not a promise or commitment to move forward with including this in the product. There are currently no plans to release ARCore Geospatial API support as part of Toolkit.
I've pushed an update to the branch that gets closer to a reasonable architecture and fixes an issue with double counting translational movement.
Currently the big remaining issue is altitude adjustment. Even with using the geoidal separation from the GPS to offset the altitude from Earth, it still consistently places me underground. There are some hacks to get reasonably good behavior, but I'd prefer to find a way to get absolute vertical positioning working properly.