VR/XR - how to rotate/move in a scene which contains an ArcGIS Map

[coryeicher]

Hi,

I'm looking for some guidance -- or nudge in the right direction -- for how to rotate (and eventually move) in a scene which contains an ArcGIS Map. I have deployed an app to Oculus Quest 2 goggles, but I can only see the map data. When I rotate the goggles the view rotates with me (not expected... since I can't look around in my view). I'm trying to get expected rotation to work as a first step towards interaction/locomotion with an Esri map.

I've followed the Unity and Oculus doc + online examples, but I've been unable to set up an XR Rig in a way which is compatible with an ArcGIS Map and ArcGIS Camera. Per Esri "the Camera game object must have the ArcGIS Camera component and be a child of an ArcGIS Map game object" (link). At a high level it seems like the support the ArcGIS Camera and Location are adding to be able to display map data is _not_ compatible with recommended pattern for supporting camera rotation/position in VR goggles.

How then to set up an XR Rig and configure the arcgis camera to rotate/move the camera in VR goggles?

This seems like a common need for the Esri user community.

Beyond what I summarized above (XR Rig/Camera), I have also configured my scene's camera (including ArcGIS Camera) with a Tracked Pose Driver, but nothing is working. What I see is either of these two states:

A) "native" Unity objects + camera rotating as expected... but no map data visible

B) ArcGIS Map data visible... but with a) no support for camera rotation and b) no support for camera movement (not really expecting this yet given what I have tried and how the Esri camera-controller is written)

What is the recommended path forward from here? Is there a recommended set-up for an XR Rig which works with an ArcGIS Camera? Or, should I be starting the process of writing my own camera controller without any XR Rig (i.e. replacing this https://developers.arcgis.com/unity/maps/camera/#camera-controller)?

Thanks,

=Cory

[coryeicher]

I was able to get most of this working by dragging the "ArcGIS Map" underneath my "XR Origin". With this setup I can:

• look around in my scene by turning my head
• "snap rotate" the scene using controller joystick
• teleport in the scene using controller button

Continuous movement is still _not_ working as expected.

FWIW there's another recent post which suggested that dragging the ArcGIS Camera under the XR Rig worked (for Steam VR/Hive). 

And so... I'm not sure I have exactly what I need, but it is doing the trick for now.

Worth mentioning: There are a couple constraints with the Esri components which narrowed my possibilities (I tired breaking these anyway... just to see). I am not sure how "hard" these constraints are. They are listed in the doc, and I see them in the source, but they may only apply to using the Map SDK UI in Unity. It would be nice to have this confirmed.

• ArcGIS Camera must be parented to ArcGIS Map.
• ArcGIS Location must be parented to ArcGIS Map. 

-Cory

[Matt_Nelson]

Hi Cory, I have been saying this in other questions but first off sorry for the delay we got busy with our Unreal Engine release and the User conference last week. 

We are going to be adding a sample for this very soon so hopefully you can just copy and paste that when it is out. 

In the meantime a couple things stuck out to me in your question.
1. You cannot have the ArcGIS Camera Controller component in your scene. That component is a sample component that is setup to handle mouse/keyboard interactions to move around. You need to be leveraging the locomotion of the XR rig. 
2. The ArcGIS Camera component should be on the same GO that is receiving your updates from the HMD. In other words as you move your head you will see that the rotation values update. If the camera component is on this you will get the rotations correctly passed to our system to load data.

[GordonCooke]

@coryeicher : I am working through same issues also.  @Matt_Nelson : Samples for VR would be VERY appreciated!

I started with a XR rig, then brought in the ArcGIS package and added a local map. The XR Rig needs to be a child of the ArcGISMap object. (If I understand correctly, the map has coordinates that are geo referenced- ie 0,0,0 is Null Island out i the ocean, so if the XR Rig is at 0,0,0 then it is other side of the planet from the map- thousands of kms away)

In the open XR Rig- I put the ArcGIS camera where the usual camera goes (under the camera offset) and deleted the normal Main Camera object. So you have the Arc GIS Camera, Right and Left controllers. 

I also had to change the ArcGIS camera near clipping plane in order to see the controllers. The default is 0.5 meters so the controllers are close enough to be clipped unless you stretch out your arm.) But I had to turn off (uncheck) Update Clipping planes so it didnt reset to default of 0.5. I dont know if there is a way to change the clipping planes (there must be) but turning the update off will be a problem if you have an use case that changes altitude (like ground to orbit) but should be ok if you have a relatively fixed altitude (walking on ground, flying at an elevation)

(I used the XR Interaction toolkit) Snap turns work. I didnt try teleport. Continuous move provider (action based) works- but the speed needs to be enough that you would notice it (100m off the ground and a speed of 1 moves the ground so slow you barely notice it, but 10 was much better.)

Currently working through the issue of ground clamping to the terrain. I can manually set an elevation and get a nice view, but moving isnt following the ground for altitude changes. I suspect its something with turning on mesh collider and doing some kind of ground clamping or maybe a collider on the XR rig and use physics(?).  Open to suggestions for best way to do that with ArcGIS so you can 'walk' on the terrain. 

EDIT: Added a Character Controller to the XR Origin rig. Turn on Mesh Colliders Enabled in the ArcGIS Map. Turn on Use Gravity in the Locomotion System>Continuous Move Provider. Now, you can move around on the ground using the thumbstick (I have it set to left). Remaining problem: how to know the initial height so you start out above the ground. (Falling in every time probably not a good solution). Raycast, script to  pull the data from the elevation-- lots of options.

[coryeicher]

Hi Gordon,

Thanks for posting this. Your hierarchy aligns with what Esri recommended to me via support...

... then set up the ArcGIS Camera component per the doc.

I have not tried this yet, but setting a flag to come back. Would love to have the joystick locomotion you wrote about above. I have this in a non-ArcGIS app working great on Oculus.

Thanks again,

-Cory

[coryeicher · Accepted solution]

I was able to get this working on my Oculus Quest 2 goggles -- including joystick continuous movement -- by following essentially what @GordonCooke posted above. This matched what Esri provided through the support channel. Pasting a snip of my hierarchy here...

... Nicely, I did not lose anything I previously had working in my scene, namely snap turn and teleport. 

Future-wise, I agree that the community will benefit from some XR examples and also some conceptual doc  on building Unity + ArcGIS Apps for XR devices. Also, this statement from the current doc doesn't exactly seem correct given the working hierarchy: "ArcGIS Camera must be parented to ArcGIS Map."

Happy to have this working, grateful for the assistance here and via the support channel.

-Cory

[Matt_Nelson]

Thank you for the feedback Cory and glad to see you got it working.

[cheeze_wizz]

@Matt_Nelson 

Have you guys had a chance to throw together a sample yet? I'm trying to implement this behavior with the Oculus Quest OVRCameraRig (instead of the OpenXR rig) but not having any success.

@GordonCooke @coryeicher Did you guys try implementing this with Oculus OVRCameraRig in addition to OpenXR rig? I'm guessing you are going with the OpenXR rig for more generic compatibility across VR devices. Does the OpenXR rig work well for the Quest 2?

[coryeicher]

Hi @cheeze_wizz 

I'm using the OpenXR plugin. I did try some things with the Oculus plugin/OVRCameraRig, but never got that working. 

At one point I was forwarded this link on the Early Adopters forum where @JadeHuang recommended "Try making OVRcamera rig a child object of ArcGISCamera".

To answer your last question: yes I am using Oculus Quest 2 goggles as my target device, and that is where I got this all working.

Hope this helps,

-Cory

[LDD_NW]

Hi Cory,

Your post and the responses have been super helpful! I just have a question, were you able to get the view to stop rotating as you move the goggles? I have been able to get continuous move with joysticks and snap turn to work but the view is still tracking with the headset as I turn. In your original post you mentioned that this was occurring.

Thanks,

Nick