topic Re: Augmented Reality: OrientationSensor to camara's transformation matrix in .NET Maps SDK Questions

Augmented Reality: OrientationSensor to camara's transformation matrix

santiagomonedero — Thu, 02 Apr 2020 20:05:05 GMT

I am trying to update the SceneView Camera using Androids OrientationSensor in Xamarin to do an AR app with no success. I get the quaternions (x,y,z,w) from the sensor and use them to create the transformation matrix but the resulting orientation does not correspond to the mobile's orientation. I have tried different combination of axes (changing order and sign in the quaternion components) assuming it was a problem with the axes definition but still with no success.

Anybody knows how to use Android's OrientationSensor to create the correct transformation matrix?

Thanks

PD: I know I can use the magnetomer and accelerometer to obtain the AR effect but prefer to use the OrientationSensor as I think it will lead to a better solution.

Re: Augmented Reality: OrientationSensor to camara's transformation matrix

NathanCastle1 — Thu, 02 Apr 2020 21:11:45 GMT

Hi,

Have you considered using AR Toolkit? It will handle control of the scene view camera using the device's sensors and AR Core. .NET Toolkit

If you need to support devices that aren't capable of using AR Core, you should consider using the AR Toolkit as a reference. AR Toolkit is open source and includes a lot of code you may be able to repurpose.

Please let me know if you have any questions.

Thank you,

Nathan C.

Re: Augmented Reality: OrientationSensor to camara's transformation matrix

dotMorten_esri — Thu, 02 Apr 2020 22:06:35 GMT

Specifically you're probably looking for this:
arcgis-toolkit-dotnet/ARSceneView.Android.cs at master · Esri/arcgis-toolkit-dotnet · GitHub

Re: Augmented Reality: OrientationSensor to camara's transformation matrix

santiagomonedero — Fri, 03 Apr 2020 01:45:29 GMT

Thank you both.

Yes I am also considering using the ARKit but wanted to have a simple solution using the orientation sensor. I was doing basically the same as your code. Anyway I just found out the problem so I write it here in case anybody is facing the same issue. To update the camera viewpoint do not set the rotation and translation at the same time, but rather 1) Use quaternions with NO translation to create the transformation matrix and a new rotated camera 2) translate the resulting camera and update viewpoint

The thing is that I assumed that translation would not affect the quaternions but apparently it does as you can see executing the snippet in the attached image.

Thanks again.