Looking at the sample scenes, it seems like the MRTK uses a pointer system instead of actually giving you virtual hands that can be manipulated by pressing buttons or triggers on the controllers, and allowing you to grab objects.
Is there a built-in way to use hands?
There is currently not a set of virtual hands build into MRTK for VR, however it would be possible to rig up a custom visual for a controller that looks like hands, and animate the hands based on things like which buttons / triggers are pressed on the controllers.
Oculus provide hand models which are rigged in their SDK, which you can appropriate for use in the MRTK as shown in the video below.
https://www.youtube.com/watch?v=F3e2lwqVPyc&list=PLCK8aOPy3e4JhG06GRdlJkmNXoITEewxV&index=4
You'd just need to do a little more work to get the hand rigging hooked up to the MRTK hands.
Related
I'm building an application that will run on both HoloLens and mobile devices (iOS/Android). I'd like to be able to use the same manipulation handlers on all devices with the goals:
Use ARFoundation for mobile device tracking and input
Use touch input with MRTK with ManipulationHandler and otherwise use touch input as normal (UI)
Simulate touch input in the editor (using a touch screen or mouse) but retain the keyboard/mouse controller for camera positioning.
So far I've tried/found:
MixedRealityPlayspace always parents the camera, so I added the ARSessionOrigin to that component, and all the default AR components to the camera (ARCameraManager, TrackedPoseDriver, ARRayCastManager, etc.)
Customizing the MRTK pointer profile to only countain MousePointer and TouchPointer.
Removing superfluous input data providers.
Disabling Hand Simulation in the InputSimulationService
Generally speaking, the method of adding the ARSessionOrigin to the MixedRealityPlayspace works as expected and ARFoundation is trivial to set up. However, I am struggling to understand how to get the ManipulationHandler to respond to touch input.
I've run into the following issues:
Dragging on a touch screen with a finger moves the camera (editor). Disabling the InputSimulationService fixes this, but then I'm unable to move the camera...
Even with the camera disabled, clicking and dragging does not affect the ManipulationHandler.
The debug rays are drawn in the correct direction, but the default touchpointer rays draw in strange positions.
I've attached a .gif explaining this. This is using touch input in the editor. The same effect is observed running on device (Android).
This also applies to Unity UI (world space canvas) whereby clicking on a UI element does not trigger (on device or in editor), which suggests to me that this is a pointer issue not a handler issue.
I would appreciate some advice on how to correctly configure the touch input and mouse input both in editor and on device, with the goal being a raycast from the screen point using the projection matrix to create the pointer, and use two-finger touch in the same way that two hand rays are used.
Interacting with Unity UI in world space on a mobile phone is supposed to work in MRTK, but there are few bugs in the input system preventing it from working. The issue is tracked here: https://github.com/microsoft/MixedRealityToolkit-Unity/issues/5390.
The fix has not been checked in, but you can apply a workaround for now (thanks largely to the work you yourself did, newske!). The workaround is posted in the issue. Please see https://gist.github.com/julenka/ccb662c2cf2655627c95ffc708cf5a69. Just replace each file in MRTK with the version in the gist.
I am working on windows 10 mobile app. If I design an UI for 5' device, it doesn't look good on 5.2' device. Also on emulators of different screen size show messed up UI. Is there any work around this? Or do I have to design it for every screen size?
Usually, please avoid to have fixed Width/Height for your controls. With a Grid layout, you can set columns/rows and place your controls inside each cells. This is a first level to adapt properly your interface. In addition, UWP provides AdaptiveTriggers if you want to adapt the layout based on the screen resolution (cf. https://channel9.msdn.com/Series/A-Developers-Guide-to-Windows-10/07) for additional information. Last but not least, please check Design&UI documention on https://developer.microsoft.com/en-us/windows/design.
The trick is with UWP that you can design for every screensize at once. It's just a responsive design you use (either HTML/CSS or XAML). But you have to use the right controls and settings.
Some guidance on how to design for various sizes can be found at the MSDN design page.
To create a dynamic layout with XAML, see this article. There are various panel-types you can use to do the layout (see this article). But if you really want to build a responsive UI (or change it dramatically in various sizes) RelativePanel is your friend.
Is it possible to implement a custom Transition in WinRT?
E.g. it would be nice to have transitions for the control visibility. So when you show/hide a part of a split view it animates the entire view by using a sliding effect.
Yes you can, using the usual WPF animation techniques, only with minor changes to WinRT.
The usual pattern is to apply double/color/easing animations to your UI elements via storyboarding - being triggered by changes in the visual state manager. You can do this either declaratively or via code-behind.
http://www.codeproject.com/Articles/23257/Beginner-s-WPF-Animation-Tutorial
http://blogs.msdn.com/b/wpfsdk/archive/2009/02/27/the-visualstatemanager-and-triggers.aspx
As far as I know you can't build your custom transitions and use them like normal WinRT Transitions, that is, inside a TransitionCollection.
<ListView.Transitions>
<TransitionCollection>
<myTransitions:PotatoeTransition/>
</TransitionCollection>
</ListView.Transitions>
You can't do the above as far as I know. (ignore the fact that I exemplified with a ListView, it applies to everything, I think)
You'll probably have to use a Storyboard that animates both the RenderTransform (TranslateTransform) and the Opacity to achieve your objective.
I think you can still create a Behavior though if you want to make it more reusable.
I need do a very special nav or tab bar for a app, I looked this but I not have idea how to make some similar, I work with Titanium mobile, someone with experience could tell me how to make some similar with Titanium, or if someone have other navigation bar more complex that the examples on the kitchensink.
This is where I'd start. Understand that that video is obviously a complex interface and will definitely take work on your part.
This (http://softlywired.com/ticustommenu/) piece of code will get you started, but you will need to go way beyond it writing that interface control. I'd then move on to creating the boxes that pop up, without animation, down the center. After that was working, I'd look at animating them and adding the event listeners to it.
You'd be a star in the Titanium community if you got it working and released the source code to it.
I'm creating a custom camera UI using UIImagePickerController cameraOverlayView property. However, I'd like to keep some of the existing UI -- specifically the flash and camera selection buttons at the top.
My thought was that I'd keep showsCameraControls = YES set the cameraOverlayView to my custom UI and then use [picker.view bringSubviewToFront:overlayView] to make sure my controls on the bottom are sitting on top of the default controls.
Alas, this does not work. I've tried moving the overlay to the front of the view hierarchy in various places without luck. Is there any UIImagePickerController hackery that could achive this? It seems a shame not to be able to reuse at least some of the camera controls while still customizing the UI.
Further investigation it appears this is not possible. You either have to use all of the default cameara UI or none of it.