I'm building an application that will run on both HoloLens and mobile devices (iOS/Android). I'd like to be able to use the same manipulation handlers on all devices with the goals:
Use ARFoundation for mobile device tracking and input
Use touch input with MRTK with ManipulationHandler and otherwise use touch input as normal (UI)
Simulate touch input in the editor (using a touch screen or mouse) but retain the keyboard/mouse controller for camera positioning.
So far I've tried/found:
MixedRealityPlayspace always parents the camera, so I added the ARSessionOrigin to that component, and all the default AR components to the camera (ARCameraManager, TrackedPoseDriver, ARRayCastManager, etc.)
Customizing the MRTK pointer profile to only countain MousePointer and TouchPointer.
Removing superfluous input data providers.
Disabling Hand Simulation in the InputSimulationService
Generally speaking, the method of adding the ARSessionOrigin to the MixedRealityPlayspace works as expected and ARFoundation is trivial to set up. However, I am struggling to understand how to get the ManipulationHandler to respond to touch input.
I've run into the following issues:
Dragging on a touch screen with a finger moves the camera (editor). Disabling the InputSimulationService fixes this, but then I'm unable to move the camera...
Even with the camera disabled, clicking and dragging does not affect the ManipulationHandler.
The debug rays are drawn in the correct direction, but the default touchpointer rays draw in strange positions.
I've attached a .gif explaining this. This is using touch input in the editor. The same effect is observed running on device (Android).
This also applies to Unity UI (world space canvas) whereby clicking on a UI element does not trigger (on device or in editor), which suggests to me that this is a pointer issue not a handler issue.
I would appreciate some advice on how to correctly configure the touch input and mouse input both in editor and on device, with the goal being a raycast from the screen point using the projection matrix to create the pointer, and use two-finger touch in the same way that two hand rays are used.
Interacting with Unity UI in world space on a mobile phone is supposed to work in MRTK, but there are few bugs in the input system preventing it from working. The issue is tracked here: https://github.com/microsoft/MixedRealityToolkit-Unity/issues/5390.
The fix has not been checked in, but you can apply a workaround for now (thanks largely to the work you yourself did, newske!). The workaround is posted in the issue. Please see https://gist.github.com/julenka/ccb662c2cf2655627c95ffc708cf5a69. Just replace each file in MRTK with the version in the gist.
Related
OK... here is the thing...
I am modeling a Star Wars Cruiser.. I'm doing pretty fine, but.. whenever I look through the camera, half of my ship disappears ?! Now you might think it's the clipping, but that doesn't make sense, since the clipping is set to 0,1 mtr - 10.000 mtr, which should be more than enough. Besides that, the clipping is the same for both the visible and now disappeared objects...
Does anyone have any clue ?
I also tried to change the Local view to Global view.. but no dice...
Here is a screenshot in not-camera view :
And here is one in camera view : i.imgur.com/XaxAHIN.png
I'm using Blender 2.82
Please let me know if you need more info or even a link to the model...
OK, I got some help from a friend... Joshua Rooijakkers from the Netherlands...
He fixed the problem in seconds...
So ... it WAS the clipping.. the setting was just buried somewhere I didn't look...
Thanks Josh ! :-)
Objects disappear in camera view modeWhile setting up my camera with the camera lock to view option selected, I noticed that on switching on to the camera view, the object had disappeared.
I tried zooming in and out, but that did not work. I exited the camera view and then selected the object in the outliner and then tried zooming in and out and then panning down and adjusting the view in the view port. I could see that the object was now visible and that the camera was positioned below the object. That was because the camera was added wrt to the position of the cursor which was snapped to the world origin. I a sure many would have encountered such issues.
I then selected the camera to ascertain its position. Then by using the grab tool, I moved the camera along the Z axis just above the object and then switched to the camera view and selected the lock camera to
view option to adjust the camera view to the desired position.
This helped me locate my object in the camera view. Do let me know in the comments if there are other ways of dealing with this.
I would like to draw some sort of window on top of all the other windows. For example, to display some debugging infos (like conky) or things like a timer.
The main thing is that I would like to able to continue using the other windows while using it (the events go through transparently).
I've tried doing it with pygtk, pyqt and others but can't find a way to make it a real overlay with no event capture.
Is there some low-level x11 solution?
I think the Composite-extension-approach will not work when a compositing manager is running (and thus Composite's overlay window is already used).
Since you explicitly mention "no event capture":
The SHAPE extension allows to set some different shapes for a window. Version 1.1 of this extension added the "input" shape. Just setting this to an empty region should pretty much do what you want.
Some concrete example of exactly what I think you ask for can be found in Conky's source code: http://sources.debian.net/src/conky/1.10.3-1/src/x11.cc/?hl=769#L764-L781
Edit: Since you said that you didn't find anything in Gtk (well, PyGtk), here is the function that you need in Gtk: https://developer.gnome.org/gdk3/stable/gdk3-Windows.html#gdk-window-input-shape-combine-region
You might need Composite extension + GetOverlayWindow request:
Version 0.3 of the protocol adds the Composite Overlay Window, which
provides compositing managers with a surface on which to draw without
interference. This window is always above normal windows and is always
below the screen saver window. It is an InputOutput window whose width
and height are the screen dimensions. Its visual is the root visual
and its border width is zero. Attempts to redirect it using the
composite extension are ignored. This window does not appear in the
reply of the QueryTree request. It is also an override redirect
window. These last two features make it invisible to window managers
and other X11 clients. The only way to access the XID of this window
is via the CompositeGetOverlayWindow request. Initially, the Composite
Overlay Window is unmapped.
CompositeGetOverlayWindow returns the XID of the Composite Overlay
Window. If the window has not yet been mapped, it is mapped by this
request. When all clients who have called this request have terminated
their X11 connections the window is unmapped.
Composite managers may render directly to the Composite Overlay
Window, or they may reparent other windows to be children of this
window and render to these. Multiple clients may render to the
Composite Overlay Window, create child windows of it, reshape it, and
redefine its input region, but the specific arbitration rules followed
by these clients is not defined by this specification; these policies
should be defined by the clients themselves.
C api : XCompositeGetOverlayWindow
PyGTK Solution:
I think the composite and shapes X extensions are sufficiently ubiquitous and shall assume here that they are active on your system. Here's PyGtk code for this:
# avoid title bar and standard window minimize, maximize, close buttons
win.set_decorated(False)
# make the window stick above all others (super button will still override it in the z-order, which is fine)
win.set_keep_above(True)
# make events pass through
region = cairo.Region(cairo.RectangleInt(0, 0, 0, 0))
my_window.input_shape_combine_region(region)
win.show_all()
# set the entire window to be semi-transparent, if we like
win.set_opacity(0.2)
Basically what this does is tell Gtk that other than pixel (0,0) the entire window my_window should not be considered part of itself in terms of event propagation. That in turn, according to my current understanding means that when the pointer moves and clicks, the events go to the underlying window under the pointer position, as if my_window was not there.
Caveat:
This does allow your overlay window being the focus window (due to user-solicited window switching or just because it pops up and gets the focus when your application starts). Which means that for example, keyboard events will still undesirably go to it up until the user has clicked through it to make it lose focus in favor of whatever window is under the cursor. I would likely use the approach described here to iron out this aspect.
If there's a different and proper approach for making a portion of the screen "display stuff but not receive events", without building an oddball window like above over it, I'm happy to learn about it.
I assume that one's particular desktop environment (gnome, unity, etc. on linux) may interfere with this solution depending on version and configuration, on some occasions.
Just started working with this awesome external but have a couple of questions.
When the control is evoked, is it always the top layer or can I have a background transparent image on top of it so I can frame the control nicely?
Also, my testing seems to read most Barcodes but when it comes down to reading Barcodes on hard drives, the control does not want to decode those.... Too dense of bar code pattern?
I am very impressed thus far with the ease of use of your externals. Makes we want to code more for mobile devices!
an overlaying transparent image is not possible, as far as i know.
but couldn´t you use
command mergZXingControlSetRect pLeft,pTop,pRight,pBottom
to define the rect of that scanner after creation
or
command mergZXingControlCreate pLeft,pTop,pRight,pBottom
to create the scanner control in the specified rect.
Set the rect smaller than the width and the height of the screen.
You could then use an underlying image, which is displayed outside of the scanner rect, to show the frame around scanner control. Did not test it myself, but i would assume that this should work.
Unfortunately the native controls in externals and the ones the engine provides are added as views on top of the LiveCode view. That means you can't intermingle LiveCode controls with them. One thing that some users have done is add a web view with a transparent background and a load a png image. If you create the barcode view first and the web view second then the web view will be on top.
I'm writing a game that asks the user to click on an image, which then reveals a different image. I'd like to make the transition between the images look like a playing card being turned over on both Android and IOS.
I've done a bit of research, but it all seems to indicate that the "curl" visual effect will do what I want, but is only available on IOS ( I can't test this as I don't have access to a MAC at the moment. )
Is there a cross platform way of doing this "turning a playing card over" sort of transition?
You might scale the (front) image control vertically until it is only 1 line and then scale the second (backside) image from 1 vertical line to its original size.
Only very few visual effects are cross platform. One of them is the reveal up/down/left/right effect. You might use this effect to display a neutral, e.g. gray or blue picture after hiding the front side image and before showing the back side image. Something like this:
lock screen for visual effect
hide img "front"
show img "intermediary"
unlock screen with visual effect reveal left fast
lock screen for visual effect
hide img "intermediary"
show img "back"
unlock screen with visual effect reveal right fast
I know it isn't ideal, but if you want it to be cross platform, you need to find a workaround. Why don't you check for the platform and write a different conditional routine for each platform?
I think the effect you want is flip and yes it's only available on iOS at the moment. There are a couple of iOS visual effects that push the image into a UIView and animate that with native methods. This blog post indicates it would be possible to implement something similar on android but it would need to be in the engine: http://www.techrepublic.com/blog/software-engineer/use-androids-scale-animation-to-simulate-a-3d-flip/
I have a problem with MFC dialog boxes that are drawn using derived MFC classes for custom drawing of controls.
One of our customers has a real slow PC with a poor graphics card and even normal Windows dialogs paint quite slow. In our case, the problem is far worse. Each individual control (e.g. buttons, group boxes, labels) can be seen to draw seperately.
In most cases I've overridden/implemented the OnPaint() handlers, thinking that drawing on whatever device context I'm provided should be the way to go.
Ideally, what I would like to do is have all controls painted on an off-screen buffer so that when a dialog repaint is required - bang - it just copies the single rendered image to the screen, rather than painting each control to the screen one by one.
Can somebody please advise me how I can achieve this kind of double-buffering?
I've sort of found the solution to my problem.. By setting the dialog extended style to WS_EX_COMPOSITED, the drawing works nicely.. The problem I'm having now concerns a continuous stream of WM_PAINT and WM_ERASEBKGND messages that I keep getting when this style is enabled.
Does anyone know how I can stop the WM_PAINT/WM_ERASEBKGND messages from continously occurring?