The new spectator view seems to have changed the way it syncs states. It doesn't use UNet anymore but each component has a state synchronization service.
I can see how this synchronizes changes in objects that were in the scene from the start, but how does it work if I want to instantiate a prefab during runtime?
In the old UNet implementation I would use the NetworkManager to instantiate it on all connected devices, and the prefab would be created everywhere with all its referenes to child components intact. What is the new way of doing this?
State synchronization logic for Spectator View should automatically assess game object creation on the HoloLens/Broadcaster device and relay creation events to spectator/Observer devices. Spectator View logic does not support an object created on the spectator device being recreated on the HoloLens device. If this aligns with your needs, the flow of logic is the following:
When a game object (flagged for synchronizing) is declared in the HoloLens's scene, it gets a TransformBroadcaster component added. This TransformBroadcaster monitors child hierarchy changes, so when a child game object of this TransformBroadcaster's game object is created, the parent TransformBroadcaster adds a TransformBroadcaster component to the child game object: https://github.com/microsoft/MixedReality-SpectatorView/blob/73414f4429048c6f0eed154fb8bbdea93b4caa81/src/SpectatorView.Unity/Assets/SpectatorView/Scripts/StateSynchronization/TransformBroadcaster.cs#L576
Each frame update, the TransformBroadcaster also observes changes to its game object. If new components have been added to the game object that have associated ComponentBroadcasterServices, it adds ComponentBroadcasters to the game object: https://github.com/microsoft/MixedReality-SpectatorView/blob/73414f4429048c6f0eed154fb8bbdea93b4caa81/src/SpectatorView.Unity/Assets/SpectatorView/Scripts/StateSynchronization/TransformBroadcaster.cs#L502
Once a ComponentBroadcaster (including the TransformBroadcaster) is added to a game object, it will assess all spectators on frame updates. If the ComponentBroadcaster detects that there is a spectator that does not know of its existence, it will send said spectator a broadcaster creation event: https://github.com/microsoft/MixedReality-SpectatorView/blob/73414f4429048c6f0eed154fb8bbdea93b4caa81/src/SpectatorView.Unity/Assets/SpectatorView/Scripts/StateSynchronization/ComponentBroadcaster.cs#L177
The creation event will be received by the spectator device in the StateSynchronizationSceneManager. When the StateSynchronizationSceneManager sees this creation event, it obtains or creates a mirror game object (essentially a copy of the game object running on the HoloLens device). It then adds the corresponding ComponentObserver to the mirror game object using the associated IComponentBroadcasterService: https://github.com/microsoft/MixedReality-SpectatorView/blob/73414f4429048c6f0eed154fb8bbdea93b4caa81/src/SpectatorView.Unity/Assets/SpectatorView/Scripts/StateSynchronization/StateSynchronizationSceneManager.cs#L241
When a ComponentObserver is added to a game object, it creates the component it is observing: https://github.com/microsoft/MixedReality-SpectatorView/blob/73414f4429048c6f0eed154fb8bbdea93b4caa81/src/SpectatorView.Unity/Assets/SpectatorView/Scripts/StateSynchronization/ComponentObserver.cs#L27
Then with each payload update, the ComponentObserver updates this attached component to mimic what is being displayed on the HoloLens device.
Once all ComponentBroadcasters register as created with the spectator device, you should have your newly created game object in the HoloLens scene recreated in the spectator scene; however, there a few other things to call out:
Game objects need to be flagged in some manner as candidates for synchronization. This can be achieved through two manners. You can declare that all objects should be synchronized in your scene through the Spectator View settings: https://github.com/microsoft/MixedReality-SpectatorView/blob/master/doc/SpectatorView.Setup.md#hololens-2--hololens
Or you can add a GameObjectHierarchyBroadcaster to your HoloLens scene (All child game objects under a GameObjectHierarchyBroadcaster component will be synchronized).
If you have custom Unity types, you may need to create your own IComponentBroadcasterService, IComponentBroadcaster and IComponentObserver. Most of the default Unity components (transforms, meshes, etc) already have component broadcaster services, but your own custom types may require additional work.
You need to have the same Asset Caches compiled into the HoloLens and spectator applications. When ComponentBroadcasters are created on spectator devices, they are often handed assetIds that are used to instantiate mirror game object components. For example, when a new MeshFilter is created it uses an assetId to look up the MeshFilter through the AssetService to know what mesh to add to the mirror game object: https://github.com/microsoft/MixedReality-SpectatorView/blob/73414f4429048c6f0eed154fb8bbdea93b4caa81/src/SpectatorView.Unity/Assets/SpectatorView/Scripts/StateSynchronization/NetworkedComponents/MeshFilter/MeshFilterObserver.cs#L15 This means that dynamically creating a mesh compared to relying on a mesh defined in one of your Assets will not work correctly.
Related
I am working on a multi threaded OpenGL application with OpenTK 3 and WinForms.
I have 2 shared GraphicsContexts:
a "main" rendering context, used for scene drawing and synchronous load operations.
a "secondary" resource loader context, used to load resources during draw.
This secondary context is used to load video frames coming from a Windows Media Foundation session (with a custom media sink). However, i have no control on what thread this media sink is running on, so i need a way, after each loading operation, to "unbind" that secondary GraphicsContext, so that it can be bound in the next thread where it will be needed.
Do I have to P/Invoke wglMakeCurrent(NULL, NULL) or is there a proper OpenTK way of doing this?
Short answer
Use OpenTK feature:
mycontext.MakeCurrent(null);
Long answer
Today's wglMakeCurrent doc has eliminated this old comment:
If hglrc is NULL, the function makes the calling thread's current
rendering context no longer current, and releases the device context
that is used by the rendering context. In this case, hdc is ignored.
I would trust that comment is still valid, due to so many code relying on it.
Pay attention to "releases the device context". Perhaps OpenTK does some action related to the device context. Perhaps the hdc is private (by using window style flag CS_OWNDC) So, let OpenTK handles this "NULL" case.
Better approach
Be aware that even when you use several shared contexts, is the GPU (normally one unique card) that does the loading, and not many cards allow loading while doing other jobs. Thus, it isn't guaranteed you get better performance. But shared contexts exist to this purpose, somehow.
Why should you use the same context in different threads?
I'd use a different thread for load video frames (without any gl-call) and for upload them to the GPU. This last thread is permanent and has its own gl-context, so it doesn't need to set as current every time it works. It sleeps or waits until the other thread has finished loading data, and after that task is completed it uploads that data to the GPU.
I would like to collect multi-touch pointer raw data in my Windows UWP application, so I can do gesture recognition.
I have done this in a Win32 application previously by using the GetPointerFrameInfo() method. It can retrieve the information of whole frame of pointer input. However, this method does not seem to be available in UWP.
What is the solution to retrieve the whole frame of pointer input?
For example, when I use three fingers to press screen, drag for a short distance, then release, I received the following event sequence in registered pointer hanbdler (onPointerPressed() / onPointerMoved() / onPointerReleased(), my handler functions)
pointer1 pressed event,
pointer2 pressed event,
pointer3 pressed event,
pointer1 moved event,
pointer2 moved event,
pointer3 moved event,
pointer1 moved event,
pointer2 moved event,
pointer3 moved event,
...
pointer1 released event,
pointer2 released event,
pointer3 released event,
Because the above events all happen in a sequential timing pattern, it is so hard to do the multi-touch processing since the total pointer number can not be known in advance.
I did notice that UWP's PointerPoint class provides a property called FrameID, used to identify the input frame, but I can not find any method to use this frame id to retrieve the whole frame of pointer input.
Because the above events all happen in a sequential timing pattern, it is so hard to do the multi-touch processing since the total pointer number can not be known in advance.
Yes you're absolutely right, but unfortunately there is no equivalent to Win32's GetPointerFrameInfo() method in UWP, we can only focus on the base Pointer events. As your test result of event sequence in registered pointer handler, a common way to solve this problem is to count the Pressed events before Moved events, and clear this count in the Released events, each pressed events represents a finger pointer, you can refer to my similar case here: How can I get touch input in uwp?
But, if your want won't be published on the store, there are methods to use Win32's API in UWP app. One way is using VS2015TemplateBrokeredComponents to build a bridge between UWP app and traditional desktop app, you can follow the steps here to try your solution out. Alternatively you can try to use PInvoke and this Win32 API in your UWP app.
Above all the ideas I provided here, I hope you may submit a request to add this new features for development through the Windows Feedback tool. I personally think this is a good feature request for UWP apps.
We are developing an app for the Spotify platform. We have a problem with the player context.
We offer our users radio stations. These radio stations are not static playlists, they are created dynamically in runtime. The radio playing process is as follows:
We play the first track with the player's playTrack method.
Then, our algorithm determines the next track to be played and sends it to the client
After the currently playing track finishes playing, we load the new track again with player's playTrack method.
The process works fine if the player has no context prior to starting our radios. But if there is a context already (for example, user starts playing a playlist on Spotify, and then starts a radio with our app), the player continues to play the previous context.
playTrack method does not change the current context of the player. Is there a way for playing a single track using the playContext method, or destroying the context of the player?
I'd say that populating a temporary playlist (Playlist.createTemporary) would be the more straightforward implementation. Then your playlist would become the context.
You can still limit the number of songs provided (seems to be desirable in your case) because you can dynamically add new songs to the end of the playlist while it is going. You can also remove songs from the beginning as you go.
I'm using performBlock on my NSManagedObjectContexts so that my changes happen on the right queue for the given context. My question is - if I'm making a lot of changes and calling methods from within performBlock - is there an easy way to ensure that I use objects from the proper context.
Example:
I have an activeAccount iVar ( created on the Main Queue ) that is a NSMangedObject for the current account in the application. I have some instance methods that use the activeAccount object to perform certain tasks - getting data, setting data. So my question is if I am doing something on a background NSManagedObjectContext and I call one of these shared methods - is there a pattern I can use so that in these methods I know to either use the current activeAccount iVar or get a new one. Also, if I needed to do something that requires a NSManagedObjectContext - how do I know which one to get/use.
One method I have for knowing which NSManagedObjectContext to use is I have a method that checks if it is running on the current thread - it then knows to return the main thread's context or the background thread's context. Also, if I'm on the background thread, am I allowed to read the Object ID of the activeAccount that lives on the main thread so that I can get a copy of it on the background thread? Thanks in advance.
Brian,
Thread confinement can be a tricky proposition to maintain. The key thing you need to maintain is using objects in their proper MOC. As every managed object maintains a link to both its host MOC and its object ID, this is really easy to ensure. For example:
NSManagedObjectContext *newMOC = NSManagedObjectContext.new;
newMOC.persistentStoreCoordinator = oldActiveAccount.managedObjectContext.persistentStoreCoordinator;
ActiveAccount *newActiveAccount = [newMOC objectWithID: oldActiveAccount.objectID];
Now every instance you access from newActiveAccount is created in the newMOC and is, hence, thread confined to that MOC. objectIDs are persistent. The -persistentStoreCoordinator is rarely, if ever, changed on the mainMOC. Hence, the above code is properly confined. There are issues with the above technique if the source MOC is transient. Hence, I cannot guarantee the above code works with respect to two background MOCs.
Andrew
I have to ask first, why are you having so many contexts in use at the same time?
I use one for background operations and one for main thread. If I need to create another one for discardable changes, I'll just create it and pass it on, so now my self.managedObjectContext points to the draft context. I will never let my managed objects to live in a scope where they could access a multitude of contexts.
It is not entirely clear if you are writing for iOS or OSX, but with iOS for example:
If I need to push a new view controller into navigation stack I will initialize my destination view controller's managedObjectContext ivar as well as any NSManagedObject subclass instances. Since in -prepareForSegue: I know whether I'll create a draft context or just pass on my current one, I also know whether I need to initialize those managed object instances by referencing them by their IDs from newly created context or I can just pass them on.
Now inside my view controller I can take it for granted that my managed objects are always tied to the self.managedObjectContext.
i read Here
and bit confused about the unregistering events. means i have button and registered for click event whether its required unregitser click event explicitly in page unload event.
Because all most UI elements will registered for some of the events.
In the model and viewmodel i can unregistered event explicitly (like loadCompleted)
Is it required to un-register the events of all UI Elements to reduce memory leak?
You only need to unregister eventhandlers that would keep the View in memory.
So if you Keep the ViewModel object (e.g. to pass it on to another view) any binding between the View and ViewModel might keep the View in memory (reachable). If you do not keep the ViewModel you ar enot in danger of this. (Yet be very certain that you do not keep a reference to the ViewModel)
A good description of this problem (and solutions) can be found here