HoloLens spatial mapping.SpatialSurfaceMesh update problem. C++/winrt - hololens

im working on the spatial mapping processing for my HoloLens project.
Somehow calling "SpatialSurfaceMesh::TryComputeLatestMeshAsync" keeps returning the same mesh data overtime.
Is there another process involved updating the observer?
void SpatialMapping::AddOrUpdateSurface(winrt::Windows::Perception::Spatial::SpatialCoordinateSystem const& coordinateSystem)
{
using namespace winrt::Windows::Perception::Spatial::Surfaces;
SpatialBoundingBox axisAlignedBoundingBox =
{
{ 0.f, 0.f, 0.f },
{ 50.f, 50.f, 50.f },
};
SpatialBoundingVolume bounds = SpatialBoundingVolume::FromBox(coordinateSystem, axisAlignedBoundingBox);
m_surfaceObserver.SetBoundingVolume(bounds);
m_surfaceObserver.ObservedSurfacesChanged(
winrt::Windows::Foundation::TypedEventHandler
<SpatialSurfaceObserver, winrt::Windows::Foundation::IInspectable>
({ this, &SpatialMapping::Observer_ObservedSurfacesChanged })
);
}
void SpatialMapping::Observer_ObservedSurfacesChanged(winrt::Windows::Perception::Spatial::Surfaces::SpatialSurfaceObserver const& sender
, winrt::Windows::Foundation::IInspectable const& object)
{
{
using namespace winrt::Windows::Perception::Spatial::Surfaces;
const auto mapContainingSurfaceCollection = sender.GetObservedSurfaces();
// Process surface adds and updates?.
for (const auto& pair : mapContainingSurfaceCollection)
{
auto id = pair.Key();
auto info = pair.Value();
InsertAsync(id, info);
}
}
}
Concurrency::task<void> SpatialMapping::InsertAsync(winrt::guid /*const&*/ id, winrt::Windows::Perception::Spatial::Surfaces::SpatialSurfaceInfo /*const&*/ newSurfaceInfo)
{
using namespace winrt::Windows::Perception::Spatial::Surfaces;
return concurrency::create_task([this, id, newSurfaceInfo]
{
const auto surfaceMesh = newSurfaceInfo.TryComputeLatestMeshAsync(m_maxTrianglesPerCubicMeter, m_surfaceMeshOptions).get();
std::lock_guard<std::mutex> guard(m_meshCollectionLock);
m_updatedSurfaces.emplace(id, surfaceMesh);
});
}
Generation works, Update does not
Manuel attempt same problem:
winrt::Windows::Foundation::IAsyncAction SpatialMapping::CollectSurfacesManuel()
{
const auto mapContainingSurfaceCollection = m_surfaceObserver.GetObservedSurfaces();
for (const auto& pair : mapContainingSurfaceCollection)
{
auto id = pair.Key();
auto info = pair.Value();
auto mesh{ co_await info.TryComputeLatestMeshAsync(m_maxTrianglesPerCubicMeter, m_surfaceMeshOptions) };
{
std::lock_guard<std::mutex> guard(m_meshCollectionLock);
m_updatedSurfaces.emplace(id, mesh);
}
}
}
MVCE:
Create a New Project with the template
"Holographic DirectX 11 App (UWP) C++/WinRT)"
Add the files:
https://github.com/lpnxDX/HL_MVCE_SpatialSurfaceMeshUpdateProblem.git
Replace m_main in AppView.h

We did some research and have some thoughts about your question right now, let me explain the findings
Has your Observer_ObservedSurfacesChanged method triggered exactly? Adding output statements or breakpoints can help you check it. Since SurfaceObserver should be always available, usually we need to check the availability of surfaceObserver in each frame and recreate a new one when necessary, sample code snippt please see here.
Have you set m_surfaceMeshOptions? It is not visible in the code you posted. If it is missing, you can configure it with the following statement:
surfaceMeshOptions-> IncludeVertexNormals = true;
Microsoft provided the Holographic spatial mapping sample, shows how to acquire spatial mapping data from Windows Perception in real-time. it is similar to your needs, to narrow down issue if it's an issue with your code, please try to check and run this example on your device
If after the above steps, you still can't solve the problem, could you provide an
MVCE, so that we can locate the issue or find a solution? Be careful to remove any privacy-related or other business function codes.

Your code has the following problems:
TryComputeLatestMeshAsync should be called from Observer_ObservedSurfacesChanged not from concurrency::create_task
TryComputeLatestMeshAsync return mesh with matrix, vertices and indices. Indices should be stored to a safe location at the first run, they don't change later. Vertices and matrix should copied as they returns. You shouldn't save the mesh itself, because its data updates from various threads.
ObservedSurfacesChanged shouldn't be called every frame. This is long running function.
Maybe it has something more. I would recommend to start from the sample mentioned earlier.

Related

How to access Object3D material after adding the GLTF

I want to duplicate my GLTF models with different positions/colors dynamically, to do so I have done:
const L_4_G = new Object3D();
...
const multiLoad_4 = (result, position) => {
const model = result.scene.children[0];
model.position.copy(position);
model.scale.set(0.05, 0.05, 0.05);
//
L_4_G.add(model.clone())
scene.add(model);
};
...
function duplicateModel4() {
L_4_G.translateX(-1.2)
L_4_G.translateY(0.0)//0.48
L_4_G.translateZ(1.2)
L_4_G.rotateY(Math.PI / 2);
scene.add(L_4_G);
}
I didn't find out how can I change the Object3D color from the documentation, can you please tell me how can I do that? thanks in advance.
Here is the full code that I'm using, and here are the models
Update
I have seen this solution, to store a set of colors in the object's userData and choose the color later:
L_2_G.userData.colors = {green : #00FF00, red : ..., ...}
L_2_G.children[0].material.color(userData.colors["green"])
But I'm getting an error that children[0] undefined, but I can see that this object has a child and a material, and color via the console: console.log(L_2_G.children), console.log(L_2_G.children.length)--> 0
Also I have tried getObjectByName as explained here:
scene.getObjectByName(name).children[0].material.color.set(color);
which also reslts: children[0] is undefined, scene.getObjectByName(name).children.length is 0.
THREE.Object3D is a base class for anything that can go in a scene graph, including lights, cameras, and empty objects. Not all Object3D instances have geometry or materials. You may be looking for the THREE.Mesh subclass which does have materials and colors.
In general, code like getObjectByName(...) and model = result.scene.children[0] is very content-specific. The file might contain many nested objects, and .children[0] just grabs the first part. It's usually best to traverse the scene graph instead, looking for the objects you want to modify (e.g. looking for all Meshes, or Meshes with a particular name).
const model = result.scene;
model.traverse((object) => {
if (object.isMesh) {
object.material.color.setHex( 0x404040 );
}
});
Then you can either add the entire group to your scene (scene.add(model)), or just add parts of it. Keep in mind that adding meshes to a new parent removes them from their previous parent, and you shouldn't do that while traversing the previous parent. Instead you can make a list of meshes, and add them in a second step:
const meshes = [];
result.scene.traverse((object) => {
if (object.isMesh) {
meshes.push(object);
}
});
for (const mesh of meshes) {
scene.add(mesh);
}
Finally, the position of an object is inherited from its parents. By removing the object from its original parents you might change its position in the scene. If you are planing to assign a new position to the object anyway, that is fine.

Is it possible to recolor a lottie animation programmatically?

If I have a lottie animation in the form of a json file, is there a way to recolor it in code or even within the json itself?
(To be clear, I hope there's a way to do it without involving After Effects. For instance if I decide to change my app's primary color, the whole app will change except the animation unless there's a way to do that.)
I figured it out. For this example, let's say I want to recolor a specific layer to Color.RED.
You'll need your LottieAnimationView, a KeyPath, and a LottieValueCallback
private LottieAnimationView lottieAnimationVIew;
private KeyPath mKeyPath;
private LottieValueCallback<Integer> mCallback;
Then in your onCreate (or onViewCreated for a fragment) you'll get the animation with findViewById, as well as "addLottieOnCompositionLoadedListener" to the lottieAnimationView, in which you will setup the "mKeyPath" and "mCallback":
lottieAnimationVIew = findViewById(R.id.animationView);
lottieAnimationView.addLottieOnCompositionLoadedListener(new LottieOnCompositionLoadedListener() {
#Override
public void onCompositionLoaded(LottieComposition composition) {
mKeyPath = getKeyPath(); // This is your own method for getting the KeyPath you desire. More on that below.
mCallback = new LottieValueCallback<>();
mCallback.setValue(Color.RED);
checkBox.addValueCallback(mKeyPath, LottieProperty.COLOR, mCallback);
}
});
The argument "LottieProperty.COLOR" specifies which property I am changing.
There's probably a better way to do this, but here's my "getKeyPath" method for finding the specific thing I want to change. It will log every KeyPath so you can see which one you want. Then it returns it once you've supplied the correct index. I saw that the one I want is the 5th in the list, hence the hard-coded index of 4.
private KeyPath getKeyPath() {
List<KeyPath> keyPaths = lottieAnimationView.resolveKeyPath(new KeyPath("Fill", "Ellipse 1", "Fill 1"));
for (int i = 0; i < keyPaths.size(); i++) {
Log.i("KeyPath", keyPaths.get(i).toString());
}
if (keyPaths.size() == 5) {
return keyPaths.get(4);
}
else {
return null;
}
}
Note that the "Fill", "Ellipse 1", "Fill 1" are strings I supplied to narrow the list down to just the ones that have those keys, because I know that the layer I want will be among those. There's likely a better way to do this as well.
There is another thread on this topic with the same approach but a bit simplified:
How to add a color overlay to an animation in Lottie?
Here's directly an example (Kotlin):
yourLottieAnimation.addValueCallback(
KeyPath("whatever_keypath", "**"),
LottieProperty.COLOR_FILTER
) {
PorterDuffColorFilter(
Color.CYAN,
PorterDuff.Mode.SRC_ATOP
)
}
You can find the names of the keypaths also in the Lottie editor.

Detect collision direction in Phaser3

I'm trying to create a 'Bomber-man' like game using Phaser3 library.
For this purpose I'd like to define a collision relationship between the player and the bricks,
and more importantly - detect the collision direction relative the the player.
I've noticed body properties like touching or blocked but they are always set to false. (please see below)
//scene.js
// bricks static group
this.scene.physics.add.staticGroup({ immovable: true });
// player defined in external file (as sprite)
this.player = new Player(this, 90, 90)
// player.js
// ...
this.physics.add.collider(
this,
scene.bricks,
function(player, brick) {
if(player.body.touching.left) { //ALWAYS FALSE!!!
this.isBlockedFromLeft = true;
}, else if(player.body.touching.right) {
this.isBlockedFromRight = true; // ALWAYS FALSE!!!
}
},
null,
this
);
I'd appreciate any help. This is driving me crazy. Maybe there is a better way to do it and i'm missing something...
Thanks in advance.
So I finally figured it out.
The major issue was the way I defined the player movement. It should be
if (this.keyboard.right.isDown) {
this.body.setVelocityX(this.speed);
}
rather than
if (this.keyboard.right.isDown) {
this.x += this.speed;
}
The second way prevent collision detection and the body.touching and body.blocked properties to be updated.
Furthermore, I've found out that when it comes to top-down tiled games, it's really easier to build up the game using the tile-map feature.
official examples can be found here:
https://phaser.io/examples/v3/search?search=map
and here is a tutorial of how to make a tiled-map using a light-weight software called 'Tiled'
https://www.youtube.com/watch?v=2_x1dOvgF1E
Thank you all!

CQRS in data-centric processes

I have got a question related to CQRS in data centric processes. Let me explain it better.
Consider we have a SOAP/JSON/whatever service, which transfers some data to our system during an integration process. It is said that in CQRS every state change must be achieved by the means of commands (or events if Event Sourcing is used).
When it comes to our integrating process we have got a great deal of structured DATA instead of a set of commands/events and I am wondering how to actually process those data.
// Some Façade service
class SomeService
{
$_someService;
public function __construct(SomeService $someService)
{
$this->_someService = $someService;
}
// Magic function to make it all good and
public function process($dto)
{
// if I get it correctly here I need somehow
// convert incoming dto (xml/json/array/etc)
// to a set of commands, i. e
$this->someService->doSomeStuff($dto->someStuffData);
// SomeStuffChangedEvent raised here
$this->someService->doSomeMoreStuff($dtom->someMoreStuffData);
// SomeMoreStuffChangedEvent raised here
}
}
My question is whether my suggestion is suitable in the given case or there may be some better methods to do what I need. Thank you in advance.
Agreed, a service may have a different interface. If you create a rest-api to update employees, you may want to provide an UpdateEmployeeMessage which contains everything that can change. In a CRUD-kind of service, this message would probably mirror the database.
Inside of the service, you can split the message into commands:
public void Update(UpdateEmployeeMessage message)
{
bus.Send(new UpdateName
{
EmployeeId = message.EmployeeId,
First = message.FirstName,
Last = message.LastName,
});
bus.Send(new UpdateAddress
{
EmployeeId = message.EmployeeId,
Street = message.Street,
ZipCode = message.ZipCode,
City = message.City
});
bus.Send(new UpdateContactInfo
{
EmployeeId = message.EmployeeId,
Phone = message.Phone,
Email = message.Email
});
}
Or you could call the aggregate directly:
public void Update(UpdateEmployeeMessage message)
{
var employee = repository.Get<Employee>(message.EmployeeId);
employee.UpdateName(message.FirstName, message.LastName);
employee.UpdateAddress(message.Street, message.ZipCode, message.City);
employee.UpdatePhone(message.Phone);
employee.UpdateEmail(message.Email);
repository.Save(employee);
}

Processing an emaillist async in MVC4

I'm trying to make my MVC4-website check to see if people should be alerted with an email because they haven't done something.
I'm having a hard time figuring out how to approach this. I checked if the shared hosting platform would allow me to activate some sort of cronjob, but this is not available.
So now my idea is to perform this check on each page-request, which already seems suboptimal (because of the overhead). But I thought that with using an async it would not be in the way of people just visiting the site.
I first tried to do this in the Application_BeginRequest method in Global.asax, but then it gets called multiple times per page-request, so that didn't work.
Next I found that I can make a Global Filter which executes on OnResultExecuted, which would seemed promising, but still it's no go.
The problem I get there is that I'm using MVCMailer to send the mails, and when I execute it I get the error: {"Value cannot be null.\r\nParameter name: httpContext"}
This probably means that mailer needs the context.
The code I now have in my global filter is the following:
public override void OnResultExecuted(ResultExecutedContext filterContext)
{
base.OnResultExecuted(filterContext);
HandleEmptyProfileAlerts();
}
private void HandleEmptyProfileAlerts()
{
new Thread(() =>
{
bool active = false;
new UserMailer().AlertFirst("bla#bla.com").Send();
DB db = new DB();
DateTime CutoffDate = DateTime.Now.AddDays(-5);
var ProfilesToAlert = db.UserProfiles.Where(x => x.CreatedOn < CutoffDate && !x.ProfileActive && x.AlertsSent.Where(y => y.AlertType == "First").Count() == 0).ToList();
foreach (UserProfile up in ProfilesToAlert)
{
if (active)
{
new UserMailer().AlertFirst(up.UserName).Send();
up.AlertsSent.Add(new UserAlert { AlertType = "First", DateSent = DateTime.Now, UserProfileID = up.UserId });
}
else
System.Diagnostics.Debug.WriteLine(up.UserName);
}
db.SaveChanges();
}).Start();
}
So my question is, am I going about this the right way, and if so, how can I make sure that MVCMailer gets the right context?
The usual way to do this kind of thing is to have a single background thread that periodically does the checks you're interested in.
You would start the thread from Application_Start(). It's common to use a database to queue and store work items, although it can also be done in memory if it's better for your app.

Resources