How is it possible to connect real objects with virtual objects using MRTK toolset from HoloLens - hololens

How can I connect real objects to virtual ones with the MRTK toolset. For example, if I place the tire of a car virtually over the real tire and then show disassembly instructions.
I found this video, how is the object detection implemented, to interact really native, thats my goal.
https://youtu.be/QTuKcm8s4QQ

Related

Is there a standard way of sending GPS navigation over BLE?

I'm developing a 'smart watch' which is connected via BLE to an app on a phone.
My idea is to show the direction on the watch so the user doesn't have to remove the phone from their pocket while using their favorite application (Waze, Google Maps, ...)
Is there any standard to send navigation information (turn left/right, ...) to a smart device over BLE?
It seems there are apps that do this, but they are fully custom: https://www.youtube.com/watch?v=naAhe7DTKYM. I'm checking on the side of Android Auto, but it seems to me it's only by USB cable.
Off the top of my head there are two services that you can use for this purpose:-
Location and Navigation Service (Used mostly for outdoor):-
"The Location and NavigationService(LN Service)exposes location and
navigation-related data from a Location and Navigation sensor
(Server)intended for outdoor activity applications."
Indoor Positioning Service (Used mostly for indoor):-
"The Indoor Positioning Service exposes location information to
support mobile devices to position themselves in an environment where
GNSS signals are not available, for example in indoor premises. The
location information is mainly exposed via advertising and the
GATT-based service is primarily intended for configuration."
If both are not 100% suitable for your application, then you can create your custom profile that contains one or both of these services in addition to a custom service (turn left/right etc). This way, any watch can connect to the phone and get the adopted services info, and for any additional info watch can add support for the custom service.
When I was looking for such "standard way" for turn-by-turn navigation (official Bluetooth specification for navigation, similar to official Battery or Heart rate services), I didn't find anything suitable, but I still have a feeling like maybe I've missed something.
Those services Youssif Saeed suggested are for different type of navigation, not turn-by-turn unfortunatery.
Some apps (Sygic for example) may integrate their own BLE service.
I guess the application you linked reads notifications (posted by navigation apps like Waze, Google Maps), extract instructions from them, and then send to an external device via BLE.

What is Kinect + Linux being used for?

An article on Hackaday piqued my curiosity, and I see Kinect + Linux questions being asked here (mostly about configuration), so I'll venture this question:
It is clear to me that Kinect can be used together with Linux on a "regular pc" -- but I can't help wondering why, that is, what might you actually use this for?
I don't suppose people really like the human/computer interface presented in movies such as "Minority Report" -- surely, nobody is actually doing text editing, coding, or business data processing by "hand-waving". So besides just games & exercises, what are examples of actual, real-world, useful (ie. 'professional') applications of such a setup?
For instance, can it be used for 3D scanning of real-world objects to obtain digital models? What sort of accuracy would such a scan yield?
The Kinect can be used for a wide variety of useful applications. I'm not sure if you are asking specifically about Linux or if Windows ("regular PC") is acceptable, but I'll provide you with some examples that come to mind.
For Linux specifically, it is likely that applications on Linux are using the sensor's raw sensor data only, rather the skeletal tracking feature. Many Kinect applications are on Windows because Microsoft's Kinect SDK is available only on Windows, and it provides the best skeletal tracking accuracy to-date.
You are right that the Kinect is rarely used where a keyboard & mouse would be faster and more accurate, but note that it is potentially relevant for accessibility.
And yes, it can be used for 3D scanning of real-world objects. I'm not sure about the exact accuracy, but I think it is acceptable for many applications. The main benefits are its low cost and speed.
For examples of 3D scanning, check out:
KinectFusion, a Microsoft Research project
Occipital Structure sensor for 3D scanning. (This is not the Kinect sensor, but provides an example application for 3D scanning. The company has a Kinect-related history as well.)
Styku - 3D body scanning for clothes fitting
Aside from 3D scanning, here are some other examples of applications:
Atlas5D - at-home patient monitoring
GestSure - 'Minority Report' interface for surgical rooms
Jintronix - games, exercises, assessments for physical therapy
There are many depth sensors like the Kinect3D on the market. The latest notable application would be iPhone X's depth sensor and FaceID. Many companies in the space are working actively in FaceID now, which would also be useful on Linux. Check out Microsoft's Window Hello biometric facial ID system - see Microsoft's official website:
Manufacturing of the Kinect sensor and adapter has been discontinued,
but the Kinect technology continues to live on in products like the
HoloLens, Cortana voice assistant, the Windows Hello biometric facial
ID system, and a context-aware user interface.
Kinect has applications in the robotics community as well, though I don't know the specifics. I assume many in robotics community use Linux when working with the Kinect. The depth and color cameras can be used to provide vision and the microphone array for audio input.
Generally, the Kinect had a big impact when it was released not just because of its technology but also because of its low price point, even if it's not the most accurate for every application. As this technology improves, I hope many other applications will emerge and become mainstream.
EDIT: also, check out this Hacker News discussion: "Microsoft Has Stopped Manufacturing The Kinect"

Live graphing multiple channels of data, platform / framework selection

What framework / library / platform would you suggest for creating a Linux, preferably multiplatform, application displaying eight oscilloscope like graphs, updating in realtime? I'm imagining a view of the eight channels scrolling steadily to the left, with the newest data coming in on the right.
The data source is a microcontroller device I'm developing right now, which AD converts the 8 channels and blasts all measurements to the PC by an USB virtual com port, I'm thinking in the order of 100 samples/sec. I can brew my own protocol for this data transfer, but again, is there some standard out there I should use to get compatibility and not reinvent the wheel?
(this is a hobby project for monitoring whats going on in my car's non-OBD motorcontroller)
Thanks,
Lars
I have used the JFreeChart package in a couple of projects. It is a very powerful and flexible tool and is free and open source.

How to programmatically use the mobile phone's IrDA to remote control a media player?

which API or library on which mobile OS is to be used when one needs to write a code to use the phone's IrDA to create the necessary impulses to remote control consumer electronics e.g. a HDD media player?
Is maybe a certain mobile OS better suited for that kind of application than others?
First you need to know that IrDA is not the best choice for remote control. It can be done, but IrDA is by design high speed/low range, you can emulate low speeds but ranges (IMO) are far from practical usage (Nokia e50 is able to control digital camera shutter from 2-3m... with very, very careful aiming). The amount of hacking needed to achieve this is shown here, you basically need to trick IrDA to send correct impulses with correct frequency.
The second thing is that CIR remote control is not as simple as you might think. There are countless standards that differ in used frequency, modulation, wavelength, command codes and so on. You need to know what you want to support. LIRC site can be very helpful in determining that http://lirc.sourceforge.net/remotes/. Approachable explanation of what it all means is available here: http://www.sbprojects.com/knowledge/ir/ir.htm
As for ready made libraries and platforms... I honestly don't know. I've seen it done on PocketPC (nevo among others) and Symbian S60 (irRemote). Haven't seen working J2ME app yet.
Last time I needed the IR remote I hacked it together using IR diode, AVR ATTiny and surprisingly short piece of assembly :)

J2ME app Vs browser on a handset

Recently I started developing a J2ME app prototype. I noticed how difficult it is to develop a good looking UI. Consider developing an app in J2ME for booking flights interacting with webservice.
A website to book flights will be easy to develop with nice ui and can be accessed by browser on a handset. I understand not all handsets have browser but all the new and upcoming ones have browser and have big screen as well.
Is it a good idea to develop such a application in j2me which need to talk to webservice for it to work? Or j2me is only suitable for standalone apps?
Advantages of J2ME:
Can access phone resources, like file system, phone book and GPS. The last is very important in map applications.
You can build richer User Interfaces. It may be difficult as you say, but there are many GUI libraries that could assist you. On the contrary the UI for a mobile browser (you can't rely on CSS and javascript working) would be very poor.
Greater flexibility on the communication logic. You can encrypt/decrypt data, compress them, use SOAP web services. With the browser, your best bet would be to develop REST services.
Disadvantages of J2ME:
Midlets need to be signed. This has some cost and there are situations that even a signed app won't run properly in specific phones.
Developing a midlet to run in all types of phones is a nightmare. On the contrary, a well designed mobile web application would be displayed properly in all recent phones.
You need to have a channel for distributing your application. People would need to download it and get charged for the required bandwidth. You would need to care for angry customers having problems with the application. Things are easier with a web site.
J2ME apps are inevitably compared with native applications (iPhone, Windows Mobile, Symbian). Compared to these, they are very poor and many would find that paying for them or even using them isn't justified.
My conclusion: Nowadays real smart phones are becoming more popular and win an ever growing market share. Under these circumstances, the advantages of J2ME can't really overcome its restrictions. The only exception I could think of, is if to have to develop a GPS application. For all other cases, a mobile web site is a better idea.
There are a lot of misunderstanding and plain wrong statements in the previous answers.
I advice you to just do your research yourself. Nowadays you CAN develop really good looking apps with J2ME without writing your own GUI framework. Take a look at LWUIT really. For example they have a virtual keyboard as one of their touch screen functionalities and this you have on devices like the N97 which itself does not have a Virtual keyboard. BTW using LWUIT you have a Blackberry and Android port included if anyone cares.
Also Apps nowadays become the center stage on many platforms not just the iPhone. Look at the recent developments in this area like OVI, RIM, Samsung, SE, Orange World they all start with app shops.
"Getting people to use a website on their mobile phone is easier than getting them to download an application." this is just a claim without proof. you cannot say that like this. It depends on a lot of other factors. - Why should users type in your mobile url into the rather small screen again?
Anyway, this answer is probably too late so I'm not gonna write much more. The mobile industry is changing fast right now but there is not yet a alternative to J2ME for crossplatform development. Maybe in the future with better browsers and widget technolgies.
Just a short note, applications like google maps or gmail mobile probably don't use WebServices to talk to their server part. A WebService has a lot of overhead, especially when considering that mobile users are usually rated by the amount of data they transmit. The best way to perform communication between your client app and its server part is to use binary data over a socket connection.
I personally think it's really hard to make a consistent and reliable J2ME application that will run across a large set of mobile phones. Based on my experience, I would only develop a J2ME application (instead of a Web application) if it's a strict requirement - for example, to be able to view your bookings without being connected to the network. There are other costs associated with J2ME applications - the applications must be downloaded, the user will be asked if the application is allowed to connect to the network when it attempts to (there are exceptions for this case but I believe the application has to be signed by 3rd party company - more $$$ involved), you will have to maintain different versions of the application running on a variety of mobile phones (more complexity to the application), and so on...
Think about it this way - if you were developing a similar thing for a computer, would you build a desktop application or a web application? With the cellphones of today (many of which can access full-html sites with javascript - which means ajax), the proposition of the question is valid.
I thing a good rule of thumb should be: If what you're trying to achieve can be done with a mobile website - go for the website.
IMHO, apps should only be used to if they cant take advantage of the mobile hardware - like location, sound, video, 3d, pictures etc...
Even if the dev costs for the app were insignificant (they usually aren't), you'd have to offer some really amazing capabilities to make the users go through the trouble of downloading it.
(All of this is essentially true for J2ME/BREW. The iPhone is a little different as apps take the center stage)
One thing worth highlighting: the only standard way of deploying a MIDlet is via OTA download so you wouldn't expect a J2ME-capable phone to not have a web browser.
Mobile web browser like Webkit and Opera are getting better faster than J2ME (at least until MIDP3.0 starts shipping, if ever).
No matter which platform you choose, you will need to test your service on many devices. I don't think switching from J2ME to webapp makes a huge difference in that regard, because phone manufacturers keep changing the binaries that go into the phones firmwares.
Getting people to use a website on their mobile phone is easier than getting them to download an application. unless that application is already installed when they buy the phone, that is.
You might want to look at LWUIT for better and easier J2ME GUI.
One thing that J2ME will accomplish for a flight booking service is save battery life by not requiring constant network data transfer, thanks to the local storage mechanisms.
there are many great j2me apps that (need to) talk to webservices. just think of the google apps, like gmail mobile and maps for mobile. they are faster and easier to use than using the services via cell phone browser. so if you can design a good app, it's definitely worth it.
EDIT: also, a j2me app makes possible features that can't be provided by a web application: integration with phone features (address book, calendar), "call this number", location api, etc.
I think for business apps, or more text/data oriented things, a mobile web/wap site might be easier to maintain, since you won't have to deal with pushing client updates out to handsets.
For UI-intensive apps (maps, games, etc.), client apps are probably the way to go, so you can handle the more of the processing and rendering on the client side.
Both options are difficult though, since there are so many compatibility issues with phones. You might be best served by narrowing down what types of phones you want to support for your app. If you think most of your customers will be iPhone or Android phones, you can target those platforms (with either client apps or web apps) and avoid old-school j2me completely.
I hate WebApps on phones. They are slow and they don't work in a semi-connected environment.
J2ME apps can do local backups, bluetooth backups, bluetooth data sharing between 2 phones and better responsive UI. However that requires money,skill,time etc.
My main gripes with MIDP though is pushing software updates and wav real time mixing. Technically those are possible within the scope of MIDP but the goons at wheel are not very creative.

Resources