I'm new to Windows development, and am looking for assistance on where to get started for a particular project.
In short, I want to create a windowed application that allows a user to send keyboard and mouse inputs to another application, by interacting with various UI controls via touch. Essentially a custom on-screen keyboard/touchpad that can be used for sending keyboard-shortcuts to other applications.
There are two applications in Windows 10 that perform exactly the way I would want my new app to - the On-Screen Keyboard and Touchpad:
https://support.microsoft.com/en-us/help/4337906/windows-10-open-the-on-screen-touchpad
https://support.microsoft.com/en-us/help/10762/windows-use-on-screen-keyboard
At the most basic level, I want to define my own interface (or allow the end user to define their own), and use the same code that the onscreen keyboard/touchpad are using for handling touch events and injecting inputs into the system.
I'm uncertain at what level I would need to start to get the functionality I need - UWP? WPF? C++?
If anyone has any insight into how the on-screen utilities were built, I think that would give me an excellent head start.
Related
For test automation I'd like to capture a virtual desktop which is not visible. It is not even accessible, as a secure desktop is shown.
I know it is possible to hook into the composite manager ("dwm") to capture each and every window on that desktop. And I kmow it is possible to send events to windows on that desktop. (I know that because otherwise the test tools wouldn't work)
Before I start to re-implement the composite manager: Is it possible to get the DesktopWindow from dwm, and if so how Do I force dwm to do its job even if a secure desktop is shown?
If I have to bite the bullet and need to implemrnt compositing myself, what is the fastest way to order all windows bottom to tom and to render them to some image?
Does the win10 capture api work for invisible desktops?
To answer the last question: No, the new win10 capture API doesnt't help. For example the program
https://github.com/robmikh/SimpleRecorder/tree/master/SimpleRecorder
cannot capture a locked desktop nor can it capture sub windows.
The above is the elaborate version of:
GDI32Util.getScreenshot(handle)
with handle being the desktop window (doent work when locked) or some other window handle (works when locked, but misses the subwindows).
So the only option is to traverse all windows in z order from bottom to top.
I need to record the RDP connection from local machine thru coded ui testing framework (Visual Studio - coded ui project).
FYI. I have Coded UI test project in my local machine and as soon as i start recording I'm going to click on Remote desktop connection and it needs to be recorded.
I've played once with such a thing. Coded UI does not support RDP. There's no known way I've heard that you can just record actions inside of remote desktop.
If you really need to do something with Remote Desktop, you may try using OpenCv Library to visually identify screen coordinates of your controls. I've done it once. The algorithm is:
make a screenshot of UI control you want to click on;
save it inside your Coded UI project;
pass the image to the OpenCV library when the control is present on screen;
OpenCV returns you coordinates' rectangle of the control;
Perform Mouse.Click(); inside the rectangle.
If you are ready to go with such a solution and need more information, please let me know...
I have created an application in which the vi has some controls and these controls are useful only during the development and on special instance can be unlocked in the application. i basically use app.kind property node to determine what environment the vi is running in and suitably hide/unhide the controls.
I have the application published on the web using the NI Web publishing tool. The computer which hosts the app works fine(and these controls remain invisible) but these controls can be sen on the web page. The vi is in "Embedd" mode. As a workaround i have pushed these controls some distance away and hence avoided the user from knowing about it. but this introduces the problem that i cannot view these controls when i unlock them.
Any help would be greatly appreciated.
You have built a stand-alone application and enabled web server, correct?
Are you sure the web panel is connecting to the stand-alone application (app.kind=2)
and it is not reaching the development LabView (app.kind=1) still listening on that web server port?
I would add an indicator to display the value of app.kind at all times.
What happens if you toggle the hidden fields on and off? I would add a button to do this on the vi.
Do they disappear/reappear reliably in the window where you have control?
Also, you said this was in Embedded mode - but are you also transferring control to the web page?
Those are some approaches I'd try to help pin this down.
I have downloaded a Window Gadget and found this function from the javascript.
What this function does? Is it malicious?
Before I run the Widget, I commented out first the function call and it seems this function is not necessary to run the widget.
function LaunchPowerApplet()
{
System.Shell.execute(System.Environment.getEnvironmentVariable("SystemRoot") + "\\system32\\mblctr.exe");
}
System.Shell.execute is a method from the Windows Sidebar API that is used to launch an application.
%WINDIR%\system32\mblctr.exe is the executable for the Windows Mobility Center, a tool that «puts commonly used laptop settings in one spot so you can access them easily on the go».
Whether it's necessary or not is hard to say without knowing what the gadget needs to accomplish. The function is called LaunchPowerApplet and Windows Mobility Center allows to configure the battery usage settings. It makes sense if the gadget is a battery level meter, it doesn't if it's a solitaire.
BTW, a gadget is «a collection of XML, HTML, Microsoft JScript, and Cascading Style Sheets (CSS) files» so it's technically not JavaScript but JScript (Microsoft's ECMAScript implementation)
Thia is just a windows mobility center exe it will help your gadget to run correctly. it will not harm your computer.
I wouldn't say it was malicious since the application it is trying to launch is a standard windows component.
Why it wants to launch the Mobility Center is a bit of a mystery, but without further context about what this widget is supposed to be able to do we can't comment.
I need to develop a browser app for use on a kiosk touchscreen -> no physical keyboard.
I would like to use Sencha-Touch.
The browser would be running on windows or linux/ununtu.
How can users input text?
I am not tied to sencha touch. I can use another framework (query mobile, etc...) if it offers such a feature.
I'm not sure if this is what you're looking for but it's a jQuery Virtual Keyboard:
http://mottie.github.com/Keyboard/ (Demo)
https://github.com/Mottie/Keyboard
I have never heard of such a feature, and Sencha Touch definitely does not include it. However, it would not take much to introduce that functionality into the framework.
You would need to:
design and create your virtual keyboard. This would be a simple component with a HTML template which had buttons for each of the keys. On each key press, you would fire an event.
in all places you need the keyboard (for example, fields), you would listen to the tap event and then trigger the virtual keyboard. You would then just update the value of the field depending on what the user types.