1. The problem I've encountered
Hi, I'm currently making a desktop application with Electron.js. Meanwhile, I have needed a feature of taking a screenshot (including the mouse cursor) but this is a problem for me because I do not know how to do this.
I think the reason for me not to be able to solve this problem is that I have no knowledge about operating systems. I think the meaning of "taking a screenshot" is "getting the image data displayed on the computer monitor", but how I can access to that?
2. What I've tried or considered
At first I tried Electron.BrowserWindow.capturePage() but its result didn't meet my want. It is because of two reasons: 1) My application has a transparent background and wherever area of transparency becomes black if I take a screenshot. 2) Mouse cursor is not captured together.
Meanwhile, I am aware of the existence of some APIs such as Screen Capture API and Media Capture and Streams API (in web browsers) and perhaps I can give it a try because I'm using Electron.js and Electron.js uses Chromium web browser and web browsers have implementations of those APIs.
However, it is still a problem that what those APIs handle is media streams (= video), which is not suitable for my case. Of course I think it is possible to take only one frame(?) out of a media stream somehow, but I think it is an overwork, given that what I desire for is just a single screenshot.
Meanwhile, because Electron.js also uses Node.js, I think it is also somehow possible to call Windows API (maybe via Foreign Function Interface?) or to invoke child_process.exec() in order to take a screenshot.
3. The question I would like to ask
How can I access to the monitor image data? So that I can implement "the screenshot feature which meets my requirements--see-through & mouse cursor" (if uses of third-party libraries needed, as least as possible).
What calculates a final image data which is going to be displayed on my computer monitor? It seems that it is a work of my graphics card because my monitor and graphics card are connected each other with a cable.
4. Miscellaneous curiosities (not much related to the question)
...Yet it is another curiosity that how, why, and where the transparent area is processed as #000000 color.
Meanwhile it is also interesting that there are some programs which do not allow me to take a screenshot of contents on them--the area where the programs are located looks black. How could the developers of this kind of programs implement this?
Thank you for reading my question.
After some internet searches, I found it difficult to access and get display data (specifically, video ram data from my graphics card). So I decided to use a workaround--It is a well-known aphorism that 'all loads lead to Rome'.
Which means,
See-through screenshots can be achieved by either "using native screenshot feature (the PrintScreen key)" or "using some scripts that take a picture of the entire screen".
Screenshots with mouse cursor can be achieved by adding (= overlaying) mouse cursor image at the coordinates where my mouse cursor is located at.
However, in my case I do not actually need to save screenshots as files, so I think it is enough to just draw a custom mouse cursor image, hide the original mouse cursor image, make it follow the mouse cursor, and take a screenshot with a manual key press. (I think it is also a feasible option to take a screenshot with the PrintScreen key press, get the screenshot data from the clipboard and do some image processings like adding effects relevant to a mouse cursor.
※ I saw a code that simulates "key press" (SendKey()) in order to take a screenshot and I think this is a good approach because of no manual key press needed.
I think whom interested in this topic may find it helpful from the following links (the numerical order does not represent importance):
Keywords mentioned: GetDC(), BitBlt(), CAPTUREBLT flag, GDI
What is the best way to take screenshots of a Window with C++ in Windows?
How can I take a screenshot in a windows application?
Keywords mentioned: DirectX, buffer
Fastest method of screen capturing on Windows
How to save backbuffer to file in DirectX 10?
Keywords mentioned: mouse cursor, cursor image, hot spot
Capture screen shot with mouse cursor
C# - Capturing the Mouse cursor image
Python - Take screenshot including mouse cursor
Keywords mentioned: PowerShell, CopyFromScreen()
How can I do a screen capture in Windows PowerShell?
Capture screenshot of active window?
Q/A about accessing to video memory
DRM Access the whole video memory
raw video memory, video driver Access the whole video memory through OpenGL programming
graphics RAM API to get the graphics or video memory
direct data write to video memory
Direct video buffer access
How to write data directly into video memory?
Is direct video card access possible? (No API)
Related
I am creating an application that allows you to take pictures. I wonder how I can access the camera itself. To take the photos I use the package image_picker, but with that I can only take the photos, and I have no influence on the camera itself. For example, I would like to adjust the size of the camera screen, turn the flash on automatically, and indicate some sort of guideline of where the object to be photographed should be. See the photo below for an example.
Can someone tell me how I could make those adjustments in the camera itself, instead of just taking a picture with it?
Use the camera plugin, it provides even more features such as making snapshots, recording video, and accessing the image within the app itself.
Need to convert mouse coordinates into PS position or row and column on mainframe emulator.
I'm using Whllapi to connect and automate mainframe emulator. I need to find a underlying field when user move mouse or click on a field at emulator screen. To identify a field on mainframe emulator i need to know row and column or PS position. I need to convert mouse position (in pixels) to emulator row and column. But there is no API in whllapi that provides such functionality.
I used whllapi api "QueryWindowCoordinates" and 'WindowStatus" to get emulator window coordinates and window hwnd. I used that handle in window API "SreenToCleint" to get mouse position with respect to emulator window. But i'm unable to translate those co-ordinates into emulator rows and column. I tried many algorithm but unable to get consistent results. I need translate mouse position precisely into PS position.
Whllap documentation has mentioned "WindowStatus" api to return font sizes for x and y but i'm unable to retrieve any value from Rumba emulator. In order to get fond height and width, I also tried window api 'GetTextMetrcies' but that was not much help either.
IBM Personal Communications for Windows offers a "Get Mouse Input" DDE function which returns PS position data (row, column) when the user clicks the mouse. There's another DDE function, Set Mouse Intercept Condition, to establish which mouse click(s) (left, right, middle, single, double, etc.) should be intercepted. I don't see a direct way to capture mere mouse movements using the DDE functions, but it might be possible (if you're very careful in your Windows programming) if you generate simulated, rate limited mouse clicks and only when the mouse pointer is moved within the emulator window.
Perhaps Rumba offers similar functions? Rumba evidently has some DDE functions, but I haven't found any DDE function reference for Rumba publicly available online.
One possible caveat is that the DDE functions are 32-bit (and 16-bit functions are also still supported since there's still some 16-bit code running on 32-bit Windows). You can use the 32-bit functions if you're doing 64-bit Windows programming, but of course you'll need to know how to do that if you don't already. Another caveat is that you probably ought to test whatever you're doing for user accessibility, for example with screen reading tools that aid vision impaired users.
Another possible approach is to embed the whole emulator within your own "wrapper" application since that might give you more programming power and control. IBM offers both ActiveX/OLE-style embedding and Java-style embedding (their "Host Access Class Libraries," a.k.a. HACL). Rumba might offer something broadly similar.
And yet another possible approach is to shift the interactions with these applications toward APIs and in favor of brand new, more portable user interfaces, usually Web and mobile interfaces. There are myriad ways to do that. If you still need terminal (3270)-driven automation -- maybe because the application source code is lost or it's otherwise really hard to create useful APIs for it? -- there are a variety of ways to shift that automation into the backend. For example, CICS Transaction Server for z/OS comes with multiple terminal automation technologies as standard included features. Look for references to "3270 bridge" and "FEPI" in IBM's Knowledge Center for CICS to explore that range of choices.
I'm aware of the existence of DDS files which allow programming of display graphics on the as/400, but is there another way?
Specifically, what I want to do is manipulate the terminal buffer directly to be able to display anything else than just text.
For example, the terminal looks like that:
Let's say, in memory, there would be a two dimensional char array: text[20][80] for the text menu and lower than that, there would be a pixel buffer array of size [200][800].
Is there a way to access either of those arrays directly?
I would like to be able to create a displayable menu entirely in C without the need of a display file and also display other kind of graphics (images) directly in the pixel buffer.
Is there a way to access either of those arrays directly?
That's easy enough, though a "display file" that has no formatted fields will still be needed. The 'file' will be the connection between the program and the physical device (or the emulator). You can define a single large area that contains whatever "text" you want your program to put into it. This can even include display field attributes that delimit input areas.
For the most control, the DDS USRDFN keyword is appropriate. But for simple stuff like lists of menu items, almost any large text field can be output to.
Outputting simple text is easy. For detailed stuff like USRDFN formatting, detailed understanding of the 5250 protocol is needed.
One kind of alternative would be to use User Interface Manager (UIM) APIs to update a PANEL's "text area" (:TEXT) via its USREXIT= application program. The UIM handles everything as far as any "display file" definition and actual I/O goes. The UIM can be thought of as a HTML interface for 5250 and uses a very similar markup language to define PANELs.
Another alternative is the Dynamic Screen Manager (DSM) APIs. These give much finer control than the UIM or DDS methods (though DDS USRDFN gets very close). But as with USRDFN, actual device control will require 5250 protocol knowledge.
...and also display other kind of graphics (images) directly in the
pixel buffer.
There is no "pixel buffer" for 5250 nor even 'pixels'. It's a character-based protocol, like telnet. If you're going for images or 'pixels', you're into browser interfaces, or perhaps Java and NAWT, or X-windows, etc.
Now, granted that with TCP/IP and sockets, you can do essentially anything that you're able to program. Whatever you can figure out how to do, including downloading/installing 3rd-party code libraries, you can do -- within the network restrictions surrounding your server. But it is in fact a server, so GUI kinds of apps generally shouldn't run on it. That's the same as for almost all types of servers. Code the GUI on the client system rather than the server. But you can do it if you really want to.
I'm not sure why you'd want to do this...
Now-a-days, it'd be much easier to simply generate your output as HTML and serve it up via the integrated apache web server.
But if you really want to do graphics via 5250, it can be done...theoretically at least. In 20+ years on the platform, I've never seen it.
But way back when (1994?), IBM added support for Graphical Data Display Manager (GDDM) and Presentation Graphics APIs into OS/400. "GDDM is a means of
displaying, printing, or plotting pictures. Presentation Graphics routines are a
means of displaying, printing, or plotting business charts."
The support is still in the OS. However, client side support is NOT available in IBM i Access for Windows or the most recently released client, IBM Access Client Solutions (ACS). It appears that the standalone IBM Personal Communications product may support GDDM.
For complete control of the character buffer, take a look at the Dynamic Screen Manager (DSM) APIs. The DSM APIs are "a set of screen I/O interfaces that provide a dynamic way to create and manage screens for the Integrated Language Environment® (ILE) high-level languages. Because the DSM interfaces are bindable, they are accessible to ILE programs only."
There is a way to do it in ILE C/C++. This was very fun to investigate since I haven't tried it myself.
The only documentation on it (page 183+) I could find is from 5.1, but you are able to cross reference the functions used to this 7.3 manual (possibly page vii/7) to see if they're still used the same.
Hope this helped!
Using breakpoints, I'm trying to figure out the working of a complex app which uses DirectX buffers.
I see many places where the Blt command draws to the back buffer/surface, and I need to know what exactly it is drawing at that point in the code.
Unfortunately, I'm able to see what was drawn only when the entire frame is drawn and the buffers are flipped.
Isn't there a way to see the back buffer/surface while it is being drawn? Before it gets flipped?
You can use PIX for Windows to debug rendering issues or performance bottlenecks in the GPU. PIX is a profiler that will give you very detailed info on what is happening at each step while rendering.
So it will allow you to see what is rendered in the back buffer at each Direct3D call. You can also select a given pixel in the final image and see what happened in the history of that pixel (shaders applied to it for example).
Here are a couple of tutorials:
http://blogs.msdn.com/b/manders/archive/2006/12/15/a-painless-introduction-to-pix-for-windows.aspx
http://www.garagegames.com/community/blogs/view/14251
http://www.youtube.com/watch?v=-3PISvNTeB0&feature=player_embedded
I'm currently designing my first ever GUI for Windows. I'm using MFC and Visual Studio 2008. The monitor I have been designing my program on has 1680x1050 native resolution. If I compile and send my program to one of my coworkers to run on their computer (generally a laptop running at 1024x768), my program will not fit on their screen.
I have been trying to read up on how to design an MFC application so that it will run on all resolutions, but I keep finding misleading information. Everywhere I look it seems that DLUs are supposed to resize your application for you, and that the only time you should run into problems is when you have an actual bitmap whose resolution you need to worry about. But if this is the case, why will my program no longer fit on my screen when I set my monitor to a lower resolution? Instead of my program "shrinking" to take up the same amount of screen real estate that it uses at 1680x1050, it gets huge and grainy.
The "obvious" solution here is to set my resolution to 1024x768 and redesign my program to fit on the screen. Except that I've already squished everything on my dialogs as much as possible to try and get my program to fit on screen running at 1024x768. My dialog fonts are set to Microsoft Sans Serif 8 but still appear huge (much larger than 8 points) when running at 1024x768.
I know there HAS to be a way to make my program keep the same scaling... right? Or is this the wrong way to approach the problem? What is the correct/standard way to go about designing an MFC program so that it can run on many resolutions, say 800x600 and up?
I assume your application GUI is dialog based (the main window is a dialog)?
In that case you have a problem, because, as you discovered, MFC has no support for resizing a dialog correctly. Your options are:
Redesign your GUI to use a SDI or MDI GUI.
Use a dialog resize extension. There are many available, for some very good suggestions see this question. Another options are this one and this one.
Don't use MFC. wxWidgets has much better support for dialog resizing.
MFC is only a thin wrapper over the Windows API. They both make an assumption which is hardly ever true: if you have a higher resolution screen, you'll adjust the DPI or font size in Windows to get larger characters. Most of the time, a larger screen size means a larger physical monitor, or a laptop where you want to squeeze as much information into a small screen as possible; people value more information over greater detail. Thus the assumption fails.
If you can't squeeze your entire UI into the smallest size screen you need to support, you'll have to find another way to make it smaller. Without knowing anything about your UI, I might suggest using tabs to group the controls into pages.
I've had good luck making my windows resizable, so that people with larger screens can see more information at once. You need to do this the hard way, responding to the WM_SIZE message to the window and deciding which controls should be made larger and which ones should just move.
There is no automatic way to resize the content of your dialogs when resolution changes. So, you need to set some boundaries.
Option 1.
If you are developing your app for customers, pick one minimum resolution (like 1024x7678), redesign you dialogs so that everything fits. Maybe break up some into several, or use tab strip control.
Option 2.
Create separate dialog forms for each resolution you'd like to support, but use the same class to handle it. At runtime detect resolution and use the appropriate form.
Option 3.
Write your own resizing functionality, so that user could adjust the size of your dialogs to his liking.