I am trying to get the color of a pixel on my screen using node.js. I want it to be returned in RGB format, e.g. (255, 0, 0). My current solution is to use screenshot-desktop to screenshot my entire screen in JPG format, decode it to get the raw pixel data, and get the color of a given pixel. However, this lags out my entire computer for 1-2 seconds as it is taking the screenshot. This is unusable as I would like to do this multiple times per second. So my question is: How can I get the color of a given pixel on the screen, without taking a full screenshot?
I am using Linux with X11. There is an X11 library for node.js, so I asssume I should use that to get the pixel color, I'm just not sure how. If you could show me how to do it in C then I can easily use node.js to do the same thing.
Thanks!
Oh my gosh I just figured it out after posting this. I was using robotjs for reading the mouse position and I totally forgot it can do screen stuff too! So, the solution would be to do
var robot = require('robotjs');
var color = robot.getPixelColor(x, y);
X11 solution using x11 node library ( I am the author ):
query windows tree with QueryTree starting at the root window
get every child geometry using GetGeometry request
if your point is not inside any child, use current window id and get 1x1 pixmap from the current image: GetImage(format, currentWindow, x, y, 1, 1, planeMask) ( 2 for format and 0xffffffff for plane mask should work ). Make sure you calculate relative x y position as you travers windows tree.
if child window covers your point query children for that window and repeat again. Note that QueryTree returns windows in bottom to top stacking order so make sure you pick last one covering your point
Once you have 1x1 pixmap from the topmost window under your point - the buffer should contain only color bytes for your image, RGB order and bit mask might depend on red_mask, green_mask, blue_mask from display.screen[0].depths[visual].
If you cache "topmost window" between requests and only start from root when no match anymore the above solution might be much more performant then the one using robotjs ( although much more low level and complicated ) Good luck!
Related
I would like to capture whole screen and then get the coordinates(x, y) of color which i provide. So i can click that coordinate with RobotJS.
A little bit more detail about your setup would be necessary. Since you tagged it, I guess you have access to OpenCV in your project? With OpenCV you can use 'inrange' to filter the channels of your image based on the color, which will give you a binary mask. You can use the mask to find blobs and their central points in your image, where the color predominates with connectedComponents or findContours.
I have an image of size 480x800 pixels and there is a icon on one corner which I need to place. What I want is that to ignore all touches on the transparent areas and detect only the area where the icon is.
I found a solution in SO to this problem but it just tells the code to be used. I need to know exactly where to put that code since I am a beginner and don't know much about cocos2d so I expect a step by step solution.
Cocos2d 2.0 - Ignoring touches to transparent areas of layers/sprites
Do not use glReadPixels because it affected by bugs in android drivers. You can translate CCTouch to CCPoint in image coordinates using convertTouchToNodeSpace, and read image pixel at given point.
Create CCImage from file that contains semi-transparent picture, and read one pixel at tap point; it should be {0,0,0,0} for transparent area.
Don't forget to check that tap is not outside picture, and create pixel index in CCImage::getData() array with formulae unsigned index = x * imageWidth + y.
I'm trying to process an image using ABBYY OCR SDK using the sample code placed in this question but I'm not able get the co-ordinates right for a specific word say "OCR" on the screenshot below.
I want to draw an overlay (yellow rectangle over the word "OCR") and sometimes the rectangle is placed very far away from the actual word.
The XML you get is synthesised according to this schema.
For each recognized character it will contain an instance of charParams element as shown in the answer you linked to. The element will contain the coordinates in page pixels - the same XML also contains a page element:
<page width="..." height="..." resolution="..." originalCoords="...">
where the image width and height are stored. So l and r for each charParams element is in range 0..width-1 of the corresponding page and t and b for each charParams element is in range 0..height-1 of the corresponding page.
Also it's worth mentioning explicitly that all coordinates are in pixels - they are completely resolution-agnostic. This is why whenever you try to highlight anything on an image you have to take zoom into account - the image will likely not be always displayed as is by your device software, but will be downscaled and so you have to map page coordinates onto your zoomed-out image coordinates and highlight appropriately.
Have you checked the DPI of the original image and also check the documentation to make sure the OCR engine is using the same DPI and not returning the image in points or some other measurement system.
It could be that rectangle you are drawing in iOS is not based on pixels but on some other measurement system also.
You just need to work through the process, testing as you go, and work out where the problem is coming from. It is most likely a uniform scaling and the distance from the actual word is proportional to the distance of the word from the top left of the page.
I have a multiviewport OpenGL modeler application. It has three different viewports : perspective, front and top. Now I want to paint a label for each viewport and not succeeding in doing it.
What is the best way to print a label for each different perspective?
EDITED : The result
Here is the result of my attempt:
I don't understand why the perspective viewport label got scrambled like that. And, Actually I want to draw it in the upper left corner. How do I accomplished this, because I think it want 3D coordinate... is that right? Here is my code of drawing the label
glColor3f(1,0,0);
glDisable(GL_DEPTH_TEST);
glDepthMask(GL_FALSE);
glRasterPos2f(0,0);
glPushAttrib(GL_LIST_BIT); // Pushes The Display List Bits
glListBase(base - 32); // Sets The Base Character to 32
glCallLists(strlen("Perspective"), GL_UNSIGNED_BYTE, "Perspective"); // Draws The Display List Textstrlen(label)
glPopAttrib();
I use the code from here http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=13
thanks
For each viewport switch into a projection that allows you to supply "viewport space" coordinates, disable depth testing (glDisable(GL_DEPTH_TEST)) and depth writes (glDepthMask(GL_FALSE)) and draw the text using one of the methods used to draw text in OpenGL (texture mapped fonts, rendering the full text into a texture drawing that one, draw glyphs as actual geometry).
Along with #datenwolf's excellent answer, I'd add just one bit of advice: rather than drawing the label in the viewport, it's usually easier (and often looks better) to draw the label just outside the viewport. This avoids the label covering anything in the viewport, and makes it easy to get nice, cleanly anti-aliased text (which you can normally do in OpenGL as well, but it's more difficult).
If you decide you need to draw the text inside the viewport anyway, I'll add just one minor detail to what #datenwolf said: since you generally do want your text anti-aliased (even if the rest of the picture isn't) you generally want to draw the label after all the other geometry of the picture itself. If you haven't turned on anti-aliasing otherwise, you generally will want to turn it on for drawing the text.
I've been stumbling over this for a while and was wondering if anyone has run into this problem before.
The application I'm working on is divided into multiple data plots and a single timeline at the bottom of the screen. Each plot (which is actually multiple textures layered on top of each other) as well as the timeline is rendered to a separate texture. The timeline texture is rendered first, followed by each of the plot textures from the top of the screen to the bottom. I am using DXUT and DirectX9 (March 2009).
After adding time text to the timeline texture, I noticed that the text was repeated in the title bar of every data plot. Here's a screencap of a portion of the display, it shows just a single plot, but the text repeats on every plot opened:
It seems like it is tied directly to the DrawText being called in the timeline's render function. I do use relative coordinates as the rect being passed to DrawText, but since I've already set the render target to the desired texture it should only affect the current texture. Not every texture rendered afterward.
Has anyone ever run into any problems similar to this using D3DXFont?
EDIT: After some more experimentation, it looks like it has something to do with the Z buffer. By adding D3DCLEAR_ZBUFFER to the clear on each texture surface, the duplicate text is gone. While the problem seems bypassed for now, I'm still curious as to why the Z buffer for a completely separate texture was being written during my DrawText call.
The Z Buffer state is persistent.
For example,
SetDepthStencilSurface(X)
SetRenderTarget(A)
Draw()
SetRenderTarget(B)
Draw()
Both Draw calls will use the same depth buffer.
The DrawText is not changing the depth buffer that you have set. It assumes you meant to do what you did.