Modify image displayed by OS on screen with OpenGL? - linux

What I want to achieve is to real time transform the image displayed by my linux system i.e. I would like to be able to achieve true gray scale so for every pixel of the screen we would have each rgb value set to be equal to average of what would be normally displayed.
I guess it would be possible to take the buffer, modify it in C and save it back, but that approach would waste lots of systems resources. Is there other way?

Related

Color management - what exactly does the monitor ICC profile do, and where does it sit in the color conversion chain?

I'm reading/watching anything I can about color management/color science and something that's not making sense to me is the scene-referred and display-referred workflows. Isn't everything display-referred, because your monitor is converting everything you see into something it can display?
While reading this article, I came across this image:
So, if I understand this right to follow a linear workflow, I should apply an inverse power function to any imported jpg/png/etc files that contain color data, to get it's gamma to be linear. I then work on the image, and when I'm ready to export, say to sRGB and save it as a png, it'll bake in the original transfer function.
But, even while it's linear, and I'm working on it, is't my monitor converting everything I see to what I can display? Isn't it basically applying it's own LUT? Isn't there already a gamma curve that the monitor itself is applying?
Also, from input to output, how many color space conversions take place, say if I'm working in the ACEScg color space. If I import a jpg texture, I linearize it and bring it into the ACEScg color space. I work on it, and when I render it out, the renderer applies a view transform to convert it from ACEScg to sRGB, and then also what I'm seeing is my monitor converting then from sRGB to my monitor's own ICC profile, right (which is always happening since everything I'm seeing is through my monitor's ICC profile)?
Finally, if I add a tone-mapping s curve, where does that conversion sit on that image?
I'm not sure your question is about programming, and the question has not much relevance to the title.
In any case:
light (photons) behave linearly. The intensity of two lights is the sum of the intensity of each light. For this reason a lot of image mangling is done in linear space. Note: camera sensors have often a near linear response.
eyes see nearly as with a gamma exponent of 2. So for compression (less noise with less bit information) gamma is useful. By accident also the CRT phosphors had a similar response (else the engineers would have found some other methods: in past such fields were done with a lot of experiments: feed back from users, on many settings).
Screens expects images with a standardized gamma correction (now it depends on the port, setting, image format). Some may be able to cope with many different colour spaces. Note: now we have no more CRT, so the screen will convert data from expected gamma to the monitor gamma (and possibly different value for each channel). So a sort of a LUT (it may just be electronically done, so without the T (table)). Screens are setup so that with a standard signal you get expected light. (There are standards (images and methods) to measure the expected bahavious, but so ... there is some implicit gamma correction of the gamma corrected values. It was always so: on old electronic monitor/TV technicians may get an internal knob to regulate single colours, general settings, etc.)
Note: Professionals outside computer graphic will use often opto-electronic transfer function (OETF) from camera (so light to signal) and the inverse electro-optical transfer function (EOTF) when you convert a signal (electric) to light, e.g. in the screen. I find this way to call the "gamma" show quickly what it is inside gamma: it is just a conversion between analogue electrical signal and light intensity.
The input image has own colour space. You now assume a JPEG, but often you have much more information (RAW or log, S-log, ...). So now you convert to your working colour space (it may be linear, as our example). If you show the working image, you will have distorted colours. But you may not able to show it, because you will use probably more then 8-bit per channel (colour). Common is 16 or 32bits, and often with half-float or single float).
And I lost some part of my answer (after last autosave). The rest was also complex, but the answer is already too long. In short. You can calibrate the monitor: two way: the best way (if you have a monitor that can be "hardware calibrated"), you just modify the tables in monitor. So it is nearly all transparent (it is just that the internal gamma function is adapted to get better colours). You still get the ICC, but for other reasons. Or you get the easy calibration, where the bytes of an image are transformed on your computer to get better colours (in a program, or now often by operating system, either directly by OS, or by telling the video card to do it). You should careful check that only one component will do colour correction.
Note: in your program, you may save the image as sRGB (or AdobeRGB), so with standard ICC profiles, and practically never as your screen ICC, just for consistency with other images. Then it is OS, or soft-preview, etc. which convert for your screen, but if the image as your screen ICC, just the OS colour management will see that ICC-image to ICC-output will be a trivial conversion (just copying the value).
So, take into account that at every step, there is an expected colour space and gamma. All programs expect it, and later it may be changed. So there may be unnecessary calculation, but it make things simpler: you should not track expectations.
And there are many more details. ICC is also use to characterize your monitor (so the capable gamut), which can be used for some colour management things. The intensions are just the method the colour correction are done, if the image has out-of-gamut colours (just keep the nearest colour, so you lose shade, but gain accuracy, or you scale all colours (and you expect your eyes will adapt: they do if you have just one image at a time). The evil is in such details.

Object tracking for velocity measurement

I am looking for a real time image processing to measure the velocity and output it to another control system.
I have attached an image of a yellow stripe. This has markings on the surface that I would like to automatically detect and use for calculation. In the first step the material moves only in one direction. Here for example to the right. For me only the horizontal part of the movement is of interest, so quasi only the velocity along an x-axis. But the material moves relatively fast. At maximum speed in 28 ms the current mark (spike) is at the position of the one in front of it.
The idea is to use a Raspberry Pi 4 with a camera at the maximum of 120 fps. So every 8.3 ms a picture is generated and should make it possible to clearly detect the movement of the marks.
My questions are:
Is it possible to process the images and detaction that fast to get the velocity in nearly real time? And which algorithem should I use for this configuration? Because it would be best if I could use two or three markers per image and average the velocity of them.
And I would like to use the velocity as an input signal for another system. What is the easiest and fastest way to send the information directly to another control system?

Gnuplot - best choice to plot very wide graph

I have a very large set of data, they are hourly measurements and I have to plot 6 years of recordings, so we have about 8760 * 6 ~ 50'000 data.
x-axes is in date-time format.
Up to now I plotted a single year data, so about 10'000 data.
In a single "screen" I want to see about 30-50 hours width, so I have setted a width of 60'000:
set terminal png size 60000,2000
In that way, gnuplot creates a ridiculously wide PNG image. Anyway it is possible to visualize it with a browser, zoom it to 100% and scroll left and right to easily analyze my data.
Now, if I have to monitor 6 years instead of 1, I need an output image 6 times greater. So I tried:
set terminal png size 360000,2000
output image is now very wide and my browser cannot open it. I think due to the big size. However I can open it with an image viewer like "gwenview".
Anyway browser would be better, because I have to send this graph to other people.
Which alternative output "terminal" could I use in gnuplot to obtain a more manageable image?
EDIT:
I tried SVG terminal. It seems better to produce images of this size:
set terminal svg size 360000,800
set output 'plot.svg'
The image plot.svg is about 11 MB, very big, but browser can open it without any problem.
I have to visualize this large amount of data just for a "quality quick check" and scroll from the left to the right within the browser is easy enough. If I has to open many split image files it would be more complicated and slow.
Any other idea is welcome, thanks!

Direct3D 9 Backbuffer sampling

I'm locking the backbuffer in direct3D 9 and copying an image to it. I noticed on one computer that when the image is stretched to the screen, it becomes blurry. On another computer I tested on, it's completely unfiltered (pixelated). Is there a way to specify how the backbuffer is sampled to the screen, or is it controlled by something else?
I've tried
Device->SetSamplerState(0, D3DSAMP_MAGFILTER, D3DTEXF_POINT);
However it had no effect; I think it only affects textures.
SetSamplerState does not affect how the backbuffer is drawn to the screen. AFAIK most drivers will use point sampling, which means pixels will be lost or doubled, resulting in bad quality. BTW, what was the GPU/driver on the machine where it looked fine (you can't/shouldn't depend on this behavior everywhere)?
The right way to do this is to copy the image to a texture and render a screen aligned quad so you can use hardware sampling to smooth the result for you.
If for whatever reason you cannot use a texture + rendering pass, you can use IDirect3DDevice9::StretchRect to filter the image when copying to the backbuffer. To actually load the image from system memory, you'll have to use another surface, either locking and copying it or using D3DXLoadSurfaceFromMemory.

Capturing only pixels from Google Glass camera

I would like to capture only a few pixels from the Google Glass camera at regular intervals, to obtain color data over time. Is there a way, to save battery life, to only capture a few pixels rather than take a full image every time and have to post-render it (which is much more intensive and battery-consuming)? Perhaps this is configured on the hardware level, and thus I cannot do such a thing.
As an alternative, I was hoping the light sensor would give RGB data, but it appears to be a monochromatic light level that is provided in units of lux.

Resources