I'm fuzzy on the relationship between Windows Imaging Component (WIC) and GDI+. I've done some work in the past that showed that, for example, WIC produces visually superior GIF encodings, but I'm surprised I don't see more people using it for image processing vs. GDI+. I know it doesn't have GDI+'s draw operations, but for encoding/decoding it seems superior. So why don't we see a migration?
The relationship (or rather the difference between) WIC and GDI+ is that WIC is an extensible imaging codec framework which allows applications implementing the framework to receive support for new image formats via provided codecs. GDI+ is a core component of Windows which supports draw operations such as lines, fonts, gradients, etc.
While GDI+ has native support for several common image formats, WIC codecs can be provided for any image format.
I understood that GDI+ uses WIC to perform certain tasks. At least, in Windows 7 it does.
Please consider the following code:
image.Write(target, ImageFormat.Gif);
When I run this code under Windows XP it will use the GDI+ Gif encoder to write the image as a Gif. When I run the same code under Windows 7 it will use the WIC Gif Encoder.
Related
I've noticed that the GPU I'm running on Vulkan doesn't support so many of the R8G8B8_UINT formats but does with the R8G8B8A8 FORMATS. Also the same thing with many others like R32G32B32_SFLOAT. I've noticed that's also the with other GPUs I've seen on OpenGPU database.
Is this normal? Why is this so? Is it normal with other graphics APIs? Is it to align values/texels to a "round/nicer/aligned" number of bytes? How are you supposed to get around this? I'm having trouble seeing how throughout your code you'll be interacting with images that you have no idea what colour format they are, which complicates things both on host code and in shaders.
Also if I have 3 channel colour format image on the host and I want to use with Vulkan and say R8G8B8 or R32G32B32, do I need to loop through the image manually and rearrange the texels?
24bpp formats are hard to optimize in graphics hardware. This is why there is no "R8G8B8" format even defined for the DirectX Graphics Infrastructure pixel formats used for Direct3D 10, 11.x, and 12. Almost all the work is around optimizing 32bpp formats which is why RGBA32, BGRA32, etc. are much more commonly supported. 64bpp and 128bpp formats are multiples of 32-bits, so it makes just the 24bpp format the 'odd-man out' in many cases. You often can find a B8G8R8X8 (32-bit) format which is about as close as you get to 24bpp.
Some hardware will find ways to support 24bpp formats (which were supported by older Direct3D releases for example), but it's generally less efficiently rendered.
Similar issues arise with 96bpp formats (R32G32B32 floating-point). For Direct3D Hardware Feature Levels, this format is always optional and when implemented it's typically done as three 32-bit floating-point planes, one for each color channel.
we are used to develop for iOS platforms using Cocos2D, and there we have plenty of choice when it comes to image formats.
I cannot find any reference about how should we save our image assets for use within Corona, especially in relation to performance.
What I found out so far is only that PVR format is not supported (obviously because of the cross-platform support of the SDK). I seemed to find small hints that Corona uses 32bit pixel format for everything, so we cannot use different pixel formats as we do with Cocos2D.
Any CoronaSDK people out there that can answer me and/or redirect me to some documentation containing more details about this?
Thanks!
I have managed to find the confirmation in the Anscamobile forums - indeed, Corona SDK currently uses only 32bit pixel format for all textures, so there is no reason to save your images as 16-bit PNGs other than the size on disk of the image.
Corona SDK now supports indexed pngs, 16-bit pngs, grayscale pngs... In fact any sort of png that you want to use.
But the only performance increase is the speed to load in memory (it GREATLY improves loading times) and space on storage.
I am using ImageOptim and ImageAlpha to optmize my PNGs :)
[Note: This is a rewrite of an earlier question that was considered inappropriate and closed.]
I need to do some pixel-level analysis of television (TV) video. The exact nature of this analysis is not pertinent, but it basically involves looking at every pixel of every frame of TV video, starting from an MPEG-2 transport stream. The host platform will be server-class, multiprocessor 64-bit Linux machines.
I need a library that can handle the decoding of the transport stream and present me with the image data in real-time. OpenCV and ffmpeg are two libraries that I am considering for this work. OpenCV is appealing because I have heard it has easy to use APIs and rich image analysis support, but I have no experience using it. I have used ffmpeg in the past for extracting video frame data from files for analysis, but it lacks image analysis support (though Intel's IPP can supplement).
In addition to general recommendations for approaches to this problem (excluding the actual image analysis), I have some more specific questions that would help me get started:
Are ffmpeg or OpenCV commonly used in industry as a foundation for real-time
video analysis, or is there something else I should be looking at?
Can OpenCV decode video frames in real time, and still leave enough
CPU left over to do nontrivial image analysis, also in real-time?
Is sufficient to use ffpmeg for MPEG-2 transport stream decoding, or
is it preferable to just use an MPEG-2 decoding library directly (and if so, which one)?
Are there particular pixel formats for the output frames that ffmpeg
or OpenCV is particularly efficient at producing (like RGB, YUV, or YUV422, etc)?
1.
I would definitely recommend OpenCV for "real-time" image analysis. I assume by real-time you are referring to the ability to keep up with TV frame rates (e.g., NTSC (29.97 fps) or PAL (25 fps)). Of course, as mentioned in the comments, it certainly depends on the hardware you have available as well as the image size SD (480p) vs. HD (720p or 1080p). FFmpeg certainly has its quirks, but you would be hard pressed to find a better free alternative. Its power and flexibility quite impressive; I'm sure that is one of the reasons that the OpenCV developers decided to use it as the back-end for video decoding/encoding with OpenCV.
2.
I have not seen issues with high-latency while using OpenCV for decoding. How much latency can your system have? If you need to increase performance, consider using separate threads for capture/decoding and image analysis. Since you mentioned having multi-processor systems, this should take greater advantage of your processing capabilities. I would definitely recommend using the latest Intel Core-i7 (or possibly the Xeon equivalent) architecture as this will give you the best performance available today.
I have used OpenCV on several embedded systems, so I'm quite familiar with your desire for peak performance. I have found many times that it was unnecessary to process a full frame image (especially when trying to determine masks). I would highly recommend down-sampling the images if you are having difficultly processing your acquired video streams. This can sometimes instantly give you a 4-8X speedup (depending on your down-sample factor). Also on the performance front, I would definitely recommend using Intel's IPP. Since OpenCV was originally an Intel project, IPP and OpenCV blend very well together.
Finally, because image-processing is one of those "embarrassingly parallel" problem fields don't forget about the possibility of using GPUs as a hardware accelerator for your problems if needed. OpenCV has been doing a lot of work on this area as of late, so you should have those tools available to you if needed.
3.
I think FFmpeg would be a good starting point; most of the alternatives I can think of (Handbrake, mencoder, etc.) tend to use ffmpeg as a backend, but it looks like you could probably roll your own with IPP's Video Coding library if you wanted to.
4.
OpenCV's internal representation of colors is BGR unless you use something like cvtColor to convert it. If you would like to see a list of the pixel formats that are supported by FFmpeg, you can run
ffmpeg -pix_fmts
to see what it can input and output.
For the 4th question only:
video streams are encoded in a 422 format: YUV, YUV422, YCbCr, etc. Converting them to BGR and back (for re-encoding) eats up lots of time. So if you can write your algorithms to run on YUV you'll get an instant performance boost.
Note 1. While OpenCV natively supports BGR images, you can make it process YUV, with some care and knowledge about its internals.
By example, if you want to detect some people in the video, just take the upper half of the decoded video buffer (it contains the grayscale representation of the image) and process it.
Note 2. If you want to access the YUV image in opencv, you must use ffmpeg API directly in your app. OpenCV force the conversion from YUV to BGR in its VideoCapture API.
I've developed a site that requires high quality resizing of uploaded photos. The site works perfectly under ASP.NET on Windows. This afternoon I tried running it under Mono/Apache/Ubuntu 10.10. To my surprise, it worked - except for the image resampling.
It seems the libraries underlying Mono's Graphics/GDI+ implementation don't implement the bi-cubic interpolation mode. (See Mono Ignores Graphics.InterpolationMode? ).
So I'm looking for a library that can do high quality image resizing. I'm willing to put in the effort to interop to it from C# since this is important functionality and I'd like to be able to run under mono if at all possible. I don't really need any other graphics processing capabilities - just resize.
Follow up: as suggested below ImageMagick works really well for this and was pretty easy to interop to. More details here: http://www.toptensoftware.com/blog/posts/17/high-quality-image-resampling-in-monolinux
ImageMagick is both a command line tool and a library with high quality interpolation and antialiasing algorithms.
I have Windows CE 5.0 device and it doesn't support any hardware accelearation.
I am looking for some good 2d graphics library to do following things.
I prefer backend programming in Compact .Net Framework.
Drawing fonts with antialiasing.
drawing lines, and simple vector objects with antialiasing.
I am not doing animation, so i don't care about frames per seconds performance.
i have looked into following libraries, but nothing suits me.
opengl (vincent 3d software rendering) - works, but api is very low level and complex.
openvg - no software implementation for windows ce.
Cairo - api is very neat, but no wince build.
Adobe Flash - installs as browser plugin , no activex support in wince.
Anti-aliased fonts in .Net CF 2.0+ can be done with Microsoft.WindowsCE.Form.LogFont -- after creating your logfont, you can use it with any WinForms widget's .Font property by converting it using System.Drawing.Font.FromLogFont().
...you might need to enable anti-aliasing in the registry for these to render properly, see this MSDN article for the right keys: [http://msdn.microsoft.com/en-us/library/ms901096.aspx][1].
There was a decent implementation of GDI+ for .Net CF 1.0 called "XrossOne Mobile GDI+", it's not longer supported, but you can get the source code here: http://www.isquaredsoftware.com/XrossOneGDIPlus.php -- Run it through the import wizard on VS2008 to build it for later versions of CF. I liked this library for its alpha transparency support without hardware acceleration, rounded rectangles and gradient support.
Someone was advertising this library in some forum. It's for Windows Mobile, but you can check it out. I have no experience with it.
link
I have Google's skia library compiling under WindowsCE, although I haven't done much with it yet :) It wasn't too hard to get working. It does support a OpenGL/ES backend.
There is also AGG (Anti Grain Geometry) which is a heavy C++ library based on templates.