Related
I've just started looking to graphics tools and how it could be made to be faster without compromising performance and then this question came to mind: what is the difference of filling the screen pixels with some colors coordinated by some optimized code to deal with "graphics" versus the actual 2D/3D graphics tools such as OpenGL, Unity, etc.?
What I want to mean by that is the following: I was seeing this video about .kkrieger fps game that used only 96kB of memory. Of course it produced huge amount of need for CPU and GPU to perform well. But what if instead of compact size files, there was a way to actually do some nice graphics with great performance without a need for a very expensive CPU+GPU combo? Is that possible anyway?
My application produces an "animation" in a per-pixel manner, so i need to efficiently draw them. I've tried different strategies/libraries with unsatisfactory results especially at higher resolutions.
Here's what I've tried:
SDL: ok, but slow;
OpenGL: inefficient pixel operations;
xlib: better, but still too slow;
svgalib, directfb, (other frame buffer implementations): they seem perfect but definitely too tricky to setup for the end user.
(NOTE: I'm maybe wrong about these assertions, if it's so please correct me)
What I need is the following:
fast pixel drawing with performances comparable to OpenGL rendering;
it should work on Linux (cross-platform as a bonus feature);
it should support double buffering and vertical synchronization;
it should be portable for what concerns the hardware;
it should be open source.
Can you please give me some enlightenment/ideas/suggestions?
Are your pixels sparse or dense (e.g. a bitmap)? If you are creating dense bitmaps out of pixels, then another option is to convert the bitmap into an OpenGL texture and use OpenGL APIs to render at some framerate.
The basic problem is that graphics hardware will be very different on different hardware platforms. Either you pick an abstraction layer, which slows things down, or code more closely to the type of graphics hardware present, which isn't portable.
I'm not totally sure what you're doing wrong, but it could be that you are writing pixels one at a time to the display surface.
Don't do that.
Instead, create a rendering surface in main memory in the same format as the display surface to render to, and then copy the whole, rendered image to the display in a single operation. Modern GPU's are very slow per transaction, but can move lots of data very quickly in a single operation.
Looks like you are confusing window manager (SDL and xlib) with rendering library (opengl).
Just pick a window manager (SDL, glut, or xlib if you like a challenge), activate double buffer mode, and make sure that you got direct rendering.
What kind of graphical card do you have? Most likely it will process pixels on the GPU. Look up how to create pixel shaders in opengl. Pixel shaders are processing per pixel.
I used DirectDraw in C and C++ years back to draw some simple 2D graphics. I was used to the steps of creating a surface, writing to it using pointers, flipping the back-buffer, storing sprites on off-screen surfaces, and so on. So today if I want write some 2D graphics programs in C or C++, what is the way to go?
Will this same method of programming still apply or do I have to have a different understanding of the video hardware abstraction?
What libraries and tools are available on Windows and Linux?
What libraries and tools are available on Windows and Linux?
SDL, OpenGL, and Qt 4 (it is gui library, but it is fast/flexible enough for 2D rendering)
Will this same method of programming still apply or do I have to have a different understanding of the video hardware abstraction?
Normally you don't write data into surface "using pointers" every frame, and instead manipulate/draw them using methods provided by API. This is because the driver will work faster with video memory than if you transfer data from system memory into video memory every frame. You still can write data into hardware surface/texture (even during every frame), if you have to, but those surfaces may need to be treated in special way to get optimal performance. For example, in DirectX you would need to tell the driver that surface data is going to change frequently and that you're going only to write data into surface, never reading it back. Also, in 3D-oriented APIs (openGL/DirectX) rendering surface on the other surface is a somewhat "special case", and you may need to use "Render Targets"(DirectX) or "Framebuffer Objects"(OpenGL). Which is different from DirectDraw (where, AFAIK, you could blit anything onto anything). The good thing is that with 3D api you get incredibly flexible way of dealing with surfaces/textures - stretching, rotating, tinting them with color, blending them together, processing them using shaders can be done on hardware.
Another thing is that modern 3D apis with hardware support frequently don't operate on 8bit palleted textures, and prefers ARGB images. 8 bit surfaces with palette may be emulated, when needed, and 2D low-level apis (SDL, DirectDraw) provide them. Also you can emulate 8bit texture on hardware using fragment/pixel shaders.
Anyway, if you want "old school" cross-platform way of using surfaces (i.e. "write data every frame using pointers" - i.e. you need software renderer or something), SDL easily allows that. If you want higher-level, more flexible operations - Qt 4 and OpenGL are for you.
On Linux you could use OpenGL, it is not only used for 3D support but also supports 2D.
SDL is also pretty easy to use, out of the box. It is also cross-platform, and includes (and has a lot of) plugins available to handle your needs. It interfaces nicely with openGL as well should you need 3D support.
Direct2D on Windows.
EGLOutput/EGLDevice or GEM depending on the GPU driver for Linux.
I've developed a running Super Mario Sprite using Visual C++ 6.0 and DirectX. But this isn't very satisfying to me (raping a 3D-Multimedia-framework for displaying a 2D sprite only), so I would like to be able to program an animated sprite using C and assembler only.
So, when looking to old games (Wolfenstein, for example) it looks that most of the game is written in C and everytime it comes to graphics output there is the use of assembler.
Unfortunatly when trying to use this old assembler code there is always the error message "NTVDM.exe has found an invalid instruction" so this things don't seem to work nowadays.
Is there any tutorial on graphics programming in assembler that is still usefull?
(I don't want to use any bloated frameworks or libraries, I just want to develop everything on my own. WinAPI would be OK for creating a full screen window and for catching user input, but not for graphics because I read GDI is too slow for fast graphics.)
I'm using WindowsXP and MASM or A86.
I totally agree with samcl
The main reason for not using assembler anymore is that you cannot access the Videomemory anymore. Back in the early days (you mentioned Castle Wolfenstein) there was a special video mode called 0x13h where your graphic was just a block of memory(each pixel was a palette color ranging from 0-255<--1 Byte) You were able to access this memory through this specific video mode, however, today things are much more complicated
Today you have very fast Videomemory and using your CPU for accessing it will just tear down all performance, as you CPU is connected through PCI-Express/AGP/PCI/VESA-LOCALBUS/ISA (<- remembering anyone!?)
Graphicsprogramming is often a lot of read and write accesses(read pixel, check if it is transparent, multiply with alpha, write pixel, etc.)
The modern memory Interfaces are much slower than direct access inside the graphic card. That's why you really should use shaders, as Robert Gould suggests. In this way you can write faster and easier to understand code and it will not stall your GFX-Memory.
IF you are more interested in GFX Programming, you can wet your appetite with shadertoy, a community dedicated to shaderbased effects complete with WebGLbased Shadercode execution.
Also your beginner assembler code will be pretty lame. in quality as in performance. Trust me. It needs a lot of time for optimizing such primitive code. So your compiled C/C++ Code will outperform your handwritten asm easily.
If you are interested in Assembler, try to code something like diskaccess. This is where you can gain a lot of performance.
It sounds like you only use Assembler because you seem to think that this is necessary. This isn't the case. If you don't have any other reason for it (i.e. wanting to learn it), don't use Assembler here, unless you know exactly what you're doing.
For your average graphics engine, Assembler programming is completely unnecessary. Especially when it comes to a Super Mario style 2D sprite engine. Even “slow” scripting languages like Python are fast enough for such things nowadays.
Adding to that, if you don't know very precisely what you're doing, Assembler will not be faster than C (in fact, chances are it will be slower because you'll re-implement existing C functions less efficiently).
I'm guessing if you are already using C with DirectX, speed is not the issue, and that this is more of a learning exercise. For 2D under Windows, C and DirectX is going to be very fast indeed, and as Konrad Rudolph points out, hand cranked assembler is unlikely to be faster than a highly optimized SDK.
From a purely educational standpoint, it is an intersting activity, but quite complex. Back in the early days of home computers and the first PCS, the graphics screen appeared pretty much as a block of memory, where bytes corresponded to one or more coloured pixels. By changing the value of the screen memory you could plot points, and hence lines, and on to sprites etc... On modern PCs this tends not to be an option, in that you program a graphics card, usually via an SDK, to do the same job. The card then does the hard work, and you are provided with a much higher level of abstraction. If you really wanted to get a feel for what it was like back in the day, I would recommend an emulator. For a modern game, stick with your SDKs.
It is possible to program your own 2D engine in a recent version of Directx, if you wish to investigate this avenue. You can create a "screen space" aligned polygon, with no perspective correction, of which is texture mapped. You can then plot your sprites on a pixel-by-pixel basis onto this texture map.
As for mode 13h (Peter Parker), it brings back some memories!
__asm
{
mov ax,0x13
int 10h // 16-bit code only, not Windows
}
But of course this will fault in a 32-bit or 64-bit Windows program; 16-bit BIOS calls are not supported by the Windows kernel (which installs its own interrupt table as part of booting and switching the CPU to 64-bit mode.)
I would tend to avoid assembler with a barge pole, it can be particulary difficult to debug, and maintain; however if you wish to explore this subject in more detail, I can recommend Michal Abrash's Graphics Programming Black Book. It's a bit old, but a good read and will give you some insight into graphics programming techniques before 3D hardware.
Assembler for graphics was because, back then, most people lacked graphics card with 3d support, so it had to be done on the CPU, not anymore. Nowadays it's about shader programming. Shader languages allow you to cuddle up with the bare metal. So if anything you should try to code your 2d graphics to be shadered base, that way the experience will have value as a career skill.
Try CUDA for a starter.
My recommendation is to experiment. Take your sprite code and write in a number of forms, starting with C/GDI and C++/DirectDraw. Don't worry about assembler yet.
DirectX is your best bet for fast action graphics. Learn it, then figure out how to micro-optimize with assembler. In general, assembler isn't going to make your API calls faster. It is going to open up flexibility for faster computation for things like 3D rotation, texture mapping, shading, etc.
Start with DirectDraw. Here's a FAQ. Technically, DirectDraw is deprecated after DirectX 7, but you can still use it and learn from it. It'll allow you direct framebuffer modification, which is what you're probably looking for.
There's some helpful tutorials and forums at TripleBuffer Software.
Also consider upgrading your compiler to Visual C++ 2008 Express. VC++ 6 has a buggy compiler that can be problematic with trying to compile certain C++ libraries.
I'm interesting in learning about the different layers of abstraction available for making graphical applications.
I see a lot of terms thrown around: At the highest level of abstraction, I hear about things like C#, .NET, pyglet and pygame. Further down, I hear about DirectX and OpenGL. Then there's DirectDraw, SDL, the Win32 API, and still other multi-platform libraries like WxWidgets.
How can I get a good sense of where one of these layers ends and where the next one begins? What is the "lowest possible level" way of creating a window in Windows, in C? What about C++? (A code sample would be divine.) What about in X11? Are the Windows implementations of OpenGL and DirectX built on top of the Win32 API? Where can I begin to learn about these things?
There's another question on SO where Programming Windows is suggested. What about for Linux? Is there an equivalent such book?
I'm aware that this is very low-level, and that there are many friendlier tools available, but I would like to at least learn the basics of what's going on beneath the surface. As much as I'd like to begin slinging windows and vectors right off the bat, starting with something like pygame is too high-level for me; I really need to make the full conceptual circuit of how you draw stuff on a computer.
I will certainly appreciate suggestions for books and resources, but I think it would be stupendously cool if the answers to this question filled up with lots of different ways to get to "Hello world" with different approaches to graphics programming. C? C++? Using OpenGL? Using DirectX? On Windows XP? On Ubuntu? Maybe I ask for too much.
The lowest level would be the graphics card's video RAM. When the computer first starts, the graphics card is typically set to the 80x25 character legacy mode.
You can write text with a BIOS provided interrupt at this point. You can also change the foreground and background color from a palette of 16 distinctive colors. You can use access ports/registers to change the display mode. At this point you could say, load a different font into the display memory and still use the 80x25 mode (OS installations usually do this) or you can go ahead and enable VGA/SVGA. It's quite complicated, that's what drivers are for.
Once the card's in the 'higher' mode you'd change what's on screen by accessing the memory mapped to the video card. It's stored horizontally pixel by pixel with some 'dirty regions' of pixels that aren't mapped to screen at the end of each line which you have to compensate for. But yeah, you could copy the pixels of an image in memory directly to the screen.
For things like DirectX, OpenGL. rather than write directly to the screen, commands are sent to the graphics card and it updates its screen automatically. Commands like "Hey you, draw this image I've loaded into the VRAM here, here and here" or "Draw these triangles with this transformation matrix..." take a fraction of the time compared to pixel by pixel . The CPU will thank you.
DirectX/OpenGL is a programmer friendly library for sending those commands to the card with all the supporting functions to help you get it done smoothly. A more direct approach would only be unproductive.
SDL is an abstraction layer so without bothering to read up on it I'd guess it would have different ways of working on each system. On one it might use semi-direct screen writing, another Direct3D, etc. Whatever's fastest as long as the code stays cross-platform..able.
The GDI/GDI+ and XWindow system. They're designed specifically to draw windows. Originally they drew using the pixel-by-pixel method (which was good enough because they'd only have to redraw when a button was pressed or a window moved, etc.) but now they use Direct3D/OpenGL for accelerated drawing (and special effects). Optimizations depend on the versions and implementations of these libraries.
So if you want the most power and speed, DirectX/openGL is the way to go. SDL is certainly useful for getting the most from a cross-platform environment and integrates with OpenGL anyway. The windowing system comes last but don't underestimate it. Especially with the stuff Microsoft's coming up with lately.
Michael Abrash's Graphics Programming 'Black Book' is a great place to start. Plus you can download it for free!
If you really want to start at the bottom then drawing a line is the most basic operation. Computer graphics is simply about filling in pixels on a grid (screen), so you need to work out which pixels to fill in to get a line that goes from (x0,y0) to (x1,y1).
Check out Bresenham's algorithm to get a feel for what is involved.
To be a good graphics and image processing programmer doesn't require this low level knowledge, but i do hate to be clueless about the insides of what i'm using. I see two ways to chase this - high-level down, or bottom-level up.
Top-down is a matter of following how the action traces from a high-level graphics operation such as to draw a circle, to the hardware. Get to know OpenGL well. Then the source to Mesa (free!) provides a peek at how OpenGL can be implemented in software. The source to Xorg would be next, first to see how the action goes from API calls through the client side to the X server. Finally you dive into a device driver that interfaces with hardware.
Bottom up: build your own graphics hardware. Think of ways it could connect to a computer - how to handle massive numbers of pixels through a few byte-size registers, how DMA would work. Write a device driver, and try designing a graphics library that might be useful for app programmers.
The bottom-up way is how i learned, years ago when it was a possibility with the slow 8-bit microprocessors. The direct experience with circuitry and hardware-software interfacing gave me a good appreciation of the difficult design decisions - e.g. to paint rectangles using clever hardware, in the device driver, or higher level. None of this is of practical everyday value, but provided a foundation of knowledge to understand newer technology.
see Open GPU Documentation section:
http://developer.amd.com/documentation/guides/Pages/default.aspx
HTH
On MSWindows it is easy: you use what the API provides, whether it is the standard windows programming API or the DirectX-family API's: that's what you use, and they are well documented.
In an X windows environment you use whatever X11-libraries that are provided. If you want to understand the principles behind windowing on X, I suggest that you do this, nevermind that many others tell you not to, it will really help you to understand graphics and windowing under X. You can read the documentation on X-programming (google for it). (After this exercise you would appreciate the higher level libraries!)
Apart from the above, at the absolutely lowest level (excluding chip-level) that you can go is to call the interrupts that switch to the various graphics modes available - there are several - and then write to the screen buffers, but for this you would have to use assembler, anything else would be too slow. Going this way will not be portable at all.
Another post mentions Abrash's Black Book - an excellent resource.
Edit: As for books on programming Linux: it is a community thing, there are many howto's around; also find a forum, join it, and as long as you act civilized you will get all the help you can ever need.
Right off the bat, I'd say "you're asking too much." From what little experience I've had, I would recommend reading some tutorials or getting a book on either directX or OpenGL to start out. To go any lower than that would be pretty complex. Most of the books I've seen in OGL or DX have pretty good introductions that explain what the functions/classes do.
Once you get the hang of one of these, you could always dig in to the libraries to see what exactly they're doing to go lower.
Or, if you really, absolutely MUST learn the LOWEST level... read the book in the above post.
libX11 is the lowest level library for X11. I believe the opengl/directx talk to the driver/hardware directly (or emulate unsupported ops), so they would be the lowest level library.
If you want to start with very low level programming, look for x86 assembly code for VGA and fire up a copy of dosbox or similar.
Vulkan api is an api which gives you very low level access to most if not all features of the gpu, computational and graphical, it works on amd and Nvidia gpus (not all)
you can also use CUDA, but it only works on Nvidia gpus and has access to computational features only, no video output.