Let me describe the "battlefield" of my task:
Multi-room audio/video chat with more than 1M users;
Custom Direct3D renderer;
What I need to implement is a TextOverVideo feature. The Text itself goes via network and is to be rendered on the recipient side with Direct3D renderer. AFAIK, it is commonly used in game development to create your own texture with letters/numbers and draw this items. Because our application must support many languages, we ought to use a standard. That's why I've been working with ID3DXFont interface but I've found out some unsatisfied limitations.
What I've faced is a lack of scalability. E.g. if user is resizing video window I have to RE-create D3DXFont with new D3DXFONT_DESC while he's doing that. I think it is unacceptable.
That is why the ONLY solution I see (due to my skills) is somehow render the text to a texture and therefore draw sprite with scaling, translation etc.
So, I'm not sure if I go into the correct direction. Please help with advice, experience, literature, sources...
Your question is a bit unclear. As I understand it, you want easily scalable font.
I think it is unacceptable
As far as I know, this is standard behavior for fonts - even for system fonts. They aren't supposed to be easily scalable.
Possible solutions:
Use ID3DXRenderTarget for rendering text onto texture. Font will be filtered when you scale it up too much. Some people will think that it looks ugly.
Write custom library that supports vector fonts. I.e. - it should be able to extract font outline from font, and build text from it. It will be MUCH slower than ID3DXFont (which is already slower than traditional "texture" fonts). Text will be easily scalable. Using this way, you are very likely to get visible artifacts ("noise") for small text. I wouldn't use that approach unless you want huge letters (40+ pixels). Freetype library may have functions for processing font outlines.
Or you could try using D3DXCreateText. This will create 3D text for ONE string. Won't be fast at all.
I'd forget about it. As long as user is happy about overall performance, improving font rendering routines (so their behavior looks nice to you) is not worth the effort.
--EDIT--
About ID3DXRenderTarget.
EVen if you use ID3DXRenderTarget, you'll need ID3DXFont. I.e. you use ID3DXFont to render text onto texture, and then use texture to blit text onto screen.
Because you said that performance is critical, you can delay creation of new ID3DXFont until user stops resizing video. I.e. When user starts resizing video, you use old font, but upscale it using texture. There will be filtering, of course. Once user stops resizing, you create new font when you have time. you probably can do that in separate thread, but I'm not sure about it. OR you could simply always render text in the same resolution as video. This way you won't have to worry about resizing it (it still will be filtered - along with the video). Some video players work this way.
Few more things about ID3DXFont. There is one problem with ID3DXFont - it is slow in situations where you need a lot of text (but you still need it, because it supports unicode, and writing texturefont with unicode support is pain). Last time I worked with it I optimized things by caching commonly used strings in the textures. I.e. any string that was drawn more than 3 frames in the row were rendered onto D3DFMT_A8R8G8B8 texture/render target, and then I've been copying that string from texture instead of using ID3DXFont. Strings that weren't rendered for a while, were removed from texture. That gave some serious boost. This solution, however is tricky - monitoring empty space in the texture, removing unused strings, and defragmenting the texture isn't exactly trivial (there is nothing exceptionally complicated, but it is easy to make a mistake). You won't need such complicated system unless your screen is literally covered by text.
ID3DXFont fonts are flat, always parallel to the screen. D3DXCreateText are meshes that can be scaled and rotated.
Texture fonts are fuzzy and don't look very clear. Not good for an app that uses lots of small text.
I am writing an app that can create 500 text meshes, each mesh averaging 3,000-5,000 vertices. The text meshes are created once, then are static. I get 700 fps on a GeForce 8800.
Related
I just started at a new company and have been asked to do what I believe impossible, but I need confirmation on this. The website that my company has allows users to view images in extremely high resolution using SVGs. We've built a custom viewer for these images that allow you to zoom and scroll around the image and it works well. Because of this, the image format "needs" to remain SVG. However, we need to include a watermark to this image. The way we're doing this right now is passing the SVG and PNG watermark to the client, inserting the watermark into the SVG, and displaying it to the user. This is very obviously easily hacked as anyone with client-side experience knows (The dev team here is mostly older developers and don't have much web experience). Even if the raw SVG isn't intercepted, they can still just delete the watermark from the source. I've convinced my boss to make the watermark be injected on the server-side, so this leaves just the problem that SVGs are editable by the end-client.
What I want to know is if one of these two things are possible:
1) Is there another image format akin to SVG that could be used to keep this highly scalable image without loosing resolution, and without it being directly editable by an end-client? The only options that people seem to discuss for the web is JPG, GIF, PNG, and SVG. I've looked at Adobe Illustrator ".ai" files and EPS (Encapsulated PostScript) as other vector options, but I can't find anywhere if I can modify these images in PHP, which is key if I want to overlay a watermark in server-side code.
2) Is there a way to obfuscate the raw SVGs content so that the use can't go and manipulate it? I've seen SVGs that have <image ...> tags inside them with PNGs represented as long and complex strings before. Something like xlink:href="data:image/png;base64...". I was wondering if there's a way to display an SVG as this string, so that the data can't be directly manipulated. I'm sure there's an algorithm to reverse these, but so long as we're sticking with SVGs, I need to make this as secure as possible with as many hoops to jump through as possible if someone wants to steal the data.
Either way would be acceptable, as long as removing this watermark is more complex than just hitting F12 and removing the element inside the dev tools.
Is there another image format akin to SVG that could be used to keep this highly scalable image without loosing resolution, and without it being directly editable by an end-client?
SVG is a vector format, and to maintain extreme scalability, you need to stick with vectors, be it any format. However, in that case it will always be possible to remove the vectors that belong to your watermark. There are of course vector formats that are stored as binary, which would make it somewhat harder for an enduser to parse and edit, but those are not editable in PHP either, and are much less compatible. So you probably don't want to do this.
Is there a way to obfuscate the raw SVGs content so that the user can't go and manipulate it?
First, it will never be "secure" in the sense that as said above, it will always be possible to remove the watermark from a vector image. (Btw the only difference to bitmap formats like JPG is that content below the watermark in bitmap is actually missing, while in SVG it's still there.)
However, depending on how "good" you want this to be, you can do a few things. I think the "goodness" here means the effort needed to remove the watermark, and you can raise the bar relatively easily. You don't have to (and you probably can't reasonably) obfuscate the whole SVG.
One thing that comes to mind is that SVG is basically just XML, it consists of tags like <rect>, <line>, <circle> etc. The order of these tags doesn't matter much (mostly, with exceptions of course). So you could entwine the tags that draw your watermark pretty much randomly among existing tags. I mean really randomly, so different downloads would produce different results. If you do this well (eg. you find line tags to "hide" your watermark line tags, and so on), it will be hard to automatically remove the watermark, because it's all over relevant data that is your actual image. Of course, the watermark could still be visually in a corner, and this is already a weakness, anything drawn to a corner could possibly be removed automatically. And it will be easy to remove by hand with any decent editor I suppose. So it depends on what the purpose is.
But I still think this sort of thing could make it hard enough in many scenarios (and would be totally inadequate in others).
I have to draw charts on browser using a python backend (which may not matter here). There are numerous libraries like JQPlot, D3, Google Charts for achieving this.
But if you classify them, they are either HTML5 Canvas based or SVG based. Both are important technologies in their own space. But
for charting as a subject, shall I go with SVG based libraries or
HTML5 Canvas based libraries. What are downside and benefits of
both approaches.
I don't have any prior experience with charting and don't want to hit the wall
after I start the project.
Projects with a large amount of data may favor canvas. SVG approaches typically create a DOM node per point (unless you make paths), which can lead to:
An explosion in the size of your DOM tree
Performance problems
Using a path, you can get around this problem, but then you lose interactivity.
Say you're building a stock chart. If you are talking about a chart with, say... max 5 years and end of trade data samples only, I think the answer is clearly SVG. If you're talking about looking at Walmart's historical data from day one of trading or doing full trade information per minute, you are going to have to really look carefully at SVG. Probably will have to employ fancy memory management and a fetch-on-demand approach as SVG will fall apart, particularly if you go one sample to one SVG node.
If interactivity is a requirement, SVG easily has the edge, given:
It is a retained mode API
You can use typical event handlers
You can add/remove nodes easily, etc.
Of course, you see that if you require full interactivity, it may go against mechanisms that allow SVG to scale, like path collapsing, so there is an inherent tension here.
There is going to be a trade-off in extremes. If size is small, the answer is SVG hands-down. If size is large and no interactivity, the answer is SVG with path drawing only or using Canvas. If size is large and interactivity is required, you have to go canvas or tricky SVG, which is complex in either case.
Some libraries out there offer both canvas and SVG renders, such as ZingChart and Dojo. Others tend to stick with just one of the two options.
Being vector based, SVG gets you scalability for free, and a side effect of this is that it's sharp on high resolution displays and sharp when printed. You can kind of get around this with canvas by rendering at 2x resolution and scaling your canvas but it's kind of a half-solution.
SVG I think is the modern way and the way to do this moving forward.
If you are concerned about rendering speed if you have many nodes consider also that if you're using canvas, you're basically using your own Javascript based rendering code which has to render those same nodes. You do get the predictability of only having to render it once, but if you only render it once that also means you lose the ability to re-render when zooming or to do various interactive things. If performance is a problem you can simplify SVG by sub-sampling your data, taking moving averages and plotting that only once per x rows, etc depending on what you're doing. But, we're talking thousands and thousands of nodes with almost no impact.
Canvas still has a place if you are building a web based raster graphics editor or something that in inherently raster-based but essentially if we are looking at charts, we're talking about something that's inherently vector based.
I've been studying 3D graphics on my own for a while now and I want to get a greater understanding of just how everything works. What I would like to do is to create a simple game without using DirectX or OpenGL. I understand most of the math I believe, but the problem I am running up against is I do not know how to get control of the pixels being displayed in a window.
How do I specify what color I want each pixel in my window to be?
I understand I will probably run into issues with buffers and image shearing and probably terrible efficiency problems, but I want to create my own program so that I could see from the very lowest level, of the high level language, how the rendering process works. I really have no idea where to start though. I've figured out how to output BMPs, but I would like to have a running program spitting out 20+ frames per second. How do I accomplish this?
You could pick a environment that allows you to fill an array with values for pixels and display it as a bitmap. This way you come closest to poking RGB values in video memory. WPF, Silverlight, HTML5/Javascript can do this. If you do not make it full screen these technologies should suffice for now.
In WPF and Silverlight, use the WriteableBitmap.
In HTML5, use the canvas
Then it is up to you to implement the logic to draw lines, circles, bezier curves, 3D projections.
This is a lot of fun and you will learn a lot.
I'm reading between the lines that you're more interested in having full control over the rendering process from a low level, rather than having a specific interest in how to achieve that on one specific platform.
If that's the case then you will probably get a good bang for your buck looking at a library like SDL which provides you with a frame buffer that you can render to directly but abstracts away a lot of the platform specifics issues. It has been around for quite a while and there are some good tutorials to give you an idea of whether it's the kind of thing you're looking for - see this tutorial and the subsequent one in the same series, which should be enough to get you up and running.
You say you want to create some kind of a rendering engine, meaning desinging you own Pipeline and matrice classes. Which you are to use to transform 3D coordinates to 2D points.
When you have got the 2D points you've been looking for. You can use say for instance on windows, you can select a brush and draw you triangle values while coloring them at the same time.
I do not know why you would need Bitmaps, but if you want to practice say Texturing you can also do that yourself although off course on a weak computer this might take your frames per second significantly.
If you aim is to understand how rendering works on the lowest level. This is with no doubt a good practice.
Jt Schwinschwiga
I'm working on a game using OpenGL displaying sprites, i.e. 2d quad-mapped graphics with no projection, that will be displayed on several different resolution screens. (i.e. iPhone retina/non-retina, iPad.. my next project the problem will expand to desktop resolutions which are far more numerous)
I'm OK with handling different aspect ratios, that can be handled by opengl and my placement of the sprites. I'm also OK with slightly different resolutions - use same art and either border the screen, or display a little bit more info.. but when things start to grow/shrink by like 50%+ it's a major issue.
What is standard procedure for generating the art assets in this situation? Generate for the largest resolution and just let OpenGL worry about resizing during it's rasterizing, or do people generate art sets for each main resolution?
Rasterized sprite art tends to get ugly when it's stretched (interpolated), so I'm concerned.. but generating different sizes really means for practical purposes I have to go with vector drawings and export several resolutions. Limits the artist and is somewhat complicated as far as loading and managing the assets
(Yes, I can "just try it" to an extent, but I already have an idea of the results. I'm looking for solutions people use and angles I maybe wouldn't have thought of. This question does have an answer(s) it's not subjective or lazy)
You are correct that scaling bitmaps tends to make sprites bad. There are a couple of ways of dealing with that:
Draw them (pixelart) at all required resolutions. That is a lot of work but gives you full control.
Draw them (vectors) and render them at all required resolutions. Less work but scaling up or down beyond 50% or 200% might give bad results.
Draw them (3D appliction) and render them at all required resolutions. Quite some work but a very consistent set of sprites.
For each of these options you are free to post-process the bitmaps to clean them up or add details but if you do this for options 2 and 3, you are breaking the chain and will have to apply the changes again when rendering the same set again.
An other option is to limit the variation of resolutions.
As far as I know it is very common in the (game) industry to make all (or the most used/visible) sprites as pixel perfect as possible. This is what they pay the artists for...
I'm working on a game in XNA for Xbox 360. The game has 3D terrain with a collection of static objects that are connected by a graph of links. I want to draw the links connecting the objects as lines projected on to the terrain. I also want to be able to change the colors etc. of links as players move their selection around, though I don't need the links to move. However, I'm running into issues making this work correctly and efficiently.
Some ideas I've had are:
1) Render quads to a separate render target, and use the texture as an overlay on top of the terrain. I currently have this working, generating the texture only for the area currently visible to the camera to minimize aliasing. However, I'm still getting aliasing issues -- the lines look jaggy, and the game chugs frequently when moving the camera EDIT: it chugs all the time, I just don't have a frame rate counter on Xbox so I only notice it when things move.
2) Bake the lines into a texture ahead of time. This could increase performance, but makes the aliasing issue worse. Also, it doesn't let me dynamically change the properties of the lines without much munging.
3) Make geometry that matches the shape of the terrain by tessellating the line-quads over the terrain. This option seems like it could help, but I'm unsure if I should spend time trying it out if there's an easier way.
Is there some magical way to do this that I haven't thought of? Is one of these paths the best when done correctly?
Your 1) is a fairly good solution. You can reduce the jagginess by filtering -- first, make sure to use bilinear sampling when using the overlay. Then, try blurring the overlay after drawing it but before using it; if you choose a proper filter, it will remove the aliasing.
If it's taking too much time to render the overlay, try reducing its resolution. Without the antialiasing filter, that would just make it jaggier, but with a good filter, it might even look better.
I don't know why the game would chug only when moving the camera. Remember, you should have a separate camera for the overlay -- orthogonal, and pointing down onto the terrain.
Does XNA have a shadowing library? If so, yo could just pretend the lines are shadows.