In follow up to my previous question, I am able to save the backbuffer of a Direct3DDevice to a surface.
I already found the D3DXSaveSurfaceToFile in the DirectX SDK and it works fine! However, I want to record the obtained surfaces to an AVI file.
I also found the AVIFile reference but they are obviously not straight up compatible with DirectX surfaces.
What would be the best way of approaching this problem? I've seen a number of GDI+/MVC based solutions of grabbing HDCs but those are out of the question. I'm also not sure what kind of data AVIFile expects and how to extract it from the D3DSurface.
Please advise! :)
edit:
Post-processing is also an option. I can capture the surface data in a number of formats, specified here, into memory with D3DXSaveSurfaceToFileInMemory. Afterwards, I could compress this data and then store to disk.
How should I be compressing my data? How should I be storing it? Do I store a timestamp along with it? After recording, how should I turn the generated data into an AVI file?
The source code at this link will show you how to do it:
http://gpalem.web.officelive.com/SimulationRecording.html
Edit: Well you don't have to do things exactly like that linked code. You have a D3DSurface so you can just lock it grab the bits and pass them into CAviFile::AppendFrameUsual ... If you want to change its format then use D3DXLoadSurfaceFromSurface. I didn't say the link was a perfect solution but it DOES show you how to write the frames into an AVi file.
Edit2: As I didn't answer your edit I should do. Firstly don't bother with compression until you have got uncompressed working. Compression is a significantly more complicated thing to get right and you won't be able to get proper compression by simply using the various D3DXSurface copying functions. They don't support the kinds of compression you are after. D3DX is for 3D rendering and NOT for video compression.
For video compression you are best off using DirectShow as you can, simply, add any compressors you wish. This will however mean you'll need to write a "source filter" that you can build your graph off. DirectShow is not an easy thing to use but its very powerful. As far as writing the "source filter" goes you can check out the "Push Source" example in the windows SDK. You will need to adapt it to take the data you are retrieving however.
As an aside, going further on my original edit you could use that code as is by intercepting more D3D9 calls. If you hook the SetRenderTarget calls then you can insert whatever render target you like in there and use the, previously, linked code directly ...
Related
Anyone knows why the image, shown below, is partially gray while loading? I assume it has something to do with progressive JPG, but I haven't seen the grayscale effect before. Is there some kind of a JPG export setting that enables that? How is it called? Can I disable it?
Direct image link
Edit:
The problem could either be the problem described in #KonradDzwinel answer OR my answer, both could look identical:
What is the problem
For some reason the file is saved or produced in way that make the file we call it Lossless JPEG.
I am not sure which software, tool or how this problem is was produced, but that won’t change the fact that your file had have some issues or even corrupted.
How to find the problem in the file
Your file was saved in way that it works identical but it had issues. To check whether your file is healthy or not, it requires a tool to scan/analysis the image file. I use a very nice and easy to use tool (software) called JPEGsnoop (link for download). This tool is free and I used it to scan and analysis your image files for problems.
Your file comes up with few places with following note, (the screen-shot btw shows only one note, but there were more):
NOTE: Scan parsing doesn’t support this SOF mode.
After I have re-saved the file correctly and analysis it again, it does not show any problems or notes:
What I did to create a problem FREE image file
For testing I have used Photoshop 5.5 to open the Lossless file and save it with new name with default jpeg (since I don't have the original file). But correctly I assume you have the original file, and I assume you have Photoshop any version does not matter OR any other image tool. Try to optimize your jpeg file best possible, analysis it and remember/note the saving setting for future wise image saving.
Here is an example of my Photoshop save settings, which result in problem FREE file:
How can I learn more about Lossless JPEG
I won’t go in details of this problem but I will leave some link that explain it in details, I hope also you find this is useful to solve your issue:
http://www.impulseadventure.com/photo/fix-corrupt-jpeg-photo.html
http://fileformats.archiveteam.org/wiki/JPEG
http://fileformats.archiveteam.org/wiki/Lossless_JPEG_%28original%29
http://www.howtogeek.com/142174/what-lossless-file-formats-are-why-you-shouldnt-convert-lossy-to-lossless/
Software JPEGsnoop
http://www.impulseadventure.com/photo/jpeg-snoop.html
Progressively encoded JPEGs contain ten scan layers by default. That means ten iterative layers of image information build on each other to deliver the final visual quality of the image. The first visible scan layer of a progressive JPEG is always highly pixelated and often black & white because it saves on color channel information.
Source (emphasis mine)
Another resource that explores this issue in lengh: http://cloudinary.com/blog/progressive_jpegs_and_green_martians
I am developing a game in j2me. How to handle images in .dat format.
I downloaded some games and extracted jar , found some dat format images and not able to open that images and images size also very less.. what tools I need to use?
Ref link
enter link description here
Not able to find solution?
A dat file could be anything. Depends what the developer felt like doing.
Some developers chose to strip PNG files of their header, and added the header back in the code. This was partly done in order to save a few bytes (because they mattered back then), and partly because of the challenge in doing it like that, and partly because it ensured all images used the exact same palette.
So that's one possibility, but it really could be anything.
As stated by mr_lou, there really isn't anything special about a .dat extension.
The steps to re-compile a file usually start with opening the file up in a hex editor and then looking at the first bits of information in the file. You then basically work from there to re-compile the data necessary for a 'normal' program to interpret the file. In particular, the first 8-16 bytes are often very helpful for determining what type of file it is "supposed" to be.
If you are looking at a png file (that's what I usually prefer to use for art assets) then you can reference http://en.wikipedia.org/wiki/Portable_Network_Graphics to see how a 'normal' png might look. When you're tweaking to save bytes you often strip unnecessary fields from png headers (things like the ancillary chunks) and using a common palette.
However, remember that it's not necessarily image data. It could be things like level data, sound, default stats or any particular amount of stuff.
I like the idea of Nodebox and Processing, and would like to generate movies to visualize some data/algorithms. However, Nodebox exports extremely bloated Quicktime files with frame by frame images, and Processing only exports Java applications. I want to be able to export movies that don't take a Gigabyte a minute of disk space. Perhaps something like SVG animations or Actionscript which stores the vector graphics definition of the animation rather than frame images would be better. Is there a framework that is as easy to program as Nodebox and Processing and can export "lean" movies?
Have you tried the MovieMaker library that ships with Processing ?
Also, it should be fairly simple to save multiple frames using saveFrame().
This is option have a couple of advantages:
If your sketch crashes at some point, you still have all the frames up to that point (unlike writing a .mov file)
It's fairly simple to put the frames back into a video file, but you also have control over playback speed and can easily do a bit of editing if needed.
You can try to a sequence of PDF file using createGraphics() to get vector output, but I'm not sure how stable/feasible this option is.
They are changing the way this works moving towards 2.0 too, as they are moving to GSVideo over Quicktime...
Daniel Shiffman posted about it recently on his blog, but it's the only place I've heard about any changes to post-2.0 tactics (though he IS part of the inner circle, I know)
You can find that post at
http://www.shiffman.net/2011/12/28/night-8-rendering-out-as-image-sequence/
Also, if you are on OSX, you can try Syphon ? See info here
https://forum.processing.org/topic/syphon-integration-with-processing
How does one combine several resources for an application (images, sounds, scripts, xmls, etc.) into a single/multiple binary file so that they're protected from user's hands? What are the typical steps (organizing, loading, encryption, etc...)?
This is particularly common in game development, yet a lot of the game frameworks and engines out there don't provide an easy way to do this, nor describe a general approach. I've been meaning to learn how to do it, but I don't know where to begin. Could anyone point me in the right direction?
There are lots of ways to do this. m_pGladiator has some good ideas, especially with seralization. I would like to make a few other comments.
First, if you are going to pack a bunch of resources into a single file (I call these packfiles), then I think that you should work to avoid loading the whole file and then deseralizing out of that file into memory. The simple reason is that it's more memory. That's really not a problem on PC's I guess, but it's good practice, and it's essential when working on the console. While we don't (currently) serialize objects as m_pGladiator has suggested, we are moving towards that.
There are two types of packfiles that you might have. One would be a file where you want arbitrary access to the contents of the files. A second type might be a collection of files where you need all of those files when loading a level. A basic example might be:
An audio packfile might contain all the audio for your game. You might only need to load certain kinds of audio for the menus or interface screens and different sets of audio for the levels. This might fall intot he first category above.
A type that falls into the second category might be all models/textures/etc for a level. You basically want to load the entire contents of this file into the game at load time because you will (likely) need all of it's contents while a player is playing that level or section.
many of the packfiles that we build fall into the second category. We basically package up the level contents, and then compresses them with something like zlib. When we load one of these at game time, we read a small amount of the file, uncompress what we've read into a memory buffer, and then repeat until the full file has been read into memory. The buffer we read into is relatively small while final destination buffer is large enough to hold the largest set of uncompressed data that we need. This method is tricky, but again, it saves on RAM, it's an interesting exercise to get working, and you feel all nice and warm inside because you are being a good steward of system resources. once the packfile has been completely uncompressed into it's destinatino buffer, we run a final pass on the buffer to fix up pointer locations, etc. This method only works when you write out your packfile as structures that the game knows. In other words, our packfile writing tools share struct (or classses) with the game code. We are basically writing out and compressing exact representations of data structures.
If you simply want to cut down on the number of files that you are shipping and installing on a users machine, you can do with something like the first kind of packfile that I describe. Maybe you have 1000s of textures and would just simply like to cut down on the sheer number of files that you have to zip up and package. You can write a small utility that will basically read the files that you want to package together and then write a header containing the files and their offsets in the packfile, and then you can write the contents of the file, one at a time, one after the other, in your large binary file. At game time, you can simply load the header of this packfile and store the filenames and offsets in a hash. When you need to read a file, you can hash the filename and see if it exists in your packfile, and if so, you can read the contents directly from the packfile by seeking to the offset and then reading from that location in the packfile. Again, this method is basically a way to pack data together without regards for encryption, etc. It's simply an organizational method.
But again, I do want to stress that if you are going a route like I or m_pGladiator suggests, I would work hard to not have to pull the whole file into RAM and then deserialize to another location in RAM. That's a waste of resources (that you perhaps have plenty of). I would say that you can do this to get it working, and then once it's working, you can work on a method that only reads part of the file at a time and then decompresses to your destination buffer. You must use a comprsesion scheme that will work like this though. zlib and lzw both do (I believe). I'm not sure about an MD5 algorithm.
Hope that this helps.
do as Java: pack it all in a zip, and use an filesystem-like API to read directly from there.
Personally, I never used the already available tools to do that. If you want to prevent your game to be hacked easily, then you have to develop your own resource manipulation engine.
First of all read about serializing objects. When you load a resource from file (graphic, sound or whatever), it is stored in some object instance in the memory. A game usually uses dozens of graphical and sound objects. You have to make a tool, which loads them all and stores them in collections in the memory. Then serialize those collections into a binary file and you have every resource there.
Then you can use for example MD5 or any other encryption algorithm to encrypt this file.
Also, you can use zlib or other compression library to make this big binary file a bit smaller.
In the game, you should load the encrypted binary file and unpack it. Then decrypt it. Then deserialize the object collections and you have all resources back in memory.
Of course you can make this more comprehensive by storing in different binary files the resources for different levels and so on - there are plenty of variants, depending on what you want. Also you can first zip, then encrypt, or make other combinations of the steps.
Short answer: yes.
In Mac OS 6,7,8 there was a substantial API devoted to this exact task. Lookup the "Resource Manager" if you are interested. Edit: So does the ROOT physics analysis package.
Not that I know of a good tool right now. What platform(s) do you want it to work on?
Edited to add: All of the two-or-three tools of this sort that I am away of share a similar struture:
The file starts with a header and index
There are a series of blocks some of which may have there own headers and indicies, some of which are leaves
Each leaf is a simple serialization of the data to be stored.
The whole file (or sometimes individual blocks) may be compressed.
Not terribly hard to implement your own, but I'd look for a good existing one that meets your needs first.
For future people, like me, who are wondering about this same topic, check out the two following links:
http://www.sfml-dev.org/wiki/en/tutorials/formatdat
http://archive.gamedev.net/reference/programming/features/pak/
Sometimes it might be useful, but mostly just looking cool or impressive to visualize log files (anything from http requests and to bandwith usage to cups of coffee drunk per day).
I know about Visitorville which I think look a bit silly, and then there's gltail.
How do you "visualize" your log files in realtime?
There is also the logstalgia tool. Visualizes Apache logs. See http://code.google.com/p/logstalgia/ for more details and a youtube video.
You may take a look at Apache Chainsaw. This nifty tool allows Log incomes from nearly everyqhere and has live filtering and colering. If you have an already written Log, I'm not sure if it can read it, it's been a while since I used it last time (was very usefull for the prototyping phase of our JBoss server)
Google has released the Visualization API that is probably flexible enough to help you:
The Google Visualization API lets you access multiple sources of structured data that you can display, choosing from a large selection of visualizations. The Google Visualization API also provides a platform that can be used to create, share and reuse visualizations written by the developer community at large.
It requires some Javascript knowledge and includes Google Docs integration, Spreadsheet integration. Check out the Gallery for some examples.
You could take a look at this. http://www.intalisys.com. 3D realtime vis app
We use Awk and Perl scripts to parse the log files and create summary reports and "databases" (technically databases in that each row corresponds to a unique event with many columns of data about that event, but not stored in a traditional database format. We're moving in that direction). I like Awk because you can very quickly search for specific strings in the log files using regex, keep counters and gather data from the log file entries, and do all kinds of calculations with that data. Then use your favorite plotting software. We use Excel, mainly because that's what was here before I started this job. I prefer MATLAB and it's open-source cousin, Octave, which is built on gnuplot.
I prefer Sawmill for visualizing data. You can basically throw any log file against it, and it will not only autodetect its structure*, but will also decide on how to analyze it. Even if you have a custom log file, you can still define what and how shall be analyzed and visualized.
I mainly use R to visualize data, but I've heard of Orange, too.
Not sure if it fits the question, but I just released this:
numStepCsvLogVis - analyze logfile data in CSV format
It uses Python's matplotlib, is motivated by the need to visualize syslog data in context of debugging kernel circular buffer operation (and variables) in C; and it visualizes by using CSV file format as intermediary to the logfile data (I cannot explain it better in brief - take a look at the README for more detail).
It has a "step" player accessed in terminal, and can handle "live" stdin input, but unfortunately, I cannot get a better response that 1 FPS when plot renders, so I wouldn't really call it "realtime" per se - but you can use it to eventually generate sonified videos of plot animations.
A simple solution is to use Logstalgia alongside the lightweight local-web-server.
First install the above. Then, from the root folder of your site visualise your logs in realtime with:
$ ws --log-format default | logstalgia -
Using SciTe, Notepad++ or other powerful text editor which have file processing routines, so you can create a script that colorizes parts of the log or just delete some non-important lines from it