Sometimes it might be useful, but mostly just looking cool or impressive to visualize log files (anything from http requests and to bandwith usage to cups of coffee drunk per day).
I know about Visitorville which I think look a bit silly, and then there's gltail.
How do you "visualize" your log files in realtime?
There is also the logstalgia tool. Visualizes Apache logs. See http://code.google.com/p/logstalgia/ for more details and a youtube video.
You may take a look at Apache Chainsaw. This nifty tool allows Log incomes from nearly everyqhere and has live filtering and colering. If you have an already written Log, I'm not sure if it can read it, it's been a while since I used it last time (was very usefull for the prototyping phase of our JBoss server)
Google has released the Visualization API that is probably flexible enough to help you:
The Google Visualization API lets you access multiple sources of structured data that you can display, choosing from a large selection of visualizations. The Google Visualization API also provides a platform that can be used to create, share and reuse visualizations written by the developer community at large.
It requires some Javascript knowledge and includes Google Docs integration, Spreadsheet integration. Check out the Gallery for some examples.
You could take a look at this. http://www.intalisys.com. 3D realtime vis app
We use Awk and Perl scripts to parse the log files and create summary reports and "databases" (technically databases in that each row corresponds to a unique event with many columns of data about that event, but not stored in a traditional database format. We're moving in that direction). I like Awk because you can very quickly search for specific strings in the log files using regex, keep counters and gather data from the log file entries, and do all kinds of calculations with that data. Then use your favorite plotting software. We use Excel, mainly because that's what was here before I started this job. I prefer MATLAB and it's open-source cousin, Octave, which is built on gnuplot.
I prefer Sawmill for visualizing data. You can basically throw any log file against it, and it will not only autodetect its structure*, but will also decide on how to analyze it. Even if you have a custom log file, you can still define what and how shall be analyzed and visualized.
I mainly use R to visualize data, but I've heard of Orange, too.
Not sure if it fits the question, but I just released this:
numStepCsvLogVis - analyze logfile data in CSV format
It uses Python's matplotlib, is motivated by the need to visualize syslog data in context of debugging kernel circular buffer operation (and variables) in C; and it visualizes by using CSV file format as intermediary to the logfile data (I cannot explain it better in brief - take a look at the README for more detail).
It has a "step" player accessed in terminal, and can handle "live" stdin input, but unfortunately, I cannot get a better response that 1 FPS when plot renders, so I wouldn't really call it "realtime" per se - but you can use it to eventually generate sonified videos of plot animations.
A simple solution is to use Logstalgia alongside the lightweight local-web-server.
First install the above. Then, from the root folder of your site visualise your logs in realtime with:
$ ws --log-format default | logstalgia -
Using SciTe, Notepad++ or other powerful text editor which have file processing routines, so you can create a script that colorizes parts of the log or just delete some non-important lines from it
Related
I'm working on the sound part of an interactive installation that would need an event to be triggered by osc an undefined number of times, making the sound linked to it overlaps instead of being rewinded and started again.
Would it be possible to do that without needing to make an array of loadings of the same sound?
I'm actually trying to do it with processing and minim library.
Do you think it would be easier to achieve it with another programming software? I've found myself in the same difficulties trying to do it with puredata. Any tip or clue would be extremely welcome.
Thanks a lot.
You will need multiple readers ([tabread~] resp [tabplay~] in Pd; i don't know about Processing/minim, but the same principle applies) to read the table multiple times (in parallel), where each one can be started separately.
However, you only need a single instance of your data array (e.g. [table]), as the various readers can access the same array independently.
Can you use Java libraries in Processing? Processing is built on Java, yes?
If you can, I have a library you can use, supporting a class I call AudioCue available via github. This is modeled on a Java Clip but with additional capabilities. It allows multiple, concurrent playback. AudioCue also has real time controls for volume, panning and playback speed, in case you want to play around with adding some more interactivity to your installation.
I would love to know if it can be used with Processing. Please follow up with me if you try this route. I'd like to see it done, and can possibly assist.
If Processing allows you to send PCM directly out for playback, then the basic algorithm is the store the audio data in an array, and create pointers or cursors (depending on your preferred terminology) that independently iterate through that array. This is the main basis of the algorithm I use for AudioCue, with the PCM being routed out via a Java SourceDataLine.
I have been creating a text editor online, just for learning experience. I was curious what the best way to store multiple versions of a text file that is consistently changing is.
I've looked at a variety of options and I am yet to see a cheap, and scale-able option.
I've looked into Google Cloud Storage and Amazon S3. The only issue is that too many requests to save the file start to add up a lot in cost. I'd like files to be saved practically instantly, and also versioned every so often. I've also looked into data deduplication which looks like a great option, but I have not yet found a way to do it without writing my own software.
Any and all advice would be greatly appreciated. Thanks!
This is a very broad question, but the basic answer is usually some flavor of Operational Transform. Basically you don't want to be constantly sending the entire document back and forth between the user(s) and the server, nor do you want to overwrite the whole of the document repeatedly. Instead you want to store diffs. Then you need to deal with the idea that multiple users might be changing the file simultaneously, but possibly in different areas, and dealing with that effectively.
Wikipedia has some good, formal discussion of the idea: https://en.wikipedia.org/wiki/Operational_transformation
You wouldn't need all of that for a document that will only be edited by one person at a time, but even then, the answer is to think in terms of diffs from previous versions and only occasionally persist whole snapshots.
In follow up to my previous question, I am able to save the backbuffer of a Direct3DDevice to a surface.
I already found the D3DXSaveSurfaceToFile in the DirectX SDK and it works fine! However, I want to record the obtained surfaces to an AVI file.
I also found the AVIFile reference but they are obviously not straight up compatible with DirectX surfaces.
What would be the best way of approaching this problem? I've seen a number of GDI+/MVC based solutions of grabbing HDCs but those are out of the question. I'm also not sure what kind of data AVIFile expects and how to extract it from the D3DSurface.
Please advise! :)
edit:
Post-processing is also an option. I can capture the surface data in a number of formats, specified here, into memory with D3DXSaveSurfaceToFileInMemory. Afterwards, I could compress this data and then store to disk.
How should I be compressing my data? How should I be storing it? Do I store a timestamp along with it? After recording, how should I turn the generated data into an AVI file?
The source code at this link will show you how to do it:
http://gpalem.web.officelive.com/SimulationRecording.html
Edit: Well you don't have to do things exactly like that linked code. You have a D3DSurface so you can just lock it grab the bits and pass them into CAviFile::AppendFrameUsual ... If you want to change its format then use D3DXLoadSurfaceFromSurface. I didn't say the link was a perfect solution but it DOES show you how to write the frames into an AVi file.
Edit2: As I didn't answer your edit I should do. Firstly don't bother with compression until you have got uncompressed working. Compression is a significantly more complicated thing to get right and you won't be able to get proper compression by simply using the various D3DXSurface copying functions. They don't support the kinds of compression you are after. D3DX is for 3D rendering and NOT for video compression.
For video compression you are best off using DirectShow as you can, simply, add any compressors you wish. This will however mean you'll need to write a "source filter" that you can build your graph off. DirectShow is not an easy thing to use but its very powerful. As far as writing the "source filter" goes you can check out the "Push Source" example in the windows SDK. You will need to adapt it to take the data you are retrieving however.
As an aside, going further on my original edit you could use that code as is by intercepting more D3D9 calls. If you hook the SetRenderTarget calls then you can insert whatever render target you like in there and use the, previously, linked code directly ...
BACKGROUND
I am using a commercial application on windows that creates a drawing
This application allows only two output options: (1) save as a bitmap file and (2) print to a printer
the bitmap is useless for my purposes - I want the vectors
Looking at the print output (I sent to the Windows XPS print driver) it seems clear based on the amount of zooming I can do without loss of detail that the underlying vectors are being send to the print driver
Once I get the vectors, I will be writing some code to transform them for some other use.
MY QUESTION
Whart are my options for geting the vectors from the print? (am open to both commercial and open source)
OPTIONS I HAVE THOUGHT OF SO FAR
Take the bitmap and use a program like VectorMagick to. I have tried this approach. It does not produce the fidelity I seek even when the original bitmap is large. Practically speaking I believe that using any tracing approach will not give me the quality vectors I need.
Print to the Adobe PDF driver. This technically works. I have Adobe CS4 so I can print to it save the resulting PDF and then import the PDF into Illustrator and then export as some other vector format. The problem with this approach is money/licensing. I own a personal copy of Adobe CS4 - so this is fine for me. But I need to capture the vectors at work for business purposes - and no I'm not going to install my personal copy of CS4 at work.
Is there a "print driver" that captures the print output directly into a vector format? I have seen some commercial ones via google. If you've used them, I would like to hear about your experience with this technique. I could write my own and in that case do you have links to any existing code that I can start with.
If this is an ongoing solution you need then you might need to buy something or build your own. If it's a onetime affair you might look to use an 'older' Lexmark PCL printer driver. I'd recommend something like the T610. If you download the PCL driver and install it you can modify the defaults and change the Graphics option from XL or Autoselect to GL/2. This will force the driver to output GL/2 output which is vector (GL/2 is a plotter language). This might do the trick for you. Other printer drivers may have the abiltiy to force GL/2 (vs. Raster) but I'm not sure. I use to work for Lexmark and have used this before for a similar requirement.
Ensure you use the Lexmark 'Custom' driver as I don't think the Microsoft-based one support this feature.
...pausing while I investigate a few things............I'm back...
Another option is to find another GL/2 driver or build you own...I just took a few minutes to search the web and came up with a few other options that might work.
Build you own:
I've built drivers (minidrivers) using the Windows Driver Development Kit (DDK), it's quite simple to construct basic drivers. Looks like there is a setting you can set to enable GL/2 output: Enabling HP-GL/2 Vector Graphics Support (PCL-5e) in the GPD
Alternate drivers:
Depending on the OS you are on there is probably a 'generic' GL/2 driver built in. I believe XP has a Hewlett-Packard HP-GL/2 Plotter. You might need to check the license (as with the Lexmark solution) but it might work for you and as it's part of the OS there shouldn't be concern about using it. It's probably written and copyrighted to Microsoft
Keep in mind you will have to do some work to convert GL/2 to whatever output you want but it should be a matter of an simple translator to convert each set of commands. There may be tools out there to help. Here is a quick link to Lexmark GL/2 reference which might be enough to get you going, check out the GL/2 information under the PCL section: Lexmark Technical Reference Guide
Postscript:
The last option I have is to use a generic Postscript driver. Postscript should output the vector images as vector graphics in the Postscript but my knowledge of this is limited at best.
Output:
If you need the output to route to file you can set the port to FILE: which requries user intervention, or install something like Redmon (or connect with me and I'll send you our port monitor that allows for automatic output to file).
Hope this helps in some way.
My favorite is the open source (GPL) PDFCreator
http://sourceforge.net/projects/emfprinter/
I need to program a util for a 3d model. What i need to do is pass the 3d model to the util and have it extract multiple information. Things like poly count, size/scale (if applicable) and anything else i can grab. Then i need to take a screen shot of the model 8 times (45deg from 0 to 360). Maybe 16. Is there an app that i can use to extract data from the model and is there another app i can use to create the image? if so i can write a php script to do this all for me :) if not i'll post more question about specifics.
My OS is debian etch
the models will be any format. I may make it 3ds only or limit it to what the app(s) support. Or i may have another app to convert one format to the other and extract the data from that format.
I haven't tried this, but you could try 3ds2pov to convert your 3DS files into POV-Ray format, and then render with POV-Ray.
The 3ds2pov program is quite old, so I don't know how compatible it is with current 3DS files. The archive comes with source code so ought to build without too much difficulty on Debian.
POV-Ray itself runs easily on Linux.
nb: other 3DS to POV converters may exist. This one just happens to be the first one returned by Googling for "3ds to pov".
Your requirements are very loose and vague but most 3d engines should have the tools you need to create your custom utility. I would start by looking at Blender or the viewer tools from OGRE, OpensceneGraph and similar tools.
You can try this: Linux based A3dsViewer -> does 3ds to pov conversion.