I've made a reinstallation of the buildserver and can no longer see the standard graphs of project statistics. The error message shown is
"Missing/Invalid statistics reports. Please check if you have enabled the Statistics Publisher, and statistics have been collected atleast once after that."
To the best of my knowledge, the ccnet.config file has not been changed in this respect and by inspection it is verified that I have a Statistics / statisticsList-section for the project. Furthermore, the values appear in the Artifacts\statistics.csv and Artifacts\report.xml files.
My guess would then be StatisticsGraph.xslt, which I have copied fresh from distribution to both Server\xlst and WebDashboard\xslt (why are they located in both places, by the way!?). Rebuild and check - still same error message.
Any hints to how to debug this would be appreciated!
Statistics report is somewhat tricky. Forget about statistics.csv. report.xml is the file you should take a closer look at.
report.xml contains header elements and n value rows. If the header elements and the row content do not match, no results are displayed at all.
The only advice I can give at the moment: Perhaps you should take a look at that. I haven't tried it myself and it might be outdated and not extremly reliable, but it targets the problem.
Related
I am trying to add Java memory profiling into my devops pipeline. I am using Eclipse MAT command line script for this purpose. Once I provide the hprof file as input to this and it generates suspect report. I need to parse this HTML report and make decision based on this.
This is cumbersome and also the parsing is not structured. I want either XML/JSON report or I want to see how the actual report is generated by Eclipse MAT. If I found that I can get the code and customize based on my need. Any suggestions on this.
Eclipse Memory Analyzer is open source, and all the code is here: https://git.eclipse.org/c/mat/org.eclipse.mat.git
The leak suspects code is in
https://git.eclipse.org/c/mat/org.eclipse.mat.git/tree/plugins/org.eclipse.mat.api/src/org/eclipse/mat/inspections/FindLeaksQuery.java
and
https://git.eclipse.org/c/mat/org.eclipse.mat.git/tree/plugins/org.eclipse.mat.api/src/org/eclipse/mat/inspections/LeakHunterQuery.java
The leak suspects report is clean HTML which passes HTML validation, which would aid parsing.
Your request for a XML/JSON report is a reasonable one, but is not currently provided by MAT. There is provision for one to be added, using the rendering extension point http://help.eclipse.org/latest/topic/org.eclipse.mat.ui.help/doc/org_eclipse_mat_report_renderer.html . There are already HTML, CSV and TXT renderers and so a JSON extension point could be added. There are then design decisions to be made about how to represent trees and tables in JSON, and how to convert HTML from say the yellow boxes of the leak suspects report to JSON.
As I know of other people who want a JSON formatted report it may be worthwhile following it up more formally with the MAT community, either on the forum https://www.eclipse.org/forums/index.php?t=thread&frm_id=186 or on Bugzilla or on the developer mailing list if you are proposing to help write code.
Another option is to implement your own extension: https://wiki.eclipse.org/MemoryAnalyzer/Extending_Memory_Analyzer#Calling_One_Query_from_Another and call the leak_hunter query. Then read through the details of the IResult and transform those to a json object.
I have been creating a text editor online, just for learning experience. I was curious what the best way to store multiple versions of a text file that is consistently changing is.
I've looked at a variety of options and I am yet to see a cheap, and scale-able option.
I've looked into Google Cloud Storage and Amazon S3. The only issue is that too many requests to save the file start to add up a lot in cost. I'd like files to be saved practically instantly, and also versioned every so often. I've also looked into data deduplication which looks like a great option, but I have not yet found a way to do it without writing my own software.
Any and all advice would be greatly appreciated. Thanks!
This is a very broad question, but the basic answer is usually some flavor of Operational Transform. Basically you don't want to be constantly sending the entire document back and forth between the user(s) and the server, nor do you want to overwrite the whole of the document repeatedly. Instead you want to store diffs. Then you need to deal with the idea that multiple users might be changing the file simultaneously, but possibly in different areas, and dealing with that effectively.
Wikipedia has some good, formal discussion of the idea: https://en.wikipedia.org/wiki/Operational_transformation
You wouldn't need all of that for a document that will only be edited by one person at a time, but even then, the answer is to think in terms of diffs from previous versions and only occasionally persist whole snapshots.
I would like to achieve the almost exact opposite of what can be
performed with command \internal. There exists a huge doxygen
documentation for a project already, but now I would like to pick out
a few blocks (functions, constants etc.) to create a very small manual
only containing the important stuff.
Instead of marking 99% of the comments as \internal it would be nice
to have a command like \external for the 1% of comments that need to
be exported in my case.
Something like disabling the "default section" (everything, which is
not part of a section) would work too, of course. Then I could use
ENABLED_SECTIONS...
Unfortunately the comments in question do not reside in one file only.
Furthermore those files contain a lot of other comments, which should
not be exported.
I already thought to move those comments into separate header files
that could be included in the original position, but this would mean
to restructure a lot and tearing files apart.
Does anybody have an idea how to solve my problem?
Thanks in advance,
Nico
I think ENABLED_SECTIONS is the way forward, but there's a couple of things that might reduce the workload.
The first is to create a separate doxyfile for your particular requirement, then you can customise that without upsetting any master one.
In that new doxyfile explicitly list, in the INPUT file list, only those files that contain content that you need. Chances are that it's currently set to pull in whole folder trees - edit that to cherry pick the individual files; not forgetting files that you may need to define the 'structure' of the document.
After that use ENABLED_SECTIONS with corresponding #if <SECTION_NAME> #endif markers to refine the selection to units smaller than a file.
During debugging a C# Consoleproject, about once an hour, I get the following error for a mind-boggling 20-30 seconds:
The weird part is that the source files are stored on a local SSD hard drive....
This is a workflow-disruptive completely unacceptable nuisance. Googling didn't amount to anything, do you know how to get rid of this?
Long delays are usually associated with network timeouts. There is one setting that can affect this. Right-click your solution node in the Solution Explorer window, the one on top of the tree, Properties, Common Properties, Debug Source Files. Verify that the list contains directories that are not stored on a disconnected network drive.
a mind-boggling 20-30 seconds
If you are sure about the length of the delay then it is not very likely to be a network timeout, it is too short. That kind of delay is associated with another scourge on programmers' machines, notably on an uptick in the past few months. It is the kind of delay you get from anti-malware. One of your files tripping the pattern of a known virus. The canonical example of such a problem is here. Also has a hint on how to detect it and the recommended workaround.
If none of this pans out then you'll need the help of the techniques outlined in Mark Russinovich's blog posts. Lots of examples of him using his tools to troubleshoot problems like this. A sampler is this post, doesn't match your problem but shows his approach.
Sometimes it might be useful, but mostly just looking cool or impressive to visualize log files (anything from http requests and to bandwith usage to cups of coffee drunk per day).
I know about Visitorville which I think look a bit silly, and then there's gltail.
How do you "visualize" your log files in realtime?
There is also the logstalgia tool. Visualizes Apache logs. See http://code.google.com/p/logstalgia/ for more details and a youtube video.
You may take a look at Apache Chainsaw. This nifty tool allows Log incomes from nearly everyqhere and has live filtering and colering. If you have an already written Log, I'm not sure if it can read it, it's been a while since I used it last time (was very usefull for the prototyping phase of our JBoss server)
Google has released the Visualization API that is probably flexible enough to help you:
The Google Visualization API lets you access multiple sources of structured data that you can display, choosing from a large selection of visualizations. The Google Visualization API also provides a platform that can be used to create, share and reuse visualizations written by the developer community at large.
It requires some Javascript knowledge and includes Google Docs integration, Spreadsheet integration. Check out the Gallery for some examples.
You could take a look at this. http://www.intalisys.com. 3D realtime vis app
We use Awk and Perl scripts to parse the log files and create summary reports and "databases" (technically databases in that each row corresponds to a unique event with many columns of data about that event, but not stored in a traditional database format. We're moving in that direction). I like Awk because you can very quickly search for specific strings in the log files using regex, keep counters and gather data from the log file entries, and do all kinds of calculations with that data. Then use your favorite plotting software. We use Excel, mainly because that's what was here before I started this job. I prefer MATLAB and it's open-source cousin, Octave, which is built on gnuplot.
I prefer Sawmill for visualizing data. You can basically throw any log file against it, and it will not only autodetect its structure*, but will also decide on how to analyze it. Even if you have a custom log file, you can still define what and how shall be analyzed and visualized.
I mainly use R to visualize data, but I've heard of Orange, too.
Not sure if it fits the question, but I just released this:
numStepCsvLogVis - analyze logfile data in CSV format
It uses Python's matplotlib, is motivated by the need to visualize syslog data in context of debugging kernel circular buffer operation (and variables) in C; and it visualizes by using CSV file format as intermediary to the logfile data (I cannot explain it better in brief - take a look at the README for more detail).
It has a "step" player accessed in terminal, and can handle "live" stdin input, but unfortunately, I cannot get a better response that 1 FPS when plot renders, so I wouldn't really call it "realtime" per se - but you can use it to eventually generate sonified videos of plot animations.
A simple solution is to use Logstalgia alongside the lightweight local-web-server.
First install the above. Then, from the root folder of your site visualise your logs in realtime with:
$ ws --log-format default | logstalgia -
Using SciTe, Notepad++ or other powerful text editor which have file processing routines, so you can create a script that colorizes parts of the log or just delete some non-important lines from it