I need to generate changelog from Trac for a specific version as XML and then process it with a custom XSL. It seems one of the default reports is the case (All Tickets By Milestone (Including closed)). However, if i request it as XML (by adding format=rss to the url) the output XML does not contain Status, Resolution, Milestone fields. How do i configure it to contain all the fields? How do you generate your changelogs from Trac to include it in release notes?
1) Please provide a copy of the Query (Click on SQL Query link at the bottom of the page)
What I find strange is that normally you get more columns in the CSV/RSS-XML reports than in the HTML version (see Wiki page TracReports and the extract below)
2) Personally I generate my changelogs directly from Trac into pdf. I personalised the SQL statement as much as possible to get what I want. I prefer to get a result quickly and economically rather than spand a lot of time to get exactly what I want.
===== TracReports extract =====
column -- Hide data. Prepending an underscore ('') to a column name instructs Trac to hide the contents from the HTML output. This is useful for information to be visible only if downloaded in other formats (like CSV or RSS/XML).
Related
In COGNOS is there a way to get the definitions (filters, selected fields) from a number of reports in a folder?
I've inherited around 500 reports defined in a folder and they all need to be checked and fixed as they have business errors (not technical errors). If it was possible to get all their definitions in a single extract that would save an enormous amount of time having to click multiple times to get that information from each report one by one.
In ACCESS this can be done with VBA (for query definitions), but I'm not sure if there is a scripting language that can be used with COGNOS to achieve a similar result.
It sounds like you may want to "validate" each of these 500 reports (effectively equivalent to pressing the "validate" button on each individual report if it was open in the authoring studio).
Validation will ensure that a report specification XML is still syntactically correct, references a package which is still present the content store, references only query items from that package which still exist, generates valid SQL vs. the underlying datasource, etc.
If that's what you're looking for, an easy way to do batch validation for all 500 reports would be to use MotioPI (its a free admin tool for Cognos). Here's a short article which walks you through the process:
http://info.motio.com/Blog/bid/70357/Batch-Validation-of-Cognos-Reports
If you're wanting to retrieve the actual report specification (XML) for each of these 500 objects, then you'd need to write a program which utilizes the Cognos SDK to retrieve the specification XML from each of the 500 report objects. After that, you'd need to add logic which examines each of these 500 XML documents, looking for whatever it is you're looking for.
We solved this by exporting the XML of the reports using a SQL query on the content store.
The output is processed with a Python script to convert XML to table layout in CSV format.
This CSV-file can easely be imported in Excel.
You might want to process the reports XML directly in a SQL query with the xmltable function. In our situation this turned out to be a heavy proces we don't want to burden the content store database with. For a small set of reports this is working fine though.
I am new to this topic, but my requirement is to parse documents of different types(Html, pdf,txt) using a crawlers. please suggest me what crawler to use for my requirement and provide me some tutorial s or some example how to parse the document using crawlers.
Thankyou.
This is a very broad question, so my answer is also very broad and only touches the surface.
It all comes down to two steps, (1) extracting the data from its source, and (2) matching and parsing the relevant data.
1a. Extracting data from the web
There are many ways to scrape data from the web. Different strategies can be used depending if the source is static or dynamic.
If the data is on static pages, you can download the HTML source for all the pages (automated, not manually) and then extract the data out of the HTML source. Downloading the HTML source can be done with many different tools (in many different languages), even a simple wget or curl will do.
If the data is on a dynamic page (for example, if the data is behind some forms that you need to do a database query to view it) then a good strategy is to use an automated web scraping or testing tool. There are many of these.
See this list of Automated Data Collection resources [1]. If you use such a tool, you can extract the data right away, you usually don't have the intermediate step of explicitly saving the HTML source to disk and then parsing it afterwards.
1b. Extracting data from PDF
Try Tabula first. It's an open source web application that lets you visually extract tabular data from PDFs.
If your PDF doesn't have its data neatly structured in simple tables or you have way too much data for Tabula to be feasible, then I recommend using the *NIX command-line tool pdftotext for converting Portable Document Format (PDF) files to plain text.
Use the command man pdftotext to see the manual page for the tool. One useful option is the -layout option which tries to preserve the original layout in the text output. The default option is to "undo" the physical layout of the document, and instead output the text in reading order.
1c. Extracting data from spreadsheet
Try xls2text for converting to text.
2. Parsing the (HTML/text) data
For parsing the data, there are also many options. For example, you can use a combination of grep and sed, or the BeautifulSoup Python library` if you're dealing with HTML source, but don't limit yourself to these options, you can use a language or tool that you're familiar with.
When you're parsing and extracting the data, you're essentially doing pattern matching.
Look for unique patterns that make it easy to isolate the data you're after.
One method of course is regular expressions. Say I want to extract email addresses from a text file named file.
egrep -io "\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\.[A-Z]{2,4}\b" file
The above command will print the email addresses [2]. If you instead want to save them to a file, append > filename to the end of the command.
[1] Note that this list is not an exhaustive list. It's missing many options.
[2] This regular expression isn't bulletproof, there are some extreme cases it will not cover.
Alternatively, you can use a script that I've created which is much better for extracting email addresses from text files. It's more accurate at finding email addresses, easier to use, and you can pass it multiple files at once. You can access it here: https://gist.github.com/dideler/5219706
I have a page with an interactive report. If I do a 'Control Break' and have an aggregate in place, is there a way I can export the results to Excel, exactly the way it appears on the page?
When I 'Download' the report, it appears as the third screen shot, which is not separated.
Interactive Report Results:
How I would like to export the data to Excel:
The format that is currently exported:
The download to excel is always in CSV format. The file extension is not .xlsxbut .CSV. So, i'd say no.
It's tough too. Even if you were to create a custom export to excel you'd have to extract the current query of the report (which is something that has finally been made easier in 4.2, but is possible in 4.0/1 with 3rd party packages). Then you'd also have to account for the control break(s) you applied, since those are not reflected in the IR query (even with APEX_IR).
I've dabbled with generating an xlsx file and made a blogpost/sample application on that if you'd like to see what it encompasses. Be aware that this is taking 'custom solution' to the extreme though (at least, in my opinion).
http://apex.oracle.com/pls/apex/f?p=10063
you could create the report in BI Publisher in Oracle, then through APEX, you can call the report with parameters.
Actually APEX Office Print (AOP) supports exporting for Interactive Report and Interactive Grid (and others) to Excel, exactly as you see on the screen (so including breaks, group by etc)
Each month I have to manually create a report for my team. For this I use a combination of numbers generated from a groovy script I've written, screenshot, cut & paste from a series of charts in our JIRA instance and cut and pasted excel charts.
I'm finding it more and more boring to do this by hand and I suspect that there is an easier way to automatically generate this report. I can generate the word document with data from the groovy script using this application (http://www.docmosis.com/) but I have not yet found a way of auto including JIRA charts in the generation or excel so my question is:
Is it possible to generate a word document that contains generated data from various sources including (in order of importance):
JIRA charts
Programmatically generate data and
Excel charts?
As you already coded your logic in Groovy, best would be skipping the Excel step and generate your report directly from JIRA using the JIRA PDF View Plugin. This plugin will reduce your work to a single click and download you a nice PDF document (PDF is more portable than Word docs).
These short tutorials help you:
Reuse your Groovy scripts for reporting logic (also see the "Charts" tutorial there)
Bring the data, charts and the template together
Update 2 years after my original answer:
JIRA PDF View Plugin 3.5.0+ can also export JIRA charts (gadgets in JIRA dashboards, to be precise). So, now you can execute your Groovy scripts, include JIRA charts and bring all the result together in a professional PDF document.
Even better, now there exists a free plugin that will make the monthly process 100% automatic for you! It will generate your PDF document and send that to your team members' mailboxes, according to the preferred schedule (CRON trigger).
Related tutorials:
Exporting JIRA dashboards to PDF
Automating JIRA PDF exports
Discl: I'm a developer working on these products, yet this is the best solution to your problem. :-)
Most of the charts generated in Jira are created from data returned from search results. So you can get data from Jira using REST or SOAP api and generate your own charts by other means.
I am not sure if there is a no-coding method of doing it. But I would write a C# program that adds data to the Word template from your sources (Jira, other programs, Excel). And would create a template that generated charts from inserted data.
For Word document generation I would have a look here -http://www.codeproject.com/Articles/20287/Generating-Word-Reports-Documents
For chart creation in the document I would have a look here - http://office.microsoft.com/en-us/word-help/create-a-chart-in-a-document-HP005190046.aspx
Easiest if you convert your JIRA chart to an image first. When you say JIRA chart, do you mean something from this plugin?
Once you have an image, your document generation tool ought to be able to include that.
There are many document generation tools which can include programmatically generated data, and some of those can include Excel charts.
There is now a JIRA plugin to generate charts in Word documents which may help you.
https://marketplace.atlassian.com/plugins/com.clariostechnology.officecharts.officecharts
There is now also Intelligent Reporter for JIRA which is a no-code method for creating word documents from your JIRA data, including native Word charts with full formatting from JIRA data.
https://marketplace.atlassian.com/plugins/com.clariostechnology.intelligentreports
Disclaimer: I work for the company that created this plugin. If there is anything you need that it cannot do, please let me know.
At the moment, we use MS WORD and MS EXCEL to mail merge documents that needs to be sent to multiple recepients.
For example, say there is a complaint form where the complainant needs to fill in his/her name, address, etc. So we have a .doc file set up with the content and the dynamic entities set up for mail merging, with the name and address details put in an excel file, from where we can happily mail merge to generate all or just the necessary forms/documents.
However, I would like to automate this process, like a form in a website where the complainant can fill in his/her name, address and other details, and we could use that to generate the complaint form automatically and offer it to be downloaded (preferrably as a pdf).
Now, the only solution that comes to mind, is Latex, so that I can just replace the needed entities and just compile to PDF. However, that bit has to be negotiated with the webhost, if they are offering Latex or not.
Is there any other solution? Any other way we could get this done, with something that shouldn't be a problem for most webhosting solutions to offer?
EDIT: I would prefer a non .NET or rather non microsoft solution since, the servers are running linux and while mono might be capable of getting the job done, none of our devs know any .NET languages. However, if required we might have to dwelve into it.
Generating PDF using an XSL. Check the following: Apoc XSL-FO
You will need to create an XML file with the required fields and transform that with this tool.
If you wish to avoid .NET then XSL-FO is worth a look. Try the FOray project.
XSLT can be a steep learn if you do not have experience already. Also users will not be able to change the templates without asking the XSLT guru to do it.
If your templates are already in MS Word and MS Excel then I would stick with generating MS docs on the server. These are now easy to work with from code since OpenXML - check out OfficeOpenXML and OpenXMLDeveloper
Apache FOP : http://xmlgraphics.apache.org/fop/
I suggest generating rtf on the server: it's easy enough to automatically generate using cpan's RTF::Writer, has converters generating good pdf, can be edited by hand in word, oo-writer & TextEdit, doesn't have any really bad compatibility issues between the main editing applications, and has decent text & resource extraction tools, with text extraction being rather better than pdf.
There's some support for moving between rtf & latex, although the best rtf -> latex converter, docx2tex, depends on the System.IO.Packaging .net module, whose mono implementation isn't yet rock solid.
Postscript — Not a recommendation: it's too much of an unwieldy sledgehammer for this job, but iText will generate the pdf directly from the form data. If you wanted to do fancy things like signed pdf, that would be the way to go.
Postscript #2 — If you break up the Word document into individual files using word's master document representation, then you can clobber one of the parts with hand-generated content. This makes it easy to do something approximating form-filling on word .doc files using just standard file-utils and some trivial rtf->doc tweaking.