can we export systrace output as CSV file, so that i can parse and generate report.
There's no built-in facility for generating reports in different formats. However, if you look at the HTML, you can see that it's split into a block of javascript code that implements the viewer, followed by the data, one entry per line. Writing a script that converts the data to the format of your choice is straightforward.
Related
I'm trying to access to my iCloud Notes with a python script using pyiCloud framework, but when I try to list the notes it seems that Documents folder is empty. Does anyone know how I should make that?
>>> api.files['com~apple~Notes']['Documents'].dir() It returns:
>>> []
It sounds like you have an authentication problem that you can't get access to your Notes from the file storage (the UbiquityService). This issue might give you some more clues.
On the other note (!), I found the following a better way to get my Notes exported. I have tried a couple of ways mentioned around the web. Although there is no in-app solution to export Notes in a format other than PDF files, I have stumbled upon the following two solutions:
Export in Markdown (or in other formats in the paid version) via the Bear app. I found this way easier and of more quality in terms of keeping the formatting, attachments, etc:
Download the Bear migration Workflow script from here and follow the instructions.
[optional] At this point, you have the HTML files with inline encoded images. Use my script to decode images to get regular HTML files with the images in an accompanying directory.
Install Bear and import the exported files from Notes.
Export the files as Markdown, HTML, or whatever format you desire from File -> Export Notes within the Bear app. Don't forget to check the "Export attachments" box in the export dialog.
Export in HTML (and then convert to Markdown if you want) via the Notes Exporter app. The app gives you HTML files with inline encoded Base64 images saved with .txt extension (?!). Although I personally like this way as it generates output files that mimic closely the original Notes, the hyperlinks are missing in the exported files (it still keeps the hyperlink coloring though):
Download Notes Exporter from here.
Export Notes to the path you choose.
[optional] Rename file extensions to .html.
Voilà, now you have your Notes as HTML files with the same formatting and images.
Decode inline Base64 images and save HTML files with images saved in a separate adjacent directory using the script that I wrote for this:
https://gist.github.com/SHi-ON/945ea2272ea4bb29e13bd0305370da90
Hope this helps to give you an idea!
In work place everyday we used different type of documents to hold data. For example, DOC, XLSX, PDF files. And sometime we use software (like adobe reader) excel to PDF converter.
As far i know another way to convert a document from excel to pdf is changing the document type from the SaveAs option (correct me if i am wrong) or changing the file extension.
My question is when we change the Document type from save as option does it change the code behind the file?
Another silly question is if we can convert the file by changing extension why we are paying for 3rd party software?!
Every document type has its format. So behind the screen, every type has its style. For example, XLSX format is a combination of XML and zip compression. PDF is a rich document representation format created by Adobe uses PostScript.
When you save a document as XLSX, the document will be saved as its standards. The saving method will be changed. As an answer to your first question, Yes the coding(method) will be changed when saving.
For the second question, the changing file format is not always an easy task. You need to change the encoding of the file when performing the conversion. When you change the extension you do not apply any conversion operation. You say your computer "This is an ... file.". But the encoding of the file is still unchanged.
I'd need something like Smooks but that could execute on Node.js.
I have the need of transforming various XML formats to/from other formats and also CSV, EDI, etc.
The tool should contain some format tool where you can define the format and then some mapping component to map input format to output format.
Have you seen this lib xml2rec?
I use an excel-based automation framework where objects' names are parameterized into the excel sheet that drives test execution.
I need to import the QTP Object Repository to Excel/Spreadsheet in a simple readable format so that I can write a macro to fetch the objects' logical name alone into the excel sheet.
Is this possible? If so please explain.
(I understand that we have the option to import in XML format, but that is not helping much.)
You cannot export the Object Repository to Excel directly. You can only export it to XML. If you do not find the XML format useful, you will have to determine what is useful for you. You could take the XML file and convert it to different formats using XML Stylesheets (XSL). You could write a script that would parse the XML nodes for the test objects and just output the names. There are many options available once you have the data in a standard format like XML.
If you need more assistance, I suggest you post a sample of an Object Repository structure you want to export to Excel, and then post a sample of how you want that data presented in Excel.
I am new to this topic, but my requirement is to parse documents of different types(Html, pdf,txt) using a crawlers. please suggest me what crawler to use for my requirement and provide me some tutorial s or some example how to parse the document using crawlers.
Thankyou.
This is a very broad question, so my answer is also very broad and only touches the surface.
It all comes down to two steps, (1) extracting the data from its source, and (2) matching and parsing the relevant data.
1a. Extracting data from the web
There are many ways to scrape data from the web. Different strategies can be used depending if the source is static or dynamic.
If the data is on static pages, you can download the HTML source for all the pages (automated, not manually) and then extract the data out of the HTML source. Downloading the HTML source can be done with many different tools (in many different languages), even a simple wget or curl will do.
If the data is on a dynamic page (for example, if the data is behind some forms that you need to do a database query to view it) then a good strategy is to use an automated web scraping or testing tool. There are many of these.
See this list of Automated Data Collection resources [1]. If you use such a tool, you can extract the data right away, you usually don't have the intermediate step of explicitly saving the HTML source to disk and then parsing it afterwards.
1b. Extracting data from PDF
Try Tabula first. It's an open source web application that lets you visually extract tabular data from PDFs.
If your PDF doesn't have its data neatly structured in simple tables or you have way too much data for Tabula to be feasible, then I recommend using the *NIX command-line tool pdftotext for converting Portable Document Format (PDF) files to plain text.
Use the command man pdftotext to see the manual page for the tool. One useful option is the -layout option which tries to preserve the original layout in the text output. The default option is to "undo" the physical layout of the document, and instead output the text in reading order.
1c. Extracting data from spreadsheet
Try xls2text for converting to text.
2. Parsing the (HTML/text) data
For parsing the data, there are also many options. For example, you can use a combination of grep and sed, or the BeautifulSoup Python library` if you're dealing with HTML source, but don't limit yourself to these options, you can use a language or tool that you're familiar with.
When you're parsing and extracting the data, you're essentially doing pattern matching.
Look for unique patterns that make it easy to isolate the data you're after.
One method of course is regular expressions. Say I want to extract email addresses from a text file named file.
egrep -io "\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\.[A-Z]{2,4}\b" file
The above command will print the email addresses [2]. If you instead want to save them to a file, append > filename to the end of the command.
[1] Note that this list is not an exhaustive list. It's missing many options.
[2] This regular expression isn't bulletproof, there are some extreme cases it will not cover.
Alternatively, you can use a script that I've created which is much better for extracting email addresses from text files. It's more accurate at finding email addresses, easier to use, and you can pass it multiple files at once. You can access it here: https://gist.github.com/dideler/5219706