CSV to XML trnformation using cloveretl - cloveretl

I want to transform my csv file to XML using cloveretl.
I gone through the basic tutorial, all explains abound direct mapping from csv to xml, csv header column names are used for xml element name.
I have one complex XSD, and I want to map csv to that XSD generated xml.
When I generate metadata using my XSD 213 fmt file generated in cloveretl.
How do map all these together ?
I saw an option to map individually, one csv metadata to one fmt. Like this I have to do 213 mapping and combine all ?

I assume you have two components: UniversalDataReader and XMLWriter. The edge between them should not have the metadata from the XSD schema (you do not have to extract metadata from the XSD at all), it should have metadata extracted from the input CSV file. Otherwise, you would not be able to read the file in the first place.
Then, in XMLWriter, you can set the XSD schema or create the mapping manually. For more information, see http://doc.cloveretl.com/documentation/UserGuide/topic/com.cloveretl.gui.docs/docs/extxmlwriter.html#xsd-mapping
If the CSV and the result XML are simple enough, you do not need the XSD schema at all.

Related

How to count values in a flat file based on the spaces? [duplicate]

Since we can query on the XML file from C# (.NET), why do we need an XSD file? I know it is metadata file of particular XML file. We can specify the relationships in XSD, but what is its functioning then?
XML
<?xml version="1.0" encoding="utf-8" ?>
<Root>
<Customers>
<Customer CustomerID="GREAL">
<CompanyName>Great Lakes Food Market</CompanyName>
<ContactName>Howard Snyder</ContactName>
<ContactTitle>Marketing Manager</ContactTitle>
<Phone>(503) 555-7555</Phone>
<FullAddress>
<Address>2732 Baker Blvd.</Address>
<City>Eugene</City>
<Region>OR</Region>
<PostalCode>97403</PostalCode>
<Country>USA</Country>
</FullAddress>
</Customer>
</Customers>
<Orders>
<Order>
<CustomerID>GREAL</CustomerID>
<EmployeeID>6</EmployeeID>
<OrderDate>1997-05-06T00:00:00</OrderDate>
<RequiredDate>1997-05-20T00:00:00</RequiredDate>
<ShipInfo ShippedDate="1997-05-09T00:00:00">
<ShipVia>2</ShipVia>
<Freight>3.35</Freight>
<ShipName>Great Lakes Food Market</ShipName>
<ShipAddress>2732 Baker Blvd.</ShipAddress>
<ShipCity>Eugene</ShipCity>
<ShipRegion>OR</ShipRegion>
<ShipPostalCode>97403</ShipPostalCode>
<ShipCountry>USA</ShipCountry>
</ShipInfo>
</Order>
<Order>
<CustomerID>GREAL</CustomerID>
<EmployeeID>8</EmployeeID>
<OrderDate>1997-07-04T00:00:00</OrderDate>
<RequiredDate>1997-08-01T00:00:00</RequiredDate>
<ShipInfo ShippedDate="1997-07-14T00:00:00">
<ShipVia>2</ShipVia>
<Freight>4.42</Freight>
<ShipName>Great Lakes Food Market</ShipName>
<ShipAddress>2732 Baker Blvd.</ShipAddress>
<ShipCity>Eugene</ShipCity>
<ShipRegion>OR</ShipRegion>
<ShipPostalCode>97403</ShipPostalCode>
<ShipCountry>USA</ShipCountry>
</ShipInfo>
</Order>
</Orders>
</Root>
I want to get data from the Order elements according to a provided CustomerID.
Also: What is the purpose of giving the relationships in XSD?
XSD files are used to validate that XML files conform to a certain format.
In that respect they are similar to DTDs that existed before them.
The main difference between XSD and DTD is that XSD is written in XML and is considered easier to read and understand.
Without XML Schema (XSD file) an XML file is a relatively free set of elements and attributes. The XSD file defines which elements and attributes are permitted and in which order.
In general XML is a metalanguage. XSD files define specific languages within that metalanguage. For example, if your XSD file contains the definition of XHTML 1.0, then your XML file is required to fit XHTML 1.0 rather than some other format.
You mention C# in your question so it may help to think of as XSD as serving a similar role to a C# interface.
It defines what the XML should 'look like' in a similar way that an interface defines what a class should implement.
XSDs constrain the vocabulary and structure of XML documents.
Without an XSD, an XML document need only follow the rules for being well-formed as given in the W3C XML Recommendation.
With an XSD, an XML document must adhere to additional constraints placed upon the names and values of its elements and attributes in order to be considered valid against the XSD per the W3C XML Schema Recommendation.
XML is all about agreement, and XSDs provide the means for structuring and communicating the agreement beyond the basic definition of XML itself.
Also questions is: What is the purpose
of giving the relationships in xsd.
Suppose you want to generate some XML for an external party's tool, or similar - how would you know what structure it is allowed to follow to be used correctly for their tool? you write to a schema. Likewise if you want other people to use your tool, you would write a schema for them to follow. It may also be useful for validating your own XML.
Before understanding the XSD(XML Schema Definition) let me explain;
What is schema?
for example; emailID: peter#gmail
You can identify the above emailID is not valid because there is no #, .com or .net or .org.
We know the email schema it looks like peter#gmail.com.
Conclusion: Schema does not validate the data, It does the validation of structure.
XSD is actually one of the implementation of XML Schema. others we have relaxng
We use XSD to validate XML data.
An XSD is a formal contract that specifies how an XML document can be formed. It is often used to validate an XML document, or to generate code from.
An XSD file is an XML Schema Definition and it is used to provide a standard method of checking that a given XML document conforms to what you expect.
An .xsd file is called an XML schema. Via an XML schema, we may require a certain structure in a given XML - which elements in which order, how many times, with which attributes, how they are nested, etc. If we have a schema for our XML input, we can verify that it contains the data we need it to contain, and nothing else, with a few lines invoking a schema validator.
The xsd file is the schema of the xml file - it defines which elements may occur and their restrictions (like amount, order, boundaries, relationships,...)

How to include an XML schema (.xsd) in a JSON schema?

I want to define a JSON API response using JSON Schema.
Embedded in part of the API response is a complete, well-formed, schema valid XML string. The XSD of this XML string is a given.
Two part question:
How do I include the XSD in the JSON Schema such that the JSON Schema will also require the XML string to be schema valid in order for the whole API response to be valid?
If this is not possible, does anyone have another suggestion how to include the XSD at least in the specification? I'm working in RAML 0.8.
How do I include the XSD in the JSON Schema such that the JSON Schema
will also require the XML string to be schema valid in order for the
whole API response to be valid?
You cannot. The only thing you can do is validate the JSON and then at a later point extract the XML and validate it separately.
If this is not possible, does anyone have another suggestion how to
include the XSD at least in the specification? I'm working in RAML
0.8.
I've only used Swagger, not RAML. Swagger is also based on JsonSchema.
The only thing you can do here is to include a detailed specification that the contained XML should be compliant against such-and-such an XSD. You can do this by using the "description" functionality in swagger (or equivalent if it exists in RAML). This allows you to create a description (which supports markdown) and attach it to any element in the definition

Flat file Schema (Delimited) creation wizard in Biztalk server 2015

i want to add nested child record with elements in Positional File (Flat File schema Creation wizard) using Biztalk,
for Eg. Generated instance to be like this: <Root><Child_Rec1><Child_Ele>ELEMENT</Child_Ele></Child_Rec1></Root>
Sorry, what you are asking for is not possible with the Flat File Disassemler.
It is not supported to have Delimited content within Positional Content.
The way to work around this is the split the delimited content in a Map and set the target fields.

How do I store the contents of an array in tablestorage

I need to store the contents of an array into Azure tablestorage. The array will have between 0 and 100 entries. I don't want to have to create 100 different elements so is there a way I can pack up the array, store it and unpack it later. Any examples would be much appreciated. I just don't know where to start :-(
You need to serialize the array into binary or xml and then use the appropriate column type to store the data (binary object or xml.)
XML will be the most flexible because you can still query the values while they are in storage. (You can't query binary data. Not easily anyway.) Here is an example of serializing and here is one for inserting the value into a table.
Some detail on XML support in Azure:
The xml Data Type
SQL Azure Database supports xml data
type that stores XML data. You can
store xml instances in a column or in
a variable of the xml type.
Support for XML Data Modification
Language
The XML data modification language
(XML DML) is an extension of the
XQuery language. The XML DML adds the
following case-sensitive keywords to
XQuery and they are supported in SQL
Azure Database:
insert (XML DML)
delete (XML DML)
replace value of (XML DML)
Support for xml Data Type Methods
You can use the xml data type methods
to query an XML instance stored in a
variable or column of the xml type.
SQL Azure Database supports the
following xml data type methods:
query() Method (xml data type)
value() Method (xml data type)
exist() Method (xml data type)
modify() Method (xml data type)
nodes() Method (xml data type)
If you really are starting out in Azure Table Storage, then there are a few nice "simple" tutorials around - e.g. http://blogs.msdn.com/b/jnak/archive/2008/10/28/walkthrough-simple-table-storage.aspx
Once you are happy with reading/writing entities then there are several ways you can map your array to Table Storage.
If you ever want to access each element of your array separately from the persistent storage, then you should create 0 to 99 separate entities - each with their own entity in the Table store.
If you don't ever want to access them separately, then you can just store the array in a single entity (row) in the table - e.g. using PartitionKey="MyArrays", RowKey="" and having another column which contains the array serialised to e.g. JSON.
As a variation on 2, you could also store the array items - 0 to 99 - in separate columns ("Array_0",..."Array_99") in the row. There are ways you could map this to a nice C# Array property using the Reading/Writing events on the table storage entity - but this might not be the best place to start if you're beginning with Azure.
Be careful, besides the 1MB entity limit there is a per field limit as well (I think it's 64kb)
Your best bet is to use the Lokad Fat Entity
http://code.google.com/p/lokad-cloud/wiki/FatEntities

How to get an ordered list parsed by XML parser?

I am using a xsd schema file; there I specified an ordered list.
When parsing an XML node of the kind...
<myOrderedList> "element_1" "element_2" "element_3" </myOrderedList>
(which is valid XML syntax)
...all XML parsers I know parse this as a single node element.
Is there a way to get the XML parser parse this list for me (return it as a list or an array or whatever) or do I always have to parse it myself?
Why not make use of XML's ability to structure your data, and put each element in it's own XML element ? e.g.
<myOrderedList>
<element>1</element>
<element>2</element>
<element>3</element>
</myOrderedList>
etc. Otherwise you're having to implement parsing (albeit in a simple fashion) on top of the parsing effort that the XML parser is performing for you ?
If you do the above, the parser will return you the ordered list without any further work, and/or you can process it more easily using standard XML tooling like XSLT/XQuery etc.
Values in a list type are always separated by whitespaces so you can easily pass that through a tokenizer to get the list of values.
There are technologies like XSLT 2.0 schema aware that will see the list of values for such an element.
Using elements as proposed in the other answer is also a solution and may ease your processing. In XML child nodes are ordered so you should not worry about that. A possible representation would be:
<myOrderedList>
<value>element_1</value>
<value>element_2</value>
<value>element_3</value>
</myOrderedList>

Resources