How to resolve: XML schema created in Excel contains denormalized data - excel

Edit/Update: By removing the <GrpHdr> element completely, Excel was able to verify the XML Map as exportable. My original question still remains, how can I solve the "Denormalized Data" error, with the <GrpHdr> included.
I am new to XML, and have been trying to import a source file (XML below) into Excel, create a schema/XML Map (unsure of the difference there) which I can then drag and drop onto two different tables:
One table contains one row of data for the Group Header: <GrpHdr> (Occurs ONCE)
One table contains multiple rows of data for the various Payments: <PmtInf> (Occurs MULTIPLE times)
I am able to successfully load the below XML into Excel using the Source button, and also to create an XML map off of it (which then appears in a "XML Source" window, showing the parent and child elements).
The problem I am having is in Verifying the XML Map for export. Excel says that the map contains "Denormalized Data". I have looked at various Microsoft resources, as well as on Stack Overflow.
Such as:
https://support.microsoft.com/en-us/office/issue-verifying-an-xml-map-for-export-fbfcdb77-c2d6-4040-b256-e584a71151b0
excel: Cannot save or export xml data. The xml map in this workbook are not exportable
Export denormalized data from excel to xml
Based on my research, I tried the following:
I have tried setting the MinOccurs and MaxOccurs attributes to be "0" and "unbounded" respectively, as I believe the default is "1" for both, and Excel's error saying that the XML Map contains "Denormalized Data" is due to having an element with the MaxOccurs set to "1".
I have also tried adding multiple <PmtInf> elements, so Excel knows (when creating a schema from the below sample file), that <PmtInf> is to occur multiple times.
Thanks!
<?xml version="1.0" encoding="utf-8"?>
<Document xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.001.03">
<CstmrCdtTrfInitn>
<GrpHdr>
<MsgId>UNIQUE MESSAGE ID 35 AN</MsgId>
<CreDtTm>2016-05-26T10:07:00</CreDtTm>
<NbOfTxs>1</NbOfTxs>
<CtrlSum>0.01</CtrlSum>
<InitgPty>
<Id>
<OrgId>
<Othr>
<Id>ABC12345678</Id>
</Othr>
</OrgId>
</Id>
</InitgPty>
</GrpHdr>
<PmtInf>
<PmtInfId>ORIGINATOR REFERENCE 35AN</PmtInfId>
<PmtMtd>TRF</PmtMtd>
<PmtTpInf>
<SvcLvl>
<Cd>SEPA</Cd>
</SvcLvl>
</PmtTpInf>
<ReqdExctnDt>2016-05-26</ReqdExctnDt>
<Dbtr>
<Nm>DEBTOR NAME 70AN</Nm>
<PstlAdr>
<StrtNm>Street Name</StrtNm>
<BldgNb>Building Number</BldgNb>
<PstCd>Post Code</PstCd>
<TwnNm>Town Name</TwnNm>
<CtrySubDvsn>County/State/Region</CtrySubDvsn>
<Ctry>LU</Ctry>
</PstlAdr>
</Dbtr>
<DbtrAcct>
<Id>
<IBAN>NL39HSBC0123456789</IBAN>
</Id>
</DbtrAcct>
<DbtrAgt>
<FinInstnId>
<BIC>HSBCNL2A</BIC>
<PstlAdr>
<Ctry>IE</Ctry>
</PstlAdr>
</FinInstnId>
</DbtrAgt>
<ChrgBr>SLEV</ChrgBr>
<CdtTrfTxInf>
<PmtId>
<InstrId>PAYMENT ID 35AN</InstrId>
<EndToEndId>UNIQUE BENEFICIARY REFERENCE 35AN</EndToEndId>
</PmtId>
<Amt>
<InstdAmt Ccy="EUR">0.01</InstdAmt>
</Amt>
<CdtrAgt>
<FinInstnId>
<BIC>MIDLGB22</BIC>
<PstlAdr>
<Ctry>GB</Ctry>
</PstlAdr>
</FinInstnId>
</CdtrAgt>
<Cdtr>
<Nm>CREDITOR NAME 70AN</Nm>
<PstlAdr>
<StrtNm>Street Name</StrtNm>
<BldgNb>Building Number</BldgNb>
<PstCd>Post Code</PstCd>
<TwnNm>Town Name</TwnNm>
<CtrySubDvsn>County/State/Region</CtrySubDvsn>
<Ctry>GB</Ctry>
</PstlAdr>
</Cdtr>
<CdtrAcct>
<Id>
<IBAN>GB94MIDL40123487654321</IBAN>
</Id>
</CdtrAcct>
<RmtInf>
<Ustrd>Remittance Info up to 140AN</Ustrd>
</RmtInf>
</CdtTrfTxInf>
</PmtInf>
</CstmrCdtTrfInitn>
</Document>

Related

How to access xml field with lxml?

Python 3.6, Lxml, Windows 10
I am getting crazy. I want to access the item field. But I always get the error:
AttributeError: 'cython_function_or_method' object has no attribute'item'
Everything else (address fields etc...) I can access without problems. How can I access the item fields (sku, amount etc...)?
I've used this code:
import requests
from lxml import objectify
url = "URL_TO_XML_FILE"
xml_content = requests.get(url).text.encode('utf-8')
xml = objectify.fromstring(xml_content)
for sale in xml.response.sales.sale:
for item in sale.items.item:
print(item.sku)
Here is the beginning of the xml:
<?xml version="1.0" encoding="ISO-8859-1"?>
<getnewsalesresult xmlns="https://pmcdn.priceminister.com/res/schema/getnewsales">
<request>
<version>2017-08-07</version>
<user>SELLER</user>
</request>
<response>
<lastversion>2017-08-07</lastversion>
<sellerid>95029358</sellerid>
<sales>
<sale>
<purchaseid>297453287592813953</purchaseid>
<purchasedate>15/12/2018-19:10</purchasedate>
<deliveryinformation>
<shippingtype>Normal</shippingtype>
<isfullrsl>N</isfullrsl>
<purchasebuyerlogin><![CDATA[LOGIN]]></purchasebuyerlogin>
<purchasebuyeremail>EMAIL</purchasebuyeremail>
<deliveryaddress>
<civility>Mme</civility>
<lastname><![CDATA[Lastname]]></lastname>
<firstname><![CDATA[Firstname]]></firstname>
<address1><![CDATA[STREET]]></address1>
<address2><![CDATA[]]></address2>
<zipcode>13570</zipcode>
<city><![CDATA[Paris]]></city>
<country><![CDATA[France]]></country>
<countryalpha2>FX</countryalpha2>
<phonenumber1></phonenumber1>
<phonenumber2>PHONENUMBER</phonenumber2>
</deliveryaddress>
</deliveryinformation>
<items>
<item>
<sku><![CDATA[SKU1]]></sku>
<advertid>411812243030</advertid>
<advertpricelisted>
<amount>15.99</amount>
<currency>EUR</currency>
</advertpricelisted>
<itemid>551131040</itemid>
<headline><![CDATA[HEADLINE]]></headline>
<itemstatus><![CDATA[REQUESTED]]></itemstatus>
<ispreorder>N</ispreorder>
<isnego>N</isnego>
<negotiationcomment></negotiationcomment>
<price>
<amount>15.99</amount>
<currency>EUR</currency>
</price>
<isrsl>N</isrsl>
<isbn></isbn>
<ean>4363745894373857474; </ean>
<paymentstatus><![CDATA[INCOMING]]></paymentstatus>
<sellerscore></sellerscore>
</item>
</items>
</sale>
<sale>
The problem is that items is actually a method of ObjectifiedElement, so the expression sale.items actually returns the method, because it has precedence.
To get the 'items' object you want, you have to be more explicit about getting the attribute of sale and not looking for methods of the class first, which is the usual python order. This is what python does behind the scene when you access an attribute, and you can do it too:
sale.__getattr__('items')
This will also work (it's a dictionary-like interface to the attributes of an object):
sale.__dict__['items']
The revised code:
import requests
from lxml import objectify
url = "URL_TO_XML_FILE"
xml_content = requests.get(url).text.encode('utf-8')
xml = objectify.fromstring(xml_content)
for sale in xml.response.sales.sale:
for item in sale.__dict__['items'].item:
print(item.sku)
Another way to deal with this is to avoid using the flaky attribute interface:
for sale in xml['response']['sales']['sale']:
for item in sale['items']['item']:
print(item['sku'])
Using the dict-like indexing interface, you never have to worry about certain attributes names (which includes such common words as items, index, keys, remove, replace, tag, set, text, and values) returning surprising results.

Avoid nesting of element in RDF/XML in Apache Jena

I am trying to writer a exporter utility to SKOS using Apache Jena. My issue is that the broader or narrower objects are getting nested. I am expecting the following xml but getting the xmls with nested elements. I am not getting any help from tutorials. Is it just a formatting issue or something to do with the way I am coding for it ?
Actual Output
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:SKOS="http://www.w3.org/2004/02/skos/core#">
<SKOS:Concept rdf:about="http://lexicon.ai/P011">
<SKOS:broader>
<SKOS:Concept>
<SKOS:narrower>
<SKOS:Concept>
<SKOS:scopeNote>testb</SKOS:scopeNote>
<SKOS:prefLabel>Disease</SKOS:prefLabel>
</SKOS:Concept>
</SKOS:narrower>
<SKOS:scopeNote>testb</SKOS:scopeNote>
<SKOS:prefLabel>Disease</SKOS:prefLabel>
</SKOS:Concept>
</SKOS:broader>
<SKOS:altLabel>alt2</SKOS:altLabel>
<SKOS:altLabel>alt1</SKOS:altLabel>
<SKOS:scopeNote>test</SKOS:scopeNote>
<SKOS:prefLabel>Disease</SKOS:prefLabel>
</SKOS:Concept>
</rdf:RDF>
Expected Output
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:SKOS="http://www.w3.org/2004/02/skos/core#">
<SKOS:Concept rdf:about="http://lexicon.ai/P011">
<SKOS:broader rdf:about="http://lexicon.ai/P012"/>
<SKOS:altLabel>alt2</SKOS:altLabel>
<SKOS:altLabel>alt1</SKOS:altLabel>
<SKOS:scopeNote>test</SKOS:scopeNote>
<SKOS:prefLabel>Disease</SKOS:prefLabel>
</SKOS:Concept>
<SKOS:Concept rdf:about="http://lexicon.ai/P012">
<SKOS:narrower rdf:about="http://lexicon.ai/P0121"/>
<SKOS:scopeNote>testb</SKOS:scopeNote>
<SKOS:prefLabel>Diseaseb</SKOS:prefLabel>
</SKOS:Concept>
<SKOS:Concept rdf:about="http://lexicon.ai/P0121">
<SKOS:scopeNote>testn</SKOS:scopeNote>
<SKOS:prefLabel>Diseasen</SKOS:prefLabel>
</SKOS:Concept>
</rdf:RDF>
Codes is as follows
Model model = ModelFactory.createDefaultModel();
model.setNsPrefix("SKOS", SKOS.uri);
Model model2 = ModelFactory.createDefaultModel();
model2.setNsPrefix("SKOS", SKOS.uri);
final Resource Entity = model.createResource(personURI);
final Resource broader1 = model.createResource();
final Resource nt1 = model.createResource();
nt1.addProperty(RDF.type, SKOS.Concept);
nt1.addProperty(SKOS.prefLabel, "Diseasen");
nt1.addProperty(SKOS.scopeNote, "testn");
broader1.addProperty(RDF.type, SKOS.Concept);
broader1.addProperty(SKOS.prefLabel, "Diseaseb");
broader1.addProperty(SKOS.scopeNote, "testb");
broader1.addProperty(SKOS.narrower, nt1);
Entity.addProperty(RDF.type, SKOS.Concept);
Entity.addProperty(SKOS.prefLabel, "Disease");
Entity.addProperty(SKOS.scopeNote, "test");
"http://lexicon.ai/P011" does not appear in the code sample Entity does not appear to be used.
There are two model.createResource() which will create 2 blank nodes.
The "actual" output shows one resource was created with createResource("http://lexicon.ai/P011") and one with a blank node. That looks like the cause of the nesting.
To get nearer to the required output, you will need to use to named resources and may be better off with the more basic writer, RDFFormat.RDFXML_PLAIN writing using RDFDataMgr.write.

Simple XSLT transformation into ABAP Object

I'm again stuck with a transformation from XML into ABAP. This time, I want to put the XML data directly into an ABAP Object.
My XML looks like this:
<qualityStatus>
<address>0</address>
<bounceRisk>0</bounceRisk>
<checked>1</checked>
<domain>1</domain>
<domainScores>
<domainScore>
<domain>gmx.de</domain>
<score>0.8333333134651184</score>
</domainScore>
<domainScore>
<domain>ggs.de</domain>
<score>0.6666666269302368</score>
</domainScore>
<domainScore>
<domain>xyz.de</domain>
<score>0.6666666269302368</score>
</domainScore>
</domainScores>
<extSyntax>1</extSyntax>
<mailserver>1</mailserver>
<mailserverDiagnosis>1</mailserverDiagnosis>
<probability>1</probability>
<syntax>1</syntax>
</qualityStatus>
Edit: I changed back to a XSLT transformation, shortened to one attribute it looks like this:
<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:sap="http://www.sap.com/sapxsl" version="1.0">
<xsl:output encoding="iso-8859-1" indent="yes" method="xml" version="1.0"/>
<xsl:strip-space elements="*"/>
<xsl:template match="/qualityStatus">
<asx:abap version="1.0" xmlns:asx="http://www.sap.com/abapxml">
<asx:values>
<ROOT href="#o26"/>
</asx:values>
<asx:heap xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:abap="http://www.sap.com/abapxml/types/built-in"
xmlns:cls="http://www.sap.com/abapxml/classes/global"
xmlns:dic="http://www.sap.com/abapxml/types/dictionary">
<cls:ZCL_ADDRESS_QUALITY id="o26" >
<local.ZCL_ADDRESS_QUALITY>
<W_ADDRESS>
<xsl:value-of select="address"/>
</W_ADDRESS>
<!--More attributes here-->
</local.ZCL_ADDRESS_QUALITY>
</cls:ZCL_ADDRESS_QUALITY>
</asx:heap>
</asx:abap>
</xsl:template>
My object attributes are all public right now, because I thought this could be the problem. However, setter and getter do exist. Yes, my class does implement the interface if_serializable_object.
DATA:
w_address TYPE char1,
w_bouncerisk TYPE char1,
w_checked TYPE char1,
w_decoded TYPE stringval,
w_domain TYPE char1,
w_domainscores TYPE z_domainscore_t, "Table type for name + score
w_extsyntax TYPE char1,
w_mailserver TYPE char1,
w_mailserverdiagnosis TYPE char1,
w_probability TYPE char1,
w_syntax TYPE char1,
w_syntaxwarnings TYPE z_syntaxwarnings_t. "Table of syntaxwarnings
Finally, I call my transformation with an instance of my class:
CALL TRANSFORMATION zst_addressquality
SOURCE XML lw_xml
RESULT result = lo_addressquality.
Now, when debugging through the transformation code, it successfully notices all fields of the given lw_xml and appears to write them into the object lo_addressquality. But the object attributes stay empty afterwards.
When testing the serialization, I can access result which contains my object, but result-w_address (and all others) are empty.
While testing, I created a structure with completely identical names and types. With it, it worked as intended.
What am I missing? Is there anything else I have to watch out for when working with transformation into ABAP Objects?
_Edit: After changing to the XSLT, I can get until W_ADDRESS before my code throws an CX_XSLT_ABAP_CALL_ERROR. So, I'm still not able to access the object'S attributes properly. :|_
Objects can be serialized/deserialized only with an XSL transformation. It's not possible to do it with a simple transformation, dixit ABAP documentation:
ST programs are restricted to the transformation of elementary and structured ABAP data, along with internal tables. The transformation of reference variables and referenced objects is not currently supported.
The XSL transformation must convert the XML into ASXML, which in short corresponds to a structure like this:
<?xml ...?>
<asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
<asx:values>
...
</asx:values>
<asx:heap>
...
</asx:heap>
</asx:abap>
The easiest way to understand what the ASXML should look like is to serialize your object reference using the identity transformation (it's an XSL transformation), and then adapt your transformation to produce the same kind of asXML:
CALL TRANSFORMATION id SOURCE anyRootName = yourObjectReference RESULT XML asXMLutf8xstring.
Example:
REPORT.
CLASS serialization_demo DEFINITION.
PUBLIC SECTION.
INTERFACES if_serializable_object.
DATA attribute TYPE i.
ENDCLASS.
START-OF-SELECTION.
DATA obj_ref TYPE REF TO serialization_demo.
DATA xstring TYPE xstring.
CREATE OBJECT obj_ref.
obj_ref->attribute = 5.
CALL TRANSFORMATION id " serialize
SOURCE root = obj_ref
RESULT XML xstring.
CLEAR obj_ref.
CALL TRANSFORMATION id " deserialize
SOURCE XML xstring
RESULT root = obj_ref.
ASXML (in the xstring variable):
<?xml version="1.0" encoding="utf-8"?>
<asx:abap version="1.0" xmlns:asx="http://www.sap.com/abapxml">
<asx:values>
<ROOT href="#o3"/>
</asx:values>
<asx:heap xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:abap="http://www.sap.com/abapxml/types/built-in" xmlns:cls="http://www.sap.com/abapxml/classes/global" xmlns:dic="http://www.sap.com/abapxml/types/dictionary">
<prg:SERIALIZATION_DEMO id="o3" xmlns:prg="http://www.sap.com/abapxml/classes/program/ZZSRO_TEST16I">
<local.SERIALIZATION_DEMO>
<ATTRIBUTE>5</ATTRIBUTE>
</local.SERIALIZATION_DEMO>
</prg:SERIALIZATION_DEMO>
</asx:heap>
</asx:abap>

How to get the Structure/Template id by Structure/Template name

I have a requirement that, Need to create JournalArticle with Structure and Template.While creating JournalArticle the method expecting the StructureId and TemplateId but these are generated by Liferay.So by name how can i get Id's of both.
Create and execute a DynamicQuery, like so (just replace Template with Structure to get structures):
DynamicQuery q = DynamicQueryFactoryUtil.forClass(DDMTemplate.class)
.add(PropertyFactoryUtil.forName("name").like("%YOUR NAME%"));
List<DDMTemplate> templates = DDMTemplateLocalServiceUtil.dynamicQuery(q);
You have to use like since the names of the structures/templates are saved like so:
<?xml version='1.0' encoding='UTF-8'?>
<root available-locales="de_DE" default-locale="de_DE">
<Name language-id="de_DE">YOUR NAME</Name>
</root>
There can be different names for different locales.
You can get StructureId (called DDMStructure) with this code
long classNameIdJournalArticle = ClassNameLocalServiceUtil.getClassNameId(JournalArticle.class);
DDMStructure ddmStructure = DDMStructureLocalServiceUtil.getStructure(groupId, classNameIdJournalArticle, "myDDMStructureName");
And TemplateId (called DDMTemplate) with this code
DDMTemplate ddmTemplate = DDMTemplateLocalServiceUtil.getTemplate(groupId, classNameIdDDMStructure, "ddmTemplateName");

Berkeley XML DB "where" analog

I'm currently studying Berkeley XML DB and got an assignment to write Python script using it. The problem I'm currently facing is to select specific node of container. For example we have container with such information
<root>
<lab>
<name>Lab1</name>
<state>Completed</state>
</lab>
<lab>
<name>Lab3</name>
<state>Not completed</state>
</lab>
</root>
How to select <lab> element with specific <name>? In SQL I'd use WHERE Name='Lab1'. Is there any way to do something like that in XML BDB?
I think you better get old document, copy data, remove document and add new with modified data.
mgr = XmlManager()
uc = mgr.createUpdateContext()
container = mgr.openContainer("labs.dbxml") # Here must be your database name
qc = mgr.createQueryContext()
document = container.getDocument("Lab11")
name = document.getName()
content = document.getContent()
# Change fields here using XPath
container.deleteDocument('La1 1', uc)
container.putDocument(name, content, uc)

Resources