How to extract CDATA without the GPath/node name - groovy

I'm trying to extract CDATA content from an XML without the using GPath (or) node name. In short, i want to find & retrieve the innerText containing CDATA section from an XML.
My XML look like:
def xml = '''<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<root>
<Test1>This node contains some innerText. Ignore This.</Test1>
<Test2><![CDATA[this is the CDATA section i want to retrieve]]></Test2>
</root>'''
From the above XML, i want to get the CDATA content alone without using the reference of its node name 'Test2'. Because the node name is not always the same in my scenario.
Also note that the XML can contain innerText in few other nodes (Test1). I dont want to retrieve that. I just need the CDATA content out of the whole XML.
I want something like below (the code below is incorrect though)
def parsedXML = new xmlSlurper().parseText(xml)
def cdataContent = parsedXML.depthFirst().findAll { it.text().startsWith('<![CDATA')}
My output should be :
this is the CDATA section i want to retrieve

As #daggett says, you can't do this with the Groovy slurper or parser, but it's not too bad to drop down and use the java classes to get it.
Note you have to set the property for CDATA to become visible, as by default it's just treated as characters.
Here's the code:
import javax.xml.stream.*
def xml = '''<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<root>
<Test1>This node contains some innerText. Ignore This.</Test1>
<Test2><![CDATA[this is the CDATA section i want to retrieve]]></Test2>
</root>'''
def factory = XMLInputFactory.newInstance()
factory.setProperty('http://java.sun.com/xml/stream/properties/report-cdata-event', true)
def reader = factory.createXMLStreamReader(new StringReader(xml))
while (reader.hasNext()) {
if (reader.eventType in [XMLStreamConstants.CDATA]) {
println reader.text
}
reader.next()
}
That will print this is the CDATA section i want to retrieve

Considering you just have one CDATA in your xml split can help here
def xml = '''<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<root>
<Test1>This node contains some innerText. Ignore This.</Test1>
<Test2><![CDATA[this is the CDATA section i want to retrieve]]></Test2>
</root>'''
log.info xml.split("<!\\[CDATA\\[")[1].split("]]")[0]
So in the above logic we split the string on CDATA start and pick the portion which is left after
xml.split("<!\\[CDATA\\[")[1]
and once we got that portion we did the split again and then got the portion which is before that pattern by using
.split("]]")[0]
Here is the proof it works

Related

Element:Text and sub elements combined in PowerBI & XML

Having an XML file like this:
<?xml version="1.0" encoding="UTF-8"?><outer>
<inner>Some text.</inner>
<inner>More text.</inner>
</outer>
and the following PowerBI script
let
Table0 = Xml.Tables(File.Contents("simple1.xml")){0}[Table]
in
Table0
you get this
Element:Text
Some text.
More text.
Now I'd like to add sub elements and keep inner.Element:Text
<?xml version="1.0" encoding="UTF-8"?><outer>
<inner>Some text.<secret>Don't care.</secret></inner>
<inner>More text.<secret>You know.</secret></inner>
</outer>
Using the same PowerBI script as above you get
secret
Don't care.
You know.
I already tried this script
let
Table0 = Xml.Tables(File.Contents("simple2.xml")),
Table1 = Table.ExpandTableColumn(Table0, "Table", {"secret"})
in
Table1
but got this
Name
secret
inner
Don't care.
inner
You know.
But I'd like to get this:
Element:Text
secret.Element:Text
Some text.
Don't care.
More text.
You know.
My current workaround (which I'd like to avoid) is to use sed to wrap the element text of an inner entry in its own sub element:
<inner><text>Some text.</text><secret>Don't care.</secret></inner>

Reading CDATA with lxml, problem with end of line

Hello I am parsing a xml document with contains bunch of CDATA sections. I was working with no problems till now. I realised that when I am reading the an element and getting the text abribute I am getting end of line characters at the beggining and also at the end of the text read it.
A piece of the important code as follow:
for comments in self.xml.iter("Comments"):
for comment in comments.iter("Comment"):
description = comment.get('Description')
if language == "Arab":
tag = self.name + description
text = comment.text
The problem is at element Comment, he is made it as follow:
<Comment>
<![CDATA[Usually made it with not reason]]>
I try to get the text atribute and I am getting like that:
\nUsually made it with not reason\n
I Know that I could do a strip and so on. But I would like to fix the problem from the root cause, and maybe there is some option before to parse with elementree.
When I am parsing the xml file I am doing like that:
tree = ET.parse(xml)
Minimal reproducible example
import xml.etree.ElementTree as ET
filename = test.xml #Place here your path test xml file
tree = ET.parse(filename)
root = tree.getroot()
Description = root[0]
text = Description.text
print (text)
Minimal xml file
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Description>
<![CDATA[Hello world]]>
</Description>
You're getting newline characters because there are newline characters:
<Comment>
<![CDATA[Usually made it with not reason]]>
</Comment>
Why else would <![CDATA and </Comment start on new lines?
If you don't want newline characters, remove them:
<Comment><![CDATA[Usually made it with not reason]]></Comment>
Everything inside an element counts towards its string value.
<![CDATA[...]]> is not an element, it's a parser flag. It changes how the XML parser is reading the enclosed characters. You can have multiple CDATA sections in the same element, switching between "regular mode" and "cdata mode" at will:
<Comment>normal text <![CDATA[
CDATA mode, this may contain <unescaped> Characters!
]]> now normal text again
<![CDATA[more special text]]> now normal text again
</Comment>
Any newlines before and after a CDATA section count towards the "normal text" section. When the parser reads this, it will create one long string consisting of the individual parts:
normal text
CDATA mode, this may contain <unescaped> Characters!
now normal text again
more special text now normal text again
I thought that when CDATA comes at xml they were coming with end of line at the beginning and at the end, like that.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Description>
<![CDATA[Hello world]]>
</Description>
But you can have it like that also.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Description><![CDATA[Hello world]]></Description>
It is the reason to get end of line characters when we are parsing the with the Elementtree library, is working perfect in both cases, you only have to strip or not strip depending how you want to process the data.
if you want to remove both '\n' just add the following code:
text = Description.text
text = text.strip('\n')

Get the index of xml element according to its attribute / Python

I need to find out the index (position) of XML element with certain attribute and namespace. In my XML there are more elements with the same name so only possible way to identify the right one is by its attribute.
This is sample of my XML document:
<mets:mets LABEL="ModernĂ­ pedagogika, 2002" TYPE="Monograph"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:mets="http://www.loc.gov/METS/"
xmlns:mods="http://www.loc.gov/mods/v3"
xmlns:ns3="http://www.openarchives.org/OAI/2.0/oai_dc/"
xmlns:ns5="http://www.w3.org/1999/xlink"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.w3.org/2001/XMLSchema-instance http://www.w3.org/2001/XMLSchema.xsd http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/mets.xsd http://www.loc.gov/mods/v3 http://www.loc.gov/standards/mods/v3/mods-3-4.xsd http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd http://www.w3.org/1999/xlink http://www.w3.org/1999/xlink.xsd">
<mets:metsHdr CREATEDATE="2012-12-05T07:42:22" LASTMODDATE="2012-12-05T07:42:22">
<mets:agent ROLE="CREATOR" TYPE="ORGANIZATION">
<mets:name>ABA001</mets:name>
</mets:agent>
<mets:agent ROLE="ARCHIVIST" TYPE="ORGANIZATION">
<mets:name>ABA001</mets:name>
</mets:agent>
</mets:metsHdr>
<mets:dmdSec ID="MODSMD_VOLUME_0001">
.....
</mets:dmdSec>
<mets:dmdSec ID="DCMD_VOLUME_0001">
.....
</mets:dmdSec>
</mets:mets>
Desired Index in this case is the index of this tag <mets:dmdSec ID="MODSMD_VOLUME_0001">
I have tried some solution regarding list(root).index(dmdSec) but without success since I am not able or do not know how to insert there details about attribute and namespace
Could someone help me with this
I'm assuming that you are using the lxml.etree library for xml parsing - if not you may have to modify things a bit - but the principle is the same:
Simply use:
Edit:
from lxml import etree
root = etree.parse(r'path\to\your\file.xml')
int(root.xpath('count(//*[#ID="MODSMD_VOLUME_0001"]/preceding-sibling::*)+1'))
Output:
2.
Note that the position is 2 and not 1 - xpath counts from 1 (unlike python, which counts from 0). Your target is the second <mets:dmdSec> node within the root.

Groovy: replaceLast() is missing

I need replaceLast() method in the Groovy script - replace the last substring. It is available in Java, but not in Groovy AFAIK. It must work with regex in the same way as the following replaceFirst.
replaceFirst(CharSequence self, Pattern pattern, CharSequence replacement)
Replaces the first substring of a CharSequence that matches the given compiled regular expression with the given replacement.
EDIT: Sorry not being specific enough. Original string is an XML file and the same key (e.g. Name) is present many times. I want to replace the last one.
<Header>
<TransactionId>1</TransactionId>
<SessionId>1</SessionId>
<User>
<Name>Bob</Name>
...
</User>
<Sender>
<Name>Joe</Name>
...
</Sender>
</Header>
...
<Context>
<Name>Rose</Name>
...
</Context>
No idea what replaceLast in Java is...it's not in the JDK... If it was in the JDK, you could use it in Groovy...
Anyway, how about using an XML parser to change your XML instead of using a regular expression?
Given some xml:
def xml = '''<Header>
<TransactionId>1</TransactionId>
<SessionId>1</SessionId>
<User>
<Name>Bob</Name>
</User>
<Sender>
<Name>Joe</Name>
</Sender>
<Something>
<Name>Tim</Name>
</Something>
</Header>'''
You can parse it using Groovy's XmlParser:
import groovy.xml.*
def parsed = new XmlParser().parseText(xml)
Then, you can do a depth first search for all nodes with the name Name, and take the last -1 one:
def lastNameNode = parsed.'**'.findAll { it.name() == 'Name' }[-1]
Then, set the value to a new string:
lastNameNode.value = 'Yates'
And print the new XML:
println XmlUtil.serialize(parsed)
<?xml version="1.0" encoding="UTF-8"?><Header>
<TransactionId>1</TransactionId>
<SessionId>1</SessionId>
<User>
<Name>Bob</Name>
</User>
<Sender>
<Name>Joe</Name>
</Sender>
<Something>
<Name>Yates</Name>
</Something>
</Header>

Groovy : How to read xml tag names having full colon

I have an Xml tag <com:Id>33638</com:Id> I want to read the value 33638 from the tag, I have parsed the xml string and stored to another string variable but cant read the value using
String id = 'com:Id'.text();
This works fine:
def t = "<com:Id>33638</com:Id>"
def xml = new XmlParser(false, false).parseText(t)
println xml.text()

Resources