Parsing XML for sub children using using python - python-3.x

I am trying to parse the below XML to get the following data. There are multiple rules following, I have shared only one rule below. Is it possible to parse the XML for these values?
name from section header, rule id value, applied-to name, source names, source values, destination name, destination value.
<?xml version="1.0" encoding="UTF-8"?>
<filteredfirewallConfiguration timestamp="1621338984151">
<contextId>globalroot</contextId>
<layer3Sections>
<section id="asdfsdf" name="production" generationNumber="132" timestamp="1621930404081" managedBy="universalroot-0" tcpStrict="false" stateless="false" useSid="false" type="LAYER3">
<rule id="1213213" disabled="false" logged="true" managedBy="universalroot-0">
<name>From Conversion Server</name>
<action>allow</action>
<appliedToList>
<appliedTo>
<name>re-int-dx</name>
<value>universalwire</value>
<type>VirtualWire</type>
<isValid>true</isValid>
</appliedTo>
<appliedTo>
<name>re-ext-ap</name>
<value>universalwire</value>
<type>VirtualWire</type>
<isValid>true</isValid>
</appliedTo>
</appliedToList>
<sectionId>sfsdfafee</sectionId>
<sources excluded="false">
<source>
<name>sdfsdf101</name>
<value>ipset-werfwefdc</value>
<type>IPSet</type>
<isValid>true</isValid>
</source>
<source>
<name>sdfsfdf102</name>
<value>ipset-4wsetgfreds</value>
<type>IPSet</type>
<isValid>true</isValid>
</source>
</sources>
<destinations excluded="false">
<destination>
<name>production-database-cluster</name>
<value>sg</value>
<type>SecurityGroup</type>
<isValid>true</isValid>
</destination>
<destination>
<name>newname</name>
<value></value>
<type>IPSet</type>
<isValid>true</isValid>
</destination>
</destinations>
<services>
<service>
<name>servicenwe</name>
<value>application-dgfsdfg</value>
<type>Application</type>
<isValid>true</isValid>
</service>
</services>
<direction>inout</direction>
<packetType>any</packetType>
</rule>
sofar, I have been able to get the section header only.
import requests
import xml.etree.ElementTree as ET
tree = ET.parse("out.xml")
root = tree.getroot()
for child in root.find('./layer3Sections'):
print(child.tag, child.attrib)

Related

hexadecimal format for message IS08583 in the server side

Actually i'm working with iso 8583 messages, and library j8583 in java.
I created a echo (0800) message to probe connection with the server, the message looks good when is printed, but in the server side the message looks like hexadecimal encoding, something like this:
The devs in the server side told me is a format error for message, the correct message will be like this:
I see the correct message format are transmitted clearly like above image.
Factory from xml field:
this.messageFactory = ConfigParser.createFromClasspathConfig("j8583-config.xml");
Iso message setting binary flags
req.setBinaryFields(true);
req.setBinaryHeader(true);
XML configuration:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE j8583-config PUBLIC "-//J8583//DTD CONFIG 1.0//EN" "http://j8583.sourceforge.net/j8583.dtd">
<j8583-config>
<!-- These are the ISO headers to be prepended to the message types specified -->
<header type="0800">6000050000</header>
<header type="0810">6000050000</header>
<template type="0800">
<field num="3" type="NUMERIC" length="6" />
<field num="7" type="DATE10" />
<field num="11" type="NUMERIC" length="6" />
<field num="24" type="NUMERIC" length="3" />
<field num="41" type="ALPHA" length="8" />
<field num="42" type="ALPHA" length="15" />
<field num="60" type="LLLLVAR" length="" />
</template>
<parse type="0800">
<field num="3" type="NUMERIC" length="6" />
<field num="7" type="DATE10" />
<field num="11" type="NUMERIC" length="6" />
<field num="24" type="NUMERIC" length="3" />
<field num="41" type="ALPHA" length="8" />
<field num="42" type="ALPHA" length="15" />
<field num="60" type="LLLLVAR" length="" />
</parse>
</j8583-config>
IsoMessage build:
final IsoMessage req = this.messageFactory.newMessage(NET_INFO_REQUEST.getValue());
req.setValue(PROCESSING_CODE, ECHO.getValue(), IsoType.NUMERIC, 6);
req.setValue(TRANSMISSION_DATE_TIME, FormatUtils.formatDate10GMT0(new Date()), IsoType.DATE10, 0);
req.setValue(SYSTEM_TRACE_AUDIT_NO, leftPad(simpleTraceGenerator.nextTrace(), 6), IsoType.NUMERIC, 6);
req.setValue(INTERNATIONAL_NETWORK_ID, command.VISA.getCode(), IsoType.NUMERIC, 4);
req.setValue(TERMINAL_ID, "72024092", IsoType.ALPHA, 8);
req.setValue(CLIENT_CODE, "03659307 ", IsoType.ALPHA, 15);
req.setValue(SOFTWARE_VERSION, "OPv1", IsoType.LLLLVAR, 0);
req.setBinary(fale);
Can help me?
Seems there's more than one problem with the message you're generating:
You set fields and headers to binary, but the bitmap is still encoded in ASCII. You should probably just call setUseBinaryMessages instead, IIRC it sets all the messages flags to binary (bitmap, headers, fields). If it doesn't then you need to set binary bitmap programmatically (there's another method for that).
You have a message header for the 0800 but from the hexdump that the other side expects, it seems it's either very different (probary binary, BCD encoded) or they don't expect a header at all. If you need a binary header, specify it in the config (binary header content should be specified as hex, e.g. <header type="800" binary="true">60011200</header>) or you can set it programatically.
For the BCD-encoded fields, just use NUMERIC fields; the values will be BCD-encoded when using binary format. Same goes for DATE fields. LVAR fields are encoded as text; if you need binary data in those, then use LBIN instead. The binary equivalent of ALPHA is BINARY.
LBCDBIN fields are only useful if you message is ASCII-encoded but you need a LBIN field with BCD-encoded length header, because the length header encoding is otherwise chosen based on the encoding of the message (BCD for binary messages, ASCII otherwise).
Hope that helps!

Powershell command -- To add a attribute if not exist in a particular node of a XML

<APPPOOL APPPOOL.NAME="Classic" RuntimeVersion="v2.0" state="Started">
<add name="Classic" autoStart="true" managedRuntimeVersion="v2.0">
<APPPOOL APPPOOL.NAME="GetServiceDet" RuntimeVersion="v2.0" state="Started">
<add name="GetServiceDet" autoStart="true">
my file has many line begining with word "add name".
I want to check if these lines have a string "managedRuntimeVersion".
if not exists, then i need to add managedRuntimeVersion="v2.0" to
that line.
Expected Result as below
<APPPOOL APPPOOL.NAME="Classic" RuntimeVersion="v2.0" state="Started">
<add name="Classic" autoStart="true" managedRuntimeVersion="v2.0">
<APPPOOL APPPOOL.NAME="GetServiceDet" RuntimeVersion="v2.0" state="Started">
<add name="GetServiceDet" autoStart="true" managedRuntimeVersion="v2.0">
I have tried with the below script.. but in the result.. it is given only the lines having "add name"
$sfile="C:\Users\subash.s\Desktop\backup\pool.xml"
(((gc "$sfile") | Select-String -Pattern "add name" |
select-string -notmatch "managedRuntimeVersion") -replace '>',' managedRuntimeVersion="v2.0">') |
Set-Content "$sfile"
with the above script. i got below result..
<add name="Classic" autoStart="true" managedRuntimeVersion="v2.0">
<add name="GetServiceDet" autoStart="true" managedRuntimeVersion="v2.0">
For Pete's sake, the configuration file is XML, not a text file! Edit it as an XML document, and you will save a lot of headaches.
There are a few ways to add attributes into such a document. As the XML in the question is a fragment - and of illegal syntax, the sample code uses a bit modified version of the same. Select all add nodes that don't have got managedRuntimeVersion attribute, create one and add attribute with values to the nodes. Like so,
# Dummy data for testing
[xml]$x = #'
<root>
<APPPOOL APPPOOL.NAME="GetServiceDet" RuntimeVersion="v2.0" state="Started">
<add name="GetServiceDet2" autoStart="true"/>
</APPPOOL>
<APPPOOL APPPOOL.NAME="Classic" RuntimeVersion="v2.0" state="Started">
<add name="Classic" autoStart="true" managedRuntimeVersion="v2.0" />
</APPPOOL>
<APPPOOL APPPOOL.NAME="GetServiceDet" RuntimeVersion="v2.0" state="Started">
<add name="GetServiceDet" autoStart="true"/>
</APPPOOL>
</root>
'#
# Select all add elements that don't have managedRuntimeVersion attribute
$nl=$x.SelectNodes('/root/APPPOOL/add[not(#managedRuntimeVersion)]')
# Add attributes to the elements
foreach($n in $nl) {
# Create new attribute and assign a value
$a = $x.CreateAttribute('managedRuntimeVersion')
$a.Value = 'v2.0'
[void]$n.Attributes.Append($a)
}
# Print modified version to console
$x.save([console]::out)
# Output
<?xml version="1.0" encoding="ibm850"?>
<root>
<APPPOOL APPPOOL.NAME="GetServiceDet" RuntimeVersion="v2.0" state="Started">
<add name="GetServiceDet2" autoStart="true" />
</APPPOOL>
<APPPOOL APPPOOL.NAME="Classic" RuntimeVersion="v2.0" state="Started">
<add name="Classic" autoStart="true" managedRuntimeVersion="v2.0" />
</APPPOOL>
<APPPOOL APPPOOL.NAME="GetServiceDet" RuntimeVersion="v2.0" state="Started">
<add name="GetServiceDet" autoStart="true" managedRuntimeVersion="v2.0" />
</APPPOOL>
</root>
Reading the actual file and saving changes to disk are left as an exercise to the reader.

How to get the required values from the below mentioned xml file?

1) i want to read below mentioned XML file and access the values, i already tried in many ways but not able to access, for example i want 'NightRaidPerformanceCPUScore' value and that is from which passIndex.
<?xml version='1.0' encoding='utf8'?>
<benchmark>
<results>
<result>
<name />
<description />
<passIndex>-1</passIndex>
<sourceId>C:\Users\dgadhipx\Documents\3DMark\3dmark-autosave-20200401155825.3dmark-result</sourceId>
<NightRaidPerformance3DMarkScore>2066</NightRaidPerformance3DMarkScore>
<NightRaidPerformanceCPUScore>1454</NightRaidPerformanceCPUScore>
<NightRaidPerformanceGraphicsScore>2233</NightRaidPerformanceGraphicsScore>
<benchmarkRunId>8045dec5-e97c-452b-abeb-54af187fd50a</benchmarkRunId>
</result>
<result>
<name />
<description />
<passIndex>0</passIndex>
<sourceId>C:\Users\dgadhipx\Documents\3DMark\3dmark-autosave-20200401155825.3dmark-result</sourceId>
<NightRaidPerformanceCPUScoreForPass>1454</NightRaidPerformanceCPUScoreForPass>
<NightRaidPerformance3DMarkScoreForPass>2066</NightRaidPerformance3DMarkScoreForPass>
<NightRaidPerformanceGraphicsScoreForPass>2233</NightRaidPerformanceGraphicsScoreForPass>
<NightRaidPerformanceGraphicsTest1>9.57</NightRaidPerformanceGraphicsTest1>
<NightRaidPerformanceGraphicsTest2>12.18</NightRaidPerformanceGraphicsTest2>
<NightRaidCpuP>395.2</NightRaidCpuP>
<benchmarkRunId>8045dec5-e97c-452b-abeb-54af187fd50a</benchmarkRunId>
</result>
</results>
</benchmark>
You can use BeautifulSoup as fellow:
with open(file_path, "r") as f:
content = f.read()
xml = BeautifulSoup(content, 'xml')
elements = xml.find_all("NightRaidPerformanceCPUScore")
for i in elements:
print(i.text)
That will print you the values of all "NightRaidPerformanceCPUScore" tags.

XML parsing for nested tags using python

i would need a help on how to parse xml files with nested tags taking the user input as attribute value of a particular tag.
for eg:
if xml code has 12 mappings and we want to select the below mapping name "m_IF1_TD_SALESORDER_STG_PR4_VBPA_BUSINESS_PARTNER" as user input and thereby printing the particular mapping details.
<MAPPING NAME="m_IF1_TD_SALESORDER_STG_PR4_VBPA_BUSINESS_PARTNER" DESCRIPTION="Data Mapping for loading Sales Order Business Partners from the staging database for PR4." OBJECTVERSION="1" ISVALID="YES" VERSIONNUMBER="1">
<TRANSFORMATION NAME="sq_STG_PR4_VBPA" DESCRIPTION="" TYPE="Source Qualifier" OBJECTVERSION="1" REUSABLE="NO" VERSIONNUMBER="1">
<TRANSFORMFIELD NAME="MANDT" DESCRIPTION="" DATATYPE="string" PORTTYPE="INPUT/OUTPUT" PRECISION="3" SCALE="0" PICTURETEXT="" DEFAULTVALUE="" EXPRESSION="MANDT" EXPRESSIONTYPE="GENERAL"/>
<TRANSFORMFIELD NAME="VBELN" DESCRIPTION="" DATATYPE="string" PORTTYPE="INPUT/OUTPUT" PRECISION="10" SCALE="0" PICTURETEXT="" DEFAULTVALUE="" EXPRESSION="VBELN" EXPRESSIONTYPE="GENERAL"/>
<TRANSFORMFIELD NAME="POSNR" DESCRIPTION="" DATATYPE="decimal" PORTTYPE="INPUT/OUTPUT" PRECISION="6" SCALE="0" PICTURETEXT="" DEFAULTVALUE="" EXPRESSION="POSNR" EXPRESSIONTYPE="GENERAL"/>
<TRANSFORMFIELD NAME="PARVW" DESCRIPTION="" DATATYPE="string" PORTTYPE="INPUT/OUTPUT" PRECISION="2" SCALE="0" PICTURETEXT="" DEFAULTVALUE="" EXPRESSION="PARVW" EXPRESSIONTYPE="GENERAL"/>
I would suggest using the XML ElementTree module of python.
It is very simple to use. For example:
import xml.etree.ElementTree as ET
tree = ET.parse('/filename.xml')
root = tree.getroot()
for subchild in root.findall('subchildName'):
print(subchild.get('subchildAttribute')

Pentaho Data Integration Mapping

I am using Pentaho Data Integration, I created a new transformation and I have 2 steps in it....1 is a CSV file of my data, the second is an Excel file with two columns one is are the state names and the other the sort form of that state name, Example ("New York" "NY")
In my CSV file I have a state columns with the state names "New York" I want to use my excel file to map "New York" with "NY"
I have googled this all day with no clear answer...can anyone help?
You can use Merge Join. Using this you can merge both the files and select the desired columns. Before merging, you have to sort those files according to fields which use are using for mapping. In your case, it will be state name.
I would recommend you to use stream lookup to do this task. Check the test transformation attached. It will do your task.
<?xml version="1.0" encoding="UTF-8"?>
<transformation-steps>
<steps>
<step>
<name>EXCEL</name>
<type>DataGrid</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<fields>
<field>
<name>State</name>
<type>String</type>
<format/>
<currency/>
<decimal/>
<group/>
<length>-1</length>
<precision>-1</precision>
<set_empty_string>N</set_empty_string>
</field>
<field>
<name>Short_state</name>
<type>String</type>
<format/>
<currency/>
<decimal/>
<group/>
<length>-1</length>
<precision>-1</precision>
<set_empty_string>N</set_empty_string>
</field>
</fields>
<data>
<line> <item>New York</item><item>TX</item> </line>
<line> <item>Texas</item><item>TX</item> </line>
</data>
<cluster_schema/>
<remotesteps> <input> </input> <output> </output> </remotesteps> <GUI>
<xloc>392</xloc>
<yloc>80</yloc>
<draw>Y</draw>
</GUI>
</step>
<step>
<name>CSV</name>
<type>DataGrid</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<fields>
<field>
<name>Full_state_name</name>
<type>String</type>
<format/>
<currency/>
<decimal/>
<group/>
<length>-1</length>
<precision>-1</precision>
<set_empty_string>N</set_empty_string>
</field>
</fields>
<data>
<line> <item>New York</item> </line>
<line> <item>Texas</item> </line>
</data>
<cluster_schema/>
<remotesteps> <input> </input> <output> </output> </remotesteps> <GUI>
<xloc>511</xloc>
<yloc>169</yloc>
<draw>Y</draw>
</GUI>
</step>
<step>
<name>Stream lookup</name>
<type>StreamLookup</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<from>EXCEL</from>
<input_sorted>N</input_sorted>
<preserve_memory>Y</preserve_memory>
<sorted_list>N</sorted_list>
<integer_pair>N</integer_pair>
<lookup>
<key>
<name>Full_state_name</name>
<field>State</field>
</key>
<value>
<name>State</name>
<rename>State</rename>
<default/>
<type>String</type>
</value>
<value>
<name>Short_state</name>
<rename>Short_state</rename>
<default/>
<type>String</type>
</value>
</lookup>
<cluster_schema/>
<remotesteps> <input> </input> <output> </output> </remotesteps> <GUI>
<xloc>510</xloc>
<yloc>79</yloc>
<draw>Y</draw>
</GUI>
</step>
</steps>
<order>
<hop> <from>EXCEL</from><to>Stream lookup</to><enabled>Y</enabled> </hop>
<hop> <from>CSV</from><to>Stream lookup</to><enabled>Y</enabled> </hop>
</order>
<notepads>
</notepads>
<step_error_handling>
</step_error_handling>
</transformation-steps>

Resources