Comparing two strings in linux KSH script - linux

Question

I have two files: file1 and file2. Here is a sample of the files content:
<TG>
<entry name="KEYNAME" val="" type="string" />
<entry name="KEYTYPE" val="" type="string" />
<entry name="TIMEZONE_OFFSET" val="-240" type="INT16" />
...
</TG>
I want to check if certain lines (those containing the string "entry name") in file1 exist in file2, and then if yes, compare if the two lines are identical or not.
I literally created file2 by copying file1, then I changed a few values. The problem is that comparing the two string variables does not return correct values. Although when displaying the two variables I can see they are identical, the result of the comparison gives that they are not. I am using Ksh. Here is my code:
while read p; do
if [[ $p == *"entry name"* ]]; then
PARAM_NAME=$(echo $p | cut -d '"' -f2)
echo $PARAM_NAME
PARAM_OLD=$(grep $PARAM_NAME file2)
if [[ $PARAM_OLD == *"entry name"* ]]; then
echo $PARAM_OLD
echo $p
if [ "$PARAM_OLD" = "$p" ]; then
echo 'Identical values'
else
echo 'Different values'
fi
else
echo "$PARAM_NAME does not exist in previous version file. Using default value"
fi
fi
done <file1
I tried all possibilities for the parenthesis, equal signs and quotations ([], [[]], = , ==, "", '""', etc.)
Here is the output I am getting:
<entry name="KEYNAME" val="" type="string" />
KEYNAME
<entry name="KEYNAME" val="" type="string" />
<entry name="KEYNAME" val="" type="string" />
Different values
<entry name="KEYTYPE" val="" type="string" />
KEYTYPE
<entry name="KEYTYPE" val="" type="string" />
<entry name="KEYTYPE" val="" type="string" />
Different values
<entry name="TIMEZONE_OFFSET" val="-24" type="INT16" />
TIMEZONE_OFFSET
<entry name="TIMEZONE_OFFSET" val="-240" type="INT16" />
<entry name="TIMEZONE_OFFSET" val="-24" type="INT16" />
Different values
Still I am getting that the strings are different! I would appreciate any explanation and help.

Answer 1

Most recent/newer OSs have support for both ksh and ksh93.
With ksh93 we can use an associative array to limit ourselves to a single pass through each file.
First some sample data:
$ cat file1
<TG>
<entry name="KEYNAME" val="" type="string" />
<entry name="KEYTYPE" val="" type="string" />
<entry name="KEYATTRIB" val="" type="string" />
<entry name="TIMEZONE_OFFSET" val="-241" type="INT16" />
</TG>
$ cat file2
<TG>
<entry name="KEYNAME" val="" type="string" />
<entry name="KEYTYPE" val="" type="stringX" />
<entry name="TIMEZONE_OFFSET" val="-240" type="INT16" />
</TG>
The ksh93 script:
$ cat my_comp
#!/bin/ksh93
unset pline
typeset -A pline
# pull unique list of 'entry name' lines from file2 and store in our associative array pline[]:
egrep "entry name" file2 | sort -u | while read line
do
# strip out the 'entry name' value
x=${line#*\"}
pname=${x%%\"*}
# use the 'entry name' value as the index for our pline[] array
pline[${pname}]=${line}
done
# for each unique 'entry name' line in file1, see if we have a match in file2 (aka our pline[] array):
egrep "entry name" file1 | sort -u | while read line
do
# again, strip out the 'entry name' value
x=${line#*\"}
pname=${x%%\"*}
# if pname does not exist in file2
[ "${pline[${pname}]}" = '' ] && \
echo "\npname = '${pname}' : Does not exist in file2. Using default value:" && \
echo "file1: ${line}" && \
continue
# if pname exists in file2 but line is different
[ "${pline[${pname}]}" = "${line}" ] && \
echo "\npname = '${pname}' : Identical values for pname" && \
echo "file1: ${line}" && \
echo "file2: ${pline[${pname}]}" && \
continue
# if pname exists in file2 and line is the same
[ "${pline[${pname}]}" != "${line}" ] && \
echo "\npname = '${pname}' : Different values for pname" && \
echo "file1: ${line}" && \
echo "file2: ${pline[${pname}]}"
done
Running the script against the sample files:
$ my_comp
pname = 'KEYATTRIB' : Does not exist in file2. Using default value:
file1: <entry name="KEYATTRIB" val="" type="string" />
pname = 'KEYNAME' : Identical values for pname
file1: <entry name="KEYNAME" val="" type="string" />
file2: <entry name="KEYNAME" val="" type="string" />
pname = 'KEYTYPE' : Different values for pname
file1: <entry name="KEYTYPE" val="" type="string" />
file2: <entry name="KEYTYPE" val="" type="stringX" />
pname = 'TIMEZONE_OFFSET' : Different values for pname
file1: <entry name="TIMEZONE_OFFSET" val="-241" type="INT16" />
file2: <entry name="TIMEZONE_OFFSET" val="-240" type="INT16" />
Back to plain ol' basic ksh:
$ cat my_comp2
#!/bin/ksh
egrep "entry name" file1 | sort -u | while read line1
do
x=${line1#*\"}
pname=${x%%\"*}
# see if we can find a matching line in file2; need to strip off leading
# spaces in order to match with line1
unset line2
line2=$( egrep "entry name.*${pname}" file2 | sed 's/^ *//g' )
# if pname does not exist in file2
[ "${line2}" = '' ] && \
echo "\npname = '${pname}' : Does not exist in file2. Using default value:" && \
echo "file1: ${line1}" && \
continue
# if pname exists in file2 but lines are different
[ "${line2}" = "${line1}" ] && \
echo "\npname = '${pname}' : Identical values for pname" && \
echo "file1: ${line1}" && \
echo "file2: ${line2}" && \
continue
# if pname exists in file2 and lines are the same
[ "${line2}" != "${line1}" ] && \
echo "\npname = '${pname}' : Different values for pname" && \
echo "file1: ${line1}" && \
echo "file2: ${line2}"
done
Running the script against the sample files:
$ my_comp2
pname = 'KEYATTRIB' : Does not exist in file2. Using default value:
file1: <entry name="KEYATTRIB" val="" type="string" />
pname = 'KEYNAME' : Identical values for pname
file1: <entry name="KEYNAME" val="" type="string" />
file2: <entry name="KEYNAME" val="" type="string" />
pname = 'KEYTYPE' : Different values for pname
file1: <entry name="KEYTYPE" val="" type="string" />
file2: <entry name="KEYTYPE" val="" type="stringX" />
pname = 'TIMEZONE_OFFSET' : Different values for pname
file1: <entry name="TIMEZONE_OFFSET" val="-241" type="INT16" />
file2: <entry name="TIMEZONE_OFFSET" val="-240" type="INT16" />

Answer 2

You have just 1 = in your if statement, change it to two:
if [ "$PARAM_OLD" = "$p" ]; then
to:
if [ "$PARAM_OLD" == "$p" ]; then
Also (isn't the problem now, but might be your next problem), surround $PARAM_OLD with " in the following line:
if [[ $PARAM_OLD == *"entry name"* ]]; then
so it becomes:
if [[ "$PARAM_OLD" == *"entry name"* ]]; then

Answer 3

Transforming from XML to Tab-Separated Key/Value Pairs
The following will transform your content into tab-separate key-value form:
xml_to_tsv() {
xmlstarlet sel -t -m '//entry[#name]' -v ./#name -o $'\t' -v ./#value -n
}
Extracting Unique Lines From Each
Thus, if you want to compare your two streams, the following will emit only lines unique to the first file:
comm -23 <(xml_to_tsv <one.xml | sort) <(xml_to_tsv <two.xml | sort)
...and the following will emit only lines unique to the second:
comm -13 <(xml_to_tsv <one.xml | sort) <(xml_to_tsv <two.xml | sort)
Doing The Same, Without XMLStarlet
If you don't have XMLStarlet installed, you can generate an XSLT template to perform the same operation. Thus, if you have the following file as extract_entries.xslt:
<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:exslt="http://exslt.org/common" version="1.0" extension-element-prefixes="exslt">
<xsl:output omit-xml-declaration="yes" indent="no"/>
<xsl:template match="/">
<xsl:for-each select="//entry[#name]">
<xsl:call-template name="value-of-template">
<xsl:with-param name="select" select="./#name"/>
</xsl:call-template>
<xsl:text> </xsl:text>
<xsl:call-template name="value-of-template">
<xsl:with-param name="select" select="./#value"/>
</xsl:call-template>
<xsl:value-of select="'
'"/>
</xsl:for-each>
</xsl:template>
<xsl:template name="value-of-template">
<xsl:param name="select"/>
<xsl:value-of select="$select"/>
<xsl:for-each select="exslt:node-set($select)[position()>1]">
<xsl:value-of select="'
'"/>
<xsl:value-of select="."/>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>
...you could use the following xml_to_tsv implementation on systems that don't have XMLStarlet at all:
xml_to_tsv() {
xsltproc extract_entries.xslt -
}

Answer 4

Using the following two data files and one transform file:
file1.xml:
<TG>
<entry name="common" value="foo"/>
<entry name="changed" value="bar"/>
<entry name="unique1" val="qux"/>
</TG>
file2.xml:
<TG>
<entry name="common" value="foo"/>
<entry name="changed" value="bar"/>
<entry name="unique2" val="quux"/>
</TG>
transform.xslt:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="yes"/>
<!-- To be passed with xsltproc's stringparam argument -->
<xsl:param name="other"/>
<!-- Convenience aliases -->
<xsl:variable name="file1" select="/"/>
<xsl:variable name="file2" select="document($other)"/>
<xsl:template match="/">
<results>
<common_entries>
<xsl:for-each select="$file1/TG/entry[#name]">
<xsl:variable name="node1" select="."/>
<xsl:variable name="node2" select="$file2/TG/entry[#name=$node1/#name]"/>
<!-- xpath 1.0, the only version people use, lacks the deep-equal() function -->
<xsl:if test="$node1/#value = $node2/#value">
<xsl:apply-templates select="$node1"/>
</xsl:if>
</xsl:for-each>
</common_entries>
<changed_entries>
<xsl:for-each select="$file1/TG/entry[#name]">
<xsl:variable name="node1" select="."/>
<xsl:variable name="node2" select="$file2/TG/entry[#name=$node1/#name]"/>
<xsl:if test="$node1/#value != $node2/#value">
<diff>
<old>
<xsl:apply-templates select="$node1"/>
</old>
<new>
<xsl:apply-templates select="$node2"/>
</new>
</diff>
</xsl:if>
</xsl:for-each>
</changed_entries>
<unique1_entries>
<xsl:for-each select="$file1/TG/entry[not(#name=$file2/TG/entry/#name)]">
<xsl:apply-templates select="."/>
</xsl:for-each>
</unique1_entries>
<unique2_entries>
<xsl:for-each select="$file2/TG/entry[not(#name=$file1/TG/entry/#name)]">
<xsl:apply-templates select="."/>
</xsl:for-each>
</unique2_entries>
</results>
</xsl:template>
<!-- Standard identity transform -->
<xsl:template match="#*|node()">
<xsl:copy>
<xsl:apply-templates select="#*|node()"/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
Then running xsltproc --stringparam other file2.xml transform.xslt file1.xml will produce:
<?xml version="1.0"?>
<results>
<common_entries>
<entry name="common" value="foo"/>
</common_entries>
<changed_entries>
<diff>
<old>
<entry name="changed" value="bar"/>
</old>
<new>
<entry name="changed" value="baz"/>
</new>
</diff>
</changed_entries>
<unique1_entries>
<entry name="unique1" val="qux"/>
</unique1_entries>
<unique2_entries>
<entry name="unique2" val="quux"/>
</unique2_entries>
</results>