Pentaho Data Integration Mapping - excel

Question

I am using Pentaho Data Integration, I created a new transformation and I have 2 steps in it....1 is a CSV file of my data, the second is an Excel file with two columns one is are the state names and the other the sort form of that state name, Example ("New York" "NY")
In my CSV file I have a state columns with the state names "New York" I want to use my excel file to map "New York" with "NY"
I have googled this all day with no clear answer...can anyone help?

Answer 1

You can use Merge Join. Using this you can merge both the files and select the desired columns. Before merging, you have to sort those files according to fields which use are using for mapping. In your case, it will be state name.

Answer 2

I would recommend you to use stream lookup to do this task. Check the test transformation attached. It will do your task.
<?xml version="1.0" encoding="UTF-8"?>
<transformation-steps>
<steps>
<step>
<name>EXCEL</name>
<type>DataGrid</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<fields>
<field>
<name>State</name>
<type>String</type>
<format/>
<currency/>
<decimal/>
<group/>
<length>-1</length>
<precision>-1</precision>
<set_empty_string>N</set_empty_string>
</field>
<field>
<name>Short_state</name>
<type>String</type>
<format/>
<currency/>
<decimal/>
<group/>
<length>-1</length>
<precision>-1</precision>
<set_empty_string>N</set_empty_string>
</field>
</fields>
<data>
<line> <item>New York</item><item>TX</item> </line>
<line> <item>Texas</item><item>TX</item> </line>
</data>
<cluster_schema/>
<remotesteps> <input> </input> <output> </output> </remotesteps> <GUI>
<xloc>392</xloc>
<yloc>80</yloc>
<draw>Y</draw>
</GUI>
</step>
<step>
<name>CSV</name>
<type>DataGrid</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<fields>
<field>
<name>Full_state_name</name>
<type>String</type>
<format/>
<currency/>
<decimal/>
<group/>
<length>-1</length>
<precision>-1</precision>
<set_empty_string>N</set_empty_string>
</field>
</fields>
<data>
<line> <item>New York</item> </line>
<line> <item>Texas</item> </line>
</data>
<cluster_schema/>
<remotesteps> <input> </input> <output> </output> </remotesteps> <GUI>
<xloc>511</xloc>
<yloc>169</yloc>
<draw>Y</draw>
</GUI>
</step>
<step>
<name>Stream lookup</name>
<type>StreamLookup</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<from>EXCEL</from>
<input_sorted>N</input_sorted>
<preserve_memory>Y</preserve_memory>
<sorted_list>N</sorted_list>
<integer_pair>N</integer_pair>
<lookup>
<key>
<name>Full_state_name</name>
<field>State</field>
</key>
<value>
<name>State</name>
<rename>State</rename>
<default/>
<type>String</type>
</value>
<value>
<name>Short_state</name>
<rename>Short_state</rename>
<default/>
<type>String</type>
</value>
</lookup>
<cluster_schema/>
<remotesteps> <input> </input> <output> </output> </remotesteps> <GUI>
<xloc>510</xloc>
<yloc>79</yloc>
<draw>Y</draw>
</GUI>
</step>
</steps>
<order>
<hop> <from>EXCEL</from><to>Stream lookup</to><enabled>Y</enabled> </hop>
<hop> <from>CSV</from><to>Stream lookup</to><enabled>Y</enabled> </hop>
</order>
<notepads>
</notepads>
<step_error_handling>
</step_error_handling>
</transformation-steps>