base64binary to string with XQUERY - base64

With Oracle ESB 11g, I need transform the structure of a webservice, but I have a problem with xml type, the source webservice has a type base64binary but my mockup is type string.
Source type
<xsd:element
name="MyDocument"
maxOccurs="1"
minOccurs="0"
type="xsd:base64Binary"
></xsd:element>
Destination type
<xs:element name="myBase64file" type="xs:string" minOccurs="0"/>
Why destination is String?? because my mockup in Java return a String
InputStream stream = this.getClass().getClassLoader().getResourceAsStream("doc3.pdf");
byte[] bytes = IOUtils.toByteArray(stream);
byte[] bytes64 = Base64.encodeBase64(bytes);
myBase64file = new String(bytes64);
But now the real service returns base64binary :(
With oepe (Eclipse more plugins for Oracle OSB) in XQuery Mapper I Tried put a hard transformation like this
{
for $MyDocument in $MyData/ns2:MyDocument
return <myBase64file>{ data($MyDocument) }</myBase64file>
}
Is this the correct way?

I think to achieve your goal it's necessary to use the XQuery because you want to change the element name from myDocument to myBase64file, even if your destination element type was xs:base64Binary you need to execute your XQuery.
However a xs:base64Binary representation is basically a string limited to 65 alphabet characters: a-z, A-Z, 0-9, the plus sign (+), the forward slash (/) and the equal sign (=), together with the space character (#x20) (base64Binary), so you can think that xs:base64binary is an xs:string with an xs:restriction where you can only use the characters described above. Therefore if you prefer to have a xs:base64Binary instead of xs:string you can change your destination type to xs:base64Binary:
<xs:element name="myBase64file" type="xs:base64Binary" minOccurs="0"/>
and your java mockup code fits perfect this type because you are encoding your data to base64 and then returning as string.
Hope this helps,

Related

Concatenate long strings from multiple records into one string

I have a situation where I need to concatenate long strings from multiple records in an Oracle database into a single string. These long strings are portions of a larger XML string, and my ultimate goal is to be able to convert this XML into something resembling query results and pull out specific values.
The data would look something like this, with the MSG_LINE_TEXT field being VARCHAR2(4000). So if the total message is less than 4000 characters, then there'd only be one record. In theory, there could be an infinite number of records for each message, although the highest I've seen so far is 14 records, which means I need to be able to handle strings that are at least 56000 characters long.
MESSAGE_ID MSG_LINE_NUMBER MSG_LINE_TEXT
---------- --------------- --------------------------------
17415414 1 Some XML snippet here
17415414 2 Some XML snippet here
17415414 3 Some XML snippet here
17415414 4 Some XML snippet here
The total XML for one MESSAGE_ID might look something like this. There could be many App_Advice_Error tags, although this specific example only contains one.
<tXML>
<Header>
<Source>MANH_prod_wmsweb</Source>
<Action_Type />
<Sequence_Number />
<Company_ID>1</Company_ID>
<Msg_Locale />
<Version />
<Internal_Reference_ID>17415414</Internal_Reference_ID>
<Internal_Date_Time_Stamp>2021-02-09 13:45:22</Internal_Date_Time_Stamp>
<External_Reference_ID />
<External_Date_Time_Stamp />
<User_ID>ESBUSER</User_ID>
<Message_Type>RESPONSE</Message_Type>
</Header>
<Response>
<Persistent_State>0</Persistent_State>
<Error_Type>2</Error_Type>
<Resp_Code>501</Resp_Code>
<Response_Details>
<Application_Advice>
<Shipper_ID />
<Imported_Object_Type>ASN</Imported_Object_Type>
<Response_Type>Error</Response_Type>
<Transaction_Date>2/9/21 13:45</Transaction_Date>
<Application_Ackg_Code>TE</Application_Ackg_Code>
<Business_Unit></Business_Unit>
<Tran_Set_Identifier_Code></Tran_Set_Identifier_Code>
<Transaction_Purpose_Code>11</Transaction_Purpose_Code>
<Imported_Message_Id></Imported_Message_Id>
<Imported_Object_Id>Reference Number Here</Imported_Object_Id>
<Additional_References>
<Additional_Reference_Info>
<Reference_Type>BusinessPartner</Reference_Type>
<Reference_ID></Reference_ID>
</Additional_Reference_Info>
</Additional_References>
<App_Advice_Errors>
<App_Advice_Error>
<App_Error_Text>Some error text here</App_Error_Text>
<Error_Message_Tokens>
<Error_Message_Token>Object that errored out</Error_Message_Token>
</Error_Message_Tokens>
<App_Err_Cond_Code>6100234</App_Err_Cond_Code>
</App_Advice_Error>
</App_Advice_Errors>
<Imported_Data></Imported_Data>
</Application_Advice>
</Response_Details>
</Response>
</tXML>
The values that I'm most interested in pulling out are the App_Err_Cond_Code, Error_Message_Token, and App_Error_Text tags. I had tried using something like this:
extractvalue(xmltype(msg_line_text), '//XPath of Tag')
This works beautifully for stuff where the entire XML is less than 4000 characters, i.e. the entire XML is stored in a single record. The problem comes when there are multiple records, because each individual snippet of XML isn't a valid XML string on its own, and so XMLTYPE throws an error, hence the reason I'm trying to concatenate them all into a single string, which I can then use with the above method.
I've tried a variety of ways to do this - LISTAGG, XMLAGG, SYS_CONNECT_BY_PATH, as well as writing a custom function something like this:
with
function get_messages(pTranLogID number) return string
is
xml varchar2;
begin
xml := '';
for msg in (
select r.msg_line_text
from tran_log_response_message r, tran_log t
where
t.message_id = r.message_id
and t.tran_log_id = pTranLogID
order by r.msg_line_number
)
loop
xml := xml || msg.msg_line_text;
end loop;
return 'test';
end;
select
tran_log_id, get_messages(tran_log_id)
from
tran_log
where
tran_log_id = '20633610';
/
The problem is that every one of these methods complained that the string was too long. Does anyone have any other ideas? Or maybe a better approach to this problem?
Thanks.

Long Integers are converting to scientific notation - ColdFusion

I am trying to pass a struct through a function, but the integers in it are converting to scientific notation.
Before deSerialization :
{"businessUnitValidList":2003051509034372557922
, "shortMessage":"Success"
, "longMessage":"Request Completed Successfully."
, "status":20001
}
After deSerialization:
businessUnitValidList 2.00305150903E+021
I have tried converting it into a string but it still gives me the same output. Any ideas?
Note: If I have more than one value in my businessUnitValidList, the numbers show up the way they are supposed to.
EDIT
This is the current code iteration:
<cfloop array="#businessUnitArray#" index="i">
<cfquery name="validatebusinessUnit" datasource="dbproduction">
select doctorid from survey.dbo.clientLocationMap
where clientbrandid = '#arguments.clientBrandid#'
and clientLocation = '#i#'
</cfquery>
<cfif validatebusinessUnit.recordcount gt 0>
<cfset businessUnitValidList = listAppend(businessUnitValidList,toString(validatebusinessUnit.doctorid),",")>
<cfelse>
<cfset businessUnitInValidList = listAppend(businessUnitInValidList,i,",")>
</cfif>
</cfloop>
<cfif businessUnitInValidList neq ''>
<cfset ResponseStruct['BusinessUnitCodes']['businessUnitMixResponse']['businessUnitInValidList'] = "#businessUnitInValidList#">
<cfset ResponseStruct['BusinessUnitCodes']['businessUnitMixResponse']['businessUnitValidList'] = "#businessUnitValidList#">
<cfreturn serializeJSON(ResponseStruct['BusinessUnitCodes']['businessUnitMixResponse'])>
<cfelse>
<cfset ResponseStruct['BusinessUnitCodes']['businessUnitSuccess']['businessUnitValidList'] = "#businessUnitValidList#">
<cfreturn serializeJSON(ResponseStruct['BusinessUnitCodes']['businessUnitSuccess'])>
</cfif>
ColdFusion's JSON serialization has issues and can vary between versions and even hotfixes. As Jedihomer Townend mentioned in the comments a leading space should force CF to treat it a string and not cast it.
I've just tried this on CF10, 11 and 2016 and preserves the input.
<cfscript>
a = {
"businessUnitValidList":" 2003051509034372557922",
"shortMessage":"Success",
"longMessage":"Request Completed Successfully.",
"status":20001
};
json = serializeJSON(a);
b = deserializeJSON(json);
writeDump(b);
</cfscript>
You can try it here:
http://trycf.com/gist/70b86fbb57f752125f35/acf?theme=monokai
(Too long for comments)
my code is producing the correct outcome is the deserializeJSON() that
is causing the issue.
Not quite. The JSON value is correct, but the serialization omits the surrounding quotes. That means the value will be handled as a numeric type during deserialization, causing the issue you observed. Unless you can force the serialization to treat the value as a string, the deserialized result will always be wrong. As Carl Von Stetten already mentioned, it is a bug. Jedihomer Townend's answer of appending a space character is probably the simplest work-around.
Longer answer:
Despite the improvements in CF11's JSON handling, CF is still a little too "helpful" ... As you noted, CF detects the value is numeric when serializing and omits the surrounding quotes. Consequently marking the value type as numeric.
{..."businessUnitValidList":2003051509034372557922 }
That all sounds great, until you try and deserialize. If the value was enclosed in quotes, it would be handled as a string, and the original value preserved. Unfortunately, without the quotes it is considered numeric, which means CF must stuff the value into one of its two numeric data types:
Real or floating point number, ie java.lang.Double or
32 bit Integer, ie java.lang.Integer
The maximum value of an Integer is 2147483647. Obviously your number is too large for that, so CF converts it into a java.lang.Double instead. That is a problem for two reasons. First, Double is an approximate type. Second, according to the rules of that class, scientific notation may be used when representing the number as a String, ie when the variable is displayed with cfoutput or cfdump. That is why the deserialized result looks different than what you were expecting. Unless you can force it to be treated as a string when serialized, the deserialized result will always be wrong.
In fairness, CF11 does contain a number improvements for JSON handling. Unfortunately most of them revolve around cfc's and query objects. Given your current structure, it won't quite work. However, if you were able to use a single query object you could resolve the issue by with the help of the new application level setting this.serialization.serializeQueryAs = "struct";. It forces a more sensible format for serialized queries than in earlier versions. Since CF11 respects column data types when serializing, the value would be preserved if the column data type is BIGDECIMAL, or you cast it as a VARCHAR. Unfortunately, CF still upper cases query column names, but the base values are preserved.
Result:
[ { "BUSINESSUNITVALIDLIST" : "2003051509034372557922",
"LONGMESSAGE" : "Request Completed Successfully.",
"SHORTMESSAGE" : "Success",
"STATUS" : 20001
} ]
Code:
qry = queryNew("");
queryAddColumn(qry, "businessUnitValidList", "varchar", ["2003051509034372557922"]);
queryAddColumn(qry, "shortMessage", "varchar", ["Success"]);
queryAddColumn(qry, "longMessage", "varchar", ["Request Completed Successfully."]);
queryAddColumn(qry, "status", "integer", [20001]);
json = serializeJSON(qry);
writeDump(deserializeJSON(json));
CF11 also introduced custom serializers/deserializers, which might work here. Though it is probably overkill for this specific task.
Having said all that, again the simplest option is to use the "append a non-numeric character" hack. Well .. either that or switch to a custom library which may do a more consistent job with JSON handling ;-)
Side Note / Efficiency:
Unless there is a specific reason you must execute a query within a loop, there are likely more efficient options (dbms specific, which you did not mention). Also, do not forget to use cfqueryparam on all variable query parameters. Among it is many benefits is boosting performance when the same query is executed multiple times - such as inside a loop.
In java we have BigInteger data type to store big values. In Coldfusion we can cast by using JavaCast to integer, long and double. But the problem with this is that even with 'long' datatype can only store 19 digits.
And by converting number to string it may be problematic to perform mathematical operations. Here we can use precisionEvaluate() while performing mathematical operations.
PrecisionEvaluate function lets you calculate arbitrarily long decimal
(BigDecimal precision) values. BigDecimal precision arithmetic accepts
and generates decimal numbers of any length.
In this example you can use like this :
<cfset ResponseStruct['BusinessUnitCodes']['businessUnitMixResponse']['businessUnitInValidList'] = "#precisionEvaluate(businessUnitInValidList)#">

XSD 1.1 assert test containing single quote

Anybody got an idea why the following won't work? The xsd is validated with Xerces
<xs:element name="myElement">
<xs:complexType>
<xs:complexContent>
<xs:extension base="myElementType">
<xs:assert test="firstname = 'George' and lastName = 'Mc&apos;Falrland'" />
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
The error hits when i use the single quote &apos; entity...
Thanks
Your problem is related to XPath string literals, as XSD 1.1 uses XPath 2.0 you can escape an apostrophe or quotes inside a string literal by writing the delimiter two times, so you can use:
<xs:assert test="firstname = 'George' and lastName = 'Mc''Falrland'" />
For more info you can use XPath 2.0 specs, section Literals:
If the literal is delimited by apostrophes, two adjacent apostrophes
within the literal are interpreted as a single apostrophe. Similarly,
if the literal is delimited by quotation marks, two adjacent quotation
marks within the literal are interpreted as one quotation mark.

Camel exceptions unmarshalling JAXB ISO-8859-1 XML file

For my route I set:
String encoding = "iso-8859-1";
JaxbDataFormat jaxb = new JaxbDataFormat( Data.class.getPackage().getName() );
if( encoding != null) {
jaxb.setEncoding( encoding );
}
from( "file://" + location + "?charset=" + encoding )
.routeId(this.getClass().getSimpleName()) // Give a nice name
. etc.
then when I provide the file in this ISO encoding I get an exception stack:
[com.ctc.wstx.exc.WstxIOException: Invalid UTF-8 start byte 0xfc (at char #3964, byte #127)]
java.io.IOException: javax.xml.bind.UnmarshalException
- with linked exception:
[com.ctc.wstx.exc.WstxIOException: Invalid UTF-8 start byte 0xfc (at char #3964, byte #127)]
at org.apache.camel.converter.jaxb.JaxbDataFormat.unmarshal(JaxbDataFormat.java:153)
at org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:57)
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:73)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
What could I be doing wrong?
According to the Camel doc, jaxb.setEncoding will not help, as this parameter is only used when marshalling XML documents but not when unmarshalling them.
In the ideal world, the encoding declaration in the prolog (first magic line in XML file) matches the actual encoding of the file:
<?xml version="1.0" encoding="ISO-8859-1"?>
This information is (or at least should be) automatically used by the file reading utility such as JAXB.
0xfc is a ISO-8859-1 encoded ü. In your case, check the prolog encoding declaration. If it doesn't say ISO-8859-1 it is faked. Ask the producer of the file (I hope, it wasn't you...) to set the declaration accordingly. Normally, this is correctly done by the XML marhalling framework.
If you cannot convince the producer of the file to set the correct declaration, then the things are getting trickier. In this case, you must know or guess the encoding and setting the camel header accordingly in the route:
.setHeader(Exchange.CHARSET_NAME, "ISO-8859-1")
According to the source code of JaxbDataFormat (here), this encoding is only taken into account if the filterNonXmlChars property of the JaxbDataFormat instance is set to true:
jaxb.setFilterNonXmlChars(true);
Alternatively, you may also set the Exchange.FILTER_NON_XML_CHARS property to true.

Deserialize XMLDocument with encoded characters in attribute names

I'm Trying to deserialize xml data into an object with c#. I have always done this using the .NET deserialize method, and that has worked well for most of what I have needed.
Now though, I have XML that is created by Sharepoint and the attribute names of the data I need to deserialize have encoded caracters, namely:
*space, º, ç ã, :, * and a hyphen as
x0020, x00ba, x007a, x00e3, x003a and x002d respectivly
I'm trying to figure out what I have to put in the attributeName parameter in the properties XmlAttribute
x0020 converts to a space well, so, for instance, I can use
[XmlAttribute(AttributeName = "ows_Nome Completo")]
to read
ows_Nome_x0020_Completo="MARIA..."
On The other hand, neither
[XmlAttribute(AttributeName = "ows_Motiva_x00e7__x00e3_o_x003a_")]
nor
[XmlAttribute(AttributeName = "ows_Motivação_x003a_")]
nor
[XmlAttribute(AttributeName = "ows_Motivação:")]
allow me to read
ows_Motiva_x00e7__x00e3_o_x003a_="text to read..."
With the first two I get no value returned, and the third gives me a runtime error for invalid caracters (the colon).
Anyway to get this working with .NET Deserialize, or do I have to build a specific deserializer for this?
Thanks!
What you are looking at (the "cryptic" data) is called XML entities. It's used by SharePoint to safekeep attribute names and similar elements.
There are a few ways of dealing with this, the most elegant ways to solve it is by extracting the List schema and match the element towards the schema. The schema contain all meta-data about your list data. A polished example of a Schema can be seen below or here http://www.bendsoft.com/documentation/camelot-php-tools/1_5/packets/schema-and-content-packets/schemas/example-list-view-schema/
If you don't want to walk that path you could start here http://msdn.microsoft.com/en-us/library/35577sxd.aspx
<Field Name="ContentType">
<ID>c042a256-787d-4a6f-8a8a-cf6ab767f12d</ID>
<DisplayName>Content Type</DisplayName>
<Type>Text</Type>
<Required>False</Required>
<ReadOnly>True</ReadOnly>
<PrimaryKey>False</PrimaryKey>
<Percentage>False</Percentage>
<RichText>False</RichText>
<VisibleInView>True</VisibleInView>
<AppendOnly>False</AppendOnly>
<FillInChoice>False</FillInChoice>
<HTMLEncode>False</HTMLEncode>
<Mult>False</Mult>
<Filterable>True</Filterable>
<Sortable>True</Sortable>
<Group>_Hidden</Group>
</Field>
<Field Name="Title">
<ID>fa564e0f-0c70-4ab9-b863-0177e6ddd247</ID>
<DisplayName>Title</DisplayName>
<Type>Text</Type>
<Required>True</Required>
<ReadOnly>False</ReadOnly>
<PrimaryKey>False</PrimaryKey>
<Percentage>False</Percentage>
<RichText>False</RichText>
<VisibleInView>True</VisibleInView>
<AppendOnly>False</AppendOnly>
<FillInChoice>False</FillInChoice>
<HTMLEncode>False</HTMLEncode>
<Mult>False</Mult>
<Filterable>True</Filterable>
<Sortable>True</Sortable>
</Field>
<Field>
...
</Field>
Well... I guess I kind of hacked a way around, which works for now. Just replaced the _x***_ charecters for nothing, and corrected the XmlAttributes acordingly. This replacement is done by first loading the xml as a string, then replacing, then loading the "clean" text as XML.
But I wopuld still like to know if it is possible to use some XmlAttribute Name for a more direct approach...
Try using System.Xml; XmlConvert.EncodeName and XmlConvert.DecodeName
I use a simply function to get the NameCol:
private string getNameCol(string colName) {
if (colName.Length > 20) colName = colName.Substring(0, 20);
return System.Xml.XmlConvert.EncodeName(colName);
}
I'm already searching for replace characters like á, é, í, ó, ú. EncodeName doesn't convert this characters.
Can use Replace:
.Replace("ó","_x00f3_").Replace("á","_x00e1_")

Resources