I am using a dynamic data structure for my project. So instead of a predefined class I am using java.util.LinkedHashMap to store my dynamic data and preserve my insertion order as well.
I am able to convert the map to json and get the map and back from Json using ``.
fasterxml.jackson.databind.ObejctMapper mapper;
LinkedHashMap<String, Object> map =
mapper.readValue(json, new TypeReference<LinkedHashMap<String, Object>>() {});
String json = mapper.writeValueAsString(map);
I am trying to do some XSLT transformation on my map data. So I also need to transform from xml to map and map to xml. As there is no direct method to convert these I wrote my own utility for map to xml.
and to convert from xml to map I used -> org.json.JSONObject. I first convert the xml to json using
org.json.XML.toJSONObject(xmlstring)
and can convert the json to map easily using object mapper.
But the problem here is I am loosing the insertion order which is crucial for my data.
How can I convert my data from xml to json so that the insertion order is preserved.
Thats a superb idea to use LinkedHashMap for dynamic data structure.
However JSONObject internally uses HashMap to create the json. So it looses the insertion order.
public JSONObject() {
// HashMap is used on purpose to ensure that elements are unordered by
// the specification.
// JSON tends to be a portable transfer format to allows the container
// implementations to rearrange their items for a faster element
// retrieval based on associative access.
// Therefore, an implementation mustn't rely on the order of the item.
this.map = new HashMap<String, Object>();
}
So If you can override the JSONObject your problem will be solved.
Enjoy!!!
You don't need to change the jar. You just need to create a new class with the same name of the class inside the jar and also have to create the new classes those are dependent on the class.
You need to copy the code from the jar class to your new class and tweek.
then access that class from you code by changing the import statement.
Related
I have a Cosmos DB with a container that contains document with varying structure.
I am using the Java SQL API for reading the documents from this container.
The issue I am having is that the API methods for querying/reading the container expects a model class as input param and will return instances of the model class. Because my container contains documents that have varying fields and depth, it is not possible for me to create a model class to represent this.
I need to be able to read/query the documents and then parse it myself and extract the values that I am looking for.
Any ideas? I have used "Object" in the API methods for e.g. queryItem and then it returns a LinkedHashMap that I can parse myself. Is this the way to do it? It looks a bit "raw" but I have not found a better way.
Below is a typical example from the SDK doc. I cannot create a "Family" model class in my code, because the structure can vary from document to document - both which fields are stored and the depth.
private void queryItems() {
CosmosQueryRequestOptions queryOptions = new CosmosQueryRequestOptions();
queryOptions.setQueryMetricsEnabled(true);
CosmosPagedIterable<Family> familiesPagedIterable = container.queryItems(
"SELECT * FROM Family WHERE Family.lastName IN ('Andersen', 'Wakefield', 'Johnson')", queryOptions, Family.class);
familiesPagedIterable.iterableByPage(10).forEach(cosmosItemPropertiesFeedResponse -> {
logger.info("Got a page of query result with {} items(s) and request charge of {}",
cosmosItemPropertiesFeedResponse.getResults().size(), cosmosItemPropertiesFeedResponse.getRequestCharge());
logger.info("Item Ids {}", cosmosItemPropertiesFeedResponse
.getResults()
.stream()
.map(Family::getId)
.collect(Collectors.toList()));
});
}
Per my understanding, it's determined by the sdk funtion's input parameters and output data type. And exactly, we can find that both sample code for java or spring are depends on the data model. So it's really good for you to use Object in your code because of the various documents.
And it's true that we can't design a data model to contain all the properties in the documents but I think it's also a good idea to set a model which contains all the properties required. I mean that maybe you have a useless property in a query, so the query model should exclude it.
I think I found the proper solution:
Create model class. Define the members with unknown depth and structure as JsonNode.
Then the model class could be used and the values of the JsonNode accessed using nice methods.
Is there a way we can use ObjectContext with DbContext's ModelBuilder? We don't want to use POCO because we have customized property code that does not modify entire object in update, but only update modified properties. Also we have lots of serialisation and auditing code that uses EntityObject.
Since poco does create a proxy with EntityObject, we want our classes to be derived from EntityObject. We don't want proxy. We also heavily use CreateSourceQuery. The only problem is EDMX file and its big connection string syntax web.config.
Is there any way I can get rid of EDMX file? It will be useful as we can dynamically compile new class based on reverse engineering database.
I would also like to use DbContext with EntityObject instead of poco.
Internal Logic
Access Modified Properties in Save Changes which is available in ObjectStateEntry and Save them onto Audit with Old and New Values
Most of times we need to only check for Any condition on Navigation Property for example
User.EmailAddresses.CreateSourceQuery()
.Any( x=> x.EmailAddress == givenAddress);
Access Property Attributes, such as XmlIgnore etc, we rely heavily on attributes defined on the properties.
A proxy for a POCO is a dynamically created class which derives from (inherits) a POCO. It adds functionality previously found in EntityObject, namely lazy loading and change tracking, as long as a POCO meets requirements. A POCO or its proxy does not contain an EntityObject as the question suggests, but rather a proxy contains functionality of EntityObject. You cannot (AFAIK) use ModelBuilder with EntityObject derivatives and you cannot get to an underlying EntityObject from a POCO or a proxy, since there isn't one as such.
I don't know what features of ObjectContext does your existing serialisation and auditing code use, but you can get to ObjectContext from a DbContext by casting a DbContext to a IObjectContextAdapter and accessing IObjectContextAdapter.ObjectContext property.
EDIT:
1. Access Modified Properties in Save Changes which is available in ObjectStateEntry and Save them onto Audit with Old and New Values
You can achieve this with POCOs by using DbContext.ChangeTracker. First you call DbContext.ChangeTracker.DetectChanges to detect the changes (if you use proxies this is not needed, but can't hurt) and then you use DbCotnext.Entries.Where(e => e.State != EntityState.Unchanged && e.State != EntityState.Detached) to get DbEntityEntry list of changed entities for auditing. Each DbEntityEntry has OriginalValues and CurrentValues and the actual Entity is in property Entity.
You also have access to ObjectStateEntry, see below.
2. Most of times we need to only check for Any condition on Navigation Property for example:
User.EmailAddresses.CreateSourceQuery().Any( x=> x.EmailAddress == givenAddress);
You can use CreateSourceQuery() with DbContext by utilizing IObjectContextAdapter as described previously. When you have ObjectContext you can get to the source query for a related end like this:
public static class DbContextUtils
{
public static ObjectQuery<TMember> CreateSourceQuery<TEntity, TMember>(this IObjectContextAdapter adapter, TEntity entity, Expression<Func<TEntity, ICollection<TMember>>> memberSelector) where TMember : class
{
var objectStateManager = adapter.ObjectContext.ObjectStateManager;
var objectStateEntry = objectStateManager.GetObjectStateEntry(entity);
var relationshipManager = objectStateManager.GetRelationshipManager(entity);
var entityType = (EntityType)objectStateEntry.EntitySet.ElementType;
var navigationProperty = entityType.NavigationProperties[(memberSelector.Body as MemberExpression).Member.Name];
var relatedEnd = relationshipManager.GetRelatedEnd(navigationProperty.RelationshipType.FullName, navigationProperty.ToEndMember.Name);
return ((EntityCollection<TMember>)relatedEnd).CreateSourceQuery();
}
}
This method uses no dynamic code and is strongly typed since it uses expressions. You use it like this:
myDbContext.CreateSourceQuery(invoice, i => i.details);
I am trying to save an Object which implements an Interface say IInterface.
private PersistentDictionary<string, IInterface> Object = new PersistentDictionary<string, IInterface>(Environment.CurrentDirectory + #"\Object");
Since many classes implement the same interface(all of which need to cached), for a generic approach I want to store an Object of type IInterface in the dictionary.
So that anywhere I can pull out that object type cast it as IInterface and use that object's internal implementation of methods etc..
But, as soon as the Esent cache is initialized it throws this error:
Not supported for SetColumn
Parameter name: TColumn
Actual value was IInterface.
I have tried to not use XmlSerializer to do the same but is unable to deserialize an Interface type.Also, [Serializable] attribute cannot be used on top of a Interface, so I am stuck.
I have also tried to make all the implementations(classes) of the Interface as [Serializable] as a dying attempt but to no use.
Does any one know a way out ? Thanks in advance !!!
The only reason that only structs are supported (as well as some basic immutable classes such as string) is that the PersistentDictionary is meant to be a drop-in replacement for Dictionary, SortedDictionary and other similar classes.
Suppose I have the following code:
class MyClass
{
int val;
}
.
.
.
var dict = new Dictionary<int,MyClass>();
var x = new MyClass();
x.val = 1;
dict.Add(0,x);
x.val = 2;
var y = dict[0];
Console.WriteLine(y.val);
The output in this case would be 2. But if I'd used the PersistentDictionary instead of the regular one, the output would be 1. The class was created with value 1, and then changed after it was added to the dictionary. Since a class is a reference type, when we retrieve the item from the dictionary, we will also have the changed data.
Since the PersistentDictionary writes the data to disk, it cannot really handle reference types this way. Serializing it, and writing it to disk is essentially the same as treating the object as a value type (an entire copy is made).
Because it's intended to be used instead of the standard dictionaries, and the fact that it cannot handle reference types with complete transparency, the developers instead opted to support only structs, because structs are value types already.
However, if you're aware of this limitation and promise to be careful not to fall into this trap, you can allow it to serialize classes quite easily. Just download the source code and compile your own version of the EsentCollections library. The only change you need to make to it is to change this line:
if (!(type.IsValueType && type.IsSerializable))
to this:
if (!type.IsSerializable)
This will allow classes to be written to the PersistentDictionary as well, provided that it's Serializable, and its members are Serializable as well. A huge benefit is that it will also allow you to store arrays in there this way. All you have to keep in mind is that it's not a real dictionary, therefore when you write an object to it, it will store a copy of the object. Therefore, updating any of your object's members after adding them to the PersistentDictionary will not update the copy in the dictionary automatically as well, you'd need to remember to update it manually.
PersistentDictionary can only store value-structs and a very limited subset of classes (string, Uri, IPAddress). Take a look at ColumnConverter.cs, at private static bool IsSerializable(Type type) for the full restrictions. You'd be hitting the typeinfo.IsValueType() restriction.
By the way, you can also try posting questions about PersistentDictionary at http://managedesent.codeplex.com/discussions .
-martin
I have used many JSON object in applicationScope, sessionScope, and viewScope to track related data. Writing and reading these in SSJS is very simple:`
//Create a app scope variable
applicationScope.put("myvarname", {p1:"part 1", p2:"part2"});
// read and use the app scope variable ...
var myvar = applicationScope.get("myvarname");
//Work with parts as myvar.p1, myvar.p2, etc...
In the Java code I have been writing I have learned to read these variables which were written using SSJS using the com.ibm.jscript.std.ObjectObject package with code like this:
ObjectObject myvar = (ObjectObject) ExtLibUtil
.getApplicationScope().get(dbkey);
FBSValue localFBS = myvar.get("p1");
String myp1 = localFBS.stringValue();
localFBS = myvar.get("p2");
String myp2 = localFBS.stringValue();
Now, of course, I want to write a new entry using the Java Bean that can then be read by SSJS and other Java Beans in the same manner. I managed to write to the scope using a Map and a Hashtable, but these crash the logic when trying to read using the ObjectObject.
So, how would I go about building a new entry in the scope using the ObjectObject and/or FBSValue packages? I cannot find how to create a new FBSValue that can then be added to an ObjectObject. I am sure it is a simple thing a Newbs like me has missed.
/Newbs
You can construct an empty ObjectObject, populate it with FBSValues, and just put it directly into the scope Map:
ObjectObject myvar = new ObjectObject();
try {
myvar.put("p1", FBSUtility.wrap("part 1"));
myvar.put("p2", FBSUtility.wrap("part 2"));
} catch (InterpretException e) {
e.printStackTrace();
}
Map<String, Object> applicationScope = ExtLibUtil.getApplicationScope();
applicationScope.put("myvarname", myvar);
When retrieving it later (as in the examples you provided), SSJS will see it as JSON, Java will see it exactly as it was stored.
If you need to store deeper hierarchies, you can put instances of ArrayObject and ObjectObject inside an ObjectObject in addition to primitives, so, just like JSON itself, you can nest these as deep as you need.
Just be sure to only include true JSON (strings, numbers, booleans, arrays, objects) if you'll be storing it anywhere higher than the requestScope; specifically, FunctionObject does not implement Serializable, so JSON is safe to store, JavaScript is not. Strictly speaking, this only becomes toxic when stored in the viewScope in 8.5.2 and 8.5.3 (and even then, only when the application's persistence option is not set to keep all pages in memory). But if IBM ever implements cluster support, then all objects stored in sessionScope and applicationScope will need to be serializable to allow for inter-server state transport... so, in the interest of future-proofing the design, it's wise to hold to this principle for anything stored longer than the duration of a single request.
Does anyone know whether it is possible to utilise XMLSlurper in a fashion that means individual sub-trees can be pulled from a very large XML document and processed individually?
Imagine you've got a huge XML feed containing a root element that has thousands of direct child elements that you can process individually. Obviously, reading the whole document into memory is a no-no but, as each child of the root is itself modestly sized, it would be nice to stream through the document but apply XMLSlurper niceness to each of the child elements in turn. As each child element is processed, garbage collection can clean up memory used to process it. In this way we get the great ease of XMLSlurper (such concise syntax) with the low memory footprint of streaming (e.g. SAX).
I'd be interested to know if anyone has ideas on this and/or whether you've come across this requirement yourselves.
Initializing an XmlSlurper instance means, calling one of its overloaded parse(..) methods (or the parseText(String) method). Upon this call, XmlSlurper will (use SAX events, at least, to) construct an in-memory GPathResult that holds the complete information on the XML elements and attributes, and their structure.
So, no, the XmlSlurper does not provide an API to parse XML document portions, only.
What can be done is, extending XmlSlurper, overwriting the parse*(..) methods, pre-processing the XML by using a custom SAX handler, gathering the desired portions of XML, and forwarding these to one of the XmlSlurper.parse*(..) methods.
You can use StAX API together with XmlSlurper to parse subtrees.
// Example of using StAX to split a large XML document and parse a single element using XmlSlurper
import javax.xml.stream.XMLInputFactory
import javax.xml.stream.XMLStreamReader
import javax.xml.transform.Transformer
import javax.xml.transform.TransformerFactory
import javax.xml.transform.sax.SAXResult
import javax.xml.transform.stax.StAXSource
def url = new URL("http://repo2.maven.org/maven2/archetype-catalog.xml")
url.withInputStream { inputStream ->
def xmlStreamReader = XMLInputFactory.newInstance().createXMLStreamReader(inputStream)
def transformer = TransformerFactory.newInstance().newTransformer()
while (xmlStreamReader.hasNext()) {
xmlStreamReader.next()
if (xmlStreamReader.isStartElement() && xmlStreamReader.getLocalName() == 'archetype') {
// Example of splitting a large XML document and parsing a single element with XmlSlurper at a time
def xmlSlurper = new XmlSlurper()
transformer.transform(new StAXSource(xmlStreamReader), new SAXResult(xmlSlurper))
def archetype = xmlSlurper.document
println "${archetype.groupId} ${archetype.artifactId} ${archetype.version}"
}
}
}