I'm using camel-olingo2 component for query SAP SuccessFactors on ODataV2 endpoints. The route is:
from("direct:start")
.to(olingoEndpoint)
.process(paging)
.loopDoWhile(simple("\${header.CamelOlingo2.\$skiptoken} != null"))
.to(olingoEndpoint)
.process(paging)
.end()
Paging processor is:
Processor paging = new Processor() {
#Override
void process(Exchange g) throws Exception {
ODataDeltaFeed feed = g.in.getMandatoryBody(ODataDeltaFeed)
if (consumer) feed.getEntries().forEach(consumer)
String next = feed.getFeedMetadata().getNextLink()
if (next) {
List<NameValuePair> lst = URLEncodedUtils.parse(new URI(next), StandardCharsets.UTF_8)
NameValuePair skiptoken = lst.find { it.name == "\$skiptoken" }
g.out.headers."CamelOlingo2.\$skiptoken" = skiptoken.value
} else {
g.out.headers.remove("CamelOlingo2.\$skiptoken")
}
}
}
Everything is OK with most of entities but there are fields for several entities with wrong nullability or data length so I got:
Caused by: org.apache.olingo.odata2.api.edm.EdmSimpleTypeException: The metadata constraints '[Nullable=true, MaxLength=16]' do not match the literal 'Bor.Kralja Petra I.16'.
at org.apache.olingo.odata2.core.edm.EdmString.internalValueOfString(EdmString.java:62)
at org.apache.olingo.odata2.core.edm.AbstractSimpleType.valueOfString(AbstractSimpleType.java:91)
at org.apache.olingo.odata2.core.ep.consumer.JsonPropertyConsumer.readSimpleProperty(JsonPropertyConsumer.java:236)
at org.apache.olingo.odata2.core.ep.consumer.JsonPropertyConsumer.readPropertyValue(JsonPropertyConsumer.java:169)
In documentation for Olingo2 camel component I cannot find the way to disable this checking or other walkaround. Can you suggest me the good way?
Please do not recommend server-side data changes, ex metadata modifiyng, it's out of scope for this task.
I have plan B: to use HTTPS requests with JSON parsing, it's quite simple but little boring.
Related
I'm playing around with reactive flows using RxJava2, Micronaut and Cassandra. I'm new to rxjava and not sure what is the correct way to return a of List Person in the best async manner?
data is coming from a Cassandra Dao interface
public interface PersonDAO {
#Query("SELECT * FROM cass_drop.person;")
CompletionStage<MappedAsyncPagingIterable<Person>> getAll();
}
that gets injected into a micronaut controller
return Single.just(personDAO.getAll().toCompletableFuture().get().currentPage())
.subscribeOn(Schedulers.io())
.map(people -> HttpResponse.ok(people));
OR
return Single.just(HttpResponse.ok())
.subscribeOn(Schedulers.io())
.map(it -> it.body(personDAO.getAll().toCompletableFuture().get().currentPage()));
OR switch to RxJava3
return Single.fromCompletionStage(personDAO.getAll())
.map(page -> HttpResponse.ok(page.currentPage()))
.onErrorReturn(throwable -> HttpResponse.ok(Collections.emptyList()));
Not a pro of RxJava nor Cassandra :
In your first and second example, you are blocking the thread executing the CompletionStage with get, even if you are doing it in the IO thread, I would not recommand doing so.
You are also using a Single wich can emit, only one value, or an error. Since you want to return a List, I would sugest to go for at least an Observable.
Third point, the result from Cassandra is paginated, I don't know if it's intentionnaly but you list only the first page, and miss the others.
I would try a solution like the one below, I kept using the IO thread (the operation may be costly in IO) and I iterate over the pages Cassandra fetch :
/* the main method of your controller */
#Get()
public Observable<Person> listPersons() {
return next(personDAO.getAll()).subscribeOn(Schedulers.io());
}
private Observable<Person> next(CompletionStage<MappedAsyncPagingIterable<Person>> pageStage) {
return Single.fromFuture(pageStage.toCompletableFuture())
.flatMapObservable(personsPage -> {
var o = Observable.fromIterable(personsPage.currentPage());
if (!personsPage.hasMorePages()) {
return o;
}
return o.concatWith(next(personsPage.fetchNextPage()));
});
}
If you ever plan to use reactor instead of RxJava, then you can give cassandra-java-driver-reactive-mapper a try.
The syntax is fairly simple and works in compile-time only.
I've done a bunch of searching and, while I've found a few hits, like Why does Spock think my Data Provider has no data?, none of them seem to be very helpful.
I've only done a data provider a couple of times, but it seems perfect for this. I have the following static method:
static List<ContactPointType> getAddressTypes() {
List<ContactPointType> result = new ArrayList<>();
for (ContactPointType cpType : ContactPointType.values()) {
if (cpType.toString().endsWith("Addr")) {
result.add(cpType);
}
}
return result;
}
And then I'm trying to use it as a data provider for calling a function on my class:
#Unroll("#cpType should be address")
def "isAddress addresses"() {
expect: "isAddress() to be true"
contactPoint.isAddress(cpType)
where:
cpType << getAddressTypes()
}
When I run this, I get:
org.spockframework.runtime.SpockExecutionException: Data provider has no data
at org.spockframework.runtime.JUnitSupervisor.afterFeature(JUnitSupervisor.java:191)
at org.spockframework.runtime.BaseSpecRunner.runFeature(BaseSpecRunner.java:236)
Like I said, it seems pretty straightforward. Does anyone have any ideas?
Well, I've tried the data provider feature and it works as expected:
#Unroll("max(1, #cpType) == #cpType")
class MyFirstSpec extends Specification {
def "let's try this!"() {
expect:
Math.max(1, cpType) == cpType
where:
cpType << dataProvider()
}
List<Integer> dataProvider() {
[2,3,4]
}
}
However if I rewrite the dataProvider function like this, I see the exception that you've mentioned:
List<Integer> dataProvider() {
[] // returns an empty list
}
Yields:
org.spockframework.runtime.SpockExecutionException: Data provider has no data
at org.spockframework.runtime.JUnitSupervisor.afterFeature(JUnitSupervisor.java:180)
at org.spockframework.runtime.BaseSpecRunner.runFeature(BaseSpecRunner.java:239)
So my idea is that probably you end up with an empty list in the data provider implementation and that's why it doesn't work
Another possible (although slightly less realistic idea to be honest) is that you've messed something up with Groovy/Java interconnection
So in terms of resolution to wrap up:
Try to use some more straightforward data provider implementation and test it
If it doesn't work - just define data provider like me in Groovy and re-test
I'm trying to apply ubiquitous language to my domain objects.
I want to convert a Data Transfer Object coming from a client into the domain object. The Aggregate's Constructor only accepts the required fields, and the rest of parameters should be passed using aggregate's API even when the Aggregate is being created(by say CreateAggregate command).
But the DTO to Aggregate mapping code becomes a bit messy:
if(DTO.RegistrantType == 0){
registrantType = RegistrantType.Person()
}
elseif(DTO.RegistrantType == 1){
registrantType = RegistrantType.Company()
}
//.....
//.....
var aggregate = new Aggregate(
title,
weight,
registrantType,
route,
callNumber,
)
//look at this one:
if(DTO.connectionType == 0){
aggregate.Route(ConnectionType.InCity(cityId))
}
elseif(DTO.connectionType == 1){
aggregate.Route(ConnectionType.Intercity(DTO.originCityId,DTO.DestinationCityId)
}
//..........
//..........
One thing I should mention is that this problem doesn't seem a domain specific problem.
How can I reduce these If-Else statements without letting my domain internals leakage, and with being sure that the aggregate(not a mapping tool) doesn't accept values that can invalide it's business rules, and with having the ubiquitous language applied?
Please don't tell me I can use AoutoMapper to do the trick. Please read the last part carefully.'
Thank you.
A typical answer would be to convert the DTO (which is effectively a message) into a Command, where the command has all of the arguments expressed as domain specific value types.
void doX(DTO dto) {
Command command = toCommand(dto)
doX(command)
}
void doX(Command command) {
// ...
aggregate.Route(command.connectionType)
}
It's fairly common for the toCommand logic use something like a Builder pattern to improve the readability of the code.
if(DTO.connectionType == 0){
aggregate.Route(ConnectionType.InCity(cityId))
}
elseif(DTO.connectionType == 1){
aggregate.Route(ConnectionType.Intercity(DTO.originCityId,DTO.DestinationCityId)
}
In cases like this one, the strategy pattern can help
ConnectionTypeFactory f = getConnectionFactory(DTO.connectionType)
ConnectionType connectionType = f.create(DTO)
Once that you recognize that ConnectionTypeFactory is a thing, you can think about building lookup tables to choose the right one.
Map<ConnectionType, ConnectionTypeFactory> lookup = /* ... */
ConnectionTypeFactory f = lookup(DTO.connectionType);
if (null == f) {
f = defaultConnectionFactory;
}
So why don't you use more inheritance
for example
class CompanyRegistration : Registration {
}
class PersonRegistraiton : Registration {
}
then you can use inheritance instead of your if/else scenario's
public class Aggregate {
public Aggregate (CompanyRegistration) {
registantType = RegistrantType.Company();
}
public Aggregate (PersonRegistration p) {
registrantType = RegistrantType.Person();
}
}
you can apply simmilar logic for say a setRoute method or any other large if/else situations.
Also, i know you don't want to hear it, you can write your own mapper (inside the aggegate) that maps and validates it's business logic
for example this idea comes from fluentmapper
var mapper = new FluentMapper.ThatMaps<Aggregate>().From<DTO>()
.ThatSets(x => x.title).When(x => x != null).From(x => x.title)
It isn't too hard to write your own mapper that allow this kind of rules and validates your properties. And i think it will improve readability
For example, if I have standard request and response DTOs, linked up via IReturn<T>, what are the reasons to have a service method signature like the following, as seen in various online examples (such as this one, although not consistently throughout):
public object Get(DTO.MyRequest request)
rather than:
public IList<DTO.MyResponse> Get(DTO.MyRequest request)
Is an object return type here simply to support service features like gzip compression of the output stream, which results in the output being a byte array? It seems that one would want to have the appropriate stronger return type from these so-called "action" calls, unless I'm missing some common scenario or use case.
It used to be a limitation that the New API only supported an object return type, but that hasn't been the case for a while where all examples on the New API wiki page now use strong-typed responses.
One of the reasons where you might want to return an object return type is if you want to decorate the response inside a HttpResult, e.g:
public object Post(Movie movie)
{
var isNew = movie.Id == null;
Db.Save(movie); //Inserts or Updates
var movie = new MovieResponse {
Movie = Db.Id<Movie>(newMovieId),
};
if (!isNew) return movie;
//Decorate the response if it was created
return new HttpResult(movie) {
StatusCode = HttpStatusCode.Created,
Headers = {
{ HttpHeaders.Location, Request.AbsoluteUri.CombineWith(movieId) }
}
};
}
It's also useful if you want to return different responses based on the request (though it's not something I recommend), e.g:
public object Get(FindMovies request)
{
if (request.Id != null)
return Db.Id<Movie>(movie.Id);
return Db.Select<Movie>();
}
If you do choose to return an object I highly recommend decorating your Request DTO with the IReturn<T> marker to give a hint to ServiceStack what the expected response of the service should be.
I have been trying to follow this example (download the source code from a link on the site or here, but I keep running into an error that seems embedded in the example.
My procedure has been as follows (after installing the AppFabric SDK and other dependencies):
Download the source
Create a Service Namespace on the AppFabric.
Import the project into a new Windows Azure project with one Worker Role, make sure that it all compiles and that the default Worker Role Run() method starts and functions.
Configure the method GetInterRoleCommunicationEndpoint in InterRoleCommunicationExtension.cs with the ServiceNameSpace and IssuerSecret from my AppFabric Service Namespace (IssuerName and ServicePath stay default). This is a hard-wiring of my own parameters.
Copy/paste the initialization logic from the "SampleWorkerRole.cs" file in the demo into the OnStart() method of my project's Worker Role
Comment-out references to Tracemanager.* as the demo code does not have the Tracemanager methods implemented and they're not crucial for this test to work. There are about 7-10 of these references (just do a Find -> "Tracemanager" in entire solution).
Build successfully.
Run on local Compute Emulator.
When I run this test, during the initialization of a new InterRoleCommunicationExtension (the first piece of the inter-role communication infrastructure to be initialized, this.interRoleCommunicator = new InterRoleCommunicationExtension();), an error is raised: "Value cannot be null. Parameter name: contractType."
Drilling into this a bit, I follow the execution down to the following method in ServiceBusHostFactory.cs (one of the files from the sample):public static Type GetServiceContract(Type serviceType)
{
Guard.ArgumentNotNull(serviceType, "serviceType");
Type[] serviceInterfaces = serviceType.GetInterfaces();
if (serviceInterfaces != null && serviceInterfaces.Length > 0)
{
foreach (Type serviceInterface in serviceInterfaces)
{
ServiceContractAttribute serviceContractAttr = FrameworkUtility.GetDeclarativeAttribute<ServiceContractAttribute>(serviceInterface);
if (serviceContractAttr != null)
{
return serviceInterface;
}
}
}
return null;
}
The serviceType parameter's Name property is "IInterRoleCommunicationServiceContract," which is one of the classes of the demo, and which extends IObservable. The call to serviceType.GetInterfaces() returns the "System.IObservable`1" interface, which is then passed into FrameworkUtility.GetDeclarativeAttribute(serviceInterface);, which has the following code:
public static IList GetDeclarativeAttributes(Type type) where T : class
{
Guard.ArgumentNotNull(type, "type");
object[] customAttributes = type.GetCustomAttributes(typeof(T), true);
IList<T> attributes = new List<T>();
if (customAttributes != null && customAttributes.Length > 0)
{
foreach (object customAttr in customAttributes)
{
if (customAttr.GetType() == typeof(T))
{
attributes.Add(customAttr as T);
}
}
}
else
{
Type[] interfaces = type.GetInterfaces();
if (interfaces != null && interfaces.Length > 0)
{
foreach (object[] customAttrs in interfaces.Select(iface => iface.GetCustomAttributes(typeof(T), false)))
{
if (customAttrs != null && customAttrs.Length > 0)
{
foreach (object customAttr in customAttrs)
{
attributes.Add(customAttr as T);
}
}
}
}
}
return attributes;
}</code><br>
It is here that the issue arises. After not finding any customAttributes on the "IObservable1" type, it calls type.GetInterfaces(), expecting a return. Even though type is "System.IObservable1," this method returns an empty array, which causes the function to return null and the exception with the above message to be raised.
I am extremely interested in getting this scenario working, as I think the Publish/Subscribe messaging paradigm is the perfect solution for my application. Has anyone been able to get this demo code (from the AppFabric CAT Team itself!) working, or can spot my error? Thank you for your help.
Answered in the original blog post (see link below). Please advise if you are still experiencing problems. We are committed to supporting our samples on best effort basis.
http://blogs.msdn.com/b/appfabriccat/archive/2010/09/30/implementing-reliable-inter-role-communication-using-windows-azure-appfabric-service-bus-observer-pattern-amp-parallel-linq.aspx#comments