I am looking for a solution for integrating Apache Oilngo with JMS.
The requirement is to have the client construct HTTP structure and send over socket connection (Synchronous) - if HTTP response take more than the specified time period (e.g. 15 minutes), then request gets forwarded to asynchronous behavior where the thread will be waiting on the socket for response. Once the response arrives, the original sender gets notified (callback like structure)
TIA.
Olingo V2 does not have these capabilities. In Olingo V4 there is an asynchronous support available.
Unfortunately there is no documentation on the olingo website about this. So I would suggest you look at the technical service which olingo uses to test features: https://github.com/apache/olingo-odata4/blob/master/lib/server-tecsvc/src/main/java/org/apache/olingo/server/tecsvc/processor/TechnicalEntityProcessor.java#L148
The TechnicalEntityProcessor in the tecsvc module can respond asynchronously. The specification part of this can be found here: http://docs.oasis-open.org/odata/odata/v4.0/errata02/os/complete/part1-protocol/odata-v4.0-errata02-os-part1-protocol-complete.html#_Toc406398358
Also questions can be asked on the olingo mailing list: user#olingo.apache.org
Related
So I have been reading about flow based programming (FBP) in the last few days and I have also been reading J. Paul Morrison's book about it. However I feel I still can't really wrap my head around it. The general concept is that you see programming as some sort of assembly line where you have components that take some packet as input and produce some packets as output. You can connect these components and packets travel through the network. While I totally see how this can work for ETL type applications or batch processing, I have no good idea how you could handle things like synchronous request/response patterns or database transactions with it.
For example let's say I have a web server implemented as FPB. This webserver has a GET /user/{id} which should return a JSON with some information about a user. It also has a POST /user/{id} where you can update the user by sending some JSON back to the server. So here is how I would imagine this flow to be looking:
I tried to have many re-usable components instead of putting the whole logic of handling a request into a single component. So there is a HTTP server component which sends out requests to a dispatcher component which then dispatches the requests into subsequent flows. In each flow the request is parsed by a generic "Request parser" component which outputs various parts of the request into the rest of the flow.
The upper part is quite straightforward, I read the entity of the user with the given ID from DB, serialize the object to JSON and then send it back. However at this point we don't really have a reference to the HTTP request anymore, so how would I know where to send this request to?
On the lower part we have some additional complexity because I would like to write to the database in a transactional way. So first a transaction is started (in parallel the request body is parsed into some object), then the user object is retrieved from the database and merged with the inputs from the request. At the end it is written back to the database and the transaction is committed. Finally some "OK" status is responded to the caller. Here I have the additional problem that when committing the transaction I really don't know which transaction to commit. And of course when sending the response I don't know which request to send it to.
So both problems seem to have something in common - a kind of "Context" that spans over many components. On one example it is a HTTP request/response context in the other a transactional context. In regular programming, these contexts are usually handled at the thread level. Since a request runs in a single thread, the transaction and request contexts are bound to a thread-local so they can be accessed everywhere as long as everything is running in the same thread.
In flow based programming, every component runs independently and ideally on separate threads. This is actually a key thing because it allows for parallelization and effective use of multiple processors. However when that thread-local context is no longer there, how can you handle these problems in flow based programming? This would get even more complicated with proper error handling (which I left out in my example).
I figure that when you do reactive style programming where most of the processing is asynchronous and multithreaded as well you will have the same issues, so I wonder if there are patterns to handle this. Do you have real life experience with either reactive style programming or flow based programming and have some hints on how I could solve this problem?
I wrote a quick answer on Twitter - thought I would post it here as well... Apologies for double-posting!
I like substreams for this/these problem(s), where the first Information Packet in the substream provides the "context" you were talking about. This may help: https://github.com/jpaulm/javafbp-websockets... HTH!
PS This loop-style network topology is also the basis of Facebook's new "Flux" technology - see Jing Chen's presentation, in which she compares this approach with MVC: https://www.youtube.com/watch?v=nYkdrAPrdcw
Hopefully this may nudge you in the right direction. I had a similar issue where I needed to perform a synchronous operation in an asynchronous microservice architecture.
How I solved it was using the Observer pattern. I have 3 components; a http server, a callback server and a timer wheel. The http server similar to yours receives the incoming request, the callback server receives the overal result after asynchronous processing and the timer wheel that queues the original http context and reconciles the response to the http request.
When an incoming request is received, the http server creates a correlation id ,appends it to the request metadata, appends the callback server url to the request metadata and finally adds the request and the original http context together into the timer wheel. Then the http server would pass the request to the dispatcher like in your case and send messages to the relevant components for asynchronous processing.
Depending on the outcome of execution of the current processing component, it will retrieve the callback url from the metadata and send the response to the callback server.In your case there's the json serialization or the database write that would do this. The callback server will then extract the correlation Id that was appended and get the corresponding http context and write the response.
NB each timer object in the timer wheel has a timeout that's configurable, that way if the asynchronous processing delays it will timeout and return a configurable message to the http client of the corresponding http context.
Warning: Please bear with me and I am fairly new with Gatling. So, apologies in advance. :P :)
I was going through the Loadrunner Asynchronous Calls Function - wb_reg_async_attributes, and I found that there are four different Asynchronous Conversation Patterns, which are:
Poll - The client polls the server periodically for information.
Long Poll - The client polls the server and waits for a response.
When the response arrives, another poll request is initiated.
Push -The client sends a request. The server response is to send updates
when there are changes to the requested information.
Cross-user - One user performs an activity that is reflected in another user's client. For example, user1 sends an email and user2 receives
notification.
Now, I have a requirement where I need to test Long-Polling using Gatling.
As far as I know, there are two ways in Gatling:
Poll
SSE
Please feel free to let me know in case I am wrong.
By using Polling function of Gatling, I am getting a Gateway Timeout Error. My theory is:
Gatling sends the request --> doesn't get a response --> Comes back with Gateway Timeout error.
Is there a way I can emulate Long Polling in Gatling? Please help me out in resolving this challenge.
Poll works in the similar fashion as LongPoll
We have the following scenario that we would like to solve using Apache Camel:
An asynchronous request arrives to an AMQP endpoint configured in Camel. This message contains a header property for a reply-to that should be used for the response. Camel must pass this message to another service using JMS and then route the response back to the reply-to queue from the AMQP request. This seems like a textbook example for using the InOut functionality in Camel but we have one problem: The reply from JMS service could take a long time, in some cases several days.
As I understand it, if we are using InOut it would mean that we would lock a thread to the the long running service. If we are unlucky, we could get several long running calls simultaneously and in the worst case scenario it could be that all threads are busy waiting for replies thus clogging the system.
What strategy should I use for solving the problem described above? At the moment, I have created to separate routes: One that listens to the AMQP endpoint and forwards the message to the JMS endpoint. The other route listens to the replyto-queue for the jms system and would be responsible for sending the reply back to the AMQP reply-to. The problem I have right now is how I should store the AMQP reply-to between these two routes and I am not sure this is a good solution overall for this problem.
Any tips or ideas on how to solve this problem would be greatly appreciated.
If you have to wait for more than a minute for reply, it's probably a good thing to treat the reply as async. and create separate request and response routes.
Since you mention several days, you might even want to survive an application restart (or even backup-restore) to correlate the response. In such cases, you need to store correlation information in a persistent store such as a database or a JMS queue using message properties - and selectors to retrieve the correlation information back.
I've used both queues and databases for long time request/reply correlation information with success.
It's always a good practice to be able to fail over/restart the server or the application at any time knowing that any ongoing processing will take up where it left off without errors.
There is a cost in complexity and performance, but robustness is often perferred to performance.
Recently I did Start To Learn Node.js abd As of That i did Figure out that nower day`s node.js is widely use cause it supports Push-Pull/Publish-subscript as well as Request-Response both kind of Programming Capability.
But After Reading 6 to 8 article still i have confusion on the diffrance between them.
Any Help Would Be Appriciated.
Thanks In Advance.
It is very simple.
Lets assume the server has an update for the client. For instance, the client receives a message.
In request-response, the client won't notice about that message until he will send a request to the server, using ajax or simply moving to another page (Executing Http request).
The main problem here is that the updates are not real-time/live. The client doesn't know if there are any updates for him until he make a further request.
On the other hand, in push-pull relation the sever pushes the updates to the client,
so the client informed about the message he receives live, without waiting for the next request to receive the update.
Pull technology vs Push technology technologies explained in wikipedia.
I'd like to implement push notification server using node.js. The basic scenario is:
Some applications sends notification messages to the server.
Notification server receives the request and forwards the message to uesr's mail or IM client based on user's preference.
In step 1, which protocol (e.g. REST, socket, HTTP/XML and so on.) would you recommend from the performance perspective?
Also in step 2, I have a plan to use node-xmpp module for IM client but for mail, which way is the best to implement? For example,
Just use SMTP. (But I think this might occur performance degradation because SMTP is an expensive communication and performance depends on SMTP server capacity.
use queue mechanism, in order to avoid drawbacks from the above. node.js app simply puts the message into the queue, and smtp server pulls the message.
other solutions...
Thanks in advance.
With regards to what to use as a protocol, i would go for a REST interface, whereby the application posting sends a POST request to a resource associated with the USER. something along the lines of "http://example.com/rest/v1/{userID}/notifications
I personally would use json as the data/content of the rest request and have node.js write this information to a message queue. (as a json string).
You can than have xmpp readers for each user, as well as an SMTP handler reading from this queue as fast as the SMTP server allows it to go.
However, this full post is what i would do in your situation, rather than a factual response on what is best. I know JMS fairly well and i've been working a lot with rest interfaces lately, therefore this is the way i would do it.