I have a scenario where a BPEL process with a parallel flow is calling an asynchronous process in parallel and waits for their callbacks. I added two correlation sets one to correlate to the calling BPEL process instance and one to correlate to the Receive in which flow path. But I am receiving a conflictingReceive fault response. And the error:
ERROR [PICK] org.apache.ode.bpel.common.FaultException: {Selector plinkInstnace= {PartnerLinkInstance partnerLinkDecl=OPartnerLink#41,scopeInstanceId=9601},ckeySet=[{CorrelationKey setId=AsynchCorr, values=[hello]}, {CorrelationKey setId=FlowCorr, values=[flow 2:]}],opName=onResult,oneWay=yes,mexId=<null>,idx=0,route=one}
I am using Apache ODE with Tomcat. Can you help me find a solution for this problem, it is driving me mad!! I can send you sample projects if you can help.
The problem was that I thought the msg is matched with the correlationSet property value .. I have defined another correlationSet with the same flow_property .. I updated the files in the comment above with the true solution. CallerProcess.bpel
Related
I've joined a legacy project, where there's virtually no logging. Few days ago we had a production release that failed massively, and we had no clear idea what's going on. That's why improving logging is one of the priorities now.
I'd like to introduce something like "correlation id", but I'm not sure what approach to take. Googling almost always brings me to the solutions that are suitable for "Microservices talking via REST" architecture, which is not my case.
Architecture is a mix of Spring Framework and NodeJS running on the same Unix box - it looks like this:
Spring receives a Request (first thread is started) and does minor processing.
Processing goes to a thread from ThreadPool (second thread is started).
Mentioned second thread starts a separate process of NodeJS that does some HTML processing.
Process ends, second thread ends, first thread ends.
Options that come to my mind are:
Generate UUID and pass it around as argument.
Generate UUID and store it in ThreadLocal, pass it when necessary when changing threads or when starting a process.
Any other ideas how it can be done correctly?
You are on the right track. Generate a UUID and pass it as a header into the request. For any of the request that do not have this header add a filter thats checks for it and add it.
Your filter will pick such a header and can put it in thread local where MDC can pick it from. There after any logging you do will have the correlation id. When making a call to any other process/request you need to make sure you pass this id as an argument/header. And the cycle repeats.
Your thread doing the task should just be aware of this ID. Its upto you to decide how you want to pass it. Try to just separate out such concerns from your biz logic (Using Aspects or any other way you see fit) and more you can keep this under the hood easier it would be for you.
You can refer to this example
I have a SoapUI test which uses an input file to read lines as input of requests. So there is a loop which reads data and execute request and write output to file. Response times are too long, so processing of this file should be done asynchronously, but I am not sure, how SoapUI can handle this. There is file attachment in SOAP requests, which is not handled by current version of JMeter.
As per the SoapUI's documentation below, both test cases or test suites can be executed in Parallel mode.
In the case of TestSuites and TestCases these can be executed either in sequence or parallell, as configured with the corresponding toolbar buttons.
In the above image, first one in the marked image stands for sequential execution and the second one (with multiple parallel arrows) stands for Parallel execution mode.
User can select either of the one before executing the tests.
Hope this helps.
Note that SOAPUI does not allows test steps to be executed in parallel. If you need any custom execution i.e., same test case and steps to be executed in Parallel, here is sample project done for that. It can be used as reference and apply it to your case.
I understood this question as requiring the ability to call a service asynchronously due to the time it takes to process. So, by this, I mean SoapUI makes a request to a web service and instead of waiting for it, it carries on. At some point later, SoapUI receives the response.
SoapUI can handle this, I haven't tried it myself, but when reading some guides recently, I noticed it can be done.
See....
Blog Guide
SoapUI Forum
In short, it involves setting up a mock service to receive the response, which can then be validated.
I am building a simple application to download a set of XML files and parse them into a database using the async module (https://npmjs.org/package/node-async) for flow control. The overall flow is as follows:
Download list of datasets from API (single Request call)
Download metadata for each dataset to get link to XML file (async.each)
Download XML for each dataset (async.parallel)
Parse XML for each dataset into JSON objects (async.parallel)
Save each JSON object to a database (async.each)
In effect, for each dataset there is a parent process (2) which sets of a series of asynchronous child processes (3, 4, 5). The challenge that I am facing is that, because so many parent processes fire before all of the children of a particular process are complete, child processes seem to be getting queued up in the event loop, and it takes a long time for all of the child processes for a particular parent process to resolve and allow garbage collection to clean everything up. The result of this is that even though the program doesn't appear to have any memory leaks, memory usage is still too high, ultimately crashing the program.
One solution which worked was to make some of the child processes synchronous so that they can be grouped together in the event loop. However, I have also seen an alternative solution discussed here: https://groups.google.com/forum/#!topic/nodejs/Xp4htMTfvYY, which pushes parent processes into a queue and only allows a certain number to be running at once. My question then is does anyone know of a more robust module for handling this type of queueing, or any other viable alternative for handling this kind of flow control. I have been searching but so far no luck.
Thanks.
I decided to post this as an answer:
Don't launch all of the processes at once. Let the callback of one request launch the next one. The overall work is still asynchronous, but each request gets run in series. You can then pool up a certain number of the connections to be running simultaneously to maximize I/O throughput. Look at async.eachLimit and replace each of your async.each examples with it.
Your async.parallel calls may be causing issues as well.
What needs to be done to make InMemoryTransientMessageService run in a background thread? I publish things inside a service using
base.MessageProducer.Publish(new RequestDto());
and they are exececuted immediately inside the service-request.
The project is self-hosted.
Here is a quick unit test showing the blocking of the current request instead of deferring it to the background:
https://gist.github.com/lmcnearney/5407097
There is nothing out of the box. You would have to build your own. Take a look at ServiceStack.Redis.Messaging.RedisMqHost - most of what you need is there, and it is probably simpler (one thread does everything) to get you going when compared to ServiceStack.Redis.Messaging.RedisMqServer (one thread for queue listening, one for each worker). I suggest you take that class and adapt it to your needs.
A few pointers:
ServiceStack.Message.InMemoryMessageQueueClient does not implement WaitForNotifyOnAny() so you will need an alternative way of getting the background thread to wait to incoming messages.
Closely related, the ServiceStack.Redis implementation uses topic subscriptions, which in this class is used to transfer the WorkerStatus.StopCommand, which means you have to find an alternative way of getting the background thread to stop.
Finally, you may want to adapt ServiceStack.Redis.Messaging.RedisMessageProducer as its Publish() method pushes the message requested to the queue and pushes the channel / queue name to the TopicIn queue. After reading the code you can see how the three points tie together.
Hope this helps...
I have implemented canFetchMore, hasChildren and fetchMore in order to allow my model to be lazy loaded. It's very simple and based on QT's: http://doc.qt.io/archives/qt-4.7/itemviews-simpletreemodel.html
My problem is that in my application fetching children is not a very quick operation, it involves a few seconds of delay on the server side while it figures out who the children actually are.
I'm unsure how to deal with that. I can't have my application locking up for several seconds every time someone expands a node. I don't know how to go about making this happen in the background. If I was to create a sub-process or thread to actually do the work of retrieving the children and updating the client side data structure, how would I go about telling the model that this had successfully completed (and for the node to finally expand).
Also, is there a way to show that the node is currently in the process of loading the data in the background?
Apologies if these are stupid questions, GUI programming is still a bit of a mystery to me and I've never used QT before.
For the record, I'm using Python, but if answers are given in C++ I can understand them.
Thanks
If I was to
create a sub-process or thread to actually do the work of retrieving
the children and updating the client side data structure, how would I
go about telling the model that this had successfully completed (and
for the node to finally expand).
You can use signal and slots. In the thread you retrieve the data you will emit a custom signal like someDataAvailable(YourdataType) and then in the gui you will handle this signal with a slot something like handleDataReadySignal(YourdataType). The signal passes the object that you give it when emitting. Apparently you need to update the gui and the list in the handleDataReadySignal slot. Of course you need to connect the slot to the signal preferably in the constructor of the window/dialog to which the list is attached