I recently had a scenario like below:
Flow_A ------> Flow_B ------> Flow_C ------> Flow_D
Where
Flow_A is the initiator and should pass messageA.
Flow_B should pass messageA+messageB.
Flow_C should pass messageA+messageB+messageC
Flow_D should pass messageA+messageB+messageC+messageD.
So, I was thinking to enhance the headers with an old message and again pass to another flow. But, it will be very bulky at the end.
Should I store the message somewhere and then pass the messageId in the header, so that the next flow can get the old message with the messageId?
What should be the best way to achieve this?
See Claim Check pattern: https://docs.spring.io/spring-integration/docs/current/reference/html/message-transformation.html#claim-check
You store a message using ClaimCheckInTransformer and get its id as an output payload.
You move this id into a header and produce the next message.
Repeat #1 and #2 steps for this second message to be ready for the third one.
And so on to prepare environment for the fourth message.
To restore those messages you need to repeat the procedure opposite direction.
Get a header from the message into a payload, remove it and call ClaimCheckOutTransformer to restore a stored message. I say "remove header" to let the stack to be restored properly: the ClaimCheckOutTransformer has a logic like this:
AbstractIntegrationMessageBuilder<?> responseBuilder = getMessageBuilderFactory().fromMessage(retrievedMessage);
// headers on the 'current' message take precedence
responseBuilder.copyHeaders(message.getHeaders());
So, without removing that header, the same message id is going to be carried into the next step and you will be is a loop - StackOverflowError.
Another is to store messages manually somewhere, e.g. MetadataStore and collect their ids in the list for payload. This way you don't need extra logic to deal with headers. Everything in a list of your payload. You can consult the store any time for any id item in that list!
Related
What is the preferred way to validate requested DICOM connection against a list of known hosts?
I can connect to the EVT_CONN_OPEN event. But in that, the event.assoc.requestor.info.ae_title element is always empty (b'').
I see from a TCP network analysis, that the name is transmitted. So, where is it?
What is the right way to validate the requesting host?
You could try using EVT_REQUESTED instead, it gets triggered after an association request is received/sent and the AE title information should be available at that point. Unfortunately EVT_CONN_OPEN is triggered on TCP connection which occurs prior to the association request.
If you don't like the host's details you can use the handler to send an association rejection message using event.assoc.acse.send_reject() or abort with event.assoc.abort().
If you're only interested in validating against the AE title you can use the AE.require_calling_aet property to restrict associations to those with matching AE titles.
For the benefit of anyone else looking this up, the correct stage to look this up is in the EVT_REQUESTED event. However you will likely find the details aren't filled in (they are populated AFTER the handler has been called).
So if you want to locate the callers AE in EVT_REQUESTED, you need to locate the A_ASSOCIATE primitive and read them from there. So for example in your handler you can do this to reject remotes:
def handle_request(event):
req_title = event.assoc.requestor.primitive.calling_ae_title.decode('ascii')
if req_title != 'MyAET':
event.assoc.acse.send_reject(0x01, 0x01, 0x03)
return
At least for 1.5.7.
Initially our flow of cimmunicating with google Pub/Sub was so:
Application accepts message
Checks that it doesn't exist in idempotencyStore
3.1 If doesn't exist - put it into idempotency store (key is a value of unique header, value is a current timestamp)
3.2 If exist - just ignore this message
When processing is finished - send acknowledge
In the acknowledge successfull callback - remove this msg from metadatastore
The point 5 is wrong because theoretically we can get duplicated message even after message has processed. Moreover we found out that sometimes message might not be removed even although successful callback was invoked( Message is received from Google Pub/Sub subscription again and again after acknowledge[Heisenbug]) So we decided to update value after message is proccessed and replace timestamp with "FiNISHED" string
But sooner or later we will encounter that this table will be overcrowded. So we have to cleanup messages in the MetaDataStore. We can remove messages which are processed and they were processed more 1 day.
As was mentioned in the comments of https://stackoverflow.com/a/51845202/2674303 I can add additional column in the metadataStore table where I could mark if message is processed. It is not a problem at all. But how can I use this flag in the my cleaner? MetadataStore has only key and value
In the acknowledge successfull callback - remove this msg from metadatastore
I don't see a reason in this step at all.
Since you say that you store in the value a timestamp that means that you can analyze this table from time to time to remove definitely old entries.
In some my project we have a daily job in DB to archive a table for better main process performance. Right, just because we don't need old data any more. For this reason we definitely check some timestamp in the raw to determine if that should go into archive or not. I wouldn't remove data immediately after process just because there is a chance for redelivery from external system.
On the other hand for better performance I would add extra indexed column with timestamp type into that metadata table and would populate a value via trigger on each update or instert. Well, MetadataStore just insert an entry from the MetadataStoreSelector:
return this.metadataStore.putIfAbsent(key, value) == null;
So, you need an on_insert trigger to populate that date column. This way you will know in the end of day if you need to remove an entry or not.
I'm configuring some requests programmatically in my test cases, I can set headers, custom properties, teardown scripts, etc. however I can't find how to set a standard json body for my put requests.
Is there any possibility from the restMethod class ?
So far I end up getting the method used :
restService = testRunner.testCase.testSuite.project.getInterfaceAt(0)
resource = restService.getOperationByName(resource_name)
request = resource.getRequestAt(0)
httpMethod = request.getMethod()
if (httpMethod.toString().equals("PUT"))
but then I'm stuck trying to find how to set a standard body for my PUT requests.
I try with the getRequestParts() method but it didn't give me what I expected ...
can anyone help, please
thank you
Alexandre
I’ve managed this. I had a tests of tests where I wanted to squirt the content of interest into the “bare bones” request. Idea being that I can wrap this in a data driven test. Then, for each row in my data spreadsheet I pull in the request body for my test. At first I simply pulled the request from a data source value in my spreadsheet, but this became unmanageable in my spreadsheet.
So, another tactic. In my test data sheet (data source) I stored the file name that contains the payload I want to squirt in.
In the test itself, I put in a groovy step immediately before the the step I want to push the payload into.
The groovy script uses the data source to firstly get the file name containing the payload, I then read the contents of the file.
In the step I want to push the data into, I just use a get from data, e.g. {groovyStep#result}.
If this doesn’t completely make sense, let me know and I’ll update with screenshot when I have access to SoapUi.
In spring-batch, data can be passed between various steps via ExecutionContext. You can set the details in one step and retrieve in the next. Do we have anything of this sort in spring-integration ?
My use case is that I have to pick up a file from ftp location, then split it based on certain business logic and then process them. Depending on the file names client id would be derived. This client id would be used in splitter, service activator and aggregator components.
From my newbie level of expertise I have in spring, I could not find anything which help me share state for a particular run.I wanted to know if spring-integration provides this state sharing context in some way.
Please let me know if there is a way to do in spring-context.
In Spring Integration applications there is no single ExecutionContext for state sharing. Instead, as Gary Russel mentioned, each message carries all the information within its payload or its headers.
If you use Spring Integration Java DSL and want to transport the clientId by message header you can use enrichHeader transformer. Being supplied with a HeaderEnricherSpec, it can accept a function which returns dynamically determined value for the specified header. As of your use case this might look like:
return IntegrationFlows
.from(/*ftp source*/)
.enrichHeaders(e -> e.headerFunction("clientId", this::deriveClientId))
./*split, aggregate, etc the file according to clientId*/
, where deriveClientId method might be a sort of:
private String deriveClientId(Message<File> fileMessage) {
String fileName = fileMessage.getHeaders().get(FileHeaders.FILENAME, String.class);
String clientId = /*some other logic for deriving clientId from*/fileName;
return clientId;
}
(FILENAME header is provided by FTP message source)
When you need to access the clientId header somewhere in the downstream flow you can do it the same way as file name mentioned above:
String clientId = message.getHeaders().get("clientId", String.class);
But make sure that the message still contains such header as it could have been lost somewhere among intermediate flow items. This is likely to happen if at some point you construct a message manually and send it further. In order not to loose any headers from the preceding message you can copy them during the building:
Message<PayloadType> newMessage = MessageBuilder
.withPayload(payloadValue)
.copyHeaders(precedingMessage.getHeaders())
.build();
Please note that message headers are immutable in Spring Integration. It means you can't just add or change a header of the existing message. You should create a new message or use HeaderEnricher for that purpose. Examples of both approaches are presented above.
Typically you convey information between components in the message payload itself, or often via message headers - see Message Construction and Header Enricher
The documentation from google is giving a 404 error right now so I can't read it.
It appears that I can put any strings I want into the MediaMetadata but when I go to retrieve them from the VCM (VideoCastManager) they aren't there. Is there any way to add custom data to the MediaMetadata?
I want this so I can have more information about the video I am playing when the application reconnects.
Thanks.
I was having the same issue....and I found the cause to be how I was constructing my MediaMetaData object in my sender app's code.
Constructing the MediaMetadata object in your receiver app's code with the constant MediaMetadata.MEDIA_TYPE_MOVIE (as is done in the CastCompanionLibrary sample) won't let you store anything with .putString(key,value) other than with the keys MediaMetadata.KEY_TITLE and MediaMetadata.KEY_SUBTITLE.
Here's what I've found....
Since I'm playing video on the chromecast, I assumed the best way to construct the MediaMetadata object was to use the MediaMetadata.MEDIA_TYPE_MOVIE constant like so :
MediaMetadata castMetaData = new MediaMetadata(MediaMetadata.MEDIA_TYPE_MOVIE);
Then....I attempted to send the following data with the MediaMetadata object:
castMetaData.putString(MediaMetadata.KEY_TITLE,"videoTitle");
castMetaData.putString(MediaMetadata.KEY_SUBTITLE,"videoSubTitle");
castMetaData.putString(MediaMetadata.KEY_ARTIST,"videoArtistName");
After sending this MediaMetadata object to the receiver (ensuring that the three KEYs and their corresponding values were in-tact) and video began to play, I attempted a call to retrieve the video's metadata from the receiver with :
MediaMetadata receiverMetadata = videoCastManager.getRemoteMediaInformation().getMetaData();
This returned a MediaMetadata object, but the MediaMetadata.KEY_ARTIST mapping was no where to be found.
After changing the MediaMetadata construct parameter to MediaMetadata.MEDIA_TYPE_GENERIC like so I was able to send more metadata in the MediaMetadata object:
MediaMetadata castMetaData = new MediaMetadata(MediaMetadata.MEDIA_TYPE_GENERIC);
Note
I don't know if constructing the MediaMetadata object with MediaMetadata.MEDIA_TYPE_GENERIC is OK when the actual task is to play video.
What is VCM? You can add any pair of key/values to the MediaMetadat, use one of the variations of put*() such as putInt(key,value), putString(key, value), etc. Then retrieve them by corresponding get*() methods. On your receiver side, you'll have a JSON with those key/value pairs so you can easily retrieve them there too.