Spring integeration DefaultSoapHeaderMapper - getStandardRequestHeaderNames - override - spring-integration

in previous si versions (si 2.11 to be specific and spring 3.1.1) getStandardRequestHeaderNames could be overrided to include Additional Application specific objects in the si message header. Our application relied on this ability (may be wrongfully so) to override this method and supply a custom POJO to be carried downstream consisting of many splitters, aggregators etc. The app used an ws inbound gateway and used the header-mapper attribute to specify the custom soap header mapper.
Any clues on the reasoning behind why getStandardRequestHeaderNames cannot be overriden?
Need some advise on how I can migrate this to the current spring release.
The requirement is to extract elements from soapHeader and map them to an SI message headers as an POJO and send it down stream.
All help appreciated.
Code Snippet: Works with older versions of spring
<int-ws:inbound-gateway id="webservice-inbound-gateway"
request-channel="input-request-channel"
reply-channel="output-response-channel"
header-mapper="CustomSoapHeaderMapper"
marshaller="marshaller"
unmarshaller="marshaller" />
#Component("CustomSoapHeaderMapper")
public class CustomSoapHeaderMapper extends DefaultSoapHeaderMapper {
private static final Logger logger = Logger.getLogger("CustomSoapHeaderMapper");
public static final String HEADER_SEARCH_METADATA = SearchMetadata.HEADER_ATTRIBUTE_NAME;
public static final String HEADER_SERVICE_AUDIT = "XXXXXXXX";
// Use simulation if security token is set to this value
public static final String SECURITY_TOKEN_SIMULATION = "XXXX";
private static final List<String> CUSTOM_HEADER_NAMES = new ArrayList<String>();
static {
CUSTOM_HEADER_NAMES.add(WebServiceHeaders.SOAP_ACTION);
CUSTOM_HEADER_NAMES.add(HEADER_SEARCH_METADATA);
}
private int version =SearchMetadata.VERSION_CURRENT;
public void setVersion(int version) {
this.version = version;
}
#Override
protected List<String> getStandardRequestHeaderNames() {
return CUSTOM_HEADER_NAMES;
}
#Override
protected Map<String, Object> extractUserDefinedHeaders(SoapMessage source) {
// logger.log(Level.INFO,"extractUserDefinedHeaders");
// call base class to extract header
Map<String, Object> map = super.extractUserDefinedHeaders(source);
Document doc = source.getDocument();
SearchMetadata searchMetadata = new SearchMetadata();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try {
source.writeTo(baos);
baos.flush();
searchMetadata.setRequestXML(baos.toString());
baos.close();
} catch (IOException e1) {
}
//logger.log(Level.WARNING, "Incoming Message " + baos.toString());
SOAPMessage soapMessage = ((SaajSoapMessage) source).getSaajMessage();
// generate TransactionID with UUID value
String transactionID = UUID.randomUUID().toString();
// logger.log(Level.WARNING, "TransactionID=" + transactionID);
Date now = new Date();
searchMetadata.setTransactionID(transactionID);
searchMetadata.setRequestType(SearchMetadata.REQUEST_TYPE_SYNCHRONOUS);
searchMetadata.setRequestTime(now);// initialize the request time
searchMetadata.setReceivedTime(now);// mark time system receives request
searchMetadata.setVersion(version);
Map<String, Object> finalHeaders = new HashMap<String, Object>();
finalHeaders.put(HEADER_SEARCH_METADATA, searchMetadata);
if (!CollectionUtils.isEmpty(map)) {
// copy from other map
finalHeaders.putAll(map);
// check if ServiceAudit is available
SoapHeaderElement serviceAuditElement = null;
for (String key : map.keySet()) {
// logger.log(Level.WARNING, "SoapHeader.{0}", key);
if (StringUtils.contains(key, HEADER_SERVICE_AUDIT)) {
serviceAuditElement = (SoapHeaderElement) map.get(key);
break;
}
}
}
return finalHeaders;
}
// GK Key Thing here for performance improvement is avoiding marshalling
public gov.dhs.ice.ess.schema.ServiceAudit ExtractAuditHeader(Document doc) {
....
}
return serviceAudit;
}
}

Share, please, some code how would you like to see that.
Maybe you can just implement your own SoapHeaderMapper and inject it into WS Inbound Gateway?
You can still reuse your logic and copy/paste the standard behavior from the DefaultSoapHeaderMapper.
UPDATE
The test-case to demonstrate how to add user-defined header manually:
#Test
public void testCustomSoapHeaderMapper() {
DefaultSoapHeaderMapper mapper = new DefaultSoapHeaderMapper() {
#Override
protected Map<String, Object> extractUserDefinedHeaders(SoapMessage source) {
Map<String, Object> headers = super.extractUserDefinedHeaders(source);
headers.put("foo", "bar");
return headers;
}
};
mapper.setRequestHeaderNames("*");
SoapMessage soapMessage = mock(SoapMessage.class);
Map<String, Object> headers = mapper.toHeadersFromRequest(soapMessage);
assertTrue(headers.containsKey("foo"));
assertEquals("bar", headers.get("foo"));
}

Related

Spring Kafka, manual committing in different threads with multiple acks?

I am trying to ack kafka Message consumed via a batchListener in a separate thread; Using #Async for the called method.
#KafkaListener( topics = "${topic.name}" ,containerFactory = "kafkaListenerContainerFactoryBatch", id ="${kafkaconsumerprefix}")
public void consume(List<ConsumerRecord<String, String>> records,Acknowledgment ack) {
records.forEach(record -> asynchttpCaller.posttoHttpsURL(record,ack));
}
and my Async code is below where KafkaConsumerException extends BatchListenerFailedException
#Async
public void posttoHttpsURL(ConsumerRecord<String, String> record,Acknowledgment ack)
{
try {
//post to http
ack.acknowledge();
}
catch(Exception ex){
throw new KafkaConsumerException("Exception occured in sending via HTTPS",record);
}
}
With the below Configuration
#Bean
public Map<String, Object> consumerConfigs() {
Map<String, Object> props = new HashMap<>();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
bootstrapServers);
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, true);
props.put(ConsumerConfig.ISOLATION_LEVEL_CONFIG,
"read_committed");
props.put(ConsumerConfig.HEARTBEAT_INTERVAL_MS_CONFIG, 10000);
props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, 30000);
props.put(ConsumerConfig.GROUP_ID_CONFIG, groupId);
props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG,
maxpollRecords);
return props;
}
#Bean
public ConsumerFactory<Object, Object> consumerFactory() {
return new DefaultKafkaConsumerFactory<>(consumerConfigs());
}
/**
* Batch Listener */
#Bean
#Primary
public ConcurrentKafkaListenerContainerFactory<Object, Object>
kafkaListenerContainerFactoryBatch (
ConcurrentKafkaListenerContainerFactoryConfigurer configurer,
ConsumerFactory<Object, Object> kafkaConsumerFactory,
KafkaOperations<? extends Object, ? extends Object> template ) {
ConcurrentKafkaListenerContainerFactory<Object, Object>
factory = new ConcurrentKafkaListenerContainerFactory<>();
configurer.configure(factory, consumerFactory());
factory.setBatchListener(true);
factory.getContainerProperties().setAckMode(AckMode.MANUAL);
DeadLetterPublishingRecoverer recoverer = new
DeadLetterPublishingRecoverer(template);
ExponentialBackOff fbo = new ExponentialBackOff();
fbo.setMaxElapsedTime(maxElapsedTime);
fbo.setInitialInterval(initialInterval);
fbo.setMultiplier(multiplier);
RecoveringBatchErrorHandler errorHandler = new
RecoveringBatchErrorHandler(recoverer, fbo);
factory.setBatchErrorHandler(errorHandler);
factory.setConcurrency(setConcurrency);
return factory;
}
This ack.acknowledge() acknowledges every record in that batch if using AckMode as MANUAL_IMMEDIATE and will ack only if all are success when AckMode is MANUAL.
The Scenario I have is --> there will be certain httpcalls that results in success and certain that gets a timeout both in the same batch; if the errored Messages has a greater offset than the successful one ;even the succesful one is not getting acknowledged and is being duplicated.
Not sure why BatchListenerFailedException always throws the whole batch though I give specifically the record that errored.
Any suggestions on how to implement this ?
You should not process asynchronously because offsets could be committed out-of-sequence.
BatchListenerFailedException will only work if thrown on the listener thread.

Spring-Integration: external routing slip

I would like to allow callers to pass an external routing slip, e.g. by posting:
POST http://localhost:8080/transform?routing-slip=capitalize&routing-slip=lowercase
Content-Type: text/plain
camelCase
It should be possible to use the given routing-slip array as external routing slip from a pojo:
#Bean
public IntegrationFlow transformerChain(RoutingSlipRouteStrategy routeStrategy) {
return IntegrationFlows.from(
Http.inboundGateway("/transform")
.headerExpression("routingSlipParam",
"#requestParams['routing-slip']")
.requestPayloadType(String.class))
.enrichHeaders(spec -> spec.header(
IntegrationMessageHeaderAccessor.ROUTING_SLIP,
new RoutingSlipHeaderValueMessageProcessor(
"#routePojo.get(request, reply)")
)
)
.logAndReply();
}
The pojo can access the routingSlipParam header and you would think it can then hold the slip as internal state, or at least that is what TestRoutingSlipRoutePojo lead me to believe, so I built this (with a slight doubt, given that there is only one instance of the pojo):
public class ExternalRoutingSlipRoutePojo {
private List<String> routingSlip;
private int i = 0;
public String get(Message<?> requestMessage, Object reply) {
if (routingSlip == null) {
routingSlip = (LinkedList)requestMessage.getHeaders()
.get("routingSlipParam");
}
try {
return this.routingSlip.get(i++);
} catch (Exception e) {
return null;
}
}
}
It turns out that this only works exactly once, which is not surprising after all - the index is incremented for every incoming message and the routing slip is never updated.
So I thought, sure, I have to hold the internal status for all incoming messages and came up with this RouteStrategy:
public class ExternalRoutingSlipRouteStrategy implements RoutingSlipRouteStrategy {
private Map<UUID, LinkedList<String>> routingSlips = new WeakHashMap<>();
private static final LinkedList EMPTY_ROUTINGSLIP = new LinkedList<>();
#Override
public Object getNextPath(Message<?> requestMessage,Object reply) {
MessageHeaders headers = requestMessage.getHeaders();
UUID id = headers.getId();
if (!routingSlips.containsKey(id)) {
#SuppressWarnings("unchecked")
List<String> routingSlipParam =
headers.get("routingSlipParam", List.class);
if (routingSlipParam != null) {
routingSlips.put(id,
new LinkedList<>(routingSlipParam));
}
}
LinkedList<String> routingSlip = routingSlips.getOrDefault(id,
EMPTY_ROUTINGSLIP);
String nextPath = routingSlip.poll();
if (nextPath == null) {
routingSlips.remove(id);
}
return nextPath;
}
}
That does not work either because the strategy is not only called for the incoming message but also for all the new messages which are created by the dynamic routing, which of course have different IDs.
But it is only called twice for the original message, so the routing slip never gets exhausted and the application runs in an endless loop.
How can I make spring-integration use an external routing slip?
UPDATE:
As suggested by Gary Russel, neither the external routing slip index nor the external routing slip itself should be stored in the Spring bean, rather one can use message headers to maintain them separately for each request:
Http.inboundGateway("/transform")
.headerExpression("routingSlipParam",
"#requestParams['routing-slip']")
.requestPayloadType(String.class))
.enrichHeaders(spec -> spec
.headerFunction("counter",h -> new AtomicInteger())
.header(IntegrationMessageHeaderAccessor.ROUTING_SLIP,
new RoutingSlipHeaderValueMessageProcessor(externalRouteStrategy)
)
)
The externalRouteStrategy is an instance of the following class:
public class ExternalRoutingSlipRouteStrategy implements
RoutingSlipRouteStrategy {
#Override
public Object getNextPath(Message<?> requestMessage, Object reply) {
List<String> routingSlip = (List<String>)
requestMessage.getHeaders().get("routingSlipParam");
int routingSlipIndex = requestMessage.getHeaders()
.get("counter", AtomicInteger.class)
.getAndIncrement();
String routingSlipEntry;
if (routingSlip != null
&& routingSlipIndex < routingSlip.size()) {
routingSlipEntry = routingSlip.get(routingSlipIndex);
} else {
routingSlipEntry = null;
}
return routingSlipEntry;
}
}
For reference, I have published the example in Github.
Go back to your first version and store i in a message header (AtomicInteger) in the header enricher.
.headerExpression("counter", "new java.util.concurrent.atomic.AtomicInteger()")
then
int i = requestMessage.getHeaders().get("counter", AtomicInteger.class).getAndIncrement();

How to write an NLog target using Signalr

I'm trying to write a target for NLog to send messages out to connected clients using SignalR.
Here's what I have now. What I'm wondering is should I be using resolving the ConnectionManager like this -or- somehow obtain a reference to the hub (SignalrTargetHub) and call a SendMessage method on it?
Are there performance ramifications for either?
[Target("Signalr")]
public class SignalrTarget:TargetWithLayout
{
public SignalR.IConnectionManager ConnectionManager { get; set; }
public SignalrTarget()
{
ConnectionManager = AspNetHost.DependencyResolver.Resolve<IConnectionManager>();
}
protected override void Write(NLog.LogEventInfo logEvent)
{
dynamic clients = GetClients();
var logEventObject = new
{
Message = this.Layout.Render(logEvent),
Level = logEvent.Level.Name,
TimeStamp = logEvent.TimeStamp.ToString("yyyy-MM-dd HH:mm:ss.fff")
};
clients.onLoggedEvent(logEventObject);
}
private dynamic GetClients()
{
return ConnectionManager.GetClients<SignalrTargetHub>();
}
}
I ended up with the basic the same basic structure that I started with. Just a few tweaks to get the information I needed.
Added exception details.
Html encoded the final message.
[Target("Signalr")]
public class SignalrTarget:TargetWithLayout
{
protected override void Write(NLog.LogEventInfo logEvent)
{
var sb = new System.Text.StringBuilder();
sb.Append(this.Layout.Render(logEvent));
if (logEvent.Exception != null)
sb.AppendLine().Append(logEvent.Exception.ToString());
var message = HttpUtility.HtmlEncode(sb.ToString());
var logEventObject = new
{
Message = message,
Logger = logEvent.LoggerName,
Level = logEvent.Level.Name,
TimeStamp = logEvent.TimeStamp.ToString("HH:mm:ss.fff")
};
GetClients().onLoggedEvent(logEventObject);
}
private dynamic GetClients()
{
return AspNetHost.DependencyResolver.Resolve<IConnectionManager>().GetClients<SignalrTargetHub>();
}
}
In my simple testing it's working well. Still remains to be seen if this adds any significant load when under stress.

How do you invoke a Runnable using Spring Framework?

I have a service that needs to invoke a runnable class.
Here are the lines of code that are being used in my service.
#Autowired
private LinkBrc2MemberProfile brcTask;
// Background Task.
SimpleAsyncTaskExecutor sate = new SimpleAsyncTaskExecutor();
sate.createThread(new LinkBrc2MemberProfile(user));
Here is my Runnable class
#Service
public class LinkBrc2MemberProfile implements Runnable {
private final Logger log = LoggerFactory.getLogger(LinkBrc2MemberProfile.class);
#Autowired
private LoyaltyDao dao;
private Member member;
public LinkBrc2MemberProfile() {
super();
}
public LinkBrc2MemberProfile(Member member) {
this.member = member;
}
public void run() {
log.debug("*** Member User Name: " + member.getString("USER_NAME"));
String emailAddress = member.getString("USER_NAME");
Map<String, Object> map = dao.findBrcByEmailAddress( emailAddress );
log.debug("==========================================================");
if( ! map.isEmpty() ) {
try {
//a.CUSTOMER_ID, a.EMAIL_ADDRESS, b.card_no
String customerId = (String) map.get("CUSTOMER_ID");
String brcCardNumber = (String) map.get("CARD_NO");
log.debug("\ncustomerId: " + customerId + " brcCardNumber: " + brcCardNumber);
if(!brcCardNumber.equals("")) {
// Add the Be Rewarded Card.
HashMap<String, String> userAttributes = new HashMap<String, String>();
String brcNumber = member.getString("BREWARDED_CARD_NO");
if (brcNumber.equals("")) {
userAttributes.put("BREWARDED_CARD_NO", brcCardNumber);
try {
member.putAll(userAttributes);
} catch (Exception e) {
String errorMessage = "Unable to save user's BRC information due to: " + e.getMessage();
log.error("{}", errorMessage);
}
}
}
} catch (Exception e) {
log.error(e.getMessage());
e.printStackTrace();
}
}
}
I'm not seeing any errors in the log but at the same time it does not appear to be invoking the Runnable class. Am I missing an annotation somewhere? Are there any good examples that you can point me to, the only ones I have found use XML files to configure the runnable class I would like to use annotations. Thanks in Advance.
I've updated my service to do the following.
Please help, my DAO is NULL so it looks like my #Autowired in my Runnable class is not wiring it in.
I've added the following bean to my bean-config.xml file.
<bean id="brcType" class="com.ws.ocp.service.LinkBrc2MemberProfile" scope="prototype"/>
I removed my #Autowired annotation and added the following to my service class.
ClassPathResource rsrc = new ClassPathResource("bean-config.xml");
XmlBeanFactory factory = new XmlBeanFactory(rsrc);
LinkBrc2MemberProfile brcTask = (LinkBrc2MemberProfile) factory.getBean("brcType");
SimpleAsyncTaskExecutor sate = new SimpleAsyncTaskExecutor();
// Set Member attribute
brcTask.setMember(user);
// Executer
sate.execute(brcTask);
Why is my dao still null?
The runnable will throw a NullPointerException, since you create it yourself (using the new operator), instead of letting Spring create it. This obviously means that the autowired DAO attribute won't be autowired, which will lead to a NPE when calling dao.findBrcByEmailAddress(...).
You should get your Runnable instance from the bean factory (as a prototype), set its member attribute, and then submit it to the executor.
To answer your question of how to properly use a Prototype-Bean, this is my favorite way:
#Component
abstract class MyBean {
/* Factory method that will be installed by Spring */
#Lookup
protected abstract YourPrototypeBean createBean();
void someCode() {
YourPrototypeBean bean = createBean();
}
}
Since it's a factory method, you can create as many instances as you like.

Does ServiceStack support binary responses?

Is there any mechanism in ServiceStack services to return streaming/large binary data? WCF's MTOM support is awkward but effective in returning large amounts of data without text conversion overhead.
I love service stack, this litle code was enough to return an Excel report from memory stream
public class ExcelFileResult : IHasOptions, IStreamWriter
{
private readonly Stream _responseStream;
public IDictionary<string, string> Options { get; private set; }
public ExcelFileResult(Stream responseStream)
{
_responseStream = responseStream;
Options = new Dictionary<string, string> {
{"Content-Type", "application/octet-stream"},
{"Content-Disposition", "attachment; filename=\"report.xls\";"}
};
}
public void WriteTo(Stream responseStream)
{
if (_responseStream == null)
return;
_responseStream.WriteTo(responseStream);
responseStream.Flush();
}
}
From a birds-eye view ServiceStack can return any of:
Any DTO object -> serialized to Response ContentType
HttpResult, HttpError, CompressedResult (IHttpResult) for Customized HTTP response
The following types are not converted and get written directly to the Response Stream:
String
Stream
IStreamWriter
byte[] - with the application/octet-stream Content Type.
Details
In addition to returning plain C# objects, ServiceStack allows you to return any Stream or IStreamWriter (which is a bit more flexible on how you write to the response stream):
public interface IStreamWriter
{
void WriteTo(Stream stream);
}
Both though allow you to write directly to the Response OutputStream without any additional conversion overhead.
If you want to customize the HTTP headers at the sametime you just need to implement IHasOptions where any Dictionary Entry is written to the Response HttpHeaders.
public interface IHasOptions
{
IDictionary<string, string> Options { get; }
}
Further than that, the IHttpResult allows even finer-grain control of the HTTP output where you can supply a custom Http Response status code. You can refer to the implementation of the HttpResult object for a real-world implementation of these above interfaces.
I had a similar requirement which also required me to track progress of the streaming file download. I did it roughly like this:
server-side:
service:
public object Get(FooRequest request)
{
var stream = ...//some Stream
return new StreamedResult(stream);
}
StreamedResult class:
public class StreamedResult : IHasOptions, IStreamWriter
{
public IDictionary<string, string> Options { get; private set; }
Stream _responseStream;
public StreamedResult(Stream responseStream)
{
_responseStream = responseStream;
long length = -1;
try { length = _responseStream.Length; }
catch (NotSupportedException) { }
Options = new Dictionary<string, string>
{
{"Content-Type", "application/octet-stream"},
{ "X-Api-Length", length.ToString() }
};
}
public void WriteTo(Stream responseStream)
{
if (_responseStream == null)
return;
using (_responseStream)
{
_responseStream.WriteTo(responseStream);
responseStream.Flush();
}
}
}
client-side:
string path = Path.GetTempFileName();//in reality, wrap this in try... so as not to leave hanging tmp files
var response = client.Get<HttpWebResponse>("/foo/bar");
long length;
if (!long.TryParse(response.GetResponseHeader("X-Api-Length"), out length))
length = -1;
using (var fs = System.IO.File.OpenWrite(path))
fs.CopyFrom(response.GetResponseStream(), new CopyFromArguments(new ProgressChange((x, y) => { Console.WriteLine(">> {0} {1}".Fmt(x, y)); }), TimeSpan.FromMilliseconds(100), length));
The "CopyFrom" extension method was borrowed directly from the source code file "StreamHelper.cs" in this project here: Copy a Stream with Progress Reporting (Kudos to Henning Dieterichs)
And kudos to mythz and any contributor to ServiceStack. Great project!

Resources