I have a stateless REST API build on Spring Boot 1.4.2. I want to log all the API calls into elk. Requests and responses data (headers, parameters, payload) need to be logged as well. I don't want to log them 1:1 - I want to filter out sensitive data etc.
I made an aspect that is intercepting my #RestController's methods invocation. I created custom annotation for method's parameter that should be logged (I use it on payloads annotated as well by #RequestBody) following this article and it gave me access to my data transfer objects in my #Around advice. I dont care about their type - I would like to call logger.debug(logObject) and send this log to logstash.
As far as I understand log message should be send as JSON with JSONLayout set in Log4j2 appender to ease things on the logstash side. So I serialize my logObject into JSON log message but during this and this only serialization I want to filter sensitive data out. I can not use transient because my controller depends on the same field.
Can I somehow create an #IgnoreForLogging annotation, that will be detected only by my custom Gson serializer that I use within logging advice and will be ignored within standard Spring's infrastructure? Is my logging into logstash approach even correct (I am trying to set it up for the first time)?
I can't believe I missed that in documentation. Here is the link
My custom annotation:
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.FIELD)
public #interface IgnoreForLogging {
}
Strategy for serializing objects:
public class LoggingExclusionStrategy implements ExclusionStrategy {
#Override
public boolean shouldSkipField(FieldAttributes fieldAttributes) {
return fieldAttributes.getAnnotation(IgnoreForLogging.class) != null;
}
#Override
public boolean shouldSkipClass(Class<?> aClass) {
return false;
}
}
Serializing log message in aspect class:
Gson gson = new GsonBuilder()
.setExclusionStrategies(new LoggingExclusionStrategy())
.create();
String json = gson.toJson(logObject);
This way Spring internally uses default serializer that doesn't know about #IgnoreForLogging and I can take advantage of my annotation in other places.
Related
I process messages from a queue. I use data from the incoming message to determine which class to use to process the message; for example origin and type. I would use the combination of origin and type to look up a FQCN and use reflection to instantiate an object to process the message. At the moment these processing objects are all simple POJOs that implement a common interface. Hence I am using a strategy pattern.
The problem I am having is that all my external resources (mostly databases accessed via JPA) are injected (#Inject) and when I create the processing object as described above all these injected objects are null. The only way I know to populate these injected resources is to make each implementation of the interface a managed bean by adding #stateless. This alone does not solve the problem because the injected members are only populated if the class implementing the interface is itself injected (i.e. container managed) as opposed to being created by me.
Here is a made up example (sensitive details changed)
public interface MessageProcessor
{
public void processMessage(String xml);
}
#Stateless
public VisaCreateClient implements MessageProcessor
{
#Inject private DAL db;
…
}
public MasterCardCreateClient implements MessageProcessor…
In the database there is an entry "visa.createclient" = "fqcn.VisaCreateClient", so if the message origin is "Visa" and the type is "Create Client" I can look up the appropriate processing class. If I use reflection to create VisaCreateClient the db variable is always null. Even if I add the #Stateless and use reflection the db variable remains null. It's only when I inject VisaCreateClient will the db variable get populated. Like so:
#Stateless
public QueueReader
{
#Inject VisaCreateClient visaCreateClient;
#Inject MasterCardCreateClient masterCardCreateClient;
#Inject … many more times
private Map<String, MessageProcessor> processors...
private void init()
{
processors.put("visa.createclient", visaCreateClient);
processors.put("mastercard.createclient", masterCardCreateClient);
… many more times
}
}
Now I have dozens of message processors and if I have to inject each implementation then register it in the map I'll end up with dozens of injections. Also, should I add more processors I have to modify the QueueReader class to add the new injections and restart the server; with my old code I merely had to add an entry into the database and deploy the new processor on the class path - didn't even have to restart the server!
I have thought of two ways to resolve this:
Add an init(DAL db, OtherResource or, ...) method to the interface that gets called right after the message processor is created with reflection and pass the required resource. The resource itself was injected into the QueueReader.
Add an argument to the processMessage(String xml, Context context) where Context is just a map of resources that were injected into the QueueReader.
But does this approach mean that I will be using the same instance of the DAL object for every message processor? I believe it would and as long as there is no state involved I believe it is OK - any and all transactions will be started outside of the DAL class.
So my question is will my approach work? What are the risks of doing it that way? Is there a better way to use a strategy pattern to dynamically select an implementation where the implementation needs access to container managed resources?
Thanks for your time.
In a similar problem statement I used an extension to the processor interface to decide which type of data object it can handle. Then you can inject all variants of the handler via instance and simply use a loop:
public interface MessageProcessor
{
public boolean canHandle(String xml);
public void processMessage(String xml);
}
And in your queueReader:
#Inject
private Instance<MessageProcessor> allProcessors;
public void handleMessage(String xml) {
MessageProcessor processor = StreamSupport.stream(allProcessors.spliterator(), false)
.filter(proc -> proc.canHandle(xml))
.findFirst()
.orElseThrow(...);
processor.processMessage(xml);
}
This does not work on a running server, but to add a new processor simply implement and deploy.
I have been working on a "paved road" for setting up asynchronous messaging between two micro services using AMQP. We want to promote the use of separate domain objects for each service, which means that each service must define their own copy of any objects passed across the queue.
We are using Jackson2JsonMessageConverter on both the producer and the consumer side and we are using the Java DSL to wire the flows to/from the queues.
I am sure there is a way to do this, but it is escaping me: I want the consumer side to ignore the __TypeID__ header that is passed from the producer, as the consumer may have a different representation of that event (and it will likely be in in a different java package).
It appears there was work done such that if using the annotation #RabbitListener, an inferredArgumentTypeargument is derived and will override the header information. This is exactly what I would like to do, but I would like to use the Java DSL to do it. I have not yet found a clean way in which to do this and maybe I am just missing something obvious. It seems it would be fairly straight forward to derive the type when using the following DSL:
return IntegrationFlows
.from(
Amqp.inboundAdapter(factory, queueRemoteTaskStatus())
.concurrentConsumers(10)
.errorHandler(errorHandler)
.messageConverter(messageConverter)
)
.channel(channelRemoteTaskStatusIn())
.handle(listener, "handleRemoteTaskStatus")
.get();
However, this results in a ClassNotFound exception. The only way I have found to get around this, so far, is to set a custom message converter, which requires explicit definition of the type.
public class ForcedTypeJsonMessageConverter extends Jackson2JsonMessageConverter {
ForcedTypeJsonMessageConverter(final Class<?> forcedType) {
setClassMapper(new ClassMapper() {
#Override
public void fromClass(Class<?> clazz, MessageProperties properties) {
//this class is only used for inbound marshalling.
}
#Override
public Class<?> toClass(MessageProperties properties) {
return forcedType;
}
});
}
}
I would really like this to be derived, so the developer does not have to really deal with this.
Is there an easier way to do this?
The simplest way is to configure the Jackson converter's DefaultJackson2JavaTypeMapper with TypeIdMapping (setIdClassMapping()).
On the sending system, map foo:com.one.Foo and on the receiving system map foo:com.two.Foo.
Then, the __TypeId__ header gets foo and the receiving system will map it to its representation of a Foo.
EDIT
Another option would be to add an afterReceiveMessagePostProcessor to the inbound channel adapter's listener container - it could change the __TypeId__ header.
I want to log exception for all of my methods using AOP. I had created an attribute for the same as following:
[AttributeUsage(AttributeTargets.All)]
public class ClsLogger : System.Attribute
{
private string _exMsg;
public ClsLogger(string exMsg)
{
//
// TODO: Add constructor logic here
//
_exMsg = exMsg;
LogError();
}
public void LogError()
{
// This methods logs exception
// Log Exception
}
}
Finally, I want to use this logging attribute to log exception messages of the methods of my application. How I can pass exception messages to the atrribute as it is not a fixed string but varrible?
Could anybody help on this?
Attributes in C# don't instantiate until you call GetCustomAttributes, and they instantiate every time you do so (see this question here on SO).
If you want to use AOP (as your title indicates), then you'll need to use some framework, like PostSharp, Fody, SheepAspect, etc.
If you are using ASP.NET MVC, then there is a built-in ActionFilter class that you can also use, but only on Controller methods.
I have some #javax.xml.bind.annotation.Xml... annotated classes here intended for a RESt web service. Jersey is setup in a spring managed web container and the web service is returning a well formatted xml. We use the maven-enunciate-plugin to document the web service and create the xsd to the returned xml documents. I now would like to use the documentation xsd file as a schemaLocation within the returned xml file so that the xml validation won't complain about missing definions. How can I get the XML serialisation configured for this?
If I remember correctly, I had to do a few of things to get namespace identifiers properly written into my generated XML.
1) Created a JaxbFactory that configs and returns a custom marshaller (and unmarshaller, too, BTW). I'm omitting the getters/and unmarshalling setup below...
//constructor
public JaxbFactory() throws Exception {
context = JAXBContext.newInstance(ResourceDto.class);
// Setup the marshaller
marshaller = context.createMarshaller();
marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
marshaller.setProperty(Marshaller.JAXB_SCHEMA_LOCATION, XmlMetadataConstants.XML_SCHEMA_LOCATION); // this schema location is used in generating the schema-location property in the xml
}
2) That factory class isn't "visible" to Jersey. To make it visible, I create a MarshallerProvider. That looks something like this:
#Provider
public class ResourceJaxbMarshallerProvider implements ContextResolver<Marshaller> {
// injected by Spring
private ResourceJaxbFactory ResourceJaxbFactory;
private ResourceStatusJaxbFactory ResourceStatusJaxbFactory;
/*
* ----------------------------------------
* Setters (for Spring injected properties)
* ----------------------------------------
*/
public void setResourceJaxbFactory(ResourceJaxbFactory ResourceJaxbFactory) {
this.ResourceJaxbFactory = ResourceJaxbFactory;
}
public void setResourceStatusJaxbFactory(ResourceStatusJaxbFactory ResourceStatusJaxbFactory) {
this.ResourceStatusJaxbFactory = ResourceStatusJaxbFactory;
}
/*
* ------------------------
* Interface Implementation
* ------------------------
*/
public Marshaller getContext(Class<?> type) {
if (type == ResourceDto.class)
return ResourceJaxbFactory.getMarshaller();
else if (type == ResourceStatusDto.class)
return ResourceStatusJaxbFactory.getMarshaller();
else
return null;
}
}
I've got Jersey wired into Spring using the Jersey/Spring Servlet so any #Provider class that gets created by Spring is automatically recognized by Jersey. In my Spring applicationContext.xml all I have to do is instantiate the resource provider. It will, in turn, go grab the marshaller from the factory.
3) The other thing that I found critical was that I had to create a package-info.java file in the root package containing my resource. Looks like this:
/*
* Note that this file is critical for ensuring that our ResourceDto object is
* marshalled/unmarshalled with the correct namespace. Without this, marshalled
* classes produce XML files without a namespace identifier
*/
#XmlSchema(namespace = XmlMetadataConstants.XML_SCHEMA_NAMESPACE, elementFormDefault = XmlNsForm.QUALIFIED)
package com.yourcompany.resource;
import javax.xml.bind.annotation.XmlNsForm;
At least I think that's everything I needed to do, I can't remember every single piece. I do remember that the package-info.java piece was the last critical cog that made it all come together.
Hope that helps. I spent wayyyy too much time digging for the info on all this. Jersey was seductively simple before I wanted it to do proper xml schema validation (and decent error reporting for schema-invalid input). Once I started down that road Jersey went from brain-dead easy to decently hard. The majority of that difficulty was sussing out all the details from the variety of posts online. Hopefully this will help get you farther, quicker. :-)
I'm wondering if it's bad practice to use log4net directly on my domain object... I'll be using ELMAH for my exceptions on the ASP.NET MVC application side, but for some informational purposes I'd like to log some data about the domain model itself.
Given the following domain object:
public class Buyer
{
private int _ID;
public int ID
{
get { return _ID; }
set
{
_ID = value;
}
}
private IList<SupportTicket> _SupportTickets=new List<SupportTicket>();
public IList<SupportTicket> SupportTickets
{
get
{
return _SupportTickets.ToList<SupportTicket>().AsReadOnly();
}
}
public void AddSupportTicket(SupportTicket ticket)
{
if (!SupportTickets.Contains(ticket))
{
_SupportTickets.Add(ticket);
}
}
}
Is adding logging behavior in the AddSupportTicketMethod a bad idea...so essentialy it'd look like this:
public class Buyer
{
protected static readonly ILog log = LogManager.GetLogger(typeof(SupportTicket));
public Buyer()
{
log4net.Config.XmlConfigurator.Configure();
}
private int _ID;
public int ID
{
get { return _ID; }
set
{
_ID = value;
}
}
private IList<SupportTicket> _SupportTickets=new List<SupportTicket>();
public IList<SupportTicket> SupportTickets
{
get
{
return _SupportTickets.ToList<SupportTicket>().AsReadOnly();
}
}
public void AddSupportTicket(SupportTicket ticket)
{
if (!SupportTickets.Contains(ticket))
{
_SupportTickets.Add(ticket);
} else {
log.Warn("Duplicate Ticket Not Added.");
}
}
}
I have used log4net and log4J directly in domain objects. This has good side effects and bad ones.
+: Logging in the domain object is simple and straightforward to code and you know you can take advantage of log4net features.
--: It means the program making use of the domain objects needs to pay attention to log4net configuration, which may or may not be a problem
--: You cannot link your domain object to a different log4net version than the calling program is using. I've seen a lot of conflicts with one item linked against log4net 1.2.0.10 and another linked against an earlier release.
Not logging in your domain object is a bad idea. The alternative is as others have suggested, dependency injection or an external framework (such as commons-logging for log4J) that allows plugging different logging frameworks or creating an interface that does the logging and logging against that interface. (The code using your domain object would need to then supply an appropriate instance of that interface for logging purposes.)
If you are going to log from your domain objects and you use an IOC container which you might want to swap out, I would recommend you use the Service Locator pattern (you could look at the Sharp# architecture for a nice implementation of a SafeServiceLocator that wraps msoft's ServiceLocator with more informative error messages).
I would also like to suggest that you consider whether you want to log the type of error you show in your example. I would tend to want to have the domain object throw an exception in that case and let the caller decide whether that was something that was expected by the application (and hence shouldn't be logged) or whether that represents a situation that the caller wants to deal with in some way.
This is a classic question!
The good way of doing this would be to introduce a class member of ILogger type and abstract the logging into this interface. In your class wherever you do a call to logg something do it through this interface. Then inject this dependency at the run-time with one of the implementation using one of the available IoC container or dependency injection farmeworks. By default you can use log4net implementation of this interface.
Here is a long list of available dependency injection frameworks:
http://www.hanselman.com/blog/ListOfNETDependencyInjectionContainersIOC.aspx
I think logging is a cross cutting concern, so it's best done in an aspect-oriented fashion. If you're using a framework like Spring.NET it's available to you.