Maven SL4J multiple bindings, previous solutions fail - apache-spark

Question

I have a project that has multiple SLF4J bindings.
I have read and tried the solutions in this SO post, this other SO post
and the slf4j website.
What I see if this when I run the code is
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/Mike/.m2/repository/org/jlab/coat/coat-libs/5.1-SNAPSHOT/coat-libs-5.1-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/Mike/.m2/repository/org/slf4j/slf4j-log4j12/1.7.16/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
However in my pom.xml file, I already have
<dependency>
<groupId>org.jlab.coat</groupId>
<artifactId>coat-libs</artifactId>
<version>5.1-SNAPSHOT</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
and in the mvn dependency:tree I do not see this dependency for the exclusion jar, only for the Spark jar, i.e.
[INFO] com.IKP:DCdatabase:jar:0.0.1-SNAPSHOT
[INFO] +- org.jlab.coat:coat-libs:jar:5.1-SNAPSHOT:compile
[INFO] +- org.apache.spark:spark-core_2.11:jar:2.2.1:compile
...
...
...
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.16:compile
[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile
[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile
[INFO] | +- log4j:log4j:jar:1.2.17:compile
[INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.16:compile
I have also tried a few more steps, such as clean out my .m2 directory, and build all sources from scratch, but I still see this double binding.
The nuance that occurs is that the log suppression for Spark does occur, i.e.
Logger.getLogger("org.apache.spark.SparkContext").setLevel(Level.WARN);
Logger.getLogger("org").setLevel(Level.OFF);
Logger.getLogger("akka").setLevel(Level.OFF);
No longer set the levels to off and I see all the levels.
Is there another way to remove this multiple binding for SLF4J?

Answer 1

It looks to me like coat-libs has been packaged as an uber jar which is why slf4j doesn't show up as a dependency when you do mvn dependency:tree. This explains why your exclusion isn't working.
I would recommend taking a look at the maven shade plugin for packaging your jar. Then, you can use a filter to exclude the slf4j dependencies from coat-libs using a filter.
For example, maybe something like:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>org.jlab.coat:coat-libs</artifact>
<excludes>
<exclude>org/slf4j/**</exclude>
</excludes>
</filter>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

Answer 2

Just like the message says, you're getting one binding from coat-libs-5.1-SNAPSHOT.jar, and another binding in slf4j-log4j12-1.7.16.jar. It's not that "coat-libs" is trying to bring in a dependency that has a binding, it is a logging binding that's trying to handle SLF4J logging. You can only use one logging binding, so you either need to remove the usage of coat-libs, or you need to exclude slf4j-log4j12 from the dependencies for spark-core, depending on which logging framework you're actually trying to use.