0

I tried to generate a single .jar for both the whole scala app and another .jar that contains some packages.

I got this error when merging the two.

[error] errors were encountered during merge
[error] java.lang.RuntimeException: deduplicate: different file contents found in the following:
[error] /proj/.ivy2/cache/org.apache.logging.log4j/log4j-core/jars/log4j-core-2.10.0.jar:META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
[error] /proj/.ivy2/cache/io.sensefly.logging.log4j/log4j-cloudwatch-appender/jars/log4j-cloudwatch-appender-1.0.1.jar:META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
[error] deduplicate: different file contents found in the following:
[error] /proj/.ivy2/cache/org.slf4j/jcl-over-slf4j/jars/jcl-over-slf4j-1.7.25.jar:org/apache/commons/logging/Log.class
[error] /proj/.ivy2/cache/commons-logging/commons-logging/jars/commons-logging-1.1.3.jar:org/apache/commons/logging/Log.class
[error] deduplicate: different file contents found in the following:
[error] /proj/.ivy2/cache/org.slf4j/jcl-over-slf4j/jars/jcl-over-slf4j-1.7.25.jar:org/apache/commons/logging/LogConfigurationException.class
Graham
  • 7,431
  • 18
  • 59
  • 84
John
  • 169
  • 1
  • 3
  • 10
  • I think this doesn't deserve a `-1` because it's unique question in a sense that there is an external JAR involved. – yǝsʞǝla Feb 19 '18 at 02:49

2 Answers2

0

It looks like you have multiple logging libraries that are in conflict. This kind of question has been asked a lot here on SO, for example: How to exclude commons-logging from a scala/sbt/slf4j project?.

You'll be the best person to judge on how to handle this because you hopefully know which logging library you do need.

There are multiple ways to address this. You can pick a single logging library or any other conflicting dependency and exclude the rest using exclude. Docs are here: https://www.scala-sbt.org/1.x/docs/Library-Management.html#Exclude+Transitive+Dependencies. If one or more of your libraries is pulling the logger dependency transitively you can exclude that transitive dependency and provide one explicitly. There is a good chance it will come from your JAR.

Another more hackerish way is to deal with conflicting files one by one using merge strategy: https://github.com/sbt/sbt-assembly#merge-strategy. You can simply take one of the files and ignore other ones. This is not always going to work and might have API compat issues. That's why it's more of a hack in this case, but could help to get pesky files out of the way.

Finally, even more hackish way would be to edit your JAR (archive) and to delete conflicting files.

Use https://github.com/jrudolph/sbt-dependency-graph to help you understand/visualize where transitive dependencies are coming from.

yǝsʞǝla
  • 16,272
  • 2
  • 44
  • 65
  • I am not sure why the logging libraries caused the conflict. I did add `).map(_.exclude("commons-logging", "commons-logging")) .map(_.exclude("log4j-cloudwatch-appender", "log4j-cloudwatch-appender"))`` – John Feb 21 '18 at 00:21
  • `error] 2 errors were encountered during merge [error] java.lang.RuntimeException: deduplicate: different file contents found in the following: [error] /.ivy2/cache/org.apache.logging.log4j/log4j-core/jars/log4j-core-2.10.0.jar:META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat ' – John Feb 21 '18 at 00:30
0

My colleague also faced the same issue. We added the following snippet in built.sbt file. It helped us.

assemblyMergeStrategy in assembly := {
 case PathList("META-INF", xs @ _*) => MergeStrategy.discard
 case x => MergeStrategy.first
}
Aakash Jain
  • 21
  • 6