Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1.3.0] sbt plugin does not work with 1.2.8 if plugin build with 1.3.0 #5049

Closed
xuwei-k opened this issue Sep 6, 2019 · 22 comments
Closed
Labels
Bug workaround exists There's a workaround
Milestone

Comments

@xuwei-k
Copy link
Member

xuwei-k commented Sep 6, 2019

workaround by eed3si9n

pluginCrossBuild / sbtVersion := "1.2.8"

original report by xuwei-k

steps

sbt version: 1.3.0

build.sbt

enablePlugins(SbtPlugin)

organization := "com.example"

name := "example-plugin"

scriptedBufferLog := false

project/build.properties

sbt.version=1.3.0

src/main/scala/example/ExamplePlugin.scala

package example

import sbt._, Keys._

object ExamplePlugin extends AutoPlugin {
  object autoImport {
    val foo = settingKey[File]("")
  }
}

src/sbt-test/a/b/build.sbt

enablePlugins(ExamplePlugin)

foo := file("bar")

src/sbt-test/a/b/project/build.properties

sbt.version=1.2.8

src/sbt-test/a/b/project/plugins.sbt

addSbtPlugin("com.example" % "example-plugin" % "0.1.0-SNAPSHOT")

src/sbt-test/a/b/test

> compile

run sbt scripted

problem

Stack trace of

[info] [error] java.lang.NoSuchMethodError: sbt.package$.singleFileJsonFormatter()Lsjsonnew/JsonFormat;
[info] [error] 	at example.ExamplePlugin$autoImport$.<init>(ExamplePlugin.scala:7)
[info] [error] java.lang.NoSuchMethodError: sbt.package$.singleFileJsonFormatter()Lsjsonnew/JsonFormat;
[info] [error] 	at example.ExamplePlugin$autoImport$.<init>(ExamplePlugin.scala:7)
[info] [error] 	at example.ExamplePlugin$autoImport$.<clinit>(ExamplePlugin.scala)
[info] [error] 	at $ccf3da47bd1d87673069$.$sbtdef(/tmp/sbt_98bd2a58/b/build.sbt:3)
[info] [error] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info] [error] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info] [error] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] [error] 	at java.lang.reflect.Method.invoke(Method.java:498)
[info] [error] 	at sbt.compiler.Eval$.getValue(Eval.scala:578)
[info] [error] 	at sbt.compiler.Eval.$anonfun$eval$1(Eval.scala:129)
[info] [error] 	at sbt.internal.EvaluateConfigurations$.$anonfun$evaluateDslEntry$1(EvaluateConfigurations.scala:249)
[info] [error] 	at sbt.internal.EvaluateConfigurations$.$anonfun$evaluateSbtFile$6(EvaluateConfigurations.scala:172)
[info] [error] 	at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:233)
[info] [error] 	at scala.collection.immutable.List.foreach(List.scala:388)�[0m
[info] [error] 	at scala.collection.TraversableLike.map(TraversableLike.scala:233)
[info] [error] 	at scala.collection.TraversableLike.map$(TraversableLike.scala:226)
[info] [error] 	at scala.collection.immutable.List.map(List.scala:294)
[info] [error] 	at sbt.internal.EvaluateConfigurations$.$anonfun$evaluateSbtFile$4(EvaluateConfigurations.scala:172)
[info] [error] 	at sbt.internal.Load$.loadSettingsFile$1(Load.scala:1137)
[info] [error] 	at sbt.internal.Load$.$anonfun$discoverProjects$2(Load.scala:1144)
[info] [error] 	at scala.collection.MapLike.getOrElse(MapLike.scala:127)
[info] [error] 	at scala.collection.MapLike.getOrElse$(MapLike.scala:125)
[info] [error] 	at scala.collection.AbstractMap.getOrElse(Map.scala:59)
[info] [error] 	at sbt.internal.Load$.memoLoadSettingsFile$1(Load.scala:1143)
[info] [error] 	at sbt.internal.Load$.$anonfun$discoverProjects$4(Load.scala:1151)
[info] [error] 	at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:233)
[info] [error] 	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:58)
[info] [error] 	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:51)
[info] [error] 	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
[info] [error] 	at scala.collection.TraversableLike.map(TraversableLike.scala:233)
[info] [error] 	at scala.collection.TraversableLike.map$(TraversableLike.scala:226)
[info] [error] 	at scala.collection.AbstractTraversable.map(Traversable.scala:104)
[info] [error] 	at sbt.internal.Load$.loadFiles$1(Load.scala:1151)
[info] [error] 	at sbt.internal.Load$.discoverProjects(Load.scala:1165)
[info] [error] 	at sbt.internal.Load$.discover$1(Load.scala:862)
[info] [error] 	at sbt.internal.Load$.loadTransitive(Load.scala:937)
[info] [error] 	at sbt.internal.Load$.loadProjects$1(Load.scala:726)
[info] [error] 	at sbt.internal.Load$.$anonfun$loadUnit$11(Load.scala:729)
[info] [error] 	at sbt.internal.Load$.timed(Load.scala:1395)
[info] [error] 	at sbt.internal.Load$.$anonfun$loadUnit$1(Load.scala:729)
[info] [error] 	at sbt.internal.Load$.timed(Load.scala:1395)
[info] [error] 	at sbt.internal.Load$.loadUnit(Load.scala:688)
[info] [error] 	at sbt.internal.Load$.$anonfun$builtinLoader$4(Load.scala:484)
[info] [error] 	at sbt.internal.BuildLoader$.$anonfun$componentLoader$5(BuildLoader.scala:176)
[info] [error] 	at sbt.internal.BuildLoader.apply(BuildLoader.scala:241)
[info] [error] 	at sbt.internal.Load$.loadURI$1(Load.scala:546)
[info] [error] 	at sbt.internal.Load$.loadAll(Load.scala:562)
[info] [error] 	at sbt.internal.Load$.loadURI(Load.scala:492)
[info] [error] 	at sbt.internal.Load$.load(Load.scala:471)
[info] [error] 	at sbt.internal.Load$.$anonfun$apply$1(Load.scala:251)
[info] [error] 	at sbt.internal.Load$.timed(Load.scala:1395)
[info] [error] 	at sbt.internal.Load$.apply(Load.scala:251)
[info] [error] 	at sbt.internal.Load$.defaultLoad(Load.scala:69)
[info] [error] 	at sbt.BuiltinCommands$.liftedTree1$1(Main.scala:829)
[info] [error] 	at sbt.BuiltinCommands$.doLoadProject(Main.scala:829)
[info] [error] 	at sbt.BuiltinCommands$.$anonfun$loadProjectImpl$2(Main.scala:800)
[info] [error] 	at sbt.Command$.$anonfun$applyEffect$4(Command.scala:142)
[info] [error] 	at sbt.Command$.$anonfun$applyEffect$2(Command.scala:137)
[info] [error] 	at sbt.Command$.process(Command.scala:181)
[info] [error] 	at sbt.MainLoop$.processCommand(MainLoop.scala:151)
[info] [error] 	at sbt.MainLoop$.$anonfun$next$2(MainLoop.scala:139)
[info] [error] 	at sbt.State$$anon$1.runCmd$1(State.scala:246)
[info] [error] 	at sbt.State$$anon$1.process(State.scala:250)
[info] [error] 	at sbt.MainLoop$.$anonfun$next$1(MainLoop.scala:139)
[info] [error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[info] [error] 	at sbt.MainLoop$.next(MainLoop.scala:139)
[info] [error] 	at sbt.MainLoop$.run(MainLoop.scala:132)
[info] [error] 	at sbt.MainLoop$.$anonfun$runWithNewLog$1(MainLoop.scala:110)
[info] [error] 	at sbt.io.Using.apply(Using.scala:22)
[info] [error] 	at sbt.MainLoop$.runWithNewLog(MainLoop.scala:104)
[info] [error] 	at sbt.MainLoop$.runAndClearLast(MainLoop.scala:59)
[info] [error] 	at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:44)
[info] [error] 	at sbt.MainLoop$.runLogged(MainLoop.scala:35)
[info] [error] 	at sbt.StandardMain$.runManaged(Main.scala:138)
[info] [error] 	at sbt.xMain.run(Main.scala:89)
[info] [error] 	at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:111)
[info] [error] 	at xsbt.boot.Launch$.withContextLoader(Launch.scala:130)
[info] [error] 	at xsbt.boot.Launch$.run(Launch.scala:111)
[info] [error] 	at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:37)
[info] [error] 	at xsbt.boot.Launch$.launch(Launch.scala:119)
[info] [error] 	at xsbt.boot.Launch$.apply(Launch.scala:20)
[info] [error] 	at xsbt.boot.Boot$.runImpl(Boot.scala:56)
[info] [error] 	at xsbt.boot.Boot$.main(Boot.scala:18)
[info] [error] 	at xsbt.boot.Boot.main(Boot.scala)
[info] [error] java.lang.NoSuchMethodError: sbt.package$.singleFileJsonFormatter()Lsjsonnew/JsonFormat;
[info] [error] Use 'last' for the full log.
[info] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? 

expectation

it should work

notes

xerial/sbt-sonatype#91

the settingKey[T] macro take OptJsonWriter[T] parameter.

sbt.singleFileJsonFormatter added since sbt 1.3.0-M3.
3319423#diff-cafb18f4282f4fff753f9efb35169d46R41

@xuwei-k
Copy link
Member Author

xuwei-k commented Sep 6, 2019

workaround

--- a/src/main/scala/example/ExamplePlugin.scala
+++ b/src/main/scala/example/ExamplePlugin.scala
@@ -4,6 +4,6 @@ import sbt._, Keys._
 
 object ExamplePlugin extends AutoPlugin {
   object autoImport {
-    val foo = settingKey[File]("")
+    val foo = SettingKey[File]("foo")(implicitly, sbt.util.NoJsonWriter())
   }
 }

@eed3si9n
Copy link
Member

eed3si9n commented Sep 6, 2019

@xuwei-k Thanks for the report!

Maybe it's ok for sbt to be backward compatible only?
This does mean we should be more careful adding stuff to package objects.

@dwijnand
Copy link
Member

dwijnand commented Sep 6, 2019

While the problem is unfortunate and super-subtle (nice detective work!) I think requiring sbt to be forward-compatible is too much to ask. The way we allow that not to be a problem is by making each sbt update reasonably easy (in comparison to scala major updates, which require all your dependencies to be rebuilt).

@dwijnand
Copy link
Member

dwijnand commented Sep 6, 2019

On the other side, you can safeguard your plugin from these problems by specifically testing for target sbt versions, so ^^1.0.0 scripted.

@eatkins
Copy link
Contributor

eatkins commented Sep 6, 2019

Another way to guard against this is to pin the sbt version in your build, e.g.

dependencyOverrides := "org.scala-sbt" % "sbt" % "1.2.8" :: Nil

@eatkins
Copy link
Contributor

eatkins commented Sep 6, 2019

I wonder if it might make sense to add a way of configuring the sbt compatibility level of SbtPlugin so you'd do something like enablePlugins(SbtPlugin.withCompatibility("1.0.4"))

@eed3si9n
Copy link
Member

eed3si9n commented Sep 6, 2019

dependencyOverrides := "org.scala-sbt" % "sbt" % "1.2.8" :: Nil

That's a nice workaround. Maybe in general we should rethink the idea of always using the sbt version used for the build as the plugin's dependency, and we should default to some old version from 1.1.x or 1.${current - 1}.x.

@dwijnand
Copy link
Member

dwijnand commented Sep 6, 2019

I'm stating the obvious, but it might be a useful comparison: this would be like building an extension to akka-actor 2.5.23 and then someone complaining it doesn't work with akka-actor 2.5.15. I think it's fine, but we could/should bring this to light, for existing plugin authors, and for future ones by adding to the plugin or scripted docs.

@eatkins
Copy link
Contributor

eatkins commented Sep 6, 2019

I agree with that. The important distinction though is that the api is opaque about what version of sbt you’re targeting. I can empathize with what happened with the sonatype plugin because I easily would have made the same kind of mistake. I think better documentation would go a long way.

@eed3si9n
Copy link
Member

eed3si9n commented Sep 6, 2019

I'm stating the obvious, but it might be a useful comparison: this would be like building an extension to akka-actor 2.5.23 and then someone complaining it doesn't work with akka-actor 2.5.15.

I guess I am considering that we decouple two sbt's:

  • the tool that plugin authors use to cobble plugins.
  • the minimum version of sbt required by the said plugin.

Plugin authors are often the most advanced sbt users, so it would be unfortunate to tell them to stay in the oldest sbt 1.x possible to maximize their plugin's compatibility.

@dwijnand
Copy link
Member

dwijnand commented Sep 7, 2019

You can already do that with sbtVersion in Global (IIRC, I would need to check).

The notion of "minimum version" I think applies to situations where users are for some reason stuck on older versions. I think that doesn't (or shouldn't) apply to sbt as it should strive to always be reasonably easy to upgrade.

@eed3si9n
Copy link
Member

eed3si9n commented Oct 4, 2019

I suggest we recommend the following setting:

pluginCrossBuild / sbtVersion := "1.2.8"
Confirmation
sbt:sbt-buildinfo> makeIvyXml
[info] :: delivering :: com.eed3si9n#sbt-buildinfo;0.9.0-81fc83a2bbbfd705d059ff54bc629fedf87bfae1-SNAPSHOT :: 0.9.0-81fc83a2bbbfd705d059ff54bc629fedf87bfae1-SNAPSHOT :: integration :: Fri Oct 04 09:37:32 EDT 2019
$ cat target/scala-2.12/sbt-1.0/ivy-0.9.0-81fc83a2bbbfd705d059ff54bc629fedf87bfae1-SNAPSHOT.xml | grep org.scala-sbt
    <dependency org="org.scala-sbt" name="sbt" rev="1.2.8" conf="provided->default(compile)"> </dependency>
    <dependency org="org.scala-sbt" name="scripted-sbt_2.12" rev="1.2.8" conf="scripted-sbt->default(compile)"> </dependency>
    <dependency org="org.scala-sbt" name="sbt-launch" rev="1.2.8" conf="scripted-sbt-launch->default(compile)"> </dependency>

I also tested it by loading it to a build using sbt 1.2.8.

Given there many plugins are falling into this pitfall, I think we should address this by default. For example we can set it to some older version by default (1.2.8, for now?) starting sbt 1.4.0, and we can let plugin authors to opt into depending on newly released sbt.

pdalpra added a commit to gatling/gatling-sbt-plugin that referenced this issue Apr 7, 2020
pdalpra added a commit to gatling/gatling-sbt-plugin that referenced this issue Apr 7, 2020
Force sbt version in plugin to 1.2.8, workaround for sbt/sbt#5049
poslegm added a commit to scalameta/sbt-scalafmt that referenced this issue Apr 10, 2020
poslegm added a commit to scalameta/sbt-scalafmt that referenced this issue Apr 10, 2020
nevillelyh added a commit to sbt/sbt-avro that referenced this issue Apr 15, 2020
@kiranbayram
Copy link

Is there any plan to fix this in v1.4 ?

@eed3si9n
Copy link
Member

eed3si9n commented Aug 6, 2020

@kiranbayram I have not taken any action on this.

Do people have opinion on how this should be addressed? Do we use "1.2.8" as the forever default plugin compatibility?

@SethTisue
Copy link
Member

SethTisue commented Aug 7, 2020

I think folks who feel they need to stick with 1.2.x, for whatever reason, ought to be okay also sticking with already-published plugin versions. I believe it is more important to keep sbt and its plugin ecosystem moving forward. If there are still critical issues with 1.3.x that prevent people from upgrading, I think it's better to put the effort into finding fixes and/or workarounds for those issues, rather than catering to users of outdated versions.

@eed3si9n
Copy link
Member

eed3si9n commented Aug 8, 2020

It's a classic Crossing the Chasm curve situation. Whenever a new 1.y feature release of sbt comes out, the adoption rate goes through a bell curve. The open source library authors and plugin maintainers likely would be in the natural earlier adopter.

Eventually in a few months or so, early majority and even late majority would adopt the new feature release, in this case 1.3.x. But the plugin authors may or may not want their audience to adopt 1.3.x immediately given that it might require some additional work, such as validating Coursier. Now 1.3.13 feels relatively stable, but when it came out it was pretty edgy.

Now, to up the ante, consider sbt 1.4.0. When I publish a new version of sbt-assembly or sbt-buildinfo, should the build users forced to immediately adopt sbt 1.4.0 (and potentially run into early issues)? Unlike Scala standard library, sbt doesn't maintain forward compatibility so I think plugins compiling against an older sbt is a valid choice. The part I am not totally sure is what that version should be, and how it's determined.

@eed3si9n
Copy link
Member

eed3si9n commented Oct 3, 2020

Here's a PR to hard code to 1.2.8 - #5918

@eed3si9n eed3si9n modified the milestones: 1.4.0, 1.something Oct 4, 2020
albuch referenced this issue in albuch/sbt-dependency-check Oct 30, 2020
@dotta
Copy link

dotta commented Dec 23, 2020

It'd be useful if plugin's author could have a hook that is called before any of the plugins' is actually loaded by sbt (the goal would be to validate plugins' compatibility before loading the project).

In the meanwhile, here is what I'm doing to help users solve their issue:

override lazy val projectSettings = {
  try internalProjectSettings
  catch {
    case _: NoSuchMethodError =>
      sys.error(s"This version of $PluginName requires sbt 1.3.0 or higher.")
  }
}

If there is a better way, please point it out :)

EDIT: I should have mentioned that I tried the original workaround suggested by @xuwei-k, but unfortunately it didn't quite work in my case (I got back an error about settings not being named properly, but I didn't investigate much further).

@AdityaSaroj
Copy link

In which file, do I add this line?

pluginCrossBuild / sbtVersion := "1.2.8"

@eed3si9n
Copy link
Member

In build.sbt of the plugin.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Bug workaround exists There's a workaround
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants