From 0855b6254ed2b0884626bf987044c9b1060aa139 Mon Sep 17 00:00:00 2001 From: Li Haoyi Date: Wed, 12 Apr 2023 05:41:08 +0800 Subject: [PATCH] Pre compile zinc compiler interface for common Scala versions (#2424) While we still the to keep the dynamic codepath necessary to compile stuff at runtime, at least for the common case of existing Scala versions we can pre-compile it, improving performance and removing some of the confusing log messages about compiling stuff that the user didn't write. This only affects the first load, but first impressions matter. Doesn't benefit Scala 3, but there's half the community is still on Scala 2, and Mill's own `build.sc` is still on Scala 2 Before, we can see that a clean `run` ends up compiling the compiler interface twice (once to compile `build.sc`, once to compile the user's `ScalaModule`) taking ~14s on my laptop ``` Compiling compiler interface... [info] compiling 1 Scala source to /Users/lihaoyi/Github/mill/example/basic/1-hello-world/out/mill-build/compile.dest/classes ... [info] done compiling [34/48] compile Compiling compiler interface... 8 warnings [info] compiling 1 Scala source to /Users/lihaoyi/Github/mill/example/basic/1-hello-world/out/compile.dest/classes ... [warn] 1 deprecation (since 2.13.0); re-run with -deprecation for details [warn] one warning found [info] done compiling [48/48] run

hello

/Users/lihaoyi/Github/mill/target/mill-release -i run --text hello 47.58s user 1.70s system 343% cpu 14.345 total ``` After, we compile the compiler interface zero times, with a clean `run` taking ~7s on my laptop ``` [build.sc] [40/47] compile [info] compiling 1 Scala source to /Users/lihaoyi/Github/mill/example/basic/1-hello-world/out/mill-build/compile.dest/classes ... [info] done compiling [34/48] compile [info] compiling 1 Scala source to /Users/lihaoyi/Github/mill/example/basic/1-hello-world/out/compile.dest/classes ... [warn] 1 deprecation (since 2.13.0); re-run with -deprecation for details [warn] one warning found [info] done compiling [48/48] run

hello

/Users/lihaoyi/Github/mill/target/mill-release -i run --text hello 19.27s user 0.85s system 297% cpu 6.768 total ``` ## Implementation I added cross modules `bridge[_]` for all supported Scala versions. These more or less follow what `ZincWorkerModule#scalaCompilerBridgeJar` already does on-the-fly: download the respective source jar from Maven Central and compile it using the respective Scala version. Note that the `META-INF` metadata is necessary for Zinc to properly pick these up, and so I manually copy the folder from the unzipped source jar into the `resources` so it can get propagated to the final jar. To avoid the slowness of compiling every bridge version during local development, we separate the `bridge[_]` publishing from the rest of the build, only enabled via the `MILL_BUILD_COMPILER_BRIDGES=true` environment variable, and published under a separate version. We expect to bump the `bridge[_]` version rarely, and want to avoid redundantly re-publishing them every Mill release. ## Testing Most of the existing unit tests use the three scala versions that we pre-compile in development mode, though some of them are left on other versions for legacy reasons e.g. they depend on a specific version of Scala to be compatible with a specific version of semanticDB or scala-native, and those continue to use the old download-compile-on-the-fly code path. I also added a new test case `mill.scalalib.HelloWorldTests.compile.nonPreCompiledBridge` that together with `.fromScratch` specifically exercises the code paths for compiled/not-compiled compiler bridges, asserting that the compiler bridge is compiled in the versions it should be compiled for (i.e. those that are not pre-compiled) and not compiled for versions it should not be compiled for (i.e. those that *are* pre-compiled) --- .github/workflows/publish-bridges.yml | 35 +++++ build.sc | 124 ++++++++++++++---- ci/mill-bootstrap.patch | 56 +++++--- ci/release-bridge-maven.sh | 25 ++++ example/basic/1-hello-world/build.sc | 2 +- example/basic/11-scala-combined/build.sc | 18 +-- example/basic/2-common-overrides/build.sc | 2 +- example/basic/3-custom-tasks/build.sc | 2 +- example/basic/4-multi-module/build.sc | 2 +- example/basic/5-nested-modules/build.sc | 2 +- example/basic/6-test-suite/build.sc | 2 +- example/basic/7-publish-module/build.sc | 2 +- example/basic/8-cross-scala-version/build.sc | 10 +- example/basic/9-sbt-compat-modules/build.sc | 8 +- .../misc/3-import-file-ivy/scalaversion.sc | 2 +- .../mill-build/src/ScalaVersion.scala | 2 +- example/misc/5-dynamic-cross-module/build.sc | 2 +- example/web/1-hello-webapp/build.sc | 2 +- example/web/2-todo-webapp/build.sc | 2 +- example/web/3-scalajs-module/build.sc | 2 +- example/web/4-webapp-scalajs/build.sc | 4 +- example/web/5-webapp-scalajs-shared/build.sc | 2 +- .../web/6-cross-platform-publishing/build.sc | 12 +- integration/failure/compile-error/repo/bar.sc | 2 +- integration/failure/parse-error/repo/bar.sc | 2 +- .../dynamic-cross-module/repo/build.sc | 2 +- integration/feature/editing/repo/build.sc | 2 +- .../feature/large-project/repo/build.sc | 2 +- main/api/src/mill/api/Result.scala | 13 ++ main/core/src/mill/eval/Evaluator.scala | 14 +- main/test/src/mill/eval/ModuleTests.scala | 6 +- .../src/mill/testkit/MillTestkit.scala | 4 +- main/util/src/mill/util/ColorLogger.scala | 2 - main/util/src/mill/util/DummyLogger.scala | 1 - main/util/src/mill/util/FileLogger.scala | 11 +- main/util/src/mill/util/MultiLogger.scala | 56 ++++---- main/util/src/mill/util/PrefixLogger.scala | 5 +- main/util/src/mill/util/PrintLogger.scala | 25 ++-- main/util/src/mill/util/ProxyLogger.scala | 1 - .../src/mill/runner/MillBuildRootModule.scala | 3 +- runner/src/mill/runner/MillMain.scala | 53 ++++---- .../mill/scalalib/api/ZincWorkerUtil.scala | 7 +- scalalib/src/mill/scalalib/ScalaModule.scala | 2 +- .../mill/scalalib/SemanticDbJavaModule.scala | 2 +- .../src/mill/scalalib/ZincWorkerModule.scala | 16 ++- .../src/mill/scalalib/HelloWorldTests.scala | 41 ++++++ .../mill/scalalib/worker/ZincWorkerImpl.scala | 2 +- .../mill/scalanativelib/ExclusionsTests.scala | 2 +- 48 files changed, 401 insertions(+), 195 deletions(-) create mode 100644 .github/workflows/publish-bridges.yml create mode 100755 ci/release-bridge-maven.sh diff --git a/.github/workflows/publish-bridges.yml b/.github/workflows/publish-bridges.yml new file mode 100644 index 00000000000..25c6f491ac6 --- /dev/null +++ b/.github/workflows/publish-bridges.yml @@ -0,0 +1,35 @@ +name: Publish Bridges + +# Manually-triggered github action to publish mill-scala-compiler-bridges jars, +# since those do not change frequently enough to be worth including in the main +# publishing workflow that runs every Mill version +on: + pull_request: + workflow_dispatch: +jobs: + publish-bridges: + runs-on: ubuntu-latest + + concurrency: publish-sonatype-${{ github.sha }} + + env: + SONATYPE_PGP_SECRET: ${{ secrets.SONATYPE_PGP_SECRET }} + SONATYPE_USERNAME: ${{ secrets.SONATYPE_DEPLOY_USER }} + SONATYPE_PASSWORD: ${{ secrets.SONATYPE_DEPLOY_PASSWORD }} + SONATYPE_PGP_PASSWORD: ${{ secrets.SONATYPE_PGP_PASSWORD }} + LANG: "en_US.UTF-8" + LC_MESSAGES: "en_US.UTF-8" + LC_ALL: "en_US.UTF-8" + MILL_BUILD_COMPILER_BRIDGES: "true" + + steps: + - uses: actions/checkout@v3 + with: + fetch-depth: 0 + + - uses: actions/setup-java@v3 + with: + java-version: 8 + distribution: temurin + + - run: ci/release-bridge-maven.sh diff --git a/build.sc b/build.sc index ac2b2785032..9475182536b 100644 --- a/build.sc +++ b/build.sc @@ -201,6 +201,62 @@ def millBinPlatform: T[String] = T { } def baseDir = build.millSourcePath +// We limit the number of compiler bridges to compile and publish for local +// development and testing, because otherwise it takes forever to compile all +// of them. Compiler bridges not in this set will get downloaded and compiled +// on the fly anyway. For publishing, we publish everything. +val buildAllCompilerBridges = interp.watchValue(sys.env.contains("MILL_BUILD_COMPILER_BRIDGES")) +val bridgeVersion = "0.0.1" +val bridgeScalaVersions = Seq( + // Our version of Zinc doesn't work with Scala 2.12.0 and 2.12.4 compiler + // bridges. We skip 2.12.1 because it's so old not to matter, and we need a + // non-supported scala versionm for testing purposes. We skip 2.13.0-2 because + // scaladoc fails on windows + /*"2.12.0",*/ /*2.12.1",*/ "2.12.2", "2.12.3", /*"2.12.4",*/ "2.12.5", "2.12.6", "2.12.7", "2.12.8", + "2.12.9", "2.12.10", "2.12.11", "2.12.12", "2.12.13", "2.12.14", "2.12.15", "2.12.16", "2.12.17", + /*"2.13.0", "2.13.1", "2.13.2",*/ "2.13.3", "2.13.4", "2.13.5", "2.13.6", "2.13.7", "2.13.8", "2.13.9", "2.13.10" +) +val buildBridgeScalaVersions = + if (!buildAllCompilerBridges) Seq() + else bridgeScalaVersions + +object bridge extends Cross[BridgeModule](buildBridgeScalaVersions: _*) +class BridgeModule(val crossScalaVersion: String) extends PublishModule with CrossScalaModule { + def scalaVersion = crossScalaVersion + def publishVersion = bridgeVersion + def artifactName = T{ "mill-scala-compiler-bridge" } + def pomSettings = commonPomSettings(artifactName()) + def crossFullScalaVersion = true + def ivyDeps = Agg( + ivy"org.scala-sbt:compiler-interface:${Versions.zinc}", + ivy"org.scala-lang:scala-compiler:${crossScalaVersion}", + ) + + def resources = T.sources { + os.copy(generatedSources().head.path / "META-INF", T.dest / "META-INF") + Seq(PathRef(T.dest)) + } + + def generatedSources = T { + import mill.scalalib.api.ZincWorkerUtil.{grepJar, scalaBinaryVersion} + val resolvedJars = resolveDeps( + T.task { Agg(ivy"org.scala-sbt::compiler-bridge:${Deps.zinc.dep.version}") }, + sources = true + )() + + val bridgeJar = grepJar( + resolvedJars.map(_.path), + s"compiler-bridge_${scalaBinaryVersion(scalaVersion())}", + Deps.zinc.dep.version, + true + ) + + mill.api.IO.unpackZip(bridgeJar, os.rel) + + Seq(PathRef(T.dest)) + } +} + trait BuildInfo extends JavaModule { /** @@ -397,15 +453,9 @@ object BuildInfo{ } } - -trait MillPublishModule extends PublishModule { - override def artifactName = "mill-" + super.artifactName() - def publishVersion = millVersion() - override def publishProperties: Target[Map[String, String]] = super.publishProperties() ++ Map( - "info.releaseNotesURL" -> Settings.changelogUrl - ) - def pomSettings = PomSettings( - description = artifactName(), +def commonPomSettings(artifactName: String) = { + PomSettings( + description = artifactName, organization = Settings.pomOrg, url = Settings.projectUrl, licenses = Seq(License.MIT), @@ -415,6 +465,15 @@ trait MillPublishModule extends PublishModule { Developer("lefou", "Tobias Roeser", "https://github.com/lefou") ) ) +} + +trait MillPublishModule extends PublishModule { + override def artifactName = "mill-" + super.artifactName() + def publishVersion = millVersion() + override def publishProperties: Target[Map[String, String]] = super.publishProperties() ++ Map( + "info.releaseNotesURL" -> Settings.changelogUrl + ) + def pomSettings = commonPomSettings(artifactName()) override def javacOptions = Seq("-source", "1.8", "-target", "1.8", "-encoding", "UTF-8") } @@ -652,25 +711,17 @@ object main extends MillModule { object testrunner extends MillModule { override def moduleDeps = Seq(scalalib.api, main.util) } -object scalalib extends MillModule with BuildInfo{ + + +object scalalib extends MillModule { override def moduleDeps = Seq(main, scalalib.api, testrunner) override def ivyDeps = Agg( Deps.scalafmtDynamic ) - def buildInfoPackageName = "mill.scalalib" - def buildInfoObjectName = "Versions" - - def buildInfoMembers = Seq( - BuildInfo.Value("ammonite", Deps.ammoniteVersion, "Version of Ammonite."), - BuildInfo.Value("zinc", Deps.zinc.dep.version, "Version of Zinc"), - BuildInfo.Value("semanticDBVersion", Deps.semanticDB.dep.version, "SemanticDB version."), - BuildInfo.Value("semanticDbJavaVersion", Deps.semanticDbJava.dep.version, "Java SemanticDB plugin version."), - BuildInfo.Value("millModuledefsVersion", Deps.millModuledefsVersion, "Mill ModuleDefs plugins version.") - ) - override def testIvyDeps = super.testIvyDeps() ++ Agg(Deps.scalaCheck) + def testArgs = T { val artifactsString = @@ -684,7 +735,7 @@ object scalalib extends MillModule with BuildInfo{ "-Djna.nosys=true", "-DMILL_SCALA_LIB=" + runClasspath().map(_.path).mkString(","), s"-DTEST_SCALAFMT_VERSION=${Deps.scalafmtDynamic.dep.version}", - s"-DMILL_EMBEDDED_DEPS=\"$artifactsString\"" + s"-DMILL_EMBEDDED_DEPS=$artifactsString" ) } object backgroundwrapper extends MillPublishModule { @@ -697,9 +748,25 @@ object scalalib extends MillModule with BuildInfo{ ) } } - object api extends MillApiModule { + object api extends MillApiModule with BuildInfo { override def moduleDeps = Seq(main.api) + + def buildInfoPackageName = "mill.scalalib.api" + + def buildInfoObjectName = "Versions" + + def buildInfoMembers = Seq( + BuildInfo.Value("ammonite", Deps.ammoniteVersion, "Version of Ammonite."), + BuildInfo.Value("zinc", Deps.zinc.dep.version, "Version of Zinc"), + BuildInfo.Value("semanticDBVersion", Deps.semanticDB.dep.version, "SemanticDB version."), + BuildInfo.Value("semanticDbJavaVersion", Deps.semanticDbJava.dep.version, "Java SemanticDB plugin version."), + BuildInfo.Value("millModuledefsVersion", Deps.millModuledefsVersion, "Mill ModuleDefs plugins version."), + BuildInfo.Value("millCompilerBridgeScalaVersions", bridgeScalaVersions.mkString(",")), + BuildInfo.Value("millCompilerBridgeVersion", bridgeVersion), + BuildInfo.Value("millVersion", millVersion(), "Mill version.") + ) } + object worker extends MillInternalModule with BuildInfo{ override def moduleDeps = Seq(scalalib.api) @@ -708,11 +775,7 @@ object scalalib extends MillModule with BuildInfo{ Deps.zinc, Deps.log4j2Core ) - def testArgs = T { - Seq( - "-DMILL_SCALA_WORKER=" + runClasspath().map(_.path).mkString(",") - ) - } + def testArgs = Seq("-DMILL_SCALA_WORKER=" + runClasspath().map(_.path).mkString(",")) def buildInfoPackageName = "mill.scalalib.worker" def buildInfoObjectName = "Versions" @@ -1092,7 +1155,10 @@ def installLocalCache() = T.command { } def installLocalTask(binFile: Task[String], ivyRepo: String = null): Task[os.Path] = { - val modules = build.millInternal.modules.collect { case m: PublishModule => m } + val modules = build.millInternal.modules.collect{ + case m: PublishModule => m + } + T.task { T.traverse(modules)(m => m.publishLocal(ivyRepo))() val millBin = assembly() diff --git a/ci/mill-bootstrap.patch b/ci/mill-bootstrap.patch index 48c89e36706..c8afac978b3 100644 --- a/ci/mill-bootstrap.patch +++ b/ci/mill-bootstrap.patch @@ -1,8 +1,8 @@ diff --git a/build.sc b/build.sc -index cbc7b7aaa..351a37fbc 100644 +index 8fcdeb538..c8155558d 100644 --- a/build.sc +++ b/build.sc -@@ -2,22 +2,12 @@ +@@ -2,27 +2,18 @@ import $file.ci.shared import $file.ci.upload import $ivy.`org.scalaj::scalaj-http:2.4.2` @@ -27,7 +27,13 @@ index cbc7b7aaa..351a37fbc 100644 import mill._ import mill.define.{Command, Source, Sources, Target, Task} import mill.eval.Evaluator -@@ -185,12 +175,8 @@ object Deps { + import mill.main.MainModule + import mill.scalalib._ ++import mill.scalalib.api.Versions + import mill.scalalib.publish._ + import mill.modules.Jvm + import mill.define.SelectMode +@@ -185,12 +176,8 @@ object Deps { val requests = ivy"com.lihaoyi::requests:0.8.0" } @@ -42,10 +48,31 @@ index cbc7b7aaa..351a37fbc 100644 def millBinPlatform: T[String] = T { val tag = millLastTag() if (tag.contains("-M")) tag -@@ -202,202 +188,6 @@ def millBinPlatform: T[String] = T { - def baseDir = build.millSourcePath +@@ -239,219 +226,23 @@ class BridgeModule(val crossScalaVersion: String) extends PublishModule with Cro + def generatedSources = T { + import mill.scalalib.api.ZincWorkerUtil.{grepJar, scalaBinaryVersion} + val resolvedJars = resolveDeps( +- T.task { Agg(ivy"org.scala-sbt::compiler-bridge:${Deps.zinc.dep.version}") }, ++ T.task { Agg(ivy"org.scala-sbt::compiler-bridge:${Deps.zinc.dep.version}").map(bindDependency()) }, + sources = true + )() + val bridgeJar = grepJar( +- resolvedJars.map(_.path), ++ resolvedJars, + s"compiler-bridge_${scalaBinaryVersion(scalaVersion())}", + Deps.zinc.dep.version, + true + ) +- mill.api.IO.unpackZip(bridgeJar, os.rel) ++ mill.api.IO.unpackZip(bridgeJar.path, os.rel) + + Seq(PathRef(T.dest)) + } + } + +- -trait BuildInfo extends JavaModule { - /** - * The package name under which the BuildInfo data object will be stored. @@ -241,11 +268,10 @@ index cbc7b7aaa..351a37fbc 100644 - } -} - -- - trait MillPublishModule extends PublishModule { - override def artifactName = "mill-" + super.artifactName() - def publishVersion = millVersion() -@@ -447,27 +237,8 @@ trait MillCoursierModule extends CoursierModule { + def commonPomSettings(artifactName: String) = { + PomSettings( + description = artifactName, +@@ -505,27 +296,8 @@ trait MillCoursierModule extends CoursierModule { ) } @@ -274,7 +300,7 @@ index cbc7b7aaa..351a37fbc 100644 } /** A Module compiled with applied Mill-specific compiler plugins: mill-moduledefs. */ -@@ -758,6 +529,7 @@ object scalajslib extends MillModule with BuildInfo{ +@@ -821,6 +593,7 @@ object scalajslib extends MillModule with BuildInfo{ } object worker extends Cross[WorkerModule]("1") class WorkerModule(scalajsWorkerVersion: String) extends MillInternalModule { @@ -282,7 +308,7 @@ index cbc7b7aaa..351a37fbc 100644 override def moduleDeps = Seq(scalajslib.`worker-api`) override def ivyDeps = Agg( Deps.Scalajs_1.scalajsLinker, -@@ -820,6 +592,7 @@ object contrib extends MillModule { +@@ -883,6 +656,7 @@ object contrib extends MillModule { object worker extends Cross[WorkerModule](Deps.play.keys.toSeq: _*) class WorkerModule(playBinary: String) extends MillInternalModule { @@ -290,7 +316,7 @@ index cbc7b7aaa..351a37fbc 100644 override def sources = T.sources { // We want to avoid duplicating code as long as the Play APIs allow. // But if newer Play versions introduce incompatibilities, -@@ -1011,6 +784,7 @@ object scalanativelib extends MillModule { +@@ -1074,6 +848,7 @@ object scalanativelib extends MillModule { object worker extends Cross[WorkerModule]("0.4") class WorkerModule(scalaNativeWorkerVersion: String) extends MillInternalModule { @@ -298,7 +324,7 @@ index cbc7b7aaa..351a37fbc 100644 override def moduleDeps = Seq(scalanativelib.`worker-api`) override def ivyDeps = scalaNativeWorkerVersion match { case "0.4" => -@@ -1171,6 +945,7 @@ trait IntegrationTestModule extends MillScalaModule { +@@ -1237,6 +1012,7 @@ trait IntegrationTestModule extends MillScalaModule { } trait IntegrationTestCrossModule extends IntegrationTestModule { @@ -306,7 +332,7 @@ index cbc7b7aaa..351a37fbc 100644 object local extends ModeModule object fork extends ModeModule object server extends ModeModule -@@ -1725,53 +1500,7 @@ def launcher = T { +@@ -1791,53 +1567,7 @@ def launcher = T { def uploadToGithub(authKey: String) = T.command { diff --git a/ci/release-bridge-maven.sh b/ci/release-bridge-maven.sh new file mode 100755 index 00000000000..7d3a60f6523 --- /dev/null +++ b/ci/release-bridge-maven.sh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash + +set -eu + +echo $SONATYPE_PGP_SECRET | base64 --decode > gpg_key + +gpg --import --no-tty --batch --yes gpg_key + +rm gpg_key + +# Build all artifacts +./mill -i __.publishArtifacts + +export MILL_BUILD_COMPILER_BRIDGES=true + +# Publish all artifacts +./mill -i \ + mill.scalalib.PublishModule/publishAll \ + --sonatypeCreds $SONATYPE_USERNAME:$SONATYPE_PASSWORD \ + --gpgArgs --passphrase=$SONATYPE_PGP_PASSWORD,--no-tty,--pinentry-mode,loopback,--batch,--yes,-a,-b \ + --publishArtifacts bridges.__.publishArtifacts \ + --readTimeout 3600000 \ + --awaitTimeout 3600000 \ + --release true \ + --signed true diff --git a/example/basic/1-hello-world/build.sc b/example/basic/1-hello-world/build.sc index 4b53a21cff2..62f263b503c 100644 --- a/example/basic/1-hello-world/build.sc +++ b/example/basic/1-hello-world/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._ object foo extends RootModule with ScalaModule { - def scalaVersion = "2.13.2" + def scalaVersion = "2.13.8" def ivyDeps = Agg( ivy"com.lihaoyi::scalatags:0.8.2", ivy"com.lihaoyi::mainargs:0.4.0" diff --git a/example/basic/11-scala-combined/build.sc b/example/basic/11-scala-combined/build.sc index 2887fbf98f0..b31083d0629 100644 --- a/example/basic/11-scala-combined/build.sc +++ b/example/basic/11-scala-combined/build.sc @@ -21,7 +21,7 @@ trait MyScalaModule extends ScalaModule with MyModule { } } -val scalaVersions = Seq("2.13.10", "3.2.2") +val scalaVersions = Seq("2.13.8", "3.2.2") object foo extends Cross[FooModule](scalaVersions:_*) class FooModule(val crossScalaVersion: String) extends MyScalaModule with CrossScalaModule{ @@ -74,17 +74,17 @@ object qux extends JavaModule with MyModule /* Example Usage > ./mill resolve __.run -bar[2.13.10].run -bar[2.13.10].test.run +bar[2.13.8].run +bar[2.13.8].test.run bar[3.2.2].run bar[3.2.2].test.run -foo[2.13.10].run -foo[2.13.10].test.run +foo[2.13.8].run +foo[2.13.8].test.run foo[3.2.2].run foo[3.2.2].test.run qux.run -> ./mill foo[2.13.10].run +> ./mill foo[2.13.8].run foo version 0.0.1 Foo.value:

hello

Bar.value:

world Specific code for Scala 2.x

@@ -116,10 +116,10 @@ Publishing Artifact(com.lihaoyi,foo_3,0.0.1) to ivy repo Publishing Artifact(com.lihaoyi,bar_3,0.0.1) to ivy repo Publishing Artifact(com.lihaoyi,qux,0.0.1) to ivy repo -> ./mill show foo[2.13.10].assembly # mac/linux -out/foo/2.13.10/assembly.dest/out.jar +> ./mill show foo[2.13.8].assembly # mac/linux +out/foo/2.13.8/assembly.dest/out.jar -> ./out/foo/2.13.10/assembly.dest/out.jar # mac/linux +> ./out/foo/2.13.8/assembly.dest/out.jar # mac/linux foo version 0.0.1 Foo.value:

hello

Bar.value:

world Specific code for Scala 2.x

diff --git a/example/basic/2-common-overrides/build.sc b/example/basic/2-common-overrides/build.sc index f30f3f92d22..a3891f17ccf 100644 --- a/example/basic/2-common-overrides/build.sc +++ b/example/basic/2-common-overrides/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._ object foo extends RootModule with ScalaModule { - def scalaVersion = "2.13.10" + def scalaVersion = "2.13.8" // You can have arbitrary numbers of third-party dependencies def ivyDeps = Agg( diff --git a/example/basic/3-custom-tasks/build.sc b/example/basic/3-custom-tasks/build.sc index c86c8f9f7bd..ae1a0dca3dd 100644 --- a/example/basic/3-custom-tasks/build.sc +++ b/example/basic/3-custom-tasks/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._ object foo extends RootModule with ScalaModule { - def scalaVersion = "2.13.10" + def scalaVersion = "2.13.8" def ivyDeps = Agg( ivy"com.lihaoyi::scalatags:0.8.2", diff --git a/example/basic/4-multi-module/build.sc b/example/basic/4-multi-module/build.sc index 2172478b0a8..6ffc0d0c6fb 100644 --- a/example/basic/4-multi-module/build.sc +++ b/example/basic/4-multi-module/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._ trait MyModule extends ScalaModule{ - def scalaVersion = "2.13.2" + def scalaVersion = "2.13.8" def ivyDeps = Agg( ivy"com.lihaoyi::scalatags:0.8.2", ivy"com.lihaoyi::mainargs:0.4.0" diff --git a/example/basic/5-nested-modules/build.sc b/example/basic/5-nested-modules/build.sc index e2b3809c664..600d9ea7785 100644 --- a/example/basic/5-nested-modules/build.sc +++ b/example/basic/5-nested-modules/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._ trait MyModule extends ScalaModule{ - def scalaVersion = "2.13.2" + def scalaVersion = "2.13.8" def ivyDeps = Agg( ivy"com.lihaoyi::scalatags:0.8.2", ivy"com.lihaoyi::mainargs:0.4.0" diff --git a/example/basic/6-test-suite/build.sc b/example/basic/6-test-suite/build.sc index d76c8ab0d27..d94da8220c1 100644 --- a/example/basic/6-test-suite/build.sc +++ b/example/basic/6-test-suite/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._ object foo extends ScalaModule { - def scalaVersion = "2.13.2" + def scalaVersion = "2.13.8" object test extends Tests { def ivyDeps = Agg(ivy"com.lihaoyi::utest:0.7.11") def testFramework = "utest.runner.Framework" diff --git a/example/basic/7-publish-module/build.sc b/example/basic/7-publish-module/build.sc index 90fc33d80f9..f46a3c4bc65 100644 --- a/example/basic/7-publish-module/build.sc +++ b/example/basic/7-publish-module/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._, publish._ object foo extends ScalaModule with PublishModule { - def scalaVersion = "2.13.10" + def scalaVersion = "2.13.8" def publishVersion = "0.0.1" def pomSettings = PomSettings( diff --git a/example/basic/8-cross-scala-version/build.sc b/example/basic/8-cross-scala-version/build.sc index f1a2a592291..198e88bde62 100644 --- a/example/basic/8-cross-scala-version/build.sc +++ b/example/basic/8-cross-scala-version/build.sc @@ -1,6 +1,6 @@ import mill._, scalalib._ -val scalaVersions = Seq("2.12.17", "2.13.10", "3.2.2") +val scalaVersions = Seq("2.12.17", "2.13.8", "3.2.2") object foo extends Cross[FooModule](scalaVersions:_*) class FooModule(val crossScalaVersion: String) extends CrossScalaModule{ @@ -28,9 +28,9 @@ class BarModule(val crossScalaVersion: String) extends CrossScalaModule > ./mill resolve __.run foo[2.12.17].run -foo[2.13.10].run +foo[2.13.8].run bar[2.12.17].run -bar[2.13.10].run +bar[2.13.8].run > ./mill foo[2.12.17].run Foo.value: Hello World Scala library version 2.12.17 @@ -38,8 +38,8 @@ Bar.value: bar-value Specific code for Scala 2.x Specific code for Scala 2.12.x -> ./mill foo[2.13.10].run -Foo.value: Hello World Scala library version 2.13.10 +> ./mill foo[2.13.8].run +Foo.value: Hello World Scala library version 2.13.8 Bar.value: bar-value Specific code for Scala 2.x Specific code for Scala 2.13.x diff --git a/example/basic/9-sbt-compat-modules/build.sc b/example/basic/9-sbt-compat-modules/build.sc index e6eaf764575..5a6e7de5c87 100644 --- a/example/basic/9-sbt-compat-modules/build.sc +++ b/example/basic/9-sbt-compat-modules/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._ object foo extends SbtModule { - def scalaVersion = "2.13.2" + def scalaVersion = "2.13.8" object test extends Tests { def ivyDeps = Agg(ivy"com.lihaoyi::utest:0.7.11") def testFramework = "utest.runner.Framework" @@ -9,7 +9,7 @@ object foo extends SbtModule { } -object bar extends Cross[BarModule]("2.12.17", "2.13.10") +object bar extends Cross[BarModule]("2.12.17", "2.13.8") class BarModule(val crossScalaVersion: String) extends CrossSbtModule{ object test extends Tests { def ivyDeps = Agg(ivy"com.lihaoyi::utest:0.7.11") @@ -51,8 +51,8 @@ compiling 1 Scala source > ./mill foo.test + foo.FooTests.hello -> ./mill bar[2.13.10].run -Hello World Scala library version 2.13.10 +> ./mill bar[2.13.8].run +Hello World Scala library version 2.13.8 > ./mill bar[2.12.17].run Hello World Scala library version 2.12.17 diff --git a/example/misc/3-import-file-ivy/scalaversion.sc b/example/misc/3-import-file-ivy/scalaversion.sc index 02915791f95..da68bc70b20 100644 --- a/example/misc/3-import-file-ivy/scalaversion.sc +++ b/example/misc/3-import-file-ivy/scalaversion.sc @@ -1 +1 @@ -def myScalaVersion = "2.13.2" \ No newline at end of file +def myScalaVersion = "2.13.8" \ No newline at end of file diff --git a/example/misc/4-mill-build-folder/mill-build/src/ScalaVersion.scala b/example/misc/4-mill-build-folder/mill-build/src/ScalaVersion.scala index 41e7b2b4559..a735fddfea8 100644 --- a/example/misc/4-mill-build-folder/mill-build/src/ScalaVersion.scala +++ b/example/misc/4-mill-build-folder/mill-build/src/ScalaVersion.scala @@ -1,4 +1,4 @@ package millbuild object ScalaVersion{ - def myScalaVersion = "2.13.2" + def myScalaVersion = "2.13.10" } diff --git a/example/misc/5-dynamic-cross-module/build.sc b/example/misc/5-dynamic-cross-module/build.sc index 3da69d78aee..17cd578dc26 100644 --- a/example/misc/5-dynamic-cross-module/build.sc +++ b/example/misc/5-dynamic-cross-module/build.sc @@ -5,7 +5,7 @@ val moduleNames = interp.watchValue(os.list(millSourcePath / "modules").map(_.la object modules extends Cross[FolderModule](moduleNames:_*) class FolderModule(name: String) extends ScalaModule{ def millSourcePath = super.millSourcePath / name - def scalaVersion = "2.13.2" + def scalaVersion = "2.13.8" } // It is sometimes necessary for the instances of a cross-module to vary based diff --git a/example/web/1-hello-webapp/build.sc b/example/web/1-hello-webapp/build.sc index 9e1ea41f47d..5df70aa1edb 100644 --- a/example/web/1-hello-webapp/build.sc +++ b/example/web/1-hello-webapp/build.sc @@ -2,7 +2,7 @@ import mill._, scalalib._ object app extends RootModule with ScalaModule{ - def scalaVersion = "2.13.10" + def scalaVersion = "2.13.8" def ivyDeps = Agg( ivy"com.lihaoyi::cask:0.9.1", ivy"com.lihaoyi::scalatags:0.12.0" diff --git a/example/web/2-todo-webapp/build.sc b/example/web/2-todo-webapp/build.sc index 00d2f45519d..19bb1e504be 100644 --- a/example/web/2-todo-webapp/build.sc +++ b/example/web/2-todo-webapp/build.sc @@ -2,7 +2,7 @@ import mill._, scalalib._ object app extends RootModule with ScalaModule{ - def scalaVersion = "2.13.10" + def scalaVersion = "2.13.8" def ivyDeps = Agg( ivy"com.lihaoyi::cask:0.9.1", ivy"com.lihaoyi::scalatags:0.12.0" diff --git a/example/web/3-scalajs-module/build.sc b/example/web/3-scalajs-module/build.sc index e1a942f02cf..a3429d83dfd 100644 --- a/example/web/3-scalajs-module/build.sc +++ b/example/web/3-scalajs-module/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._, scalajslib._ object foo extends ScalaJSModule { - def scalaVersion = "2.13.2" + def scalaVersion = "2.13.8" def scalaJSVersion = "1.13.0" def ivyDeps = Agg(ivy"com.lihaoyi::scalatags::0.12.0") object test extends Tests { diff --git a/example/web/4-webapp-scalajs/build.sc b/example/web/4-webapp-scalajs/build.sc index 6b0a64e6550..20864260f00 100644 --- a/example/web/4-webapp-scalajs/build.sc +++ b/example/web/4-webapp-scalajs/build.sc @@ -2,7 +2,7 @@ import mill._, scalalib._, scalajslib._ object app extends RootModule with ScalaModule{ - def scalaVersion = "2.13.10" + def scalaVersion = "2.13.8" def ivyDeps = Agg( ivy"com.lihaoyi::cask:0.9.1", ivy"com.lihaoyi::scalatags:0.12.0" @@ -28,7 +28,7 @@ object app extends RootModule with ScalaModule{ } object client extends ScalaJSModule { - def scalaVersion = "2.13.10" + def scalaVersion = "2.13.8" def scalaJSVersion = "1.13.0" def ivyDeps = Agg(ivy"org.scala-js::scalajs-dom::2.2.0") } diff --git a/example/web/5-webapp-scalajs-shared/build.sc b/example/web/5-webapp-scalajs-shared/build.sc index 566a57f6ce2..980f71d9d67 100644 --- a/example/web/5-webapp-scalajs-shared/build.sc +++ b/example/web/5-webapp-scalajs-shared/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._, scalajslib._ trait AppScalaModule extends ScalaModule{ - def scalaVersion = "2.13.10" + def scalaVersion = "2.13.8" } trait AppScalaJSModule extends AppScalaModule with ScalaJSModule { diff --git a/example/web/6-cross-platform-publishing/build.sc b/example/web/6-cross-platform-publishing/build.sc index 81c6703bc6f..5412962e819 100644 --- a/example/web/6-cross-platform-publishing/build.sc +++ b/example/web/6-cross-platform-publishing/build.sc @@ -1,6 +1,6 @@ import mill._, scalalib._, scalajslib._, publish._ -object foo extends Cross[FooModule]("2.13.10", "3.2.2") +object foo extends Cross[FooModule]("2.13.8", "3.2.2") class FooModule(val crossScalaVersion: String) extends CrossScalaModule.Base { trait Shared extends CrossScalaModule with PlatformScalaModule with PublishModule { def publishVersion = "0.0.1" @@ -45,16 +45,16 @@ class FooModule(val crossScalaVersion: String) extends CrossScalaModule.Base { } // This example demonstrates how to publish Scala modules which are both -// cross-version and cross-platform: running on both Scala 2.13.10/3.2.2 as +// cross-version and cross-platform: running on both Scala 2.13.8/3.2.2 as // well as Scala-JVM/JS. /* Example Usage -> ./mill show foo[2.13.10].bar.jvm.sources +> ./mill show foo[2.13.8].bar.jvm.sources foo/bar/src foo/bar/src-jvm -foo/bar/src-2.13.10 -foo/bar/src-2.13.10-jvm +foo/bar/src-2.13.8 +foo/bar/src-2.13.8-jvm foo/bar/src-2.13 foo/bar/src-2.13-jvm foo/bar/src-2 @@ -70,7 +70,7 @@ foo/qux/src-3.2-js foo/qux/src-3 foo/qux/src-3-js -> ./mill foo[2.13.10].qux.jvm.run +> ./mill foo[2.13.8].qux.jvm.run Bar.value:

world Specific code for Scala 2.x

Parsing JSON with ujson.read Qux.main: Set(

i

,

cow

,

me

) diff --git a/integration/failure/compile-error/repo/bar.sc b/integration/failure/compile-error/repo/bar.sc index 538ca121252..8398ac65b8f 100644 --- a/integration/failure/compile-error/repo/bar.sc +++ b/integration/failure/compile-error/repo/bar.sc @@ -1,4 +1,4 @@ -def myScalaVersion = "2.13.2" +def myScalaVersion = "2.13.8" println(doesntExist) \ No newline at end of file diff --git a/integration/failure/parse-error/repo/bar.sc b/integration/failure/parse-error/repo/bar.sc index 6d9d210b00d..5b999607e56 100644 --- a/integration/failure/parse-error/repo/bar.sc +++ b/integration/failure/parse-error/repo/bar.sc @@ -1,4 +1,4 @@ -def myScalaVersion = "2.13.2" +def myScalaVersion = "2.13.8" println(doesntExist}) \ No newline at end of file diff --git a/integration/feature/dynamic-cross-module/repo/build.sc b/integration/feature/dynamic-cross-module/repo/build.sc index 901a8b90c3b..bb5be08b021 100644 --- a/integration/feature/dynamic-cross-module/repo/build.sc +++ b/integration/feature/dynamic-cross-module/repo/build.sc @@ -5,6 +5,6 @@ val moduleNames = interp.watchValue(os.list(millSourcePath / "modules").map(_.la object modules extends Cross[FolderModule](moduleNames:_*) class FolderModule(name: String) extends ScalaModule{ def millSourcePath = super.millSourcePath / name - def scalaVersion = "2.13.2" + def scalaVersion = "2.13.8" } diff --git a/integration/feature/editing/repo/build.sc b/integration/feature/editing/repo/build.sc index a5b1689a3e3..f9c79d55173 100644 --- a/integration/feature/editing/repo/build.sc +++ b/integration/feature/editing/repo/build.sc @@ -2,7 +2,7 @@ import mill._, scalalib._ import scalatags.Text.all._ object foo extends ScalaModule { - def scalaVersion = "2.13.2" + def scalaVersion = "2.13.8" def forkEnv = Map( "snippet" -> frag(h1("hello"), p("world"), p(constant.Constant.scalatagsVersion)).render diff --git a/integration/feature/large-project/repo/build.sc b/integration/feature/large-project/repo/build.sc index 27844c90095..ba1a46fed5b 100644 --- a/integration/feature/large-project/repo/build.sc +++ b/integration/feature/large-project/repo/build.sc @@ -1,7 +1,7 @@ import mill._, scalalib._ trait TModule extends SbtModule { - def scalaVersion = "2.12.7" + def scalaVersion = "2.12.17" } object foo extends Module { diff --git a/main/api/src/mill/api/Result.scala b/main/api/src/mill/api/Result.scala index 50b6ea0f894..5ec9a6e99c3 100644 --- a/main/api/src/mill/api/Result.scala +++ b/main/api/src/mill/api/Result.scala @@ -78,6 +78,19 @@ object Result { case class Exception(throwable: Throwable, outerStack: OuterStack) extends Failing[Nothing] { def map[V](f: Nothing => V): Exception = this def flatMap[V](f: Nothing => Result[V]): Exception = this + + override def toString: String = { + var current = List(throwable) + while (current.head.getCause != null) { + current = current.head.getCause :: current + } + current.reverse + .flatMap(ex => + Seq(ex.toString) ++ + ex.getStackTrace.dropRight(outerStack.value.length).map(" " + _) + ) + .mkString("\n") + } } class OuterStack(val value: Seq[StackTraceElement]) { diff --git a/main/core/src/mill/eval/Evaluator.scala b/main/core/src/mill/eval/Evaluator.scala index 6805efd41cf..a608f5b2a3c 100644 --- a/main/core/src/mill/eval/Evaluator.scala +++ b/main/core/src/mill/eval/Evaluator.scala @@ -578,7 +578,7 @@ class Evaluator private ( val out = System.out val in = System.in val err = System.err - mill.util.Util.withStreams(multiLogger.systemStreams){ + mill.util.Util.withStreams(multiLogger.systemStreams) { try task.evaluate(args) catch { case NonFatal(e) => @@ -904,17 +904,7 @@ object Evaluator { case Right(t) => t.segments.render } val fss = fs.map { - case Result.Exception(t, outerStack) => - var current = List(t) - while (current.head.getCause != null) { - current = current.head.getCause :: current - } - current.reverse - .flatMap(ex => - Seq(ex.toString) ++ - ex.getStackTrace.dropRight(outerStack.value.length).map(" " + _) - ) - .mkString("\n") + case ex: Result.Exception => ex.toString case Result.Failure(t, _) => t } s"$ks ${fss.iterator.mkString(", ")}" diff --git a/main/test/src/mill/eval/ModuleTests.scala b/main/test/src/mill/eval/ModuleTests.scala index 92d7256684f..a0b0e057040 100644 --- a/main/test/src/mill/eval/ModuleTests.scala +++ b/main/test/src/mill/eval/ModuleTests.scala @@ -18,18 +18,18 @@ object ModuleTests extends TestSuite { def z = T { ExternalModule.x() + ExternalModule.inner.y() } } val tests = Tests { - os.remove.all(TestUtil.externalOutPath) "externalModuleTargetsAreNamespacedByModulePackagePath" - { val check = new TestEvaluator(Build) + os.remove.all(check.outPath) val zresult = check.apply(Build.z) assert( zresult == Right((30, 1)), os.read(check.evaluator.outPath / "z.json").contains("30"), os.read( - TestUtil.externalOutPath / "mill" / "eval" / "ModuleTests" / "ExternalModule" / "x.json" + check.outPath / "mill" / "eval" / "ModuleTests" / "ExternalModule" / "x.json" ).contains("13"), os.read( - TestUtil.externalOutPath / "mill" / "eval" / "ModuleTests" / "ExternalModule" / "inner" / "y.json" + check.outPath / "mill" / "eval" / "ModuleTests" / "ExternalModule" / "inner" / "y.json" ).contains("17") ) } diff --git a/main/testkit/src/mill/testkit/MillTestkit.scala b/main/testkit/src/mill/testkit/MillTestkit.scala index dc63918fcdc..c6a1645e418 100644 --- a/main/testkit/src/mill/testkit/MillTestkit.scala +++ b/main/testkit/src/mill/testkit/MillTestkit.scala @@ -20,8 +20,6 @@ trait MillTestKit { def targetDir: os.Path = defaultTargetDir - def externalOutPath: os.Path = targetDir / "external" - def staticTestEvaluator(module: => mill.define.BaseModule)(implicit fullName: sourcecode.FullName ) = { @@ -100,7 +98,7 @@ trait MillTestKit { val evaluator = Evaluator( mill.api.Ctx.defaultHome, outPath, - externalOutPath, + outPath, module, logger, 0 diff --git a/main/util/src/mill/util/ColorLogger.scala b/main/util/src/mill/util/ColorLogger.scala index 6dac0c6f218..576c518f6d9 100644 --- a/main/util/src/mill/util/ColorLogger.scala +++ b/main/util/src/mill/util/ColorLogger.scala @@ -2,9 +2,7 @@ package mill.util import mill.api.Logger - trait ColorLogger extends Logger { def infoColor: fansi.Attrs def errorColor: fansi.Attrs } - diff --git a/main/util/src/mill/util/DummyLogger.scala b/main/util/src/mill/util/DummyLogger.scala index 0bc4501130b..12a6c88defc 100644 --- a/main/util/src/mill/util/DummyLogger.scala +++ b/main/util/src/mill/util/DummyLogger.scala @@ -4,7 +4,6 @@ import mill.api.{Logger, SystemStreams} import java.io.{ByteArrayInputStream, PrintStream} - object DummyLogger extends Logger { def colored = false diff --git a/main/util/src/mill/util/FileLogger.scala b/main/util/src/mill/util/FileLogger.scala index d7b77fd6905..475aa3320ad 100644 --- a/main/util/src/mill/util/FileLogger.scala +++ b/main/util/src/mill/util/FileLogger.scala @@ -5,11 +5,12 @@ import mill.api.{Logger, SystemStreams} import java.io.{OutputStream, PrintStream} import java.nio.file.{Files, StandardOpenOption} - -class FileLogger(override val colored: Boolean, - file: os.Path, - override val debugEnabled: Boolean, - append: Boolean = false) extends Logger { +class FileLogger( + override val colored: Boolean, + file: os.Path, + override val debugEnabled: Boolean, + append: Boolean = false +) extends Logger { private[this] var outputStreamUsed: Boolean = false lazy val fileStream = { diff --git a/main/util/src/mill/util/MultiLogger.scala b/main/util/src/mill/util/MultiLogger.scala index ff5f03097f0..dd278718c3c 100644 --- a/main/util/src/mill/util/MultiLogger.scala +++ b/main/util/src/mill/util/MultiLogger.scala @@ -4,11 +4,13 @@ import mill.api.{Logger, SystemStreams} import java.io.{InputStream, OutputStream, PrintStream} -class MultiLogger(val colored: Boolean, - val logger1: Logger, - val logger2: Logger, - val inStream0: InputStream, - override val debugEnabled: Boolean) extends Logger { +class MultiLogger( + val colored: Boolean, + val logger1: Logger, + val logger2: Logger, + val inStream0: InputStream, + override val debugEnabled: Boolean +) extends Logger { lazy val systemStreams = new SystemStreams( new MultiStream(logger1.systemStreams.out, logger2.systemStreams.out), @@ -41,25 +43,25 @@ class MultiLogger(val colored: Boolean, } class MultiStream(stream1: OutputStream, stream2: OutputStream) - extends PrintStream(new OutputStream { - def write(b: Int): Unit = { - stream1.write(b) - stream2.write(b) - } - override def write(b: Array[Byte]): Unit = { - stream1.write(b) - stream2.write(b) - } - override def write(b: Array[Byte], off: Int, len: Int) = { - stream1.write(b, off, len) - stream2.write(b, off, len) - } - override def flush() = { - stream1.flush() - stream2.flush() - } - override def close() = { - stream1.close() - stream2.close() - } - }) + extends PrintStream(new OutputStream { + def write(b: Int): Unit = { + stream1.write(b) + stream2.write(b) + } + override def write(b: Array[Byte]): Unit = { + stream1.write(b) + stream2.write(b) + } + override def write(b: Array[Byte], off: Int, len: Int) = { + stream1.write(b, off, len) + stream2.write(b, off, len) + } + override def flush() = { + stream1.flush() + stream2.flush() + } + override def close() = { + stream1.close() + stream2.close() + } + }) diff --git a/main/util/src/mill/util/PrefixLogger.scala b/main/util/src/mill/util/PrefixLogger.scala index cd37ac23a79..73d39789699 100644 --- a/main/util/src/mill/util/PrefixLogger.scala +++ b/main/util/src/mill/util/PrefixLogger.scala @@ -4,9 +4,8 @@ import mill.api.SystemStreams import java.io.PrintStream - class PrefixLogger(out: ColorLogger, context: String, tickerContext: String = "") - extends ColorLogger { + extends ColorLogger { override def colored = out.colored def infoColor = out.infoColor @@ -35,8 +34,6 @@ class PrefixLogger(out: ColorLogger, context: String, tickerContext: String = "" override def debugEnabled: Boolean = out.debugEnabled } - - object PrefixLogger { def apply(out: ColorLogger, context: String, tickerContext: String = ""): PrefixLogger = new PrefixLogger(out, context, tickerContext) diff --git a/main/util/src/mill/util/PrintLogger.scala b/main/util/src/mill/util/PrintLogger.scala index 8a3bb138815..6e6b2268873 100644 --- a/main/util/src/mill/util/PrintLogger.scala +++ b/main/util/src/mill/util/PrintLogger.scala @@ -16,7 +16,6 @@ class PrintLogger( printLoggerState: PrintLogger.State ) extends ColorLogger { - def info(s: String) = synchronized { printLoggerState.value = PrintLogger.State.Newline systemStreams.err.println(infoColor(context + s)) @@ -54,14 +53,14 @@ class PrintLogger( copy(systemStreams = new SystemStreams(outStream, systemStreams.err, systemStreams.in)) private def copy( - colored: Boolean = colored, - enableTicker: Boolean = enableTicker, - infoColor: fansi.Attrs = infoColor, - errorColor: fansi.Attrs = errorColor, - systemStreams: SystemStreams = systemStreams, - debugEnabled: Boolean = debugEnabled, - context: String = context, - printLoggerState: PrintLogger.State = printLoggerState, + colored: Boolean = colored, + enableTicker: Boolean = enableTicker, + infoColor: fansi.Attrs = infoColor, + errorColor: fansi.Attrs = errorColor, + systemStreams: SystemStreams = systemStreams, + debugEnabled: Boolean = debugEnabled, + context: String = context, + printLoggerState: PrintLogger.State = printLoggerState ): PrintLogger = new PrintLogger( colored, enableTicker, @@ -81,7 +80,7 @@ class PrintLogger( } } -object PrintLogger{ +object PrintLogger { def wrapSystemStreams(systemStreams0: SystemStreams, printLoggerState: State) = { new SystemStreams( @@ -90,9 +89,8 @@ object PrintLogger{ systemStreams0.in ) } - class StateStream(wrapped: OutputStream, - setprintLoggerState0: State.Value => Unit) extends OutputStream { - + class StateStream(wrapped: OutputStream, setprintLoggerState0: State.Value => Unit) + extends OutputStream { private[this] def setprintLoggerState(c: Char) = setprintLoggerState0( c match { @@ -133,4 +131,3 @@ object PrintLogger{ case object Middle extends Value } } - diff --git a/main/util/src/mill/util/ProxyLogger.scala b/main/util/src/mill/util/ProxyLogger.scala index ec91d14fff9..beacdfd621b 100644 --- a/main/util/src/mill/util/ProxyLogger.scala +++ b/main/util/src/mill/util/ProxyLogger.scala @@ -2,7 +2,6 @@ package mill.util import mill.api.Logger - /** * A Logger that forwards all logging to another Logger. Intended to be * used as a base class for wrappers that modify logging behavior. diff --git a/runner/src/mill/runner/MillBuildRootModule.scala b/runner/src/mill/runner/MillBuildRootModule.scala index 64d9aa883cc..6e42a9aa1ea 100644 --- a/runner/src/mill/runner/MillBuildRootModule.scala +++ b/runner/src/mill/runner/MillBuildRootModule.scala @@ -3,7 +3,8 @@ package mill.runner import mill._ import mill.api.{Loose, PathRef, Result, internal} import mill.define.{Caller, Discover, Target, Task} -import mill.scalalib.{BoundDep, DepSyntax, Lib, ScalaModule, Versions} +import mill.scalalib.{BoundDep, DepSyntax, Lib, ScalaModule} +import mill.scalalib.api.Versions import os.Path import pprint.Util.literalize diff --git a/runner/src/mill/runner/MillMain.scala b/runner/src/mill/runner/MillMain.scala index 06d700f9eb3..fd1b2b6b73d 100644 --- a/runner/src/mill/runner/MillMain.scala +++ b/runner/src/mill/runner/MillMain.scala @@ -32,15 +32,16 @@ object MillMain { val (result, _) = try main0( - args, - RunnerState.empty, - mill.util.Util.isInteractive(), - initialSystemStreams, - System.getenv().asScala.toMap, - b => (), - userSpecifiedProperties0 = Map(), - initialSystemProperties = sys.props.toMap - ) finally { + args, + RunnerState.empty, + mill.util.Util.isInteractive(), + initialSystemStreams, + System.getenv().asScala.toMap, + b => (), + userSpecifiedProperties0 = Map(), + initialSystemProperties = sys.props.toMap + ) + finally { openStreams.foreach(_.close()) } System.exit(if (result) 0 else 1) @@ -58,7 +59,7 @@ object MillMain { ): (Boolean, RunnerState) = { val printLoggerState = new PrintLogger.State() val streams = PrintLogger.wrapSystemStreams(streams0, printLoggerState) - Util.withStreams(streams){ + Util.withStreams(streams) { MillCliConfigParser.parse(args) match { // Cannot parse args case Left(msg) => @@ -74,22 +75,22 @@ object MillMain { streams.out.println( s"""Mill Build Tool version ${BuildInfo.millVersion} |Java version: ${p("java.version", "" - )} + "file.encoding", + "" + )} |OS name: "${p("os.name")}", version: ${p("os.version")}, arch: ${p( - "os.arch" - )}""".stripMargin + "os.arch" + )}""".stripMargin ) (true, RunnerState.empty) case Right(config) - if ( - config.interactive.value || config.repl.value || config.noServer.value || config.bsp.value + if ( + config.interactive.value || config.repl.value || config.noServer.value || config.bsp.value ) && streams.in == DummyInputStream => // because we have stdin as dummy, we assume we were already started in server process streams.err.println( @@ -98,12 +99,12 @@ object MillMain { (false, RunnerState.empty) case Right(config) - if Seq( - config.interactive.value, - config.repl.value, - config.noServer.value, - config.bsp.value - ).count(identity) > 1 => + if Seq( + config.interactive.value, + config.repl.value, + config.noServer.value, + config.bsp.value + ).count(identity) > 1 => streams.err.println( "Only one of -i/--interactive, --repl, --no-server or --bsp may be given" ) diff --git a/scalalib/api/src/mill/scalalib/api/ZincWorkerUtil.scala b/scalalib/api/src/mill/scalalib/api/ZincWorkerUtil.scala index ec54376c32d..a4fe90c8c72 100644 --- a/scalalib/api/src/mill/scalalib/api/ZincWorkerUtil.scala +++ b/scalalib/api/src/mill/scalalib/api/ZincWorkerUtil.scala @@ -2,6 +2,7 @@ package mill.scalalib.api import mill.api.Loose.Agg import mill.api.PathRef +import mill.scalalib.api.Versions trait ZincWorkerUtil { @@ -105,9 +106,13 @@ trait ZincWorkerUtil { case _ => true } + lazy val millCompilerBridgeScalaVersions: Set[String] = + Versions.millCompilerBridgeScalaVersions.split(",").toSet + /** @return true if the compiler bridge can be downloaded as an already compiled jar */ def isBinaryBridgeAvailable(scalaVersion: String) = - scalaVersion match { + if (millCompilerBridgeScalaVersions.contains(scalaVersion)) true + else scalaVersion match { case DottyNightlyVersion(major, minor, _, _) => major.toInt > 0 || minor.toInt >= 14 // 0.14.0-bin or more (not 0.13.0-bin) case DottyVersion(minor, _) => minor.toInt >= 13 // 0.13.0-RC1 or more diff --git a/scalalib/src/mill/scalalib/ScalaModule.scala b/scalalib/src/mill/scalalib/ScalaModule.scala index 979de2d2772..00bfc70cb4b 100644 --- a/scalalib/src/mill/scalalib/ScalaModule.scala +++ b/scalalib/src/mill/scalalib/ScalaModule.scala @@ -7,7 +7,7 @@ import mill.api.{DummyInputStream, JarManifest, PathRef, Result, internal} import mill.modules.Jvm import mill.modules.Jvm.createJar import mill.api.Loose.Agg -import mill.scalalib.api.{CompilationResult, ZincWorkerUtil} +import mill.scalalib.api.{CompilationResult, ZincWorkerUtil, Versions} import scala.jdk.CollectionConverters._ import mainargs.Flag diff --git a/scalalib/src/mill/scalalib/SemanticDbJavaModule.scala b/scalalib/src/mill/scalalib/SemanticDbJavaModule.scala index 088ed7795e5..19e8592f77f 100644 --- a/scalalib/src/mill/scalalib/SemanticDbJavaModule.scala +++ b/scalalib/src/mill/scalalib/SemanticDbJavaModule.scala @@ -2,7 +2,7 @@ package mill.scalalib import mill.api.{PathRef, Result, experimental} import mill.define.{Input, Target, Task} -import mill.scalalib.api.ZincWorkerUtil +import mill.scalalib.api.{Versions, ZincWorkerUtil} import mill.util.Version import mill.{Agg, BuildInfo, T} diff --git a/scalalib/src/mill/scalalib/ZincWorkerModule.scala b/scalalib/src/mill/scalalib/ZincWorkerModule.scala index 8f17ab6fdcf..567b2378680 100644 --- a/scalalib/src/mill/scalalib/ZincWorkerModule.scala +++ b/scalalib/src/mill/scalalib/ZincWorkerModule.scala @@ -9,7 +9,7 @@ import mill.api.{Ctx, FixSizedCache, KeyedLockedCache, PathRef, Result} import mill.define.{Command, Discover, ExternalModule, Input, Target, Worker} import mill.scalalib.Lib.resolveDependencies import mill.scalalib.api.ZincWorkerUtil.{isBinaryBridgeAvailable, isDotty, isDottyOrScala3} -import mill.scalalib.api.{ZincWorkerApi, ZincWorkerUtil} +import mill.scalalib.api.{ZincWorkerApi, ZincWorkerUtil, Versions} import os.Path object ZincWorkerModule extends ExternalModule with ZincWorkerModule with CoursierModule { @@ -55,6 +55,7 @@ trait ZincWorkerModule extends mill.Module with OfflineSupportModule { self: Cou classpath().map(_.path.toNIO.toUri.toURL).iterator.to(Vector), getClass.getClassLoader ) + val cls = cl.loadClass("mill.scalalib.worker.ZincWorkerImpl") val instance = cls.getConstructor( classOf[ @@ -73,7 +74,12 @@ trait ZincWorkerModule extends mill.Module with OfflineSupportModule { self: Cou Left(( T.ctx(), (x: String, y: String) => - scalaCompilerBridgeJar(x, y, repositoriesTask()).asSuccess.get.value + scalaCompilerBridgeJar(x, y, repositoriesTask()) + .asSuccess + .getOrElse( + throw new Exception(s"Failed to load compiler bridge for $x $y") + ) + .value )), ZincWorkerUtil.grepJar(_, "scala-library", _, sources = false), ZincWorkerUtil.grepJar(_, "scala-compiler", _, sources = false), @@ -101,6 +107,11 @@ trait ZincWorkerModule extends mill.Module with OfflineSupportModule { self: Cou else "scala3-sbt-bridge" val version = scalaVersion (ivy"$org:$name:$version", name, version) + } else if (ZincWorkerUtil.millCompilerBridgeScalaVersions.contains(scalaVersion0)) { + val org = "com.lihaoyi" + val name = s"mill-scala-compiler-bridge_$scalaVersion" + val version = Versions.millCompilerBridgeVersion + (ivy"$org:$name:$version", name, version) } else { val org = "org.scala-sbt" val name = "compiler-bridge" @@ -111,6 +122,7 @@ trait ZincWorkerModule extends mill.Module with OfflineSupportModule { self: Cou version ) } + val useSources = !isBinaryBridgeAvailable(scalaVersion) val bridgeJar = resolveDependencies( diff --git a/scalalib/test/src/mill/scalalib/HelloWorldTests.scala b/scalalib/test/src/mill/scalalib/HelloWorldTests.scala index c37bfb4cb8d..7939c650da4 100644 --- a/scalalib/test/src/mill/scalalib/HelloWorldTests.scala +++ b/scalalib/test/src/mill/scalalib/HelloWorldTests.scala @@ -10,6 +10,7 @@ import mill.api.Result import mill.define.{Input, NamedTask, Target} import mill.eval.{Evaluator, EvaluatorPaths} import mill.modules.Assembly +import mill.scalalib.api.ZincWorkerUtil import mill.scalalib.publish.{VersionControl, _} import mill.util.{TestEvaluator, TestUtil} import utest._ @@ -43,6 +44,11 @@ object HelloWorldTests extends TestSuite { object HelloWorld extends HelloBase { object core extends HelloWorldModule } + object HelloWorldNonPrecompiledBridge extends HelloBase { + object core extends HelloWorldModule { + override def scalaVersion = "2.12.1" + } + } object CrossHelloWorld extends HelloBase { object core extends Cross[HelloWorldCross]( scala210Version, @@ -504,7 +510,42 @@ object HelloWorldTests extends TestSuite { val Right((_, unchangedEvalCount)) = eval.apply(HelloWorld.core.compile) assert(unchangedEvalCount == 0) + + // Make sure we *do not* end up compiling the compiler bridge, since + // it's using a pre-compiled bridge value + assert(!os.exists( + eval.outPath / "mill" / "scalalib" / "ZincWorkerModule" / "worker.dest" / "zinc-1.8.0" + )) } + + "nonPreCompiledBridge" - workspaceTest(HelloWorldNonPrecompiledBridge) { eval => + val Right((result, evalCount)) = eval.apply(HelloWorldNonPrecompiledBridge.core.compile) + + val classesPath = eval.outPath / "core" / "compile.dest" / "classes" + + val analysisFile = result.analysisFile + val outputFiles = os.walk(result.classes.path) + val expectedClassfiles = compileClassfiles.map(classesPath / _) + assert( + result.classes.path == classesPath, + os.exists(analysisFile), + outputFiles.nonEmpty, + outputFiles.forall(expectedClassfiles.contains), + evalCount > 0 + ) + + // don't recompile if nothing changed + val Right((_, unchangedEvalCount)) = eval.apply(HelloWorldNonPrecompiledBridge.core.compile) + + assert(unchangedEvalCount == 0) + + // Make sure we *do* end up compiling the compiler bridge, since it's + // *not* using a pre-compiled bridge value + assert(os.exists( + eval.outPath / "mill" / "scalalib" / "ZincWorkerModule" / "worker.dest" / "zinc-1.8.0" + )) + } + "recompileOnChange" - workspaceTest(HelloWorld) { eval => val Right((_, freshCount)) = eval.apply(HelloWorld.core.compile) assert(freshCount > 0) diff --git a/scalalib/worker/src/mill/scalalib/worker/ZincWorkerImpl.scala b/scalalib/worker/src/mill/scalalib/worker/ZincWorkerImpl.scala index 9493bc91531..b5aca5931e1 100644 --- a/scalalib/worker/src/mill/scalalib/worker/ZincWorkerImpl.scala +++ b/scalalib/worker/src/mill/scalalib/worker/ZincWorkerImpl.scala @@ -2,7 +2,7 @@ package mill.scalalib.worker import mill.api.Loose.Agg import mill.api.{CompileProblemReporter, KeyedLockedCache, PathRef, Result, internal} -import mill.scalalib.api.{CompilationResult, ZincWorkerApi, ZincWorkerUtil} +import mill.scalalib.api.{CompilationResult, ZincWorkerApi, ZincWorkerUtil, Versions} import sbt.internal.inc.{ Analysis, CompileFailed, diff --git a/scalanativelib/test/src/mill/scalanativelib/ExclusionsTests.scala b/scalanativelib/test/src/mill/scalanativelib/ExclusionsTests.scala index c90df86730e..7116e4e271a 100644 --- a/scalanativelib/test/src/mill/scalanativelib/ExclusionsTests.scala +++ b/scalanativelib/test/src/mill/scalanativelib/ExclusionsTests.scala @@ -11,7 +11,7 @@ object ExclusionsTests extends TestSuite { object Exclusions extends TestUtil.BaseModule { object scala213 extends ScalaNativeModule { def scalaNativeVersion = "0.4.3" - def scalaVersion = "2.13.8" + def scalaVersion = "2.13.10" override def ivyDeps = super.ivyDeps() ++ Agg( ivy"com.github.scopt:scopt_native0.4_3:4.0.1" )