Support for scala 2.12 #9

machaval · 2022-07-27T14:18:38Z

Can we have support for scala 2.12?

dubinsky · 2022-07-28T01:21:30Z

Scala 2.12?! Sure, I'll take a look.

What happens when you try the plugin with Scala 2.12 project? How does it break?
Do you have a minimal project I can use to reproduce the breakage?

Thank you!

dubinsky · 2022-07-28T03:12:35Z

@machaval I tried the plugin with Scala 2.12.16, and it seems to work. What problems did you encounter? Thanks!

dubinsky · 2022-07-28T05:16:25Z

@machaval sorry, I spoke too soon (forgot to clean out the Gradle Daemons before testing).
I now see java.lang.NoSuchMethodError: 'scala.collection.mutable.WrappedArray scala.Predef$.wrapRefArray(java.lang.Object[])

If that is what you see, I know what the problem is and how to address it.
It will take at least a couple of weeks, but internals of the plugin will be a bit cleaner as a result...

Thank you for bringing this to my attention!

machaval · 2022-07-28T13:50:35Z

The exception that is throwing me is

Cause 3: java.lang.NoSuchMethodError: 'scala.collection.mutable.Set scala.collection.convert.AsScalaConverters.asScala$(scala.collection.convert.AsScalaConverters, java.util.Set)'
        at scala.collection.convert.AsScalaExtensions$SetHasAsScala.asScala(AsScalaExtensions.scala:63)
        at org.opentorah.build.Dependency.getFromConfiguration(Dependency.scala:34)
        at org.opentorah.build.ScalaLibrary$.getFromConfiguration(ScalaLibrary.scala:34)
        at org.podval.tools.scalajs.ScalaJSPlugin.apply$$anonfun$1(ScalaJSPlugin.scala:32)
        at org.gradle.configuration.internal.DefaultUserCodeApplicationContext$CurrentApplication$1.execute(DefaultUserCodeApplicationContext.java:123)
        at org.gradle.configuration.internal.DefaultListenerBuildOperationDecorator$BuildOperationEmittingAction$1.run(DefaultListenerBuildOperationDecorator.java:171)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)

My scripts looks like


buildscript {
    repositories {
        maven {
            name "mule-releases"
            url "https://repository-master.mulesoft.org/nexus/content/repositories/releases"
        }
        mavenCentral()
    }
    dependencies {
//        classpath 'com.github.machaval:scalajs-plugin:sjs0.6_2.12_0.3.7'
        classpath group: 'commons-io', name: 'commons-io', version: '2.5'
        classpath "com.google.guava:guava:21.0"
    }
}

plugins {
    id 'org.podval.tools.scalajs' version '0.1.0'
}

sourceSets {
    main {
        scala {
            srcDirs = ['src/main/scala', 'build/genSources', 'build/parser-common']
        }
    }
}

dependencies {
    implementation group: 'org.scala-lang', name: 'scala-library', version: '2.13.8'
    implementation group: 'org.scala-js', name: 'scalajs-library_2.13', version: scalaJsVersion
    implementation group: 'org.parboiled', name: 'parboiled_sjs0.6_2.13', version: '2.1.8'
}

task genSources {
    doLast {
        def wlangResources = new File("$project.projectDir/../wlang/src/main/resources")
        def genDirectory = new File("$project.buildDir/genSources")
        def outputFile = new File(genDirectory, "org/mule/weave/v2/resources/JSResourceLoader.scala")
        def coreModulesResources = new File("$project.projectDir/../core-modules/src/main/resources")
        new ResourceBuilder().run(outputFile, wlangResources, coreModulesResources);
    }

}

task generatePomFileForMavenJavaPublication {}

//ext {
//    d = true
//    testLogLevel = "Debug" //Use a string
//    sourceMap = true
//    commonJSModule = true
//    r = 'dw-parser-js/src/'
//    o = 'dw-parser-js/src/Parser.js'
//    s = true
//}

link {
    optimization     = 'Fast'          // one of: 'Fast', 'Full'
    moduleKind       = 'NoModule'      // one of: 'NoModule', 'ESModule', 'CommonJSModule'
    moduleSplitStyle = 'FewestModules' // one of: 'FewestModules', 'SmallestModules'
    prettyPrint      = false
}


task cleanupGen(type: Delete) {
    delete 'build/genSources'
    delete 'build/parser-common/'
}

task copySrc(type: Sync) {
    from '../parser/src/main/scala'
    into 'build/parser-common/'

    from '../parser/build/genresource'
    into 'build/parser-common'
}

//defaultTasks += FullOptJS

compileScala.dependsOn(copySrc).dependsOn(genSources).dependsOn(cleanupGen)

//build.dependsOn(FullOptJS)

import org.apache.commons.io.FilenameUtils
import org.apache.commons.io.IOUtils

public class ResourceBuilder {

    public void run(File outputScalaFile, File... resourceDir) throws Exception {
        def parentFile = outputScalaFile.getParentFile()
        if (!parentFile.exists()) {
            parentFile.mkdirs()
        }
        final PrintWriter outputPrinter = new PrintWriter(new FileWriter(outputScalaFile));
        try {
            outputPrinter.println("package org.mule.weave.v2.resources\n" +
                    "\n" +
                    "object " + FilenameUtils.getBaseName(outputScalaFile.getName()) + " {");

            final HashMap<String, String> resourcesMap = new HashMap<>();
            for (File resource : resourceDir) {
                generate("", resource, outputPrinter, resourcesMap);
            }
            outputPrinter.print("\n\tval _resources = Map(");
            boolean first = true;
            for (Map.Entry<String, String> resourcesEntry : resourcesMap.entrySet()) {
                if (!first) {
                    outputPrinter.print(",\n \t\t")
                }
                outputPrinter.print("(\"" + resourcesEntry.getKey() + "\", " + resourcesEntry.getValue() + ")");
                first = false;
            }
            outputPrinter.println(")");
            outputPrinter.println("  def getResource(name: String): String = {\n" +
                    "    _resources.get(name).orNull\n" +
                    "  }\n" +
                    "}");
        }
        finally {
            outputPrinter.close();
        }

    }

    private void generate(String basePath, File resourceDir, PrintWriter outputPrinter, HashMap<String, String> resources) throws IOException {
        final File[] files = resourceDir.listFiles(new FileFilter() {
            @Override
            boolean accept(File pathname) {
                return pathname.isDirectory() || pathname.getName().endsWith(".dwl")
            }
        });
        for (File file : files) {
            final String resourcePath = basePath.isEmpty() ? file.getName().replace("/", "::") : basePath + "::" + file.getName().replace(".dwl", "");
            if (file.isDirectory()) {
                generate(resourcePath, file, outputPrinter, resources);
            } else {
                final String variableName = FilenameUtils.getBaseName(file.getName());
                resources.put(resourcePath, variableName);
                final List<String> lines = IOUtils.readLines(new FileInputStream(file), "UTF-8");
                outputPrinter.println("  def " + variableName + " =");
                outputPrinter.println("    Array(");
                for (String line : lines) {
                    outputPrinter.println("\"\"\"" + line + "\"\"\", ");
                }
                outputPrinter.println("\"\").mkString(\"\\n\")");
            }
        }
    }
}

Even if I try with scala 13 so looks like there is something wrong with my configuration. Also I've noticed that gradle 7 is required if not there is another exception with a method not found.

Cause 3: java.lang.NoSuchMethodError: 'org.gradle.api.tasks.SourceSetContainer org.gradle.api.plugins.JavaPluginExtension.getSourceSets()'

This may be good to add this to the documentation.

Thanks for all the replays really appreciate.

dubinsky · 2022-07-28T15:46:19Z

I see a different exception from yours. Will take a look at your setup reasonably soon ;)
Good point about Gradle version requirements. I will document those in the next release.
Thanks!

dubinsky · 2022-07-28T17:49:44Z

Looking at your setup:

you declare buildscript.repositories, but plugin needs to be able to resolve dependencies it adds, so there should be a repositories block with mavenCentral() in it that applies to the project as a whole, not just buildScript; I usually put it in the settings.gradle, inside dependencyResolutionManagement; I'll document this in the next release (you probably have this in your overall setup somewhere)
implementation group: 'org.scala-js', name: 'scalajs-library_2.13', version: scalaJsVersion won't work since scalaJsVersion is not defined in build.gradle; I'll adjust the documentation to not mention undefined variables in new release :)
unless you want to use specific version of ScalaJS, you do not need to specify this dependency explicitly - plugin does it for you;
if you do want specific ScalaJS version, it must be at least 1.0; I'll document this requirement in the next release;
that means that parboiled_sjs0.6_2.13 is not going to work with the plugin; I do not know if there is a version for ScalaJS 1.0, but there should be;
I had to comment out the code-copying parts (since I have nothing to copy), and do see some problems - but not the exception you see... will look some more later.

Thanks!

dubinsky · 2022-07-28T18:35:59Z

As to Scala 3 - notice that the library artifact name is different, not just the version:
implementation group: 'org.scala-lang', name: 'scala3-library_3', version: '3.1.3'.
What used to be scala-library is now scala3-library_3.
This caused a lot of issues for tools, including Scala 3 support in Gradle in general, and is not specific to my plugin ;)

dubinsky · 2022-07-31T19:17:00Z

@machaval Scala 2.12 is supported in version 0.2.0 of the plugin

Thank you for your encouragement!

machaval · 2022-08-01T19:15:31Z

Thanks! I will try it out. I'm updating my build to latest gradle and scalajs 🤞🏾

machaval · 2022-08-01T19:42:28Z

Hi I'm trying it but this error still occurs

Cause 3: java.lang.NoSuchMethodError: 'scala.collection.mutable.ArrayBuffer scala.collection.mutable.ArrayBuffer$.empty()'
        at org.opentorah.build.Gradle$.collectClassPath(Gradle.scala:119)
        at org.podval.tools.scalajs.ScalaJSPlugin.apply$$anonfun$1(ScalaJSPlugin.scala:44)
        at org.gradle.configuration.internal.DefaultUserCodeApplicationContext$CurrentApplication$1.execute(DefaultUserCodeApplicationContext.java:123)
        at org.gradle.configuration.internal.DefaultListenerBuildOperationDecorator$BuildOperationEmittingAction$1.run(DefaultListenerBuildOperationDecorator.java:171)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68)
        at org.gradle.configuration.internal.DefaultListenerBuildOperationDecorator$BuildOperationEmittingAction.execute(DefaultListenerBuildOperationDecorator.java:168)
        at org.gradle.internal.event.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:95)
        at org.gradle.internal.event.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:83)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:43)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:245)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:157)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:61)
        at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:346)
        at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:249)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:141)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
        at com.sun.proxy.$Proxy41.afterEvaluate(Unknown Source)
        at org.gradle.configuration.project.LifecycleProjectEvaluator$NotifyAfterEvaluate$1.execute(LifecycleProjectEvaluator.java:247)
        at org.gradle.configuration.project.LifecycleProjectEvaluator$NotifyAfterEvaluate$1.execute(LifecycleProjectEvaluator.java:244)
        at org.gradle.api.internal.project.DefaultProject.stepEvaluationListener(DefaultProject.java:1434)
        at org.gradle.configuration.project.LifecycleProjectEvaluator$NotifyAfterEvaluate.run(LifecycleProjectEvaluator.java:253)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68)
        at org.gradle.configuration.project.LifecycleProjectEvaluator$EvaluateProject.lambda$run$0(LifecycleProjectEvaluator.java:114)
        at org.gradle.api.internal.project.DefaultProjectStateRegistry$ProjectStateImpl.lambda$applyToMutableState$0(DefaultProjectStateRegistry.java:351)
        at org.gradle.api.internal.project.DefaultProjectStateRegistry$ProjectStateImpl.lambda$fromMutableState$1(DefaultProjectStateRegistry.java:374)
        at org.gradle.internal.work.DefaultWorkerLeaseService.withReplacedLocks(DefaultWorkerLeaseService.java:345)
        at org.gradle.api.internal.project.DefaultProjectStateRegistry$ProjectStateImpl.fromMutableState(DefaultProjectStateRegistry.java:374)
        at org.gradle.api.internal.project.DefaultProjectStateRegistry$ProjectStateImpl.applyToMutableState(DefaultProjectStateRegistry.java:350)
        at org.gradle.configuration.project.LifecycleProjectEvaluator$EvaluateProject.run(LifecycleProjectEvaluator.java:100)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68)
        at org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:72)
        at org.gradle.api.internal.project.DefaultProject.evaluate(DefaultProject.java:761)
        at org.gradle.api.internal.project.DefaultProject.evaluate(DefaultProject.java:152)
        at org.gradle.api.internal.project.ProjectLifecycleController.lambda$ensureSelfConfigured$1(ProjectLifecycleController.java:63)
        at org.gradle.internal.model.StateTransitionController.lambda$doTransition$12(StateTransitionController.java:236)
        at org.gradle.internal.model.StateTransitionController.doTransition(StateTransitionController.java:247)
        at org.gradle.internal.model.StateTransitionController.doTransition(StateTransitionController.java:235)
        at org.gradle.internal.model.StateTransitionController.lambda$maybeTransitionIfNotCurrentlyTransitioning$9(StateTransitionController.java:196)
        at org.gradle.internal.work.DefaultSynchronizer.withLock(DefaultSynchronizer.java:34)
        at org.gradle.internal.model.StateTransitionController.maybeTransitionIfNotCurrentlyTransitioning(StateTransitionController.java:192)
        at org.gradle.api.internal.project.ProjectLifecycleController.ensureSelfConfigured(ProjectLifecycleController.java:63)
        at org.gradle.api.internal.project.DefaultProjectStateRegistry$ProjectStateImpl.ensureConfigured(DefaultProjectStateRegistry.java:325)
        at org.gradle.execution.TaskPathProjectEvaluator.configure(TaskPathProjectEvaluator.java:33)
        at org.gradle.execution.TaskPathProjectEvaluator.configureHierarchy(TaskPathProjectEvaluator.java:49)
        at org.gradle.configuration.DefaultProjectsPreparer.prepareProjects(DefaultProjectsPreparer.java:50)
        at org.gradle.configuration.BuildTreePreparingProjectsPreparer.prepareProjects(BuildTreePreparingProjectsPreparer.java:64)
        at org.gradle.configuration.BuildOperationFiringProjectsPreparer$ConfigureBuild.run(BuildOperationFiringProjectsPreparer.java:52)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68)
        at org.gradle.configuration.BuildOperationFiringProjectsPrepar

I'm not even able to run dependencies once I've added your plugin. But just to make sure I've comment other plugins that uses scala (like scoverage) to avoid any conflict. Do you have any clue why it may occur

dubinsky · 2022-08-01T19:59:27Z

My only guess is - something brings onto your Gradle buildscript classpath a version of Scala incompatible with the one my plugin uses (3.1.3); apparently, you did not comment them all out ;)

A Scala 2.12 project that I use for testing is a part of the repository: https://github.com/dubinsky/scalajs-gradle/tree/master/test-projects/tutorial-Scala2.12

dubinsky · 2022-08-01T20:12:47Z

@machaval wait a minute, it is not the plugins (or not only the plugins) - it is your copying code!
You build it using Scala 2.12 and put it onto the buildscript classpath.
My plugin supports projects that use Scala 2.12, but not the situation where Scala 2.12 is on the buildscript classpath.
Maybe you can prepare the files to be copied separately and then copy them using Gradle's project.copy()?
Or add the directories needed to appropriate sourceSet?
Anyway, I won't be able to make things work with both Scala 2.13 and 2.12 present on the buildscript classpath...

dubinsky self-assigned this Jul 31, 2022

dubinsky closed this as completed Jul 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for scala 2.12 #9

Support for scala 2.12 #9

machaval commented Jul 27, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 28, 2022

machaval commented Jul 28, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 31, 2022

machaval commented Aug 1, 2022

machaval commented Aug 1, 2022

dubinsky commented Aug 1, 2022

dubinsky commented Aug 1, 2022

Support for scala 2.12 #9

Support for scala 2.12 #9

Comments

machaval commented Jul 27, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 28, 2022

machaval commented Jul 28, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 28, 2022

dubinsky commented Jul 31, 2022

machaval commented Aug 1, 2022

machaval commented Aug 1, 2022

dubinsky commented Aug 1, 2022

dubinsky commented Aug 1, 2022