Kotlin 1.3 support

[Globegitter]

I tried to use these rules using the latest kotlin 1.3 release by manually setting kotlin repositories(compiler release={... }) It downloaded the compiler fine but then complained about a missing dependency. I am on my mobile now so unfortunately do not have the error message but will update once on my desktop tomorrow. Just wanted to report this in the meantime.

[cgruber]

I'm using mildly modified versions of these rules with 1.3 successfully. Please do add the error in question. That'll help with diagnosis.

…

On Thu, 8 Nov 2018 at 11:46 Markus Padourek ***@***.***> wrote: I tried to use these rules using the latest kotlin 1.3 release by manually setting kotlin repositories(compiler release={... }) It downloaded the compiler fine but then complained about a missing dependency. I am on my mobile now so unfortunately do not have the error message but will update once on my desktop tomorrow. Just wanted to report this in the meantime. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#159>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUN4qnOLrr2hd8F9G-rH_3UoX7A2kZmks5utIotgaJpZM4YVXGC> .

[Globegitter]

Back on my machine now. The error is:

bazel run --verbose_failures //examples:hello
INFO: Invocation ID: ea40c621-eb51-424c-a75d-12169d03538e
INFO: Analysed target //examples:hello (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: missing input file '@com_github_jetbrains_kotlin//:lib/kotlin-runtime.jar'
ERROR: /path/examples/BUILD.bazel:10:1: //examples:hello: missing input file '@com_github_jetbrains_kotlin//:lib/kotlin-runtime.jar'
Target //examples:hello failed to build
ERROR: /path/examples/BUILD.bazel:10:1 1 input file(s) do not exist
INFO: Elapsed time: 0.871s, Critical Path: 0.06s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

The BUILD file is:

kt_jvm_binary(
    name = "hello",
    main_class = "examples.Hello",
    srcs = [
        "Hello.kt",
    ]
)

And yeah if I switch back to the default kotlin compiler the application runs as expected.

Maybe also relevant I am running on the latest bazel rc: release 0.20.0rc1

Edit:
The url I am providing is: https://github.com/JetBrains/kotlin/releases/download/v1.3.0/kotlin-compiler-1.3.0.zip

[cgruber]

Can you post (or exerpt) your WORKSPACE file as well? Bazel itself shouldn't be an issue, as this is nearly certainly a matter of how the toolchain is specified, and what ends up imported where.

[Globegitter]

@cgruber at its simplest it is

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

rules_kotlin_version = "d6711b288f4066954aedb3c800131aa8e970b443"
rules_kotlin_compiler_release = {
    "urls": [
        "https://github.com/JetBrains/kotlin/releases/download/v1.3.0/kotlin-compiler-1.3.0.zip",
    ],
    "sha256": "ff851cb84dd12df6078ae1f4a5424de9be6dcb4ac578b35455eeb7106dc52592",
}

http_archive(
    name = "io_bazel_rules_kotlin",
    urls = ["https://github.com/globegitter/rules_kotlin/archive/%s.zip" % rules_kotlin_version],
    type = "zip",
    strip_prefix = "rules_kotlin-%s" % rules_kotlin_version,
    sha256 = "2c4f2ecc8184243a893be21ac684ea0d0bdf93877d6163659a17c00092b0d99e",
)

load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kotlin_repositories", "kt_register_toolchains")
kotlin_repositories(compiler_release=rules_kotlin_compiler_release)
kt_register_toolchains()

[pierreis]

I'm also stuck at the same point using Kotlin 1.3, with the same compiler release (basically same workspace as above), but with a different error:

ERROR: /private/var/tmp/_bazel_pmatri/c08565c5c6c1bed750978f81a404fb89/external/io_bazel_rules_kotlin/src/main/kotlin/io/bazel/kotlin/builder/BUILD:34:1: Building external/io_bazel_rules_kotlin/src/main/kotlin/io/bazel/kotlin/builder/libbuilder.jar (2 source files) and running annotation processors (ComponentProcessor) failed (Exit 1).
error: cannot access NotNull
  class file for org.jetbrains.annotations.NotNull not found
  Consult the following stack trace for details.
  com.sun.tools.javac.code.Symbol$CompletionFailure: class file for org.jetbrains.annotations.NotNull not found
Target //services/HelloWorldService/src/kotlin/com/hurow/services/HelloWorld:HelloWorld failed to build
Use --verbose_failures to see the command lines of failed build steps.

Seems that there is a mismatch between the third-party dependencies of the rules and those of the project sources.

[Globegitter]

After commenting out:

• rules_kotlin/kotlin/internal/repositories/BUILD.com_github_jetbrains_kotlin

  Line 64 in a4e902d

  "runtime",

• rules_kotlin/kotlin/internal/jvm/jvm.bzl

  Line 137 in 46b867e

  default = Label("@" + _KT_COMPILER_REPO + "//:kotlin-runtime"),

• rules_kotlin/kotlin/internal/toolchains.bzl

  Line 115 in a4e902d

  default = Label("@" + _KT_COMPILER_REPO + "//:kotlin-runtime"),

I get the same error as @pierreis above. There seem to be a few other people who have run into this issue: https://discuss.kotlinlang.org/t/org-jetbrains-annotations-notnull-problem-with-android-build/629 & JakeWharton/timber#295 but have not been able to fix it with rules_maven. I tried to put it on the omit list but no luck.

[cgruber]

Huh. After a bit of looking at our own rules (at Square), I realize it wasn't working, we weren't pulling in what I thought we were. I'll need to look into this next week and see if I end up in the same situation.

…

On Sat, 10 Nov 2018 at 14:39 Markus Padourek ***@***.***> wrote: After commenting out: - https://github.com/bazelbuild/rules_kotlin/blob/a4e902d33504b9a3fe2c46901c67973a74b50f40/kotlin/internal/repositories/BUILD.com_github_jetbrains_kotlin#L64 - https://github.com/bazelbuild/rules_kotlin/blob/46b867ed1907e2394ce1d6bdb5cfeb5c151f9c5c/kotlin/internal/jvm/jvm.bzl#L137 - https://github.com/bazelbuild/rules_kotlin/blob/a4e902d33504b9a3fe2c46901c67973a74b50f40/kotlin/internal/toolchains.bzl#L115 I get the same error as @pierreis <https://github.com/pierreis> above. There seem to be a few other people who have run into this issue: https://discuss.kotlinlang.org/t/org-jetbrains-annotations-notnull-problem-with-android-build/629 & JakeWharton/timber#295 <JakeWharton/timber#295> but have not been able to fix it with rules_maven. I tried to put it on the omit list but no luck. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#159 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUN4oKNXD65yNL7Fhaj7_8i24JzHYuIks5ut1W8gaJpZM4YVXGC> .

[Globegitter]

Ah the error even happens for a hello world example. And I am seeing that org.jetbrains:annotations is a transitive dependency specified by rules_kotlin.

So adding "//third_party/jvm/org/jetbrains:annotations", to:
https://github.com/bazelbuild/rules_kotlin/blob/master/src/main/kotlin/io/bazel/kotlin/builder/BUILD#L46

I seem ot be getting to the next error:

INFO: Invocation ID: 524a8a96-f56b-4f0a-84ee-798868b6d6e2
INFO: Analysed target //:tempjar (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
INFO: From Compiling Kotlin to JVM //:tempjar { kt: 1, java: 0, srcjars: 0 }:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/home/user/.cache/bazel/_bazel_user/98954e6e9031b033f581d48ad4da2ceb/external/io_bazel_rules_kotlin_com_google_protobuf_protobuf_java/jar/io_bazel_rules_kotlin_com_google_protobuf_protobuf_java.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Target //:tempjar up-to-date:
  bazel-bin/tempjar.jar
INFO: Elapsed time: 3.377s, Critical Path: 3.26s
INFO: 1 process: 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
INFO: Build completed successfully, 2 total actions
Exception in thread "main" java.lang.NoClassDefFoundError: kotlin/jvm/internal/Intrinsics
	at digester.ServerKt.main(Server.kt)
Caused by: java.lang.ClassNotFoundException: kotlin.jvm.internal.Intrinsics
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)

So the build actually works and I am getting a runtime error.

[Globegitter]

@cgruber Getting a bit off point here (happy to open up a separate issue) but just trying to use https://github.com/square/moshi with the kotlin-codegen plugin. But for some reason the generated does not seem to be picked up by the resulting jar. Do you know by any chance how to get this to work? Is there something missing in bazel for it to include the generated code?

[cgruber]

@Globegitter - sadly no. You'd have to open an issue on square's moshi project and explore it there.

[pierreis]

@Globegitter, before I try it myself, did you try updating the dependencies in third_party/jvm/workspace.bzl based on the dependency YAML file?

[Globegitter]

@pierreis not all of them, in fact I only tried to update kotlinx-coroutines and it did seem like it got me a bit further but I wonder if any of the source code needs updating like https://github.com/bazelbuild/rules_kotlin/blob/master/src/main/kotlin/io/bazel/kotlin/compiler/BazelK2JVMCompiler.kt or anything within https://github.com/bazelbuild/rules_kotlin/tree/master/src/main/kotlin/io/bazel/kotlin/builder

[Globegitter]

@pierreis you can actually see my progress here: https://github.com/Globegitter/rules_kotlin/tree/exports

[Kernald]

KT-27895 (the annotations bug) as been fixed in Kotlin 1.3.10.

[Globegitter]

@pierreis @cgruber using the patched repo from https://github.com/Globegitter/rules_kotlin/tree/exports with 1.3.10 as well as manually adding "@com_github_jetbrains_kotlin//:kotlin-stdlib", as a dep to a kt_jvm_binary, or runtime_dep to a java_binary let's me compile and run an app with 1.3

Not sure why that is necessary but at least it is running.

[pierreis]

@Globegitter awesome, thanks.
Any fix planned from a repo maintainer?

[cgruber]

Yeah - thanks @Globegitter. No love from maintainers yet. I believe there are plans for more focus/ownership on the kotlin side, but it's all informal discussions I'm having at this point, with no firm commitments yet. To be fair, some people were out for personal reasons, and are only getting back recently. Hopefully we'll see more action soon.

[psartini]

@Globegitter thanks for your effort!

To be able to compile sourcecode with 1.3 features, I had to add the version in toolchains.bzl.
Not sure how to do this without setting the default version.

Index: kotlin/internal/toolchains.bzl
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- kotlin/internal/toolchains.bzl	(revision 170:95be9131aa2d04c6206a46c3b5077b507a1547f3)
+++ kotlin/internal/toolchains.bzl	(revision 170+:95be9131aa2d+)
@@ -80,18 +80,20 @@
         ),
         "language_version": attr.string(
             doc = "this is the -languag_version flag [see](https://kotlinlang.org/docs/reference/compatibility.html)",
-            default = "1.2",
+            default = "1.3",
             values = [
                 "1.1",
                 "1.2",
+                "1.3",
             ],
         ),
         "api_version": attr.string(
             doc = "this is the -api_version flag [see](https://kotlinlang.org/docs/reference/compatibility.html).",
-            default = "1.2",
+            default = "1.3",
             values = [
                 "1.1",
                 "1.2",
+                "1.3",
             ],
         ),
         "debug": attr.string_list(

Without this, I got an error:

error: the feature "inline classes" is only available since language version 1.3

[cgruber]

I will at least update my fork. I'm hoping thomas and co can come on-stream soon and start taking maintenance pulls here, even while they look at re-doing the whole ruleset.

[cgruber]

Sorry, I hvaen't updated my fork, as there are quite a few other issues going on, not hte least of which is that the 1.3 kotlinc package doesn't inclue kotlin-runtime (which is now in kotlin-stdlib) and so the rule needs some special handling in order to handle that case. I have prototyped it a bit, but in our square internal fork. Once I settle on an actual working version, I'll throw it into my fork.

It may not matter, insofar as the #174 suggests the google-internal rules may be open-sourced soon, but not knowing the timelines yet on availability, I'll probably still do it as a stop-gap, so folks can use the legacy rules with 1.3 until the harmonized super-awesome rules are in place.

[trevorsummerssmith]

Hi @cgruber we're hoping to upgrade to Kotlin 1.3 -- are you still planning on updating your fork with a working version? If so would be greatly appreciated.
Thanks!
Trevor

[cgruber]

Hey - sorry, I'm just working on our internal fork now, for 1.3 support. I can back-port what I do there to my repo, in the near-term. But also, Globegitter has a 1.3-based fork here: https://github.com/Globegitter/rules_kotlin/tree/exports

I haven't tried it, but it should work. Regardless, I'm deep in the structure of the present rules, making them 1.3 workable, and so once I have that fixed internally, I"ll externalize those changes. Hopefully I'll have something external by next week. Hoping for something working internally in the next couple of days.

[trevorsummerssmith]

@cgruber thanks so much I appreciate your update and work. Looking forward to seeing what you create.

[keith]

We've been using this fork, with this patch applied https://github.com/loreto/rules_kotlin/pull/1

[nfisher]

Seems org.jetbrains.kotlin:kotlin-runtime was moved/renamed to org.jetbrains.kotlin:std-lib. I'm guessing a bunch of other things have broken in the process too otherwise it would've been an easy fix.

I'm able to download the artifact with the following change:

_COMPILER_RELEASE = {
    "urls": [
        "https://github.com/JetBrains/kotlin/releases/download/v1.3.31/kotlin-compiler-1.3.31.zip",
    ],
    "sha256": "107325d56315af4f59ff28db6837d03c2660088e3efeb7d4e41f3e01bb848d6a",
}

kotlin_repositories(compiler_release=_COMPILER_RELEASE)

It is an incomplete build graph though yielding the error @Globegitter mentioned. Looking at the maven repo it looks like the std-lib has been back filled for all versions of Kotlin... in theory maybe that means it could be used as a maven_jar dep instead of kotlin-runtime? Slight waste in bandwidth perhaps but 🤷‍♂.

[cgruber]

Again, internally I have this fixed, and am under pretty intense deadline pressure. I'll try to push something out, but it's very likely that it won't support both 1.2 and 1.3 kotlinc. (That is, you can use kotlinc 1.3 in 1.2 language mode, but it is frustrating to make the change agnostic to the kotlinc version.

[shs96c]

I note with interest that the @Globegitter rules are failing for me when using bazel 0.25.2 with:

ERROR: /private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/internal/js/js.bzl:57:17: Traceback (most recent call last):
	File "/private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/internal/js/js.bzl", line 50
		rule(attrs = {"srcs": attr.label_list...")}, <4 more arguments>)
	File "/private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/internal/js/js.bzl", line 57, in rule
		attr.label_list(allow_files = True, default = [], cf...")
cfg must be either 'host' or 'target'.
ERROR: error loading package '': in /private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/kotlin.bzl: Extension file 'kotlin/internal/js/js.bzl' has errors
ERROR: error loading package '': in /private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/kotlin.bzl: Extension file 'kotlin/internal/js/js.bzl' has errors

and the "official" bazel rules don't yet support 1.3. Any chance of the merged ruleset being released in the near future?

[cgruber]

I'm actively (like, at this literal moment) working on making my fork work. I'm happy to port any of the changes back to the legacy branch, though I'll need a sponsor on the google side for reviews and merging.

[shs96c]

Thanks to @cgruber there's a way forward for people, but is there an ETA on the google version of these rules being updated? Or are there plans to merge @cgruber's rules_kotlin into this tree?

[timpeut]

Hi everyone,

The timelines for the new Kotlin rules have slipped. Due to these delays in releasing we’ve decided to accept @cgruber’s fork as a stop-gap measure to support the community.

We’re actively working with him to get this merged, and will be open to receiving pull requests for features and bug fixes until the Google rules are released.

Thank you for your patience through this process, and your support for an awesome Kotlin + Bazel experience!

[cgruber]

Pressure's on, but I'll be looking at starting this process next week.

[cgruber]

So... this week apparently. :) Sorry for the slip.

[cgruber]

Fixed by #205

[nkoroste]

Is there an ETA for a new release which will includes this fix?