java.lang.IllegalStateException from rule using new C++ toolchain + returning CcInfo

[jmillikin]

Using the new C++ Starlark API (#4570, #7036), I get a large Java stack trace when trying to build a hello world -ish cc_library equivalent.

Version + environment: Bazel 0.22rc1 on macOS 10.12.6.

Stack trace:

$ ./bazel-0.22.0rc1-darwin-x86_64 build //tests:hello_c_bin
Starting local Bazel server and connecting to it...
INFO: Invocation ID: a8940993-1551-474c-9650-a00917bd7219
Internal error thrown during build. Printing stack trace: java.lang.RuntimeException: Unrecoverable error while evaluating node '//tests:hello_c BuildConfigurationValue.Key[1e08ea8021436b7c999c252f16135df1] false' (requested by nodes '//tests:hello_c_bin BuildConfigurationValue.Key[1e08ea8021436b7c999c252f16135df1] false')
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:514)
	at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:368)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.localPopAndExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: java.lang.IllegalArgumentException: Invalid EvalException:
java.lang.IllegalStateException
	at com.google.common.base.Preconditions.checkState(Preconditions.java:491)
	at com.google.devtools.build.lib.rules.cpp.LinkerInputs$CompoundLibraryToLink.<init>(LinkerInputs.java:333)
	at com.google.devtools.build.lib.rules.cpp.LinkerInputs$CompoundLibraryToLink.<init>(LinkerInputs.java:287)
	at com.google.devtools.build.lib.rules.cpp.LinkerInputs.newInputLibrary(LinkerInputs.java:539)
	at com.google.devtools.build.lib.rules.cpp.LibraryToLinkWrapper.getPicStaticLibraryToLink(LibraryToLinkWrapper.java:990)
	at com.google.devtools.build.lib.rules.cpp.LibraryToLinkWrapper.buildStaticModeParamsForDynamicLibraryCcLinkParams(LibraryToLinkWrapper.java:886)
	at com.google.devtools.build.lib.rules.cpp.LibraryToLinkWrapper.toCcLinkingInfo(LibraryToLinkWrapper.java:346)
	at com.google.devtools.build.lib.rules.cpp.LibraryToLinkWrapper$CcLinkingContext.toCcLinkingInfo(LibraryToLinkWrapper.java:179)
	at com.google.devtools.build.lib.rules.cpp.CcInfo$Provider.createInfo(CcInfo.java:150)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at com.google.devtools.build.lib.syntax.MethodDescriptor.call(MethodDescriptor.java:134)
	at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:990)
	at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:897)
	at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:69)
	at com.google.devtools.build.lib.syntax.ListLiteral.doEval(ListLiteral.java:106)
	at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:69)
	at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:89)
	at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
	at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
	at com.google.devtools.build.lib.analysis.skylark.SkylarkRuleConfiguredTargetUtil.buildRule(SkylarkRuleConfiguredTargetUtil.java:123)
	at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createRule(ConfiguredTargetFactory.java:308)
	at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createConfiguredTarget(ConfiguredTargetFactory.java:207)
	at com.google.devtools.build.lib.skyframe.SkyframeBuildView.createConfiguredTarget(SkyframeBuildView.java:758)
	at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.createConfiguredTarget(ConfiguredTargetFunction.java:780)
	at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.compute(ConfiguredTargetFunction.java:326)
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:437)
	at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:368)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.localPopAndExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

	at com.google.devtools.build.lib.syntax.EvalException.<init>(EvalException.java:112)
	at com.google.devtools.build.lib.syntax.EvalException$EvalExceptionWithJavaCause.<init>(EvalException.java:209)
	at com.google.devtools.build.lib.syntax.EvalException$EvalExceptionWithJavaCause.<init>(EvalException.java:217)
	at com.google.devtools.build.lib.syntax.MethodDescriptor.call(MethodDescriptor.java:167)
	at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:990)
	at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:897)
	at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:69)
	at com.google.devtools.build.lib.syntax.ListLiteral.doEval(ListLiteral.java:106)
	at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:69)
	at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:89)
	at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
	at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
	at com.google.devtools.build.lib.analysis.skylark.SkylarkRuleConfiguredTargetUtil.buildRule(SkylarkRuleConfiguredTargetUtil.java:123)
	at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createRule(ConfiguredTargetFactory.java:308)
	at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createConfiguredTarget(ConfiguredTargetFactory.java:207)
	at com.google.devtools.build.lib.skyframe.SkyframeBuildView.createConfiguredTarget(SkyframeBuildView.java:758)
	at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.createConfiguredTarget(ConfiguredTargetFunction.java:780)
	at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.compute(ConfiguredTargetFunction.java:326)
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:437)
	... 6 more

rule (trimmed let me know if you want the whole thing):

load(
    "@bazel_tools//tools/build_defs/cc:action_names.bzl",
    _C_COMPILE = "C_COMPILE_ACTION_NAME",
)
def _cc_compile(ctx, source, header, output, use_pic):
    cc_toolchain = ctx.attr._cc_toolchain[cc_common.CcToolchainInfo]

    features = cc_common.configure_features(
        cc_toolchain = cc_toolchain,
    )
    cc_ctx = cc_common.create_compilation_context(
        headers = depset([header]),
    )
    cc_compiler = cc_common.get_tool_for_action(
        feature_configuration = features,
        action_name = _C_COMPILE,
    )
    cc_vars = cc_common.create_compile_variables(
        cc_toolchain = cc_toolchain,
        feature_configuration = features,
        source_file = source.path,
        output_file = output.path,
        use_pic = use_pic,
    )
    cc_argv = cc_common.get_memory_inefficient_command_line(
        feature_configuration = features,
        action_name = _C_COMPILE,
        variables = cc_vars,
    )
    cc_env = cc_common.get_environment_variables(
        feature_configuration = features,
        action_name = _C_COMPILE,
        variables = cc_vars,
    )
    ctx.actions.run(
        inputs = depset([source, header]) + ctx.attr._cc_toolchain[DefaultInfo].files,
        outputs = [output],
        executable = cc_compiler,
        arguments = cc_argv,
        mnemonic = "CppCompile",
        progress_message = "testing cc toolchain",
        env = cc_env,
    )

def _flex_lexer_impl(ctx):
    c_src = ctx.actions.declare_file("{}.c".format(ctx.attr.nam))
    c_hdr = ctx.actions.declare_file("{}.h".format(ctx.attr.name))

    # ... actions to create c_src and c_hdr ...
    
    out_obj = ctx.actions.declare_file("{}.o".format(ctx.attr.name))
    out_obj_pic = ctx.actions.declare_file("{}.pic.o".format(ctx.attr.name))
    _cc_compile(ctx, out_src, out_hdr, out_obj, use_pic = False)
    _cc_compile(ctx, out_src, out_hdr, out_obj_pic, use_pic = True)

    cc_toolchain = ctx.attr._cc_toolchain[cc_common.CcToolchainInfo]

    features = cc_common.configure_features(
        cc_toolchain = cc_toolchain,
    )
    linking_context = cc_common.create_linking_context(
        libraries_to_link = [cc_common.create_library_to_link(
            actions = ctx.actions,
            feature_configuration = features,
            cc_toolchain = cc_toolchain,
            static_library = out_obj,
            pic_static_library = out_obj_pic,
        )],
    )

    return CcInfo(linking_context = linking_context)

BUILD file:

load("//flex:flex.bzl", "flex_lexer")

flex_lexer(
    name = "hello_c",
    src = "hello.l",
)

cc_binary(
    name = "hello_c_bin",
    deps = [":hello_c"],
)

[oquenchil]

I see that you are passing a .o file. We want to eventually allow .o files to be passed to the linking API as well, but for now could you try to create a .a file with an action that calls the archiver?

[jmillikin]

Thanks -- performing a static link fixed the failing input validation crash, and allowed the resulting CcInfo to be successfully depended on by a cc_binary().

For those following along at home, here's the rules required to run ar to convert .o -> .a. I suspect some adjustments will be needed for toolchains with different file extensions, such as using .obj and .lib when building with MSVC.

    out_a = ctx.actions.declare_file("{}.a".format(ctx.attr.name))

    ar = cc_common.get_tool_for_action(
        feature_configuration = features,
        action_name = CPP_LINK_STATIC_LIBRARY_ACTION_NAME,
    )
    ar_vars = cc_common.create_link_variables(
        cc_toolchain = cc_toolchain,
        feature_configuration = features,
        output_file = out_a.path,
        is_using_linker = False,
        is_static_linking_mode=True,
    )
    ar_argv = cc_common.get_memory_inefficient_command_line(
        feature_configuration = features,
        action_name = CPP_LINK_STATIC_LIBRARY_ACTION_NAME,
        variables = ar_vars,
    )
    ar_env = cc_common.get_environment_variables(
        feature_configuration = features,
        action_name = CPP_LINK_STATIC_LIBRARY_ACTION_NAME,
        variables = ar_vars,
    )
    ctx.actions.run(
        inputs = depset([out_obj]) + ctx.attr._cc_toolchain[DefaultInfo].files,
        outputs = [out_a],
        executable = ar,
        arguments = ar_argv + [out_obj.path],
        mnemonic = "StaticLink",
        progress_message = "testing cc toolchain",
        env = ar_env,
    )

[hlopko]

We have a mechanism in the c++ toolchain how to configure the extension used, and setters are going to be exposed to Starlark for #5380 in https://source.bazel.build/bazel/+/2dccff01ae0711fa6729fd149ed66845ec6b0f54:tools/cpp/cc_toolchain_config_lib.bzl;l=509. TIL getters are not exposed, and they should be. If you need this, please file a new issue, and we'll reprioritize.

Thanks for testing the API!

[jmillikin]

OK, filed #7170 for file extension accessors.