.swift{module,doc} files not included in framework

[steeve]

When building an ios_framework or ios_static_framework around a swift_library, several issues arise:

• module.modulemap is only created if there are hdrs in the bundle or objc deps with sdk_framework
• swiftmodule and swiftdoc files are not bundled

In the end, the XCode complains that the framework is not useable.

[steeve]

Here is my framework layout:

MyFramework.framework
MyFramework.framework/MyFramework
MyFramework.framework/Headers
MyFramework.framework/Headers/MyFrameworkObjc.h
MyFramework.framework/Headers/MyFramework.h
MyFramework.framework/Modules
MyFramework.framework/Modules/module.modulemap

It was created like this:

swift_library(
    name = "MyFramework",
    srcs = glob(["*.swift"]),
    copts = [
        "-embed-bitcode",  # build with bitcode
    ],
    module_name = "MyFramework",
    visibility = ["//visibility:public"],
)

ios_static_framework(
    name = "MyFramework.framework",
    hdrs = [
        "MyFrameworkObjc.h",
    ],
    bundle_name = "MyFramework",
    families = [
        "iphone",
        "ipad",
    ],
    minimum_os_version = "10.0",
    visibility = ["//visibility:public"],
    deps = [
        ":MyFramework",
    ],
)

Note that doing a ios_framework doesn't help either.

[steeve]

If I remove the Objc.h dummy header, there is only the MyFramework archive, no modulemap or no umbrella header (which XCode complains about).

I'm unable to import it, then.

[steeve]

Here is what I'm expecting:

MyFramework.framework/
MyFramework.framework/MyFramework
MyFramework.framework/Headers
MyFramework.framework/Headers/MyFramework-Swift.h
MyFramework.framework/Headers/MyFramework.h
MyFramework.framework/Modules
MyFramework.framework/Modules/module.modulemap
MyFramework.framework/Modules/MyFramework.swiftmodule
MyFramework.framework/Modules/MyFramework.swiftmodule/x86_64.swiftdoc
MyFramework.framework/Modules/MyFramework.swiftmodule/x86_64.swiftmodule
MyFramework.framework/Modules/MyFramework.swiftmodule/arm.swiftdoc
MyFramework.framework/Modules/MyFramework.swiftmodule/arm.swiftmodule
MyFramework.framework/Modules/MyFramework.swiftmodule/arm64.swiftdoc
MyFramework.framework/Modules/MyFramework.swiftmodule/arm64.swiftmodule

[sergiocampama]

yes, swift_library is not yet supported on ios_static_framework. The only workaround that I can think of is somehow making an ipa_post_processor that creates and adds the missing files.

[steeve]

Does it support ios_multi_cpus for swift_library ?
EDIT: after trying it and digging the split does indeed happen.

I'm guessing it's going to be simpler to add it in rules_apple rather than escape the sandbox to find the swift{doc,module} files in the bazel build folder

[sergiocampama]

The tricky part is that each swift_library creates a module, so you'll need to collate all the transitive swift_library targets and gather their swiftmodule files. At that point, you'll be providing all of those modules as part of the framework, and there will be multiple entry points into the symbols provided by the framework. I'm not sure if there's a way to create an umbrella module that proxies all the underlying modules so that there is a single entrypoint.

[allevato]

Multi-arch Swift modules in a framework are represented as a directory instead of a single file:

Foo.swiftmodule/
  armv7.swiftmodule
  arm64.swiftmodule
  i386.swiftmodule
  x86_64.swiftmodule

So theoretically, an ios_framework rule that supported Swift modules would need to collect the module files from the various split transitions and rearrange them into that format, with the correct names.

Fundamentally, however, to address the original issue, this is another side effect of the fact that ios_framework is currently designed with different use cases in mind. Not creating distributable frameworks, but rather creating app-specific frameworks that allow code to be shared across targets in the same app. Under those circumstances, packaging the .swiftmodule files isn't something that needs to happen.

(ios_static_framework, on the other hand, is meant for creating distributable bundles, but Swift was also not something the original authors were interested in so it didn't come up.)

[steeve]

In the end I got it working without a post processor because the rule needs a split configuration, and the post processor is compiled with the host configuration (as it should be).

What I'm doing is leveraging SwiftInfo, apple_common.multi_arch_split and zipper to repack the framework with the swift{doc,module} inside the framework.

I'm also doing a pre-processor to extract the -Swift.h headers (thanks to apple_common.Objc) and add it the ios_static_framework invocation.

It may not handle all the cases, but it's working fine for my needs.

I didn't feel like patching rules_apple because it was kind of scary with my timeframe.

I cannot yet share the code as it's part of our internal bazel rules.

[keith]

I'm looking into this now but I'm not entirely sure how to collect the multiple swiftmodule files in order to output them in the expected format shown above.

I'm able to access the SwiftInfo from the ios_static_framework's deps, but this only vends a single arch in direct_swiftmodules. I'm not clear on how split transitions are related to this, but is there a way for us to access each of these from the skylark rules?

[keith]

It looks like ctx.split_attr in this rule

rules_apple/apple/bundling/ios_rules.bzl

Line 402 in 0e593c7

def _ios_static_framework_impl(ctx):

is a empty struct

[steeve]

Here is my rule:

load("@build_bazel_rules_swift//swift:swift.bzl", "SwiftInfo")
load("@build_bazel_rules_apple//apple:providers.bzl", "AppleBundleInfo")
load(
    "@build_bazel_rules_apple//apple:ios.bzl",
    _ios_static_framework = "ios_static_framework",
)


_CPUS = {
    "ios_armv7": "arm",
    "ios_arm64": "arm64",
    "ios_i386": "x86",
    "ios_x86_64": "x86_64",
}

_SCRIPT = """\
{zipper} x {framework}
{zipper} c {new_framework} $(find {framework_name} -type f) $@
rm -rf {framework}
"""

def _module_zipper_arg(framework, module_name, cpu, file):
    return "{framework}/Modules/{module_name}.swiftmodule/{cpu}.{ext}={file_path}".format(
        framework = framework,
        module_name = module_name,
        cpu = cpu,
        ext = file.extension,
        file_path = file.path,
    )

def _objc_headers_impl(ctx):
    headers = []
    for dep in ctx.attr.deps:
        objc_headers = dep[apple_common.Objc].header.to_list()
        for hdr in objc_headers:
            if hdr.owner == dep.label:
                headers.append(hdr)
    return [
        DefaultInfo(
            files = depset(headers),
        ),
    ]

_objc_headers = rule(
    _objc_headers_impl,
    attrs = {
        "deps": attr.label_list(
            providers = [SwiftInfo],
        ),
    },
)

def _framework_swift_postprocess_impl(ctx):
    bundle_info = ctx.attr.framework[AppleBundleInfo]
    framework_name = bundle_info.bundle_name + bundle_info.bundle_extension
    new_framework = ctx.actions.declare_file(ctx.label.name + ".zip")
    infiles = [
        ctx.file.framework,
    ]
    zipper_args = []
    for arch, deps in ctx.split_attr.deps.items():
        cpu = _CPUS.get(arch)
        if not cpu:
            continue
        for d in deps:
            objc_info = d[apple_common.Objc]
            swift_info = d[SwiftInfo]
            swiftmodule = swift_info.direct_swiftmodules[0]
            swiftdoc = swift_info.direct_swiftdocs[0]
            infiles.extend([swiftmodule, swiftdoc])
            zipper_args.extend([
                _module_zipper_arg(framework_name, swift_info.module_name, cpu, swiftmodule),
                _module_zipper_arg(framework_name, swift_info.module_name, cpu, swiftdoc),
            ])

    ctx.actions.run_shell(
        inputs = infiles,
        outputs = [new_framework],
        mnemonic = "SwiftFrameworkPostProcess",
        progress_message = "Postprocessing %s for Swift support" % framework_name,
        command = _SCRIPT.format(
            framework = ctx.file.framework.path,
            framework_name = framework_name,
            new_framework = new_framework.path,
            zipper = ctx.executable._zipper.path,
        ),
        arguments = zipper_args,
        tools = [
            ctx.executable._zipper,
        ],
    )
    return [
        DefaultInfo(
            files = depset([new_framework]),
        ),
    ]

_framework_swift_postprocess = rule(
    _framework_swift_postprocess_impl,
    attrs = {
        "platform_type": attr.string(),
        "minimum_os_version": attr.string(),
        "framework": attr.label(
            providers = [AppleBundleInfo],
            allow_single_file = True,
        ),
        "deps": attr.label_list(
            providers = [SwiftInfo],
            cfg = apple_common.multi_arch_split,
        ),
        "_zipper": attr.label(
            default = "@bazel_tools//tools/zip:zipper",
            cfg = "host",
            executable = True,
        ),
    },
)

def ios_static_framework(name, minimum_os_version, hdrs=[], deps=[], visibility=None, **kwargs):
    _objc_headers(
        name = name + ".hdrs",
        deps = deps,
    )
    _ios_static_framework(
        name = name + ".intermediate",
        hdrs = hdrs + [name + ".hdrs"],
        minimum_os_version = minimum_os_version,
        deps = deps,
        **kwargs
    )
    _framework_swift_postprocess(
        name = name,
        platform_type = "ios",
        minimum_os_version = minimum_os_version,
        framework = name + ".intermediate",
        deps = deps,
        visibility = visibility,
)
    ```

[steeve]

Forgive my poor copy paste as I'm on my phone

[keith]

Thanks for sharing. It looks like I'm currently not hitting the codepath I expected here

rules_apple/apple/bundling/rule_factory.bzl

Lines 476 to 489 in 0e593c7

	else:
	binary_dep_attrs = {
	"deps": attr.label_list(
	aspects = [
	apple_bundling_aspect,
	swift_usage_aspect,
	new_apple_resource_aspect,
	framework_import_aspect,
	],
	cfg = apple_common.multi_arch_split,
	),
	# Required by apple_common.multi_arch_split on 'deps'.
	"platform_type": attr.string(mandatory = True),
	}

because use_binary_rule is always True for ios_static_framework

[kastiglione]

@steeve do you use slack? There's a slack channel that's focused on Bazel for iOS / swift, if you're interested in joining.

[steeve]

I do, happy to join in :)

[sergiocampama]

@kastiglione mind inviting me as well? :)

[keith]

@steeve looks like you have a typo in your _CPUS dictionary "ios_i386": "x86", for swiftmodule's should be i386 instead I think?

[steeve]

@keith not sure, do you have any doc/example?

[keith]

If it works I guess everything is fine, but if you build a framework for a 32 bit simulator in Xcode you end up with i386.swiftdoc and i386.swiftmodule

[steeve]

Ah ! So indeed it's a typo of mine

On Fri 5 Oct 2018 at 01:51, Keith Smiley ***@***.***> wrote: If it works I guess everything is fine, but if you build a framework for a 32 bit simulator in Xcode you end up with i386.swiftdoc and i386.swiftmodule — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#223 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIY-yBLAAHkrn-rzVPu-cgc9AVprQczks5uhp8bgaJpZM4WVTeV> .

-- twitter.com/steeve github.com/steeve linkd.in/smorin

[keith]

We've pushed the rule we've started using for this here https://github.com/ios-bazel-users/ios-bazel-users/tree/master/prebuilt_swift_static_framework

[thii]

@keith Were you able to use static frameworks built with the prebuilt_swift_static_framework rule from Objective-C code? Looks like the -Swift.h header and module.modulemap are not bundled.

[keith]

We don't have that use case in our code, but if you added that to the files being zipped it should work fine!

[thii]

Thanks. I'll try to figure that out.

[sergiocampama]

#355 for the ios_framework part. Keeping this open to track the ios_static_framework part.

[keith]

FYI at this point swiftinterface and swiftdoc files are in the ios_static_framework, so this issue as it currently stands might be closable. But there might need to be a new issue for #223 (comment)

[keith]

I think the original issue is fixed here, if there are still issues with ObjC interop we should open a new issue