Failing to build macos dylib

[matthewn4444]

System information (Please provide as much relevant information as possible)

• OS Platform and Distribution (e.g. Linux Ubuntu 16.04, Android 11, iOS 14.4): MacOS 11.4
• Compiler version (e.g. gcc/g++ 8 /Apple clang version 12.0.0): XCode 12.3
• Programming Language and version ( e.g. C++ 14, Python 3.6, Java ): bazel
• Installed using virtualenv? pip? Conda? (if python): 3.9.1
• MediaPipe version: top of tree or 0.8.3.1 (for 0.8.3.1 you need to modify WORKSPACE rules urls because they dead)
• Bazel version: bazel 3.7.2-homebrew
• XCode and Tulsi versions (if iOS): 12.3

Describe the problem:
I am trying to build a dylib of mediapipe for external use on MacOS platform. I am trying to build the framework and a cpu calculator from tensorflow, you can see my BUILD snippet below

load("@build_bazel_rules_apple//apple:macos.bzl", "macos_dylib")
macos_dylib (
  name = "mediapipe_framework.dylib",
    minimum_os_version = "10.3",
  deps = [
     ":mediapipe_framework",
     "@macos_opencv//:opencv",
  ],
)

cc_library(
    name = "mediapipe_framework",
    linkopts = select({
        "//conditions:default": [],
    }),
    visibility = ["//visibility:public"],
    deps = [
        "//mediapipe/framework:calculator_framework",
        "//mediapipe/framework/tool:calculator_graph_template_cc_proto",
        "//mediapipe/framework/port:logging",
        "//mediapipe/framework/formats:classification_cc_proto",
        "//mediapipe/framework/formats:landmark_cc_proto",
        "//mediapipe/framework/formats:detection_cc_proto",
        "//mediapipe/framework/formats:rect_cc_proto",

    ]  + select({
        "//conditions:default": [
            "//mediapipe/graphs/hand_tracking:mobile_calculators",
           "//mediapipe/graphs/object_detection:mobile_calculators",
        ],
    
        "//mediapipe/gpu:disable_gpu": [
            "//mediapipe/graphs/hand_tracking:desktop_tflite_calculators",
            "//mediapipe/graphs/object_detection:desktop_tflite_calculators",
        ],
    }),
    alwayslink = 1,
)

I am not sure why it doesnt work but then i get a message on top of tree master when building

DEBUG: Rule 'rules_foreign_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "c2cdcf55ffaf49366725639e45dedd449b8c3fe22b54e31625eb80ce3a240f1e"
DEBUG: Repository rules_foreign_cc instantiated at:
  /mediapipe/WORKSPACE:42:13: in <toplevel>
Repository rule http_archive defined at:
  /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
WARNING: Download from http://mirror.tensorflow.org/github.com/tensorflow/runtime/archive/44c890cd46fd9aad2b3bf4b13c5f36d8cff795c3.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'build_bazel_rules_apple' because it already exists.
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'build_bazel_rules_swift' because it already exists.
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'build_bazel_apple_support' because it already exists.
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'bazel_skylib' because it already exists.
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'pybind11' because it already exists.
DEBUG: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/tf_runtime/third_party/cuda/dependencies.bzl:51:10: The following command will download NVIDIA proprietary software. By using the software you agree to comply with the terms of the license agreement that accompanies the software. If you do not agree to the terms of the license agreement, do not use the software.
DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "02413e4a077107711ecc21458432e7c9ba63e75e63f3fa84ae4da2d6b2c82700"
DEBUG: Repository rules_cc instantiated at:
mediapipe/WORKSPACE:36:13: in <toplevel>
Repository rule http_archive defined at:
  /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
ERROR: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/cpuinfo/BUILD.bazel:100:11: Configurable attribute "srcs" doesn't match this configuration (would a default condition help?).
Conditions checked:
 @cpuinfo//:linux_x86_64
 @cpuinfo//:linux_arm
 @cpuinfo//:linux_armhf
 @cpuinfo//:linux_armv7a
 @cpuinfo//:linux_armeabi
 @cpuinfo//:linux_aarch64
 @cpuinfo//:linux_mips64
 @cpuinfo//:linux_riscv64
 @cpuinfo//:linux_s390x
 @cpuinfo//:macos_x86_64
 @cpuinfo//:macos_arm64
 @cpuinfo//:windows_x86_64
 @cpuinfo//:android_armv7
 @cpuinfo//:android_arm64
 @cpuinfo//:android_x86
 @cpuinfo//:android_x86_64
 @cpuinfo//:ios_x86_64
 @cpuinfo//:ios_x86
 @cpuinfo//:ios_armv7
 @cpuinfo//:ios_arm64
 @cpuinfo//:ios_arm64e
 @cpuinfo//:watchos_x86_64
 @cpuinfo//:watchos_x86
 @cpuinfo//:watchos_armv7k
 @cpuinfo//:watchos_arm64_32
 @cpuinfo//:tvos_x86_64
 @cpuinfo//:tvos_arm64
 @cpuinfo//:emscripten_wasm
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/google/ruy/archive/d37128311b445e758136b8602d1bbd2a755e115d.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
ERROR: Analysis of target '//mediapipe/external:mediapipe_framework.dylib2' failed; build aborted: /private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/cpuinfo/BUILD.bazel:100:11: Configurable attribute "srcs" doesn't match this configuration (would a default condition help?).
Conditions checked:
 @cpuinfo//:linux_x86_64
 @cpuinfo//:linux_arm
 @cpuinfo//:linux_armhf
 @cpuinfo//:linux_armv7a
 @cpuinfo//:linux_armeabi
 @cpuinfo//:linux_aarch64
 @cpuinfo//:linux_mips64
 @cpuinfo//:linux_riscv64
 @cpuinfo//:linux_s390x
 @cpuinfo//:macos_x86_64
 @cpuinfo//:macos_arm64
 @cpuinfo//:windows_x86_64
 @cpuinfo//:android_armv7
 @cpuinfo//:android_arm64
 @cpuinfo//:android_x86
 @cpuinfo//:android_x86_64
 @cpuinfo//:ios_x86_64
 @cpuinfo//:ios_x86
 @cpuinfo//:ios_armv7
 @cpuinfo//:ios_arm64
 @cpuinfo//:ios_arm64e
 @cpuinfo//:watchos_x86_64
 @cpuinfo//:watchos_x86
 @cpuinfo//:watchos_armv7k
 @cpuinfo//:watchos_arm64_32
 @cpuinfo//:tvos_x86_64
 @cpuinfo//:tvos_arm64
 @cpuinfo//:emscripten_wasm
INFO: Elapsed time: 0.179s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 0 targets configured)
    Fetching @clog; fetching
    Fetching @XNNPACK; fetching

I have also tried tensorflow/tensorflow#41039 to change the darwin value. Sometimes if i modify the cpuinfo bazel file I get errors to XNNPack not having dependencies on line 4278.

/private/var/tmp/_bazel_me/b39a9c2497bafb052e53d56396554a9a/external/XNNPACK/BUILD.bazel:4278:26: Configurable attribute "deps" doesn't match this configuration (would a default condition help?).

I believe it has something to do with XNNPack. I dont think I need XNNPack (or is it even supported?) on MacOS. This might be an issue with TensorFlow trying to build MacOS with XNNPack.

I also tried to change the macos_dylib to cc_binary to build a ".dylib" and it builts but I cannot test it because I built with a newer version of MacOS and I get linking issues related to using this library built with the newest version of MacOS (not even sure it works anyways) because I cannot set the minimum macos version unless I use macos_dylib or even apple_binary.

Any reason why this doesn't work?

**[Provide the exact sequence of commands / steps that you executed before running into the problem]

1. Clone mediapipe
2. Create an external folder in the root after mediapipe
3. Add the BUILD file with above build settings
4. Run command bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 --copt -fvisibility=hidden --linkopt -s --strip always --cxxopt=-std=c++17 --apple_platform_type=macos mediapipe/external:mediapipe_framework.dylib

[ahmadyan]

@sgowroji this looks like it is for building hand tracking on mac, not objectron.

[matthewn4444]

Yes I am trying to build it for mac which is where the issue lies.

[JacobMuchow]

Any success with this @matthewn4444 ? I'm trying to do something similar right now and am having the same issue.

[JacobMuchow]

This might be unrelated to macos_dylib itself, and maybe a bigger issue with compiling for macos/darwin config (?).

For reference, I get the same issue building with objc_library

The commented out line breaks the build.

bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 --config=darwin_x86_64 mediapipe/examples/desktop/objctest:MacOSTestObjcLibrary

Note: same result with --config=macos and not including --config as well.

My test.h/test.mm are really simple Obj-C files - a class with an NSString property. They work on their own. Adding the face_mesh calculator dependency busts the whole thing. Same error output as Matthew.

---

For more info, from the face_mesh desktop example I can get a binary that runs with:

bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1  mediapipe/examples/desktop/face_mesh:face_mesh_cpu

But adding --config=darwin_x86_64 gives me the same error output as above.

[tomleader]

@JacobMuchow I had met the same problem...I try to modify the --cpu option to darwin/darwin_x86_64 but still same error occur

[ROBYER1]

Think I am having the same issue on MacOS for both IOS and Mac targets

[AravindPrabhs]

Hi,

From the issue on tensorflow (here), it's a problem with the cpuinfo.

I managed to get my build compiling by adding a patch to tensorflow and manually editing XNNPack/cpuinfo.BUILD (when the error showed up). However simply renaming to 'darwin_x86_64' did not work (since setting cpu to either makes no difference), so I just removed the cpu dependency like below. This obviously isn't a good solution but I am unsure how the cpu variable is different being global in bazel when the cpuinfo build file is executed.

diff --git a/third_party/cpuinfo/BUILD.bazel b/third_party/cpuinfo/BUILD.bazel
index eb2937d2..d370ef53 100644
--- a/third_party/cpuinfo/BUILD.bazel
+++ b/third_party/cpuinfo/BUILD.bazel
@@ -236,7 +236,6 @@ config_setting(
     name = "macos_x86_64",
     values = {
         "apple_platform_type": "macos",
-        "cpu": "darwin",
     },
 )

[sureshdagooglecom]

Hi @matthewn4444 ,
Are you still looking for the resolution on this issue.

[matthewn4444]

I am not anymore, I gave up and moved onto another project because I didnt think it was possible

[alexbeattie42]

I'm hoping the mediapipe team addresses this issue soon. Until then, I wrote this article on how you can patch tensorflow so you can use @build_bazel_rules_apple rules with mediapipe.

[kuaashish]

Hello @matthewn4444,
We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediaPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions.

[github-actions]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[alexbeattie42]

Hello @kuaashish ,

Your response has appeared in many older mediapipe issues. Although there may be a new MediaPipe API, unless the MediaPipe team plans to remove tensorflow and bazel from MediaPipe, this issue will remain.

The full problem is described in the linked issue above.

TL;DR Fix

Add the --incompatible_enable_apple_toolchain_resolution flag to your build command and modify@build_bazel_rules_apple with the following patch:

diff --git a/apple/internal/transition_support.bzl b/apple/internal/transition_support.bzl
index 65f51b89..7da7ade3 100644
--- a/apple/internal/transition_support.bzl
+++ b/apple/internal/transition_support.bzl
@@ -102,6 +102,9 @@ def _cpu_string(*, cpu, platform_type, settings = {}):
             return "ios_sim_arm64"
         return "ios_x86_64"
     if platform_type == "macos":
+        host_cpu = settings["//command_line_option:host_cpu"]
+        if host_cpu == "darwin": 
+            return "darwin"
         if cpu:
             return "darwin_{}".format(cpu)
         macos_cpus = settings["//command_line_option:macos_cpus"]
@@ -342,6 +345,7 @@ _apple_rule_common_transition_inputs = [
     "//command_line_option:apple_crosstool_top",
 ]
 _apple_rule_base_transition_inputs = _apple_rule_common_transition_inputs + [
+    "//command_line_option:host_cpu",
     "//command_line_option:cpu",
     "//command_line_option:ios_multi_cpus",
     "//command_line_option:macos_cpus",

[schmidt-sebastian]

I am not able to reproduce the issue with the given BUILD file. I do however have to change the minimum iOS version to 11.x to get the build to succeed. I am using XCode 14.2.

That being said, the fix mentioned in #2324 (comment). I will follow up on bazelbuild/rules_apple#1954

Edit: I see that this only affects x86 Macs. I can try to repro this via our CI (which still has some Intel Macs).

[github-actions]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions]

This issue was closed due to lack of activity after being marked stale for past 7 days.