Unable to include s4nnc inside bazel project

Question

Unable to include s4nnc inside bazel project

davidw0311 opened this issue a year ago · 4 comments

When adding the dependency to s4nnc in my bazel project for iOS, my compilation errors with

ld: Undefined symbols:
  _cblas_dgemm, referenced from:
      _ccv_gemm in libccv_algebra.a[2](ccv_algebra.o)
  _cblas_sgemm, referenced from:
      __ccv_nnc_gbmm_and_bias in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gbmm_and_bias in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gbmm in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[10](_ccv_nnc_gemm_cpu_sys.o)
      ...

I am running on a Mac M2, and my WORKSPACE includes the following

git_repository(
    name = "s4nnc",
    commit = "f4a6c3255163a2acc9b75e8e850fd8e346667bae",
    remote = "https://github.com/liuliu/s4nnc.git"
)
load("@s4nnc//:deps.bzl", "s4nnc_deps")
s4nnc_deps()

My BUILD file is as follows:

load("@build_bazel_rules_apple//apple:ios.bzl", "ios_application")
load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")


swift_library(
    name = "diffusion",
    srcs = glob(["src/*.swift"]),
    module_name = "Diffusion",
    visibility = ["//visibility:public"],
    deps = [
        "@SwiftNumerics//:Numerics",
        "@s4nnc//nnc",
    ],
)

ios_application(
    name = "iOSApp",
    bundle_id = "build.bazel.rules-apple-example",
    families = [
        "iphone",
        "ipad",
    ],
    infoplists = ["Resources/Info.plist"],
    minimum_os_version = "17.0",
    visibility = ["//visibility:public"],
    deps = [":diffusion"],
)

any tips for debugging this would be greatly appreciated, thank you!

Answer 1 · 2024-02-06T17:24:00.000Z

Thanks for your interest!

On macOS, you need to explicitly set the ccv_settings. Here is mine (in WORKSPACE file just under s4nnc_deps() call):

git_repository(
    name = "s4nnc",
    commit = "37636f0f88452e5086bf2dc7bef671539f6cc40b",
    remote = "https://github.com/liuliu/s4nnc.git",
    shallow_since = "1706641861 -0500",
)

load("@s4nnc//:deps.bzl", "s4nnc_deps")

s4nnc_deps()

load("@ccv//config:ccv.bzl", "ccv_deps", "ccv_setting")

ccv_deps()

load("@build_bazel_rules_cuda//gpus:cuda_configure.bzl", "cuda_configure")
load("@build_bazel_rules_cuda//nccl:nccl_configure.bzl", "nccl_configure")

cuda_configure(name = "local_config_cuda")

nccl_configure(name = "local_config_nccl")

ccv_setting(
    name = "local_config_ccv",
    have_accelerate_framework = True,
    have_pthread = True,
)

load("@s4nnc//:deps.bzl", "s4nnc_extra_deps")

s4nnc_extra_deps() # This is optional, I believe it check in dependencies for PythonKit, Protobuf (TensorBoard support) etc.

Note that on CUDA enabled machine, it is a bit different and you can see swift-diffusion repo for how to setup there.

If you use NCHW tensor format throughout, you might want to disable Metal FlashAttention GEMM kernel and MixedMPSGEMM. To disable both flags:

DynamicGraph.flags = [.disableMixedMPSGEMM, .disableMFAGEMM]

Answer 2 · 2024-02-07T05:30:23.000Z

Thank you! I did include the ccv_setting for my build, this is my full WORKSPACE file
( I followed the instructions from rules_apple to deploy the bazel project for iOS) :

load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository", "new_git_repository")
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_python",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.4.0/rules_python-0.4.0.tar.gz",
    sha256 = "954aa89b491be4a083304a2cb838019c8b8c3720a7abb9c4cb81ac7a24230cea",
)

load("@rules_python//python:pip.bzl", "pip_install")

git_repository(
    name = "s4nnc",
    commit = "37636f0f88452e5086bf2dc7bef671539f6cc40b",
    remote = "https://github.com/liuliu/s4nnc.git",
    shallow_since = "1706641861 -0500",
)

load("@s4nnc//:deps.bzl", "s4nnc_deps")

s4nnc_deps()

load("@ccv//config:ccv.bzl", "ccv_deps", "ccv_setting")

ccv_deps()

load("@build_bazel_rules_cuda//gpus:cuda_configure.bzl", "cuda_configure")
load("@build_bazel_rules_cuda//nccl:nccl_configure.bzl", "nccl_configure")

cuda_configure(name = "local_config_cuda")

nccl_configure(name = "local_config_nccl")

ccv_setting(
    name = "local_config_ccv",
    have_accelerate_framework = True,
    have_pthread = True,
)

load("@s4nnc//:deps.bzl", "s4nnc_extra_deps")

s4nnc_extra_deps()

git_repository(
    name = "swift-fickling",
    commit = "296c8eb774332a3a49c8c403fdbec373d9fb2f96",
    remote = "https://github.com/liuliu/swift-fickling.git",
    shallow_since = "1675031846 -0500",
)

load("@swift-fickling//:deps.bzl", "swift_fickling_deps")

swift_fickling_deps()

git_repository(
    name = "swift-sentencepiece",
    commit = "2c4ec57bea836f8b420179ee7670304a4972c572",
    remote = "https://github.com/liuliu/swift-sentencepiece.git",
    shallow_since = "1683864360 -0400",
)

load("@swift-sentencepiece//:deps.bzl", "swift_sentencepiece_deps")

swift_sentencepiece_deps()

new_git_repository(
    name = "SwiftPNG",
    build_file = "swift-png.BUILD",
    commit = "075dfb248ae327822635370e9d4f94a5d3fe93b2",
    remote = "https://github.com/kelvin13/swift-png",
    shallow_since = "1645648674 -0600",
)

new_git_repository(
    name = "ZIPFoundation",
    build_file = "zip-foundation.BUILD",
    commit = "642436f3684009ca7a5e3d6b30f2ecea26f8f772",
    remote = "https://github.com/weichsel/ZIPFoundation.git",
    shallow_since = "1665504317 +0200",
)

git_repository(
    name = "build_bazel_rules_swift",
    commit = "3bc7bc164020a842ae08e0cf071ed35f0939dd39",
    remote = "https://github.com/bazelbuild/rules_swift.git",
    shallow_since = "1654173801 -0500",
)


new_git_repository(
    name = "SwiftArgumentParser",
    build_file = "swift-argument-parser.BUILD",
    commit = "9f39744e025c7d377987f30b03770805dcb0bcd1",
    remote = "https://github.com/apple/swift-argument-parser.git",
    shallow_since = "1661571047 -0500",
)

new_git_repository(
    name = "SwiftSystem",
    build_file = "swift-system.BUILD",
    commit = "025bcb1165deab2e20d4eaba79967ce73013f496",
    remote = "https://github.com/apple/swift-system.git",
    shallow_since = "1654977448 -0700",
)

new_git_repository(
    name = "SwiftToolsSupportCore",
    build_file = "swift-tools-support-core.BUILD",
    commit = "286b48b1d73388e1d49b2bb33aabf995838104e3",
    remote = "https://github.com/apple/swift-tools-support-core.git",
    shallow_since = "1670947584 -0800",
)

new_git_repository(
    name = "SwiftSyntax",
    build_file = "swift-syntax.BUILD",
    commit = "cd793adf5680e138bf2bcbaacc292490175d0dcd",
    remote = "https://github.com/apple/swift-syntax.git",
    shallow_since = "1676877517 +0100",
)

new_git_repository(
    name = "SwiftFormat",
    build_file = "swift-format.BUILD",
    commit = "9f1cc7172f100118229644619ce9c8f9ebc1032c",
    remote = "https://github.com/apple/swift-format.git",
    shallow_since = "1676404655 +0000",
)

new_git_repository(
    name = "SwiftProtobuf",
    build_file = "swift-protobuf.BUILD",
    commit = "7cbb5279dd7e997c8f0f5537e46d4513be894ff1",
    remote = "https://github.com/apple/swift-protobuf.git",
    shallow_since = "1658527939 -0700",
)



# buildifier is written in Go and hence needs rules_go to be built.
# See https://github.com/bazelbuild/rules_go for the up to date setup instructions.

http_archive(
    name = "io_bazel_rules_go",
    sha256 = "099a9fb96a376ccbbb7d291ed4ecbdfd42f6bc822ab77ae6f1b5cb9e914e94fa",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.35.0/rules_go-v0.35.0.zip",
        "https://github.com/bazelbuild/rules_go/releases/download/v0.35.0/rules_go-v0.35.0.zip",
    ],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_rules_dependencies()

go_register_toolchains(version = "1.19.1")

http_archive(
    name = "bazel_gazelle",
    sha256 = "501deb3d5695ab658e82f6f6f549ba681ea3ca2a5fb7911154b5aa45596183fa",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.26.0/bazel-gazelle-v0.26.0.tar.gz",
        "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.26.0/bazel-gazelle-v0.26.0.tar.gz",
    ],
)

load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")

gazelle_dependencies()

git_repository(
    name = "com_github_bazelbuild_buildtools",
    commit = "174cbb4ba7d15a3ad029c2e4ee4f30ea4d76edce",
    remote = "https://github.com/bazelbuild/buildtools.git",
    shallow_since = "1607975103 +0100",
)


############# apple rules WORKSPACE file #############

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "build_bazel_rules_apple",
    sha256 = "9c4f1e1ec4fdfeac5bddb07fa0e872c398e3d8eb0ac596af9c463f9123ace292",
    url = "https://github.com/bazelbuild/rules_apple/releases/download/3.2.1/rules_apple.3.2.1.tar.gz",
)

load(
    "@build_bazel_rules_apple//apple:repositories.bzl",
    "apple_rules_dependencies",
)

apple_rules_dependencies()

load(
    "@build_bazel_rules_swift//swift:repositories.bzl",
    "swift_rules_dependencies",
)

swift_rules_dependencies()

load(
    "@build_bazel_rules_swift//swift:extras.bzl",
    "swift_rules_extra_dependencies",
)

swift_rules_extra_dependencies()

load(
    "@build_bazel_apple_support//lib:repositories.bzl",
    "apple_support_dependencies",
)

apple_support_dependencies()

###########################################################

and here is my BUILD file:

load("@build_bazel_rules_apple//apple:ios.bzl", "ios_application")
load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")

swift_library(
    name = "diffusion",
    srcs = glob(["src/*.swift"]),
    module_name = "Diffusion",
    visibility = ["//visibility:public"],
    data = [
        "Resources/Main.storyboard",
    ],
    deps = [
        "@SwiftNumerics//:Numerics",
        "@s4nnc//nnc",
    ],
)


ios_application(
    name = "iOSApp",
    bundle_id = "build.bazel.rules-apple-example",
    families = [
        "iphone",
        "ipad",
    ],
    infoplists = ["Resources/Info.plist"],
    minimum_os_version = "16.0",
    visibility = ["//visibility:public"],
    deps = [":diffusion"],
)

However, when I try to build the app, I am running into the following error:

Linking iOSApp_bin failed: (Exit 1): wrapped_clang failed: error executing command (from target //:iOSApp) external/local_config_cc/wrapped_clang @bazel-out/ios-sim_arm64-min16.0-applebin_ios-ios_sim_arm64-fastbuild-ST-af7ff2f43918/bin/iOSApp_bin-2.params

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
ld: Undefined symbols:
  _OBJC_CLASS_$_MPSCommandBuffer, referenced from:
       in libnnc_mps_compat.a[2](ccv_nnc_mps.o)
  _OBJC_CLASS_$_MPSGraph, referenced from:
       in libcmd_mps.a[4](ccv_nnc_add_mps.o)
  _OBJC_CLASS_$_MPSGraphCompilationDescriptor, referenced from:
       in libnnc_mps_compat.a[2](ccv_nnc_mps.o)
  _OBJC_CLASS_$_MPSGraphConvolution2DOpDescriptor, referenced from:
       in libcmd_mps.a[8](ccv_nnc_conv_mps.o)
  _OBJC_CLASS_$_MPSGraphDevice, referenced from:
       in libnnc_mps_compat.a[2](ccv_nnc_mps.o)
  _OBJC_CLASS_$_MPSGraphExecutionDescriptor, referenced from:
       in libnnc_mps_compat.a[2](ccv_nnc_mps.o)
  _OBJC_CLASS_$_MPSGraphPooling2DOpDescriptor, referenced from:
       in libcmd_mps.a[19](ccv_nnc_avg_pool_mps.o)
  _OBJC_CLASS_$_MPSGraphRandomOpDescriptor, referenced from:
       in libcmd_mps.a[21](ccv_nnc_rand_normal_mps.o)
  _OBJC_CLASS_$_MPSGraphShapedType, referenced from:
       in libcmd_mps.a[2](ccv_nnc_adam_mps.o)
  _OBJC_CLASS_$_MPSGraphTensorData, referenced from:
       in libnnc_mps_compat.a[2](ccv_nnc_mps.o)
  _OBJC_CLASS_$_MPSMatrix, referenced from:
       in libcmd_mps.a[6](ccv_nnc_gemm_mps.o)
  _OBJC_CLASS_$_MPSMatrixDescriptor, referenced from:
       in libcmd_mps.a[6](ccv_nnc_gemm_mps.o)
  _OBJC_CLASS_$_MPSMatrixMultiplication, referenced from:
       in libcmd_mps.a[6](ccv_nnc_gemm_mps.o)
  _OBJC_CLASS_$_MPSMatrixNeuron, referenced from:
       in libcmd_mps.a[6](ccv_nnc_gemm_mps.o)
  _OBJC_CLASS_$_MPSMatrixSoftMax, referenced from:
       in libcmd_mps.a[32](ccv_nnc_softmax_mps.o)
  _OBJC_CLASS_$_MPSVector, referenced from:
       in libcmd_mps.a[6](ccv_nnc_gemm_mps.o)
  _OBJC_CLASS_$_MPSVectorDescriptor, referenced from:
       in libcmd_mps.a[6](ccv_nnc_gemm_mps.o)
  _cblas_dgemm, referenced from:
      _ccv_gemm in libccv_algebra.a[2](ccv_algebra.o)
  _cblas_sgemm, referenced from:
      __ccv_nnc_gbmm_and_bias in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gbmm_and_bias in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gbmm in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      __ccv_nnc_gemm_back_cpu_sys in libcmd.a[12](_ccv_nnc_gemm_cpu_sys.o)
      ...
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error in child process '/usr/bin/xcrun'. 1
Target //:iOSApp failed to build

Is there some other configuration which I am missing? Thanks!

Answer 3 · 2024-02-07T05:34:58.000Z

Ha! Great you've made it this far already. Yes, Bazel is weird that the "automatic" framework inclusion available in Xcode doesn't work for iOS apps. So you need to include these manually. Here is what we do in Draw Things app's target:

ios_application(
# All your normal stuff.
    sdk_frameworks = [
        "Accelerate",
        "Metal",
        "MetalPerformanceShaders",
        "MetalPerformanceShadersGraph",
    ]
)

You will notice that this is not an issue if you just build an ordinary swift_binary target due to automatic framework inclusion.

Answer 4 · 2024-02-07T05:52:39.000Z

I see! That solves the issue thank you!