cmd/go: cgo builds don't work with "zig cc"

Question

cmd/go: cgo builds don't work with "zig cc"

andrewrk opened this issue 4 years ago · 20 comments

Hello, I am filing this issue on behalf of an issue reported upstream here: ziglang/zig#7342

@lu4p writes:

I would like to use "zig cc" for easily cross compiling cgo dependent go code, however the "drop-in" c compiler replacement doesn't work.

Example cgo code

main.go

package main

//int Add(int a, int b){
//    return a+b;
//}
import "C"
import "fmt"

func main() {
	a := C.int(10)
	b := C.int(20)
	c := C.Add(a, b)
	fmt.Println(c) // 30
}

other compilers

go build main.go //build works fine (uses gcc)
CC="clang" go build main.go //build works fine (uses clang)

zig cc

$ CC="zig cc" go build main.go
# runtime/cgo
info: Usage: zig [command] [options]

Commands:

  build            Build project from build.zig
  build-exe        Create executable from source or object files
  build-lib        Create library from source or object files
  build-obj        Create object from source or assembly
  cc               Use Zig as a drop-in C compiler
  c++              Use Zig as a drop-in C++ compiler
  env              Print lib path, std path, compiler id and version
  fmt              Parse file and render in canonical zig format
  init-exe         Initialize a `zig build` application in the cwd
  init-lib         Initialize a `zig build` library in the cwd
  libc             Display native libc paths file or validate one
  run              Create executable and run immediately
  translate-c      Convert C code to Zig code
  targets          List available compilation targets
  test             Create and run a test build
  version          Print version number and exit
  zen              Print zen of zig and exit

General Options:

  --help           Print command-specific usage

error: unknown command: -E

go version go1.15.5 linux/amd64

So this looks like go is inserting flags before it passes cc. The user will have to work around this by creating a shell script like this:

#!/bin/sh
zig cc $@

And then passing that as CC. This way zig gets invoked with zig cc -E ... instead of zig -E ...

This issue is a feature request for go to honor the entire command line in CC, appending flags rather than prepending them. Or, at least adding explicit detection for zig cc and passing the cc as the first arg.

seankhliao commented 4 years ago

oops

😄1

Answer 1 · 2020-12-08T20:53:27.000Z

This seems closely related to #41400, but with the extra caveat that the first argument is not actually the full C compiler command.

Answer 2 · 2020-12-08T20:53:59.000Z

CC (ha!) @ianlancetaylor @jayconrod @matloob

Answer 3 · 2020-12-08T21:27:39.000Z

Related topic, can anyone help shed some light on why zig cc gets EACCES when it tries to create a project-local zig-cache/ directory?

mkdirat(AT_FDCWD, "zig-cache", 0755 = -1 EACCES (Permission denied)

Edit: hmm, it looks like go changes the cwd when invoking the c compiler for some reason.

Answer 4 · 2020-12-08T21:55:26.000Z

@andrewrk run go build -x to show the build commands
It should be cding to a temp dir before running cc

When I do CC="gcc gcc2" go build -x (ignore the invalid CC string)
I get ... TERM='dumb' gcc -I "c:\\go\\src\\runtime\\cgo" gcc2 -m64 -mthreads -fmessage-length=0 "-fdebug-prefix-map=$WORK\\b057=/tmp/go-build" -gno-record-gcc-switches -I "$WORK\\b057\\" -g -O2 -Wall -Werror -o "$WORK\\b057\\_x001.o" -c _cgo_export.c

After skimming the CL for #41400 it does look like the fix would be in the same places

Answer 5 · 2020-12-08T22:07:49.000Z

@bcmills @ianlancetaylor

~~If we use get rid of string splitting for CC/CXX completely (return []string{os.Getenv("CC")}) we could prob fix this and #41400 without too much complexity.~~

EDIT: nvm forgot that exec needs to split out the exe

Answer 6 · 2020-12-08T22:18:39.000Z

@andrewrk run go build -x to show the build commands

Hmm I got this:

[nix-shell:~/misc/gozig]$ CC="zcc" go build -x main.go
WORK=/run/user/1000/go-build735552327
mkdir -p $WORK/b001/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile command-line-arguments=/home/andy/.cache/go-build/24/2472355c95ff5bd2c8379130112402791828be5fa411080b10c62f6c57556a65-d
packagefile fmt=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/fmt.a
packagefile runtime/cgo=/home/andy/.cache/go-build/9b/9b3a75664ee02a7fb873a1dc93cf7958c170dedef5fc1c46e2de82b40856bdec-d
packagefile syscall=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/syscall.a
packagefile runtime=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime.a
packagefile errors=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/errors.a
packagefile internal/fmtsort=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/fmtsort.a
packagefile io=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/io.a
packagefile math=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/math.a
packagefile os=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/os.a
packagefile reflect=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/reflect.a
packagefile strconv=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/strconv.a
packagefile sync=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/sync.a
packagefile unicode/utf8=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/unicode/utf8.a
packagefile internal/bytealg=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/bytealg.a
packagefile internal/oserror=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/oserror.a
packagefile internal/race=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/race.a
packagefile internal/unsafeheader=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/unsafeheader.a
packagefile internal/cpu=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/cpu.a
packagefile runtime/internal/atomic=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime/internal/atomic.a
packagefile runtime/internal/math=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime/internal/math.a
packagefile runtime/internal/sys=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime/internal/sys.a
packagefile internal/reflectlite=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/reflectlite.a
packagefile sort=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/sort.a
packagefile math/bits=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/math/bits.a
packagefile internal/poll=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/poll.a
packagefile internal/syscall/execenv=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/syscall/execenv.a
packagefile internal/syscall/unix=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/syscall/unix.a
packagefile internal/testlog=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/testlog.a
packagefile sync/atomic=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/sync/atomic.a
packagefile time=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/time.a
packagefile unicode=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/unicode.a
EOF
mkdir -p $WORK/b001/exe/
cd .
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=YCcuUkfsSqMNZpM94Hzz/WNU9rIi2KCHzwXiokjH7/pwW5amiz-sk8p-Zpyuhd/YCcuUkfsSqMNZpM94Hzz -extld=zcc /home/andy/.cache/go-build/24/2472355c95ff5bd2c8379130112402791828be5fa411080b10c62f6c57556a65-d
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/buildid -w $WORK/b001/exe/a.out # internal
# command-line-arguments
warning: unsupported linker arg: --compress-debug-sections=zlib-gnu
cp $WORK/b001/exe/a.out main
rm -r $WORK/b001/

I don't see the zig cc build commands

Answer 7 · 2020-12-08T22:20:18.000Z

@andrewrk if it's been build cached you'd need to run go build -x -a to force a rebuild

Answer 8 · 2020-12-08T23:01:47.000Z

Change https://golang.org/cl/276412 mentions this issue: cmd/go: Retain CC/CXX arg order

Answer 9 · 2020-12-08T23:11:21.000Z

OK here's what I'm seeing:

~/my-project-directory$ CC="zcc" go build -x -a main.go
...
cd /nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/src/syscall
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/asm -p syscall -trimpath "$WORK/b029=>" -I $WORK/b029/ -I /nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/include -D GOOS_linux -D GOARCH_amd64 -o $WORK/b029/asm_linux_amd64.o ./asm_linux_amd64.s
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/pack r $WORK/b029/_pkg_.a $WORK/b029/asm_linux_amd64.o # internal
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/buildid -w $WORK/b029/_pkg_.a # internal
# runtime/cgo
attempt to unwrap error: ReadOnlyFileSystem
...

User runs command with cwd set to $HOME/my-project-directory
Go changes the cwd to /share/go/src/syscall
Zig does this:

mkdirat(AT_FDCWD, "zig-cache", 0755) = -1 EROFS (Read-only file system)

Although go does its own caching, it would be best for the user experience to allow zig to do its own caching as well, for a couple reasons:

There is a global cache as well, which speeds up linking for subsequent builds for the same target, even for different projects. This is for things such as libc and compiler_rt. Actually this part is working fine though, it is the local cache that is problematic.
zig cc does in fact allow .zig files to be thrown into the mix, and zig does its own caching of that part of the compilation. In this use case the user would do something like CC=zig cc foo.zig.
Zig disables its own per-file caching when the -MD flags (et all) are used, which Go is passing, so it actually integrates kind of perfectly.

I can open another issue for this but some informal guidance on how the go project sees this use case would be useful too. My question is, is this to be expected that the cwd should be completely ignored by the C compiler?

Answer 10 · 2020-12-10T06:25:15.000Z

Just a passing thought, but assuming that the compiler's working directory is writeable might be flawed? At least, it seems reasonable enough to execute a C compiler from a non-writeable directory (maybe you're compiling sources in someone else's home directory?) as long as the output file you've specified is writeable. Maybe it would generally make sense for zig cc to gracefully handle cases where it can't create a local cache in the working directory in general, and not just for cgo?

Answer 11 · 2020-12-10T23:55:11.000Z

I solved the cache directory issue with ziglang/zig@f7d6006. So, as soon as https://golang.org/cl/276412 lands, I believe this issue can be considered to be solved.

Answer 12 · 2020-12-11T20:53:08.000Z

@andrwrk, as a workaround in the meantime you could probably set CC to a small script that invokes zig cc with the same arguments.

Answer 13 · 2020-12-12T04:32:43.000Z

@bcmills that workaround is mentioned in the original issue description. :)

Answer 14 · 2021-01-26T19:57:29.000Z

resolved with #43808 and CL 284780

Answer 15 · 2021-01-26T19:59:14.000Z

@seankhliao #43808 is a separate issue?

Answer 16 · 2021-03-02T13:26:26.000Z

resolved with #43808 and CL 284780

yeah running with CL 284780 and that definitely does NOT solve this issue.

Answer 17 · 2021-07-15T00:08:31.000Z

Change https://golang.org/cl/334732 mentions this issue: [dev.cmdgo] cmd: support space and quotes in CC and CXX

Answer 18 · 2021-08-12T22:42:54.000Z

Change https://golang.org/cl/341936 mentions this issue: cmd: support space and quotes in CC and CXX

Answer 19 · 2021-08-24T12:11:08.000Z

I can confirm this fixes the issue.