Go bindings broken with go 1.11
justinfx opened this issue · 4 comments
justinfx commented
As of go 1.11 the bindings no longer build:
go get -u github.com/valyala/ybc/bindings/go/ybc
go: finding github.com/valyala/ybc/bindings/go/ybc latest
go: finding github.com/valyala/ybc/bindings/go latest
go: finding github.com/valyala/ybc/bindings latest
go: finding github.com/valyala/ybc latest
# github.com/valyala/ybc/bindings/go/ybc
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:663:7: could not determine kind of name for C.YBC_DE_NOTFOUND
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:667:7: could not determine kind of name for C.YBC_DE_SUCCESS
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:659:7: could not determine kind of name for C.YBC_DE_WOULDBLOCK
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:483:5: could not determine kind of name for C.go_item_set
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:326:39: could not determine kind of name for C.uint64_t
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:712:2: could not determine kind of name for C.ybc_clear
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:469:2: could not determine kind of name for C.ybc_close
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:254:8: could not determine kind of name for C.ybc_config_destroy
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:329:3: could not determine kind of name for C.ybc_config_disable_overwrite_protection
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:56:19: could not determine kind of name for C.ybc_config_get_size
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:296:2: could not determine kind of name for C.ybc_config_init
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:313:3: could not determine kind of name for C.ybc_config_set_data_file
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:302:3: could not determine kind of name for C.ybc_config_set_data_file_size
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:319:3: could not determine kind of name for C.ybc_config_set_de_hashtable_size
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:317:2: could not determine kind of name for C.ybc_config_set_hot_data_size
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:316:2: could not determine kind of name for C.ybc_config_set_hot_items_count
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:307:3: could not determine kind of name for C.ybc_config_set_index_file
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:299:3: could not determine kind of name for C.ybc_config_set_max_items_count
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:326:3: could not determine kind of name for C.ybc_config_set_sync_interval
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:57:19: could not determine kind of name for C.ybc_get_size
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:59:19: could not determine kind of name for C.ybc_item_get_size
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:859:2: could not determine kind of name for C.ybc_item_release
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:272:5: could not determine kind of name for C.ybc_open
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:287:2: could not determine kind of name for C.ybc_remove
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:753:2: could not determine kind of name for C.ybc_set_txn_commit
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:58:19: could not determine kind of name for C.ybc_set_txn_get_size
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:832:3: could not determine kind of name for C.ybc_set_txn_get_value
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:761:2: could not determine kind of name for C.ybc_set_txn_rollback
../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:819:2: could not determine kind of name for C.ybc_set_txn_update_value_size
cgo:
gcc errors for preamble:
In file included from ./ybc_go_glue.c:1:0,
from ../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:19:
./ybc.h:1:1: error: expected identifier or '(' before '.' token
../../../ybc.h
^
In file included from ../go/pkg/mod/github.com/valyala/ybc@v0.0.0-20170629132908-7f424c60224c/bindings/go/ybc/ybc.go:19:0:
./ybc_go_glue.c:21:38: error: unknown type name 'size_t'
const void *const key_ptr, const size_t key_size)
^
./ybc_go_glue.c: In function 'go_item_remove':
./ybc_go_glue.c:23:16: error: variable 'key' has initializer but incomplete type
const struct ybc_key key = {
^
./ybc_go_glue.c:24:5: error: unknown field 'ptr' specified in initializer
.ptr = key_ptr,
^
...
If I manually disable the module support, it will work:
GO111MODULE=off -u github.com/valyala/ybc/bindings/go/ybc
But this means I cannot use go 1.11 module vendoring support. I believe this has to do with the way source data is cached. The go tool isn't copying across the required c source files for ybc when it uses module logic.
valyala commented
I fixed similar issue in gozstd - see valyala/gozstd@1628254 . The basic idea is just copying all the required *.h files into a directory with *.go files. Feel free sending a PR that fixes this.
In the mean time try fastcache instead of ybc. It should work better for items smaller than 64KB.
justinfx commented
Yea I just realized it was because of the symlinks to the parent directory that get flattened into empty files. I can see how manually copying them is a workaround, but unfortuntately it means a project using ybc as a dependency now requires a makefile instead of just the go tool.
I will look into switching to fastcache, thanks
valyala commented
Additionally ybc is vulnerable to mmap stalls, which may be noticeable on big caches under high load.
The only valid reason to use ybc is for caching big entries in a cache exceeding available RAM size.
valyala commented
I can see how manually copying them is a workaround, but unfortuntately it means a project using ybc as a dependency now requires a makefile instead of just the go tool.
Just send a PR which copies the corresponding *.h files into a directory with *.go files. Then ybc will remain go get'able on go1.11+