linking against ipv4 module fails with missing checksum symbol

Question

linking against ipv4 module fails with missing checksum symbol

yomimono opened this issue 7 years ago · 10 comments

This is visible in the Travis output for #320 , but I've reproduced it locally as well:

home/dothraki/.opam/4.05.0+trunk+afl/lib/tcpip/ipv4/tcpip_ipv4.a(ipv4_packet.o): In function `camlIpv4_packet__unsafe_fill_1734':
/home/dothraki/.opam/4.05.0+trunk+afl/build/tcpip.3.1.4/_build/default/src/ipv4/ipv4_packet.ml:59: undefined reference to `caml_tcpip_ones_complement_checksum'
/home/dothraki/.opam/4.05.0+trunk+afl/lib/tcpip/ipv4/tcpip_ipv4.a(ipv4_packet.o): In function `camlIpv4_packet__check_1879':
/home/dothraki/.opam/4.05.0+trunk+afl/build/tcpip.3.1.4/_build/default/src/ipv4/ipv4_packet.ml:145: undefined reference to `caml_tcpip_ones_complement_checksum_list'
/home/dothraki/.opam/4.05.0+trunk+afl/lib/tcpip/tcpip.a(tcpip_checksum.o): In function `camlTcpip_checksum__fun_1277':
:(.text+0x37): undefined reference to `caml_tcpip_ones_complement_checksum_list'
/home/dothraki/.opam/4.05.0+trunk+afl/lib/tcpip/tcpip.a(tcpip_checksum.o): In function `camlTcpip_checksum__fun_1279':
:(.text+0x87): undefined reference to `caml_tcpip_ones_complement_checksum'
/home/dothraki/.opam/4.05.0+trunk+afl/lib/tcpip/tcpip.a(tcpip_checksum.o): In function `camlTcpip_checksum__2':
:(.data+0x58): undefined reference to `caml_tcpip_ones_complement_checksum_list'
:(.data+0x60): undefined reference to `caml_tcpip_ones_complement_checksum'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Command exited with code 2.
pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-tag' 'debug'
     '-build-dir' '_build' 'pkg/META' 'CHANGES.md' 'LICENSE.md' 'README.md'
     'src/fuzz_ethif.native' 'src/fuzz_ipv4.native' 'src/fuzz_icmpv4.native'
     'src/fuzz_tcp.native' 'src/fuzz_udp.native']: exited with 10

Answer 1 · 2017-07-11T11:36:03.000Z

Any clues on where to start investigating this would be appreciated. It blocks fuzz testing the tcp stack with crowbar.

Answer 2 · 2017-07-11T12:32:05.000Z

This is using the trunk version of tcpip, so it's probably due to the port to jbuilder. I'll look at this shortly.

Answer 3 · 2017-07-11T15:07:03.000Z

It looks like a depedency on tcpip.unix is missing from the binary being built. However, where is it being built? I cant find a PR with the fuzz testing branch

Answer 4 · 2017-07-11T16:08:18.000Z

Yes, that's because crowbar is not released, so adding a dependency on it seemed unwise. There is code playing with it in https://github.com/yomimono/crowbar .

Answer 5 · 2017-07-11T16:09:37.000Z

Sorry, https://GitHub.com/yomimono/somerandompacket ; forgive the spam.

Answer 6 · 2017-08-03T01:55:50.000Z

Including tcpip.unix fixes this for unix, but the corresponding tcpip.xen doesn't help when building MirageOS unikernels targeting non-Unix. Here's the relevant bit of an attempt to build something with mirage configure -t xen, after tweaking mirage to always include tcpip.xen when it'll be required for checksums:

ocamlfind ocamlopt -c -g -g -bin-annot -safe-string -principal -strict-sequence -package tcpip.xen -package tcpip.udp -package tcpip.tcp -package tcpip.stack-direct -package tcpip.ipv4 -package tcpip.icmpv4 -package tcpip.ethif -package tcpip.arpv4 -package tcpip -package nocrypto.mirage -package nocrypto -package mirage-xen -package mirage-types-lwt -package mirage-types -package mirage-runtime -package mirage-random -package mirage-net-xen -package mirage-logs -package mirage-http -package mirage-console-xen -package mirage-conduit -package mirage-clock-freestanding -package mirage-bootvar-xen -package mirage-block-ramdisk -package lwt -package logs-syslog.mirage -package logs-syslog -package io-page -package functoria-runtime -predicates mirage_xen -w A-4-41-42-44 -color always -o main.cmx main.ml
ocamlfind ocamlopt -g -dontlink unix -dontlink str -dontlink num -dontlink threads -linkpkg -output-obj -package tcpip.xen -package tcpip.udp -package tcpip.tcp -package tcpip.stack-direct -package tcpip.ipv4 -package tcpip.icmpv4 -package tcpip.ethif -package tcpip.arpv4 -package tcpip -package nocrypto.mirage -package nocrypto -package mirage-xen -package mirage-types-lwt -package mirage-types -package mirage-runtime -package mirage-random -package mirage-net-xen -package mirage-logs -package mirage-http -package mirage-console-xen -package mirage-conduit -package mirage-clock-freestanding -package mirage-bootvar-xen -package mirage-block-ramdisk -package lwt -package logs-syslog.mirage -package logs-syslog -package io-page -package functoria-runtime -predicates mirage_xen key_gen.cmx static1.cmx unikernel.cmx main.cmx -o main.native.o
_build/main.native.o: In function `camlIcmpv4_packet__unsafe_fill_1855':
/home/dothraki/.opam/4.04.2/build/tcpip.3.2.0/_build/default/src/icmp/icmpv4_packet.ml:98: undefined reference to `caml_tcpip_ones_complement_checksum_list'
_build/main.native.o: In function `camlIpv4_packet__unsafe_fill_1727':
/home/dothraki/.opam/4.04.2/build/tcpip.3.2.0/_build/default/src/ipv4/ipv4_packet.ml:59: undefined reference to `caml_tcpip_ones_complement_checksum'
_build/main.native.o: In function `camlIpv4_packet__check_1871':
/home/dothraki/.opam/4.04.2/build/tcpip.3.2.0/_build/default/src/ipv4/ipv4_packet.ml:145: undefined reference to `caml_tcpip_ones_complement_checksum_list'
_build/main.native.o: In function `camlIpv4_common__adjust_output_header_1199':
/home/dothraki/.opam/4.04.2/build/tcpip.3.2.0/_build/default/src/ipv4/ipv4_common.ml:9: undefined reference to `caml_tcpip_ones_complement_checksum'
_build/main.native.o: In function `camlIpv4_common__checksum_1660':
/home/dothraki/.opam/4.04.2/build/tcpip.3.2.0/_build/default/src/ipv4/ipv4_common.ml:40: undefined reference to `caml_tcpip_ones_complement_checksum_list'
_build/main.native.o: In function `camlTcp__Tcp_packet__unsafe_fill_1648':
/home/dothraki/.opam/4.04.2/build/tcpip.3.2.0/_build/default/src/tcp/tcp_packet.ml:102: undefined reference to `caml_tcpip_ones_complement_checksum_list'
_build/main.native.o: In function `camlUdp_packet__unsafe_fill_1539':
/home/dothraki/.opam/4.04.2/build/tcpip.3.2.0/_build/default/src/udp/udp_packet.ml:58: undefined reference to `caml_tcpip_ones_complement_checksum_list'
_build/main.native.o: In function `camlTcpip_checksum__fun_1272':
(.text+0x182f1a): undefined reference to `caml_tcpip_ones_complement_checksum_list'
_build/main.native.o: In function `camlTcpip_checksum__fun_1274':
(.text+0x182f4a): undefined reference to `caml_tcpip_ones_complement_checksum'
_build/main.native.o: In function `camlTcpip_checksum__2':
:(.data+0x13b1c0): undefined reference to `caml_tcpip_ones_complement_checksum_list'
:(.data+0x13b1c8): undefined reference to `caml_tcpip_ones_complement_checksum'
run ['ld' '-d' '-static' '-nostdlib' '_build/main.native.o'
     '-L/home/dothraki/.opam/4.04.2/lib/io-page-xen' '-lio_page_xen_stubs'
     '-L/home/dothraki/.opam/4.04.2/lib/zarith' '-lzarith-xen'
     '-L/home/dothraki/.opam/4.04.2/lib/gmp-xen' '-lgmp-xen'
     '-L/home/dothraki/.opam/4.04.2/lib/nocrypto'
     '-lnocrypto_stubs+mirage-xen'
     '-L/home/dothraki/.opam/4.04.2/lib/mirage-entropy'
     '-lmirage-entropy_stubs+mirage-xen' '-L/home/dothraki/.opam/4.04.2/lib'
     '-L/home/dothraki/.opam/4.04.2/lib/pkgconfig/../../lib/minios-xen'
     '/home/dothraki/.opam/4.04.2/lib/pkgconfig/../../lib/mirage-xen/libxencamlbindings.a'
     '/home/dothraki/.opam/4.04.2/lib/pkgconfig/../../lib/mirage-xen-ocaml/libxenasmrun.a'
     '/home/dothraki/.opam/4.04.2/lib/pkgconfig/../../lib/mirage-xen-ocaml/libxenotherlibs.a'
     '/home/dothraki/.opam/4.04.2/lib/pkgconfig/../../lib/mirage-xen-posix/libxenposix.a'
     '-lopenlibm' '-lminios'
     '-T/home/dothraki/.opam/4.04.2/lib/pkgconfig/../../lib/minios-xen/libminios.lds'
     '-m' 'elf_x86_64' '-lx86_64' '-o' 'all_devices.xen']: exited with 1
Makefile:18: recipe for target 'build' failed
make: *** [build] Error 1

You can see that -package tcpip.xen is invoked, but it doesn't do us any good.

Answer 7 · 2017-08-03T09:39:53.000Z

This reproduces for me.

In the very short term I think we need to add the xen_linkopts back to the META file. I'll take a quick look.

IMHO the handling of the checksum stubs is annoyingly complicated. I'll try to summarise how I think it currently works (-- let me know if you spot a mistake) (I'm writing this to force myself to page it back in -- feel free to ignore)

on unix we use the normal OCaml mechanism: the stubs are associated with the OCaml objects and the ocamlopt invocation links them into the final binary
on Xen we perform the final link ourselves in the mirage tool, and rather than query the OCaml objects to discover the stubs (see mirage/mirage#832) we add a custom xen_linkopts field inside the META file which is used by the mirage tool to pull in the objects
on Solo5 we have a customised copy of the stubs in the base platform library which is unconditionally linked

I think we've not pushed through any of these models to all libraries * all backends because they all seem a bit odd, leaving us in an inconsistent state.

I think the current consensus long-term view is that we should cross-compile everything (including OCaml code) for each backend that the user has selected to avoid future problems and use the regular OCaml mechanism to link stubs rather than a custom one. I hope that the jbuilder "workspace" concept will help us -- we can build everything n times with different flags. The C stubs would then live in the same repo as the OCaml code, perhaps with #ifdefs to use the different low-level platform C APIs? I guess the small set of stubs that are portable across backends are either pure functions (like crypto and checksums) or bindings to things like posix_memalign. The former doesn't require an #ifdef and the latter... maybe we should standardise the posix_memalign interface at the C level anyway.

Answer 8 · 2017-08-03T10:05:05.000Z

OK, I got stuck -- see my attempt at #332. If I hand-edit the META file to put the xen_linkopts inside the package "xen" block then it works. I don't know how to achieve a similar effect at the top-level in the META file.

One option is to split tcpip.xen out into tcpip-xen like we did for io-page-xen. Another option is to put some more special-case directory magic into the mirage tool.

Answer 9 · 2017-08-03T10:46:00.000Z

OK with help from @samoht I've made it link for me. I think we need

to merge #332
the tweak to mirage that @yomimono mentioned above

Answer 10 · 2017-08-03T17:08:23.000Z

I've added that change to the PR at mirage/mirage#856 .