test_pr_purve fails due to slightly different floating point values
Mic92 opened this issue · 2 comments
Mic92 commented
Describe the bug
I get the following test errors for v2.4 and master for python3.9 on x86_64 linux:
FAILED tests/test_pr_curve.py::PRCurveTest::test_pr_purve - AssertionError
FAILED tests/test_pr_curve.py::PRCurveTest::test_pr_purve_raw - AssertionError
For PRCurveTest::test_pr_purve:
(Pdb) print(str_to_compare)
value {
tag: "tag"
tensor {
dtype: DT_FLOAT
tensor_shape {
dim {
size: 6
}
dim {
size: 1
}
}
float_val: 57.0
float_val: 43.0
float_val: 0.0
float_val: 0.0
float_val: 0.5699999928474426
float_val: 1.0
}
metadata {
plugin_data {
plugin_name: "pr_curves"
content: "\020\001"
}
}
}
(Pdb) print(expected)
value {
tag: "tag"
tensor {
dtype: DT_FLOAT
tensor_shape {
dim {
size: 6
}
dim {
size: 1
}
}
float_val: 57.0
float_val: 43.0
float_val: 0.0
float_val: 0.0
float_val: 0.57
float_val: 1.0
}
metadata {
plugin_data {
plugin_name: "pr_curves"
content: "\020\001"
}
}
for PRCurveTest::test_pr_purve_raw:
(Pdb) print(str_to_compare)
value {
tag: "prcurve with raw data"
tensor {
dtype: DT_FLOAT
tensor_shape {
dim {
size: 6
}
dim {
size: 5
}
}
float_val: 75.0
float_val: 64.0
float_val: 21.0
float_val: 5.0
float_val: 0.0
float_val: 150.0
float_val: 105.0
float_val: 18.0
float_val: 0.0
float_val: 0.0
float_val: 0.0
float_val: 45.0
float_val: 132.0
float_val: 150.0
float_val: 150.0
float_val: 0.0
float_val: 11.0
float_val: 54.0
float_val: 70.0
float_val: 75.0
float_val: 0.3333333134651184
float_val: 0.3786981999874115
float_val: 0.5384616255760193
float_val: 1.0
float_val: 0.0
float_val: 1.0
float_val: 0.8533334136009216
float_val: 0.2800000011920929
float_val: 0.06666669994592667
float_val: 0.0
}
metadata {
plugin_data {
plugin_name: "pr_curves"
content: "\020\001"
}
}
}
(Pdb) print(expected)
value {
tag: "prcurve with raw data"
tensor {
dtype: DT_FLOAT
tensor_shape {
dim {
size: 6
}
dim {
size: 5
}
}
float_val: 75.0
float_val: 64.0
float_val: 21.0
float_val: 5.0
float_val: 0.0
float_val: 150.0
float_val: 105.0
float_val: 18.0
float_val: 0.0
float_val: 0.0
float_val: 0.0
float_val: 45.0
float_val: 132.0
float_val: 150.0
float_val: 150.0
float_val: 0.0
float_val: 11.0
float_val: 54.0
float_val: 70.0
float_val: 75.0
float_val: 0.3333333
float_val: 0.3786982
float_val: 0.5384616
float_val: 1.0
float_val: 0.0
float_val: 1.0
float_val: 0.8533334
float_val: 0.28
float_val: 0.0666667
float_val: 0.0
}
metadata {
plugin_data {
plugin_name: "pr_curves"
content: "\020\001"
}
}
}
Minimal runnable code to reproduce the behavior
the tests in question
Expected behavior
no test failures
Environment
This is the content of my PYTHONPATH (you can interfere all python library versions from it)
/run/user/1000/tmp.7ykAbLnxEP/lib/python3.9/site-packages
/nix/store/rppr9s436950i1dlzknbmz40m2xqqnxc-python3-3.9.9/lib/python3.9/site-packages
/nix/store/0kk44adw7gbvgskds4ic56fr5zahbh8j-python3.9-setuptools-57.2.0/lib/python3.9/site-packages
/nix/store/0jkmi9zajsfvnabfkm2bq9snzsail204-python3.9-wheel-0.36.2/lib/python3.9/site-packages
/nix/store/11wvwr8f2dp4x8xjnrgqn3inmh418apn-python3.9-pip-21.1.3/lib/python3.9/site-packages
/nix/store/msjbsd34j8qh5q61b4i0qfmrcpmmjrlr-python3.9-boto3-1.20.13/lib/python3.9/site-packages
/nix/store/sx74hxvfjdza013p1jwinyk1hxsybry7-python3.9-botocore-1.23.14/lib/python3.9/site-packages
/nix/store/dvh8r3p061s7n4bmbdfssp6nxqxwr8n0-python3.9-python-dateutil-2.8.2/lib/python3.9/site-packages
/nix/store/bxk11pnsmb10j7igg2227d861f9m0a73-python3.9-six-1.16.0/lib/python3.9/site-packages
/nix/store/dfjhdx51az0d40skbgpn81gs02qwziv8-python3.9-jmespath-0.10.0/lib/python3.9/site-packages
/nix/store/a1gqcxk3s7qaw1zpd5giwjjr9simcv8r-python3.9-ply-3.11/lib/python3.9/site-packages
/nix/store/q9g3z692dc6lnp06zq348bns4ziqsq69-python3.9-docutils-0.17.1/lib/python3.9/site-packages
/nix/store/33dvahw0rply43zk1z1spqvf425nay5y-python3.9-ordereddict-1.1/lib/python3.9/site-packages
/nix/store/dy4k7a0amd074i80zaa87fc071wadi5f-python3.9-simplejson-3.17.6/lib/python3.9/site-packages
/nix/store/1v6056f7131ihxydss1w35ciaf6kwb6b-python3.9-urllib3-1.26.7/lib/python3.9/site-packages
/nix/store/1i3i630z8c1g77pr3ay1af0ainj3h6xz-python3.9-brotli-1.0.9/lib/python3.9/site-packages
/nix/store/pwjy912kf7r21706i09930gq0sgn0fb2-python3.9-pysocks-1.7.1/lib/python3.9/site-packages
/nix/store/d9jpxxmjlqb8x0yp8n2vawql9p0vk9dr-python3.9-s3transfer-0.5.0/lib/python3.9/site-packages
/nix/store/1b5c96l71p5szh3n15zxq5drsc5pjacg-python3.9-matplotlib-3.5.1/lib/python3.9/site-packages
/nix/store/2jknfdah23418pky61yj6hy6k1nghbd1-python3.9-certifi-2021.10.08/lib/python3.9/site-packages
/nix/store/89shxlnrb3irqkm526bw0rzhlsnfzqhn-python3.9-cycler-0.11.0/lib/python3.9/site-packages
/nix/store/d2qj1kqm7jncjf878mil2ld39zz96rfh-python3.9-fonttools-4.26.2/lib/python3.9/site-packages
/nix/store/cibm34dn0hcv94b9p5r9hp4yifvmmvsf-python3.9-kiwisolver-1.3.2/lib/python3.9/site-packages
/nix/store/iwnvfxnnsg525m3zgbraqgvjmm4kcp8i-python3.9-mock-4.0.3/lib/python3.9/site-packages
/nix/store/qff7hii60jmd2p6njgwvgpkjils9vaws-python3.9-pbr-5.8.0/lib/python3.9/site-packages
/nix/store/147gai4wzhanfa6ng7iqd2zm6vzd39cd-python3.9-numpy-1.21.4/lib/python3.9/site-packages
/nix/store/7czibm80c4kfa5f6r9bmlmvvv8804jns-python3.9-packaging-20.9/lib/python3.9/site-packages
/nix/store/hvzc59kvsd3vk418rsz0f1vl2dwd5cmx-python3.9-pyparsing-2.4.7/lib/python3.9/site-packages
/nix/store/fgpv1g18dfxbqnzvylp3yihrbjhiydwh-python3.9-Pillow-8.4.0/lib/python3.9/site-packages
/nix/store/ry0z0a1qk8yv65ldnlk03mrvpyzrv63g-python3.9-olefile-0.46/lib/python3.9/site-packages
/nix/store/b6f5pyhdykd43kx2adfnz8sjg0adyzav-python3.9-defusedxml-0.7.1/lib/python3.9/site-packages
/nix/store/cqjxlp7dwm2rfsb5n4dr6v9rpfp2ckbn-python3.9-pytz-2021.3/lib/python3.9/site-packages
/nix/store/lpkgqn92439cfdp554jrg28sa8z4h8rp-python3.9-tornado-6.1/lib/python3.9/site-packages
/nix/store/l2gq9cjvy6nw94x087qr7y12h1s2jypd-python3.9-tkinter-3.9.9/lib/python3.9/site-packages
/nix/store/kag7sarwlxjr92bkszsmkv3mfxmimzc0-python3.9-moto-1.3.16/lib/python3.9/site-packages
/nix/store/l02898b46ap4xj5nr4zskpxrfmzgx5m3-python3.9-aws-xray-sdk-2.8.0/lib/python3.9/site-packages
/nix/store/kkwysi6iv6aj0n1hlvckbvw63n3wp52c-python3.9-jsonpickle-2.0.0/lib/python3.9/site-packages
/nix/store/pzrbsdkwchkcrr49hlnjvzsn37h26zcf-python3.9-importlib-metadata-4.8.2/lib/python3.9/site-packages
/nix/store/23ws96riyvhb3x11zjqdshqmdificqv8-python3.9-toml-0.10.2/lib/python3.9/site-packages
/nix/store/cf0y560zi5fjncvmwphza5vskpywkgnd-python3.9-zipp-3.6.0/lib/python3.9/site-packages
/nix/store/m1pmgdcrm930csq2x42y91xrwx1ycn0z-python3.9-more-itertools-8.12.0/lib/python3.9/site-packages
/nix/store/9gmg1af7qz1xn6xk70phw39padd72k2v-python3.9-wrapt-1.13.3/lib/python3.9/site-packages
/nix/store/47w9wdpsz6arsd5zf7sic5d4sj7y5gkr-python3.9-requests-2.26.0/lib/python3.9/site-packages
/nix/store/azx03chpb7bbip81yl7lqy7bs4bzl6i2-python3.9-idna-3.3/lib/python3.9/site-packages
/nix/store/9mj2dv31fsv49kcdfwfywsrjwzakyiwa-python3.9-chardet-4.0.0/lib/python3.9/site-packages
/nix/store/ywqh8g28byagf31ggxz8k34p10cfydj5-python3.9-brotlicffi-1.0.9.2/lib/python3.9/site-packages
/nix/store/7qvbgr72c44gwnbs63mnxxbm0209dlcz-python3.9-pycparser-2.21/lib/python3.9/site-packages
/nix/store/ljjnmcbmqhifhwblscw4nbnky6la2sfi-python3.9-cffi-1.15.0/lib/python3.9/site-packages
/nix/store/80f60wqxm4vw3fn05aaavw19gxiqfjn9-python3.9-charset-normalizer-2.0.8/lib/python3.9/site-packages
/nix/store/k6q72pramqcrpb2j8yiw670xs8nb9fxy-python3.9-future-0.18.2/lib/python3.9/site-packages
/nix/store/gxiqk0kbi4phlwnyj9q8p13y1zw3svh0-python3.9-cfn-lint-0.56.2/lib/python3.9/site-packages
/nix/store/lqrbv54xws8adsy4mw79115zhphnvlqr-python3.9-aws-sam-translator-1.42.0/lib/python3.9/site-packages
/nix/store/zpv7zf16f4a17fcmsbvdfhk380zz3866-python3.9-jsonschema-3.2.0/lib/python3.9/site-packages
/nix/store/d83z4zbjg301g3x570k7yg34ms8c9yir-python3.9-attrs-21.2.0/lib/python3.9/site-packages
/nix/store/xvpwk6r5h5nzgj1f0yzvrqrnkv7dvzrx-python3.9-pyrsistent-0.18.0/lib/python3.9/site-packages
/nix/store/ndiln6n8wlzdnb0aqp1x12vrx21krvyk-python3.9-jschema-to-python-1.2.3/lib/python3.9/site-packages
/nix/store/cfgnrzxl27m9jz7nfc5d92sjzgn5pqzh-python3.9-jsonpatch-1.32/lib/python3.9/site-packages
/nix/store/88hwpgr0hj4cw2g5gj5haz1fnkpyrjjr-python3.9-jsonpointer-2.2/lib/python3.9/site-packages
/nix/store/ws88xhm7iwv4vq1iw8bg5zjp8nq8fzk3-python3.9-junit-xml-1.9/lib/python3.9/site-packages
/nix/store/cwbi0zg1bqs6wa6dkw14xkyixhqddy3z-python3.9-networkx-2.6.3/lib/python3.9/site-packages
/nix/store/257mz2v8sw9d3q5cj4vg6msrnk3m020p-python3.9-decorator-5.1.0/lib/python3.9/site-packages
/nix/store/7kny1gp03xyzip08cjs2bsnavqwx2h0i-python3.9-pathlib2-2.3.6/lib/python3.9/site-packages
/nix/store/9lzf47n706k2cdnyrg0xzb9ffz65x2d6-python3.9-PyYAML-5.4.1.1/lib/python3.9/site-packages
/nix/store/4y0zrmlsdf738lxijsqh76xbkwghsy6k-python3.9-sarif-om-1.0.4/lib/python3.9/site-packages
/nix/store/qyl39n1xyy39zasjckkljv01333l7wgw-python3.9-docker-5.0.3/lib/python3.9/site-packages
/nix/store/dnzban7p9pmp270v84nnvcrs8xycrsb1-python3.9-paramiko-2.8.0/lib/python3.9/site-packages
/nix/store/09khq7gqfwwy05n4hjd5csqychibx0zg-python3.9-bcrypt-3.2.0/lib/python3.9/site-packages
/nix/store/94g2d5w7f1kw35qxq3i4lvwpnr4g43h2-python3.9-cryptography-3.4.8/lib/python3.9/site-packages
/nix/store/ybd69djaw96qx8apkkg49yy7vzcr88hz-python3.9-pynacl-1.4.0/lib/python3.9/site-packages
/nix/store/0dyklfn2d21aqz858f4px08fl8ahncpc-python3.9-pyasn1-0.4.8/lib/python3.9/site-packages
/nix/store/k0qjfvyrw4h8v3g3lji039fq9mjffjq5-python3.9-websocket-client-1.2.1/lib/python3.9/site-packages
/nix/store/y4r5r3jkmsdraaqvak1fn1259x2my3sw-python3.9-python-socks-2.0.0/lib/python3.9/site-packages
/nix/store/08d0r35xk4j4fmnl9jb89316665lvfdn-python3.9-trio-0.19.0/lib/python3.9/site-packages
/nix/store/d4cgf4cgicbz572m0l758xg3m04dg88a-python3.9-sortedcontainers-2.4.0/lib/python3.9/site-packages
/nix/store/6drid6fdrgnizxfqdw0ckvsf9lcr8yr0-python3.9-async_generator-1.10/lib/python3.9/site-packages
/nix/store/q4f8ca11758wdck9rq7a4xl6gkndi27c-python3.9-outcome-1.1.0/lib/python3.9/site-packages
/nix/store/c6kbrga1p60n0f0zim1kdz6pl82hz3y2-python3.9-sniffio-1.2.0/lib/python3.9/site-packages
/nix/store/2y3hk6gj4az1lm9haqnasjy5mgip4gz8-python3.9-curio-1.5/lib/python3.9/site-packages
/nix/store/cfjbhsghgfbp59y1x7d0n1v6sv11n6ib-python3.9-async-timeout-4.0.1/lib/python3.9/site-packages
/nix/store/bw5nl4r0pqwf9jsig7rqzrhx9m98ajzv-python3.9-typing_extensions-3.10.0.2/lib/python3.9/site-packages
/nix/store/vf78d6b7s1wk59a8xlgv0dlq4izixjyv-python3.9-Flask-2.0.2/lib/python3.9/site-packages
/nix/store/55gl146996h1j1gj5mydv3m7imq932kn-python3.9-asgiref-3.4.1/lib/python3.9/site-packages
/nix/store/y8rv23h44a3wjdipj006cyzqy7r6lf7h-python3.9-python-dotenv-0.19.2/lib/python3.9/site-packages
/nix/store/626nk2dqsr3amls4f7lxirvpxyp0lx5v-python3.9-click-8.0.3/lib/python3.9/site-packages
/nix/store/48c9r6zqbb06qqhks06iqhfz0hy418fb-python3.9-itsdangerous-2.0.1/lib/python3.9/site-packages
/nix/store/ap9bcfhiyv7xj7dq79ik6vs5ab2w4ha4-python3.9-Jinja2-3.0.3/lib/python3.9/site-packages
/nix/store/vw4jbwa0vjy1x012j9ybymw7v9x6hamh-python3.9-Babel-2.9.1/lib/python3.9/site-packages
/nix/store/k7f01l2sx0mc14gj15d4kndds9sdclww-python3.9-markupsafe-2.0.1/lib/python3.9/site-packages
/nix/store/dixgra62cj6ysssdxyypgpibks7gjh5y-python3.9-werkzeug-2.0.2/lib/python3.9/site-packages
/nix/store/rlrchrj5hf0d11qi89vhfkh0sfvrv1kj-python3.9-watchdog-2.1.6/lib/python3.9/site-packages
/nix/store/5p892g7fd5szqz1p591nsjafzbvv0ap9-python3.9-pathtools-0.1.2/lib/python3.9/site-packages
/nix/store/glcyi2g4dpxny8jl5ckyf77hzaaa00rd-python3.9-jsondiff-1.3.0/lib/python3.9/site-packages
/nix/store/hp5vyzrs3b9461vjzclh7pr9ihrgmlg6-python3.9-pyaml-21.10.1/lib/python3.9/site-packages
/nix/store/5nvc930xycgrw9s1a18sjd508ii1wdzk-python3.9-python-jose-3.3.0/lib/python3.9/site-packages
/nix/store/rd9xpxn8v62r1rh8iq2402sic7hw7ql9-python3.9-ecdsa-0.17.0/lib/python3.9/site-packages
/nix/store/7pshr2wq62hqzxgxg4w3x0jj9znzk68a-python3.9-pycrypto-3.11.0/lib/python3.9/site-packages
/nix/store/4g1869djqbzqspqgmxmwqa5qllxw8q1a-python3.9-pycryptodome-3.11.0/lib/python3.9/site-packages
/nix/store/pxkbq1xm3cjprnsx2bjbwsld1nim3w9p-python3.9-rsa-4.8/lib/python3.9/site-packages
/nix/store/z478jj7jd7cjll22gj8jprcpkrn1fsn3-python3.9-responses-0.16.0/lib/python3.9/site-packages
/nix/store/n2sdr8n23rp7nlmynrrsasc6szi71q8n-python3.9-sshpubkeys-3.3.1/lib/python3.9/site-packages
/nix/store/hvfif1xsv71f56icv5rs8h1rnyb9avzk-python3.9-xmltodict-0.12.0/lib/python3.9/site-packages
/nix/store/a9zygd4sj2z0p33zl2mf437q5iw71jsv-python3.9-pytest-6.2.5/lib/python3.9/site-packages
/nix/store/9mccn35i9m1bwg1nmf2v2p7x4rm4zqbs-python3.9-atomicwrites-1.4.0/lib/python3.9/site-packages
/nix/store/c7v4hmk32m8i09bxm97z0clq50rgw4w9-python3.9-iniconfig-1.1.1/lib/python3.9/site-packages
/nix/store/5x48f3534j5aqr3jfqlbwwlwbnxcgipx-python3.9-pluggy-1.0.0/lib/python3.9/site-packages
/nix/store/mz5b96xzzpf2f0w15k23ylj50s6gs40p-python3.9-py-1.11.0/lib/python3.9/site-packages
/nix/store/zgjvyamy1vms8ffqhlhvdr3akj4l7bnm-python3.9-wcwidth-0.2.5/lib/python3.9/site-packages
/nix/store/rk5887yi2p0x4j6j60wa9wdn2ypzl8vy-python3.9-tensorflow-tensorboard-2.6.0/lib/python3.9/site-packages
/nix/store/njy0484n2vc0ayk3g36c34gkyqr3km5a-python3.9-absl-py-0.15.0/lib/python3.9/site-packages
/nix/store/k4wrwdjymhc1rqpr0a76d034r5qcxz7j-python3.9-protobuf-3.19.0/lib/python3.9/site-packages
/nix/store/kjjjx6v879b6y97lkcwlf2d29p1cniks-python3.9-grpcio-1.43.0/lib/python3.9/site-packages
/nix/store/w8pj50g9s481qbzh3bwshypx6039apk9-python3.9-google-auth-oauthlib-0.4.6/lib/python3.9/site-packages
/nix/store/6xpcpvxm5dg2cqrjr822w15qc07wnhqs-python3.9-google-auth-2.3.3/lib/python3.9/site-packages
/nix/store/mbqr69xgbpn4amlb9sqgc78b8ilq49yy-python3.9-cachetools-4.2.4/lib/python3.9/site-packages
/nix/store/rr7n06vij4xly1yyamzrrwcm9a0wk3qf-python3.9-pyasn1-modules-0.2.8/lib/python3.9/site-packages
/nix/store/z9pxxdgrnr75yvipb1w9j5pxa9z6glbv-python3.9-pyopenssl-20.0.1/lib/python3.9/site-packages
/nix/store/vbzpzwgr99igfa67glfpnifpf22024y3-python3.9-pyu2f-0.1.5a/lib/python3.9/site-packages
/nix/store/xq8sp29326x1si6my6a3ilrrwiw4sxrq-python3.9-requests-oauthlib-1.3.0/lib/python3.9/site-packages
/nix/store/vrw2i1q68yz82mn470cidb7gsbhcaldg-python3.9-oauthlib-unstable-2020-05-08/lib/python3.9/site-packages
/nix/store/yasd2qpq2v7v04c0glzajrccyxidj7mg-python3.9-blinker-1.4/lib/python3.9/site-packages
/nix/store/p5rdgfh9g2bpfryfg6s5x51xn8wbqkxx-python3.9-pyjwt-2.1.0/lib/python3.9/site-packages
/nix/store/ncqydwr9zb5bcwf43dvgw4jrqz54ajy6-python3.9-markdown-3.3.5/lib/python3.9/site-packages
/nix/store/r9d9ysc6irkcixmsx775cm494941765s-python3.9-tensorboard-data-server-0.6.1/lib/python3.9/site-packages
/nix/store/3ii32grmp2hwjk2zwdgh4k7fsqgiyz53-python3.9-tensorboard_plugin_profile-2.5.0/lib/python3.9/site-packages
/nix/store/7snwvl8705pp0wzhr421nhr5z22jp3xp-python3.9-gviz_api-1.10.0/lib/python3.9/site-packages
/nix/store/dviq8h85sxjfjs918dkdzl353jhd7f90-python3.9-tensorboard_plugin_wit-1.7.0/lib/python3.9/site-packages
/nix/store/sb2qvpc6mwz95a4aga20sx8mnh6f9icp-python3.9-pytorch-1.9.0/lib/python3.9/site-packages
/nix/store/dafv82cf7m6645cxn7lp2fhh41qw0xr8-python3.9-torchvision-0.11.2/lib/python3.9/site-packages
/nix/store/qsv40hj71s2flw6gbafzacpzna1pi6w2-python3.9-scipy-1.7.3/lib/python3.9/site-packages
/nix/store/f6mm7vp4vjs9w1hpxgwkqzzwjx7ig5cz-python3.9-crc32c-2.2.post0/lib/python3.9/site-packages
/nix/store/hlm3hnan6syz7jw76vi5pf91b8yljy73-python3.9-soundfile-0.10.3.post1/lib/python3.9/site-packages
Python environment
python3.9, no virtualenv, just python + PYTHONPATH
Additional context
Also see this failing CI build: https://hydra.nixos.org/build/161775882/nixlog/1
Mic92 commented
found the issue: NixOS/nixpkgs@f4f802c
Mic92 commented
I had to set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION