Meta-ticket: Fix unstable random doctests detected after #29935
kliem opened this issue · 43 comments
- #32543: Mark unstable tests due do #29956.
- #32763: Increase tolerance in doctests of
modular/modform/numerical.py - #33353: increase tolerance of doctests in
modular/modform/numerical.py - #32771: Fix incorrect doctest introduced in #29978
- #32772: Fix two random test failures in graphs
- #32773: Another test failure with KlyachkoBundle_class.random_deformation (duplicate: #32849)
- #32774: Bug in continue fraction doctest
- #32783: random doctest failure in
src/sage/graphs/generic_graph.py - #32816: random doctest failure in
rings/polynomial/polynomial_element_generic.py - #32817: Fix two tests failing at random
- #32850: random doctest failure in
tests/books/computational-mathematics-with-sagemath/sol/linsolve_doctest.py - #32890: Unstable doctest in hmm/chmm.pyx
- #32891: Unstable doctest geometry/hyperbolic_space/hyperbolic_model.py
- #32862: Failure in sage/modular/modform/numerical.py for special seed
- #33015: Random failure in number_field_element.pyx
- #33035: random doctest failure in
src/sage/tests/book_stein_ent.py - #33036: random doctest failure in
src/sage/crypto/util.py - #33039: Random doctest failure in continued_fraction_gosper
- #33072: Random failure in matrix_gap.pyx
- #33129: src/sage/plot/plot.py: random doctest failure
- #33628: random failure in
graphs.RandomToleranceGraph - #33751: random doctest failure in
src/sage/matrix/matrix_integer_dense_hnf.py - #33880: Another unstable doctest in
hmm/chmm.pyx - #34520: remove randomly failing doctest in
src/sage/graphs/generators/random.py - #34762: fix random chain complex doctest
- #34901: random doctest failure in
src/doc/en/thematic_tutorials/group_theory.rst - #32773: random doctest failure in
src/sage/schemes/toric/sheaf/klyachko.py
Component: doctest framework
Issue created by migration from https://trac.sagemath.org/ticket/32544
I see failures
./sage -t --long --warn-long 101.7 --random-seed=260367842280804525169259077536186393366 src/sage/modular/modform/numerical.py
In detail
Doctesting 1 file.
sage -t --long --warn-long 101.7 --random-seed=260367842280804525169259077536186393366 src/sage/modular/modform/numerical.py
**********************************************************************
File "src/sage/modular/modform/numerical.py", line 74, in sage.modular.modform.numerical.NumericalEigenforms
Failed example:
n.ap(2) # rel tol 2e-14
Expected:
[3.0, -1.6180339887498947, 0.6180339887498968]
Got:
[3.0, -1.6180339887498947, 0.6180339887498804]
Tolerance exceeded in 1 of 3:
0.6180339887498968 vs 0.6180339887498804, tolerance 3e-14 > 2e-14
**********************************************************************
File "src/sage/modular/modform/numerical.py", line 76, in sage.modular.modform.numerical.NumericalEigenforms
Failed example:
n.systems_of_eigenvalues(7) # rel tol 2e-14
Expected:
[
[-1.6180339887498947, 2.2360679774997894, -3.2360679774997894],
[0.6180339887498968, -2.236067977499788, 1.2360679774997936],
[3.0, 4.0, 6.0]
]
Got:
[
[-1.6180339887498947, 2.2360679774997894, -3.2360679774997894],
[0.6180339887498804, -2.2360679774998293, 1.2360679774997607],
[3.0, 4.0, 6.0]
]
Tolerance exceeded in 2 of 9:
0.6180339887498968 vs 0.6180339887498804, tolerance 3e-14 > 2e-14
1.2360679774997936 vs 1.2360679774997607, tolerance 3e-14 > 2e-14
**********************************************************************
File "src/sage/modular/modform/numerical.py", line 88, in sage.modular.modform.numerical.NumericalEigenforms
Failed example:
n.eigenvalues([2,3,5]) # rel tol 2e-14
Expected:
[[3.0, -1.6180339887498947, 0.6180339887498968],
[4.0, 2.2360679774997894, -2.236067977499788],
[6.0, -3.2360679774997894, 1.2360679774997936]]
Got:
[[3.0, -1.6180339887498947, 0.6180339887498804],
[4.0, 2.2360679774997894, -2.2360679774998293],
[6.0, -3.2360679774997894, 1.2360679774997607]]
Tolerance exceeded in 2 of 9:
0.6180339887498968 vs 0.6180339887498804, tolerance 3e-14 > 2e-14
1.2360679774997936 vs 1.2360679774997607, tolerance 3e-14 > 2e-14
**********************************************************************
1 item had failures:
3 of 7 in sage.modular.modform.numerical.NumericalEigenforms
[46 tests, 3 failures, 0.26 s]
----------------------------------------------------------------------
sage -t --long --warn-long 101.7 --random-seed=260367842280804525169259077536186393366 src/sage/modular/modform/numerical.py # 3 doctests failed
Description changed:
---
+++
@@ -1 +1,2 @@
- #32543: Mark unstable tests due do #29956.
+- #32763: Increase tolerance in doctests of `modular/modform/numerical.py` sage -t --long --random-seed=180924489930967511277954046005685261728 src/sage/graphs/base/sparse_graph.pyx
**********************************************************************
File "src/sage/graphs/base/sparse_graph.pyx", line 1163, in sage.graphs.base.sparse_graph._test_adjacency_sequence_out
Failed example:
_test_adjacency_sequence_out() # long time
Exception raised:
Traceback (most recent call last):
File "/home/jonathan/Applications/sage/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 694, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/home/jonathan/Applications/sage/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
exec(compiled, globs)
File "<doctest sage.graphs.base.sparse_graph._test_adjacency_sequence_out[1]>", line 1, in <module>
_test_adjacency_sequence_out() # long time
File "sage/graphs/base/sparse_graph.pyx", line 1189, in sage.graphs.base.sparse_graph._test_adjacency_sequence_out (build/cythonized/sage/graphs/base/sparse_graph.c:9302)
u = randint(low, n - 1)
File "/home/jonathan/Applications/sage/local/lib/python3.8/site-packages/sage/misc/prandom.py", line 137, in randint
return _pyrand().randint(a, b)
File "/home/jonathan/Applications/sage/local/lib/python3.8/site-packages/sage/cpython/_py2_random.py", line 220, in randint
return self.randrange(a, b+1)
File "/home/jonathan/Applications/sage/local/lib/python3.8/site-packages/sage/cpython/_py2_random.py", line 195, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" %
ValueError: empty range for randrange() (0,0, 0)
sage -t --long --random-seed=180924489930967511277954046005685261728 src/sage/graphs/generators/families.py
**********************************************************************
File "src/sage/graphs/generators/families.py", line 1477, in sage.graphs.generators.families.FriendshipGraph
Failed example:
G.diameter()
Expected:
2
Got:
1
sage -t --long --random-seed=23747002002644886606785003174022326205 src/sage/modules/free_module_integer.py
**********************************************************************
File "src/sage/modules/free_module_integer.py", line 370, in sage.modules.free_module_integer.FreeModule_submodule_with_basis_integer.LLL
Failed example:
new_min <= old_min
Expected:
True
Got:
False
sage -t --long --random-seed=23747002002644886606785003174022326205 src/sage/schemes/toric/sheaf/klyachko.py
**********************************************************************
File "src/sage/schemes/toric/sheaf/klyachko.py", line 951, in sage.schemes.toric.sheaf.klyachko.KlyachkoBundle_class.random_deformation
Failed example:
Vtilde.cohomology(dim=True, weight=(0,))
Expected:
(1, 0)
Got:
(0, 0)
sage -t --long --random-seed=77478494819088915365500074763386376542 src/sage/rings/continued_fraction.py
**********************************************************************
File "src/sage/rings/continued_fraction.py", line 265, in sage.rings.continued_fraction.rat_interval_cf_list
Failed example:
for prec in range(10,54):
R = RealIntervalField(20)
for _ in range(100):
x = R.random_element() * R.random_element() + R.random_element() / 100
l = x.lower().exact_rational()
u = x.upper().exact_rational()
cf = rat_interval_cf_list(l,u)
a = continued_fraction(cf).value()
b = continued_fraction(cf+[1]).value()
if a > b:
a,b = b,a
assert a <= l
assert b >= u
Exception raised:
Traceback (most recent call last):
File "/home/jonathan/Applications/sage/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 694, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/home/jonathan/Applications/sage/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
exec(compiled, globs)
File "<doctest sage.rings.continued_fraction.rat_interval_cf_list[2]>", line 8, in <module>
a = continued_fraction(cf).value()
File "/home/jonathan/Applications/sage/local/lib/python3.8/site-packages/sage/rings/continued_fraction.py", line 2625, in continued_fraction
x1, x2 = check_and_reduce_pair(x)
File "/home/jonathan/Applications/sage/local/lib/python3.8/site-packages/sage/rings/continued_fraction.py", line 2254, in check_and_reduce_pair
raise ValueError("continued fraction can not represent infinity")
ValueError: continued fraction can not represent infinity
**********************************************************************
1 item had failures:
1 of 4 in sage.rings.continued_fraction.rat_interval_cf_list
[437 tests, 1 failure, 5.09 s]
-------------------------------------------------------------
Created tickets for reported failures so far.
Description changed:
---
+++
@@ -1,2 +1,6 @@
- #32543: Mark unstable tests due do #29956.
-- #32763: Increase tolerance in doctests of `modular/modform/numerical.py`
+- #32763: Increase tolerance in doctests of `modular/modform/numerical.py`
+- #32771: Fix incorrect doctest introduced in #29978
+- #32772: Fix two random test failures in graphs
+- #32773: Another test failure with KlyachkoBundle_class.random_deformation
+- #32774: Bug in continue fraction doctest Description changed:
---
+++
@@ -4,3 +4,5 @@
- #32772: Fix two random test failures in graphs
- #32773: Another test failure with KlyachkoBundle_class.random_deformation
- #32774: Bug in continue fraction doctest
+- #32783: random doctest failure in `src/sage/graphs/generic_graph.py`
+sage -t --long --random-seed=321172385432269463934777057410284981568 src/sage/rings/polynomial/multi_polynomial_ring_base.pyx
**********************************************************************
File "src/sage/rings/polynomial/multi_polynomial_ring_base.pyx", line 994, in sage.rings.polynomial.multi_polynomial_ring_base.MPolynomialRing_base.random_element
Failed example:
len(list(f))
Expected:
1
Got:
0
sage -t --long --random-seed=324642231642420052385517156149041216749 src/sage/dynamics/arithmetic_dynamics/projective_ds.py
**********************************************************************
File "src/sage/dynamics/arithmetic_dynamics/projective_ds.py", line 7076, in sage.dynamics.arithmetic_dynamics.projective_ds.?.conjugating_set
Failed example:
D6.conjugating_set(D6)
Expected:
[
[1 0] [0 1] [0 2] [4 0] [2 0] [0 4]
[0 1], [1 0], [1 0], [0 1], [0 1], [1 0]
]
Got:
[
[4 0] [2 0] [0 4] [1 0] [0 1] [0 2]
[0 1], [0 1], [1 0], [0 1], [1 0], [1 0]
]
Description changed:
---
+++
@@ -5,4 +5,4 @@
- #32773: Another test failure with KlyachkoBundle_class.random_deformation
- #32774: Bug in continue fraction doctest
- #32783: random doctest failure in `src/sage/graphs/generic_graph.py`
-
+- #32816: random doctest failure in rings/polynomial/polynomial_element_generic.pyDescription changed:
---
+++
@@ -6,3 +6,4 @@
- #32774: Bug in continue fraction doctest
- #32783: random doctest failure in `src/sage/graphs/generic_graph.py`
- #32816: random doctest failure in rings/polynomial/polynomial_element_generic.py
+- #32817: Fix two tests failing at random Created tickets for reported failures so far.
Description changed:
---
+++
@@ -5,5 +5,7 @@
- #32773: Another test failure with KlyachkoBundle_class.random_deformation
- #32774: Bug in continue fraction doctest
- #32783: random doctest failure in `src/sage/graphs/generic_graph.py`
-- #32816: random doctest failure in rings/polynomial/polynomial_element_generic.py
-- #32817: Fix two tests failing at random
+- #32816: random doctest failure in `rings/polynomial/polynomial_element_generic.py`
+- #32817: Fix two tests failing at random
+- #32849: random doctest failure in `schemes/toric/sheaf/klyachko.py`
+- #32850: random doctest failure in `tests/books/computational-mathematics-with-sagemath/sol/linsolve_doctest.py`Description changed:
---
+++
@@ -2,10 +2,9 @@
- #32763: Increase tolerance in doctests of `modular/modform/numerical.py`
- #32771: Fix incorrect doctest introduced in #29978
- #32772: Fix two random test failures in graphs
-- #32773: Another test failure with KlyachkoBundle_class.random_deformation
+- #32773: Another test failure with KlyachkoBundle_class.random_deformation (duplicate: #32849)
- #32774: Bug in continue fraction doctest
- #32783: random doctest failure in `src/sage/graphs/generic_graph.py`
- #32816: random doctest failure in `rings/polynomial/polynomial_element_generic.py`
- #32817: Fix two tests failing at random
-- #32849: random doctest failure in `schemes/toric/sheaf/klyachko.py`
- #32850: random doctest failure in `tests/books/computational-mathematics-with-sagemath/sol/linsolve_doctest.py`Description changed:
---
+++
@@ -8,3 +8,4 @@
- #32816: random doctest failure in `rings/polynomial/polynomial_element_generic.py`
- #32817: Fix two tests failing at random
- #32850: random doctest failure in `tests/books/computational-mathematics-with-sagemath/sol/linsolve_doctest.py`
+- #32890: Unstable doctest in hmm/chmm.pyx Description changed:
---
+++
@@ -9,3 +9,4 @@
- #32817: Fix two tests failing at random
- #32850: random doctest failure in `tests/books/computational-mathematics-with-sagemath/sol/linsolve_doctest.py`
- #32890: Unstable doctest in hmm/chmm.pyx
+- #32891: Unstable doctest geometry/hyperbolic_space/hyperbolic_model.py Description changed:
---
+++
@@ -10,3 +10,4 @@
- #32850: random doctest failure in `tests/books/computational-mathematics-with-sagemath/sol/linsolve_doctest.py`
- #32890: Unstable doctest in hmm/chmm.pyx
- #32891: Unstable doctest geometry/hyperbolic_space/hyperbolic_model.py
+- #32862: Failure in sage/modular/modform/numerical.py for special seed Description changed:
---
+++
@@ -10,4 +10,5 @@
- #32850: random doctest failure in `tests/books/computational-mathematics-with-sagemath/sol/linsolve_doctest.py`
- #32890: Unstable doctest in hmm/chmm.pyx
- #32891: Unstable doctest geometry/hyperbolic_space/hyperbolic_model.py
-- #32862: Failure in sage/modular/modform/numerical.py for special seed
+- #32862: Failure in sage/modular/modform/numerical.py for special seed
+- #33015: Random failure in number_field_element.pyx Description changed:
---
+++
@@ -12,3 +12,4 @@
- #32891: Unstable doctest geometry/hyperbolic_space/hyperbolic_model.py
- #32862: Failure in sage/modular/modform/numerical.py for special seed
- #33015: Random failure in number_field_element.pyx
+- #33035: random doctest failure in `src/sage/tests/book_stein_ent.py`Description changed:
---
+++
@@ -13,3 +13,4 @@
- #32862: Failure in sage/modular/modform/numerical.py for special seed
- #33015: Random failure in number_field_element.pyx
- #33035: random doctest failure in `src/sage/tests/book_stein_ent.py`
+- #33036: random doctest failure in `src/sage/crypto/util.py`Description changed:
---
+++
@@ -14,3 +14,4 @@
- #33015: Random failure in number_field_element.pyx
- #33035: random doctest failure in `src/sage/tests/book_stein_ent.py`
- #33036: random doctest failure in `src/sage/crypto/util.py`
+- #33039: Random doctest failure in continued_fraction_gosper See also this discussion:
Description changed:
---
+++
@@ -14,4 +14,5 @@
- #33015: Random failure in number_field_element.pyx
- #33035: random doctest failure in `src/sage/tests/book_stein_ent.py`
- #33036: random doctest failure in `src/sage/crypto/util.py`
-- #33039: Random doctest failure in continued_fraction_gosper
+- #33039: Random doctest failure in continued_fraction_gosper
+- #33072: Random failure in matrix_gap.pyxbump to 9.6
Description changed:
---
+++
@@ -1,5 +1,6 @@
- #32543: Mark unstable tests due do #29956.
- #32763: Increase tolerance in doctests of `modular/modform/numerical.py`
+- #33353: increase tolerance of doctests in `modular/modform/numerical.py`
- #32771: Fix incorrect doctest introduced in #29978
- #32772: Fix two random test failures in graphs
- #32773: Another test failure with KlyachkoBundle_class.random_deformation (duplicate: #32849)Description changed:
---
+++
@@ -17,3 +17,4 @@
- #33036: random doctest failure in `src/sage/crypto/util.py`
- #33039: Random doctest failure in continued_fraction_gosper
- #33072: Random failure in matrix_gap.pyx
+- #33628: random failure in `graphs.RandomToleranceGraph`Description changed:
---
+++
@@ -18,3 +18,4 @@
- #33039: Random doctest failure in continued_fraction_gosper
- #33072: Random failure in matrix_gap.pyx
- #33628: random failure in `graphs.RandomToleranceGraph`
+- #33751: random doctest failure in `src/sage/matrix/matrix_integer_dense_hnf.py`Description changed:
---
+++
@@ -19,3 +19,5 @@
- #33072: Random failure in matrix_gap.pyx
- #33628: random failure in `graphs.RandomToleranceGraph`
- #33751: random doctest failure in `src/sage/matrix/matrix_integer_dense_hnf.py`
+- #33880: Another unstable doctest in `hmm/chmm.pyx`
+2 random errors with 9.7.beta5 (both macOS and fedora):
sage -t --long --random-seed=224943010850339078223674527953664126031 src/sage/plot/plot.py
**********************************************************************
File "src/sage/plot/plot.py", line 1819, in sage.plot.plot.plot
Failed example:
plot(f, (x, -3.5, 3.5), detect_poles='show', exclude=[-3..3], ymin=-5, ymax=5)
Expected:
Graphics object consisting of 12 graphics primitives
Got:
Graphics object consisting of 13 graphics primitives
and
sage -t --long --random-seed=224943010850339078223674527953664126031 src/sage/schemes/toric/sheaf/klyachko.py
**********************************************************************
File "src/sage/schemes/toric/sheaf/klyachko.py", line 951, in sage.schemes.toric.sheaf.klyachko.KlyachkoBundle_class.random_deformation
Failed example:
Vtilde.cohomology(dim=True, weight=(0,))
Expected:
(1, 0)
Got:
(0, 0)
The failures in comment:29 already have tickets: the plot failure is #33129 and the Vtilde.cohomology failure is #32773 (which is already in the ticket description). So I added #33129 to the ticket description.
Description changed:
---
+++
@@ -17,6 +17,7 @@
- #33036: random doctest failure in `src/sage/crypto/util.py`
- #33039: Random doctest failure in continued_fraction_gosper
- #33072: Random failure in matrix_gap.pyx
+- #33129: src/sage/plot/plot.py: random doctest failure
- #33628: random failure in `graphs.RandomToleranceGraph`
- #33751: random doctest failure in `src/sage/matrix/matrix_integer_dense_hnf.py`
- #33880: Another unstable doctest in `hmm/chmm.pyx`sage -t --random-seed=124699541928353137721504742146487601068 src/sage/graphs/generators/random.py
**********************************************************************
File "src/sage/graphs/generators/random.py", line 817, in sage.graphs.generators.random.RandomHolmeKim
Failed example:
G.subgraph_search(C3)
Expected:
Subgraph of (): Graph on 3 vertices
Got:
<BLANKLINE>
**********************************************************************
1 item had failures:
1 of 6 in sage.graphs.generators.random.RandomHolmeKim
[197 tests, 1 failure, 1.92 s]
This doctest checks for the existence of a triangle in graphs.RandomHolmeKim(8, 2, 0.5). To reduce the probability of getting a triangle free graph, we can go for a larger graph, or simply mark this test as random.
It's fine with me to mark this test as random, but I think it might be better to delete it, because it doesn't seem to be providing a very meaningful test of the Holme-Kim algorithm. (I am -1 on enlarging the graph or increasing the probability of triangle-formation.) Even if 0.5 is changed to 0 (which I think means that the triangle-formation step is being skipped), the graph still has at least one triangle with high probability.
sage: count = sum(graphs.RandomHolmeKim(8, 2, 0).triangles_count() > 0 for _ in range(1000))
sage: print(f"{round(count/10,1)}% of the graphs have at least one triangle")
99.1% of the graphs have at least one triangle
sage: [graphs.RandomHolmeKim(8, 2, 0).triangles_count() for _ in range(10)]
[4, 4, 3, 4, 3, 4, 3, 2, 2, 3]
Ideally, we would have a better test that the algorithm is working correctly, but I don't have any suggestions.
Description changed:
---
+++
@@ -21,4 +21,5 @@
- #33628: random failure in `graphs.RandomToleranceGraph`
- #33751: random doctest failure in `src/sage/matrix/matrix_integer_dense_hnf.py`
- #33880: Another unstable doctest in `hmm/chmm.pyx`
+- #34520: remove randomly failing doctest in `src/sage/graphs/generators/random.py`
I opened #34520. I agree that it's better to completely remove the doctest and I don't have better tests in mind yet.
with sage 9.8.beta3
sage -t --long --warn-long 41.8 --random-seed=133497414867200611466007568756298202280 src/sage/homology/tests.py
**********************************************************************
File "src/sage/homology/tests.py", line 50, in sage.homology.tests.random_chain_complex
Failed example:
C
Expected:
Chain complex with at most ... nonzero terms over Integer Ring
Got:
Trivial chain complex over Integer Ring
**********************************************************************
File "src/sage/homology/tests.py", line 52, in sage.homology.tests.random_chain_complex
Failed example:
len(C.nonzero_degrees()) in [1, 2]
Expected:
True
Got:
False
**********************************************************************
Description changed:
---
+++
@@ -22,4 +22,4 @@
- #33751: random doctest failure in `src/sage/matrix/matrix_integer_dense_hnf.py`
- #33880: Another unstable doctest in `hmm/chmm.pyx`
- #34520: remove randomly failing doctest in `src/sage/graphs/generators/random.py`
-
+- #34762: fix random chain complex doctestReplying to David Coudert:
with sage 9.8.beta3
sage -t --long --warn-long 41.8 --random-seed=133497414867200611466007568756298202280 src/sage/homology/tests.py ********************************************************************** File "src/sage/homology/tests.py", line 50, in sage.homology.tests.random_chain_complex Failed example: C Expected: Chain complex with at most ... nonzero terms over Integer Ring Got: Trivial chain complex over Integer Ring ********************************************************************** File "src/sage/homology/tests.py", line 52, in sage.homology.tests.random_chain_complex Failed example: len(C.nonzero_degrees()) in [1, 2] Expected: True Got: False **********************************************************************
I have a fix in #34762.
Description changed:
---
+++
@@ -23,3 +23,5 @@
- #33880: Another unstable doctest in `hmm/chmm.pyx`
- #34520: remove randomly failing doctest in `src/sage/graphs/generators/random.py`
- #34762: fix random chain complex doctest
+- #34901: random doctest failure in `src/doc/en/thematic_tutorials/group_theory.rst`
+Description changed:
---
+++
@@ -24,4 +24,5 @@
- #34520: remove randomly failing doctest in `src/sage/graphs/generators/random.py`
- #34762: fix random chain complex doctest
- #34901: random doctest failure in `src/doc/en/thematic_tutorials/group_theory.rst`
+- #34914: random doctest failure in `src/sage/schemes/toric/sheaf/klyachko.py`
Description changed:
---
+++
@@ -24,5 +24,5 @@
- #34520: remove randomly failing doctest in `src/sage/graphs/generators/random.py`
- #34762: fix random chain complex doctest
- #34901: random doctest failure in `src/doc/en/thematic_tutorials/group_theory.rst`
-- #34914: random doctest failure in `src/sage/schemes/toric/sheaf/klyachko.py`
+- #32773: random doctest failure in `src/sage/schemes/toric/sheaf/klyachko.py`