[BUG] matching on tuple with list and nil element

Question

[BUG] matching on tuple with list and nil element

mindreframer opened this issue a year ago · 9 comments

Found another one! :)

use Mneme, action: :accept, default_pattern: :last
...
auto_assert({["test/some/user_test.exs", "test/some/admin_test.exs"], nil})

Error message:

 test/matcher_test.exs:7
     ** (Mneme.InternalError) Mneme encountered an internal error. This is likely a bug in Mneme.

     Please consider reporting this error at https://github.com/zachallaun/mneme/issues. Thanks!

     ** (FunctionClauseError) no function clause matching in Sourceror.Zipper.node/1
         (sourceror 0.12.2) lib/sourceror/zipper.ex:89: Sourceror.Zipper.node(nil)
         (mneme 0.3.1) lib/mneme/patcher.ex:136: Mneme.Patcher.prepare_assertion/2
         (mneme 0.3.1) lib/mneme/patcher.ex:85: Mneme.Patcher.patch!/4
         (mneme 0.3.1) lib/mneme/server.ex:183: Mneme.Server.do_patch_assertion/2
         (mneme 0.3.1) lib/mneme/server.ex:171: Mneme.Server.handle_continue/2
         (stdlib 4.2) gen_server.erl:1123: :gen_server.try_dispatch/4
         (stdlib 4.2) gen_server.erl:865: :gen_server.loop/7
         (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3


     code: auto_assert({["test/some/user_test.exs", "test/some/admin_test.exs"], nil})
     stacktrace:
       (mneme 0.3.1) lib/mneme/assertion.ex:146: Mneme.Assertion.handle_assertion/4
       (mneme 0.3.1) lib/mneme/assertion.ex:99: Mneme.Assertion.run/3
       test/matcher_test.exs:18: (test)

Answer 1 · 2023-04-15T12:17:34.000Z

It works without action: :accept option.

Answer 2 · 2023-04-15T12:22:09.000Z

It works fine with 0.3.0. Something big changed in 0.3.1

Answer 3 · 2023-04-15T13:43:00.000Z

Hmm, I'm not able to reproduce this. Can you provide some more context? Is it only failing on this particular assertion, or every assertion? Here's what I've tried to reproduce so far:

Elixir 1.14.4 and 1.14.3
Running in clean install with only :mneme as a dep
Adding :maxo_test_iex and running via MIX_ENV=test iex -S mix
Running with and without action: :accept

Answer 4 · 2023-04-15T13:45:03.000Z

Also: This particular error means that Mneme wasn't able to find the assertion in the AST it parsed from the test file. I'm going to improve the error message to log some additional detail when that happens.

Is it possible that the file contents changed between when you first ran the test and when it hit that assertion?

Answer 5 · 2023-04-15T17:01:37.000Z

Hmm... It might be a meta-bug.

I was testing it here:

https://github.com/maxohq/maxo_test_iex/blob/main/test/matcher_test.exs
With TestIex running it's own tests :)

Maybe this led to some un-intended code-caching between the runs.
If this does not affect other projects, it might be OK.

Repro:

$ git clone https://github.com/maxohq/maxo_test_iex
$ git checkout e74dcfbf77df7256f315b966676bdc7cf8c89b26
$ change mix.exs deps to mneme 0.3.1

$ MIX_ENV iex -S mix
iex> TestIex.run()
# delete asserted values in MatcherTest
iex> TestIex.run()

On multiple runs the updates seem to accumulate and not update the source code:

iex(18)> TestIex.run
[test_iex]  Running ["test/matcher_test.exs", "test/test_iex2_test.exs", "test/test_iex_test.exs"]

TestIexTest [test/test_iex_test.exs]
  * test works (0.00ms) [L#4]

TestIex2Test [test/test_iex2_test.exs]
  * test works2 (0.00ms) [L#8]
  * test works (0.00ms) [L#4]

MatcherTest [test/matcher_test.exs]
  * test match matches single file with given line (0.2ms) [L#22]
  * test match matches multiple files (0.5ms) [L#9]


[Mneme] 14 new <--- looks weird

Deleting more than one assertion lead to errors:

iex(23)> TestIex.run
[test_iex]  Running ["test/matcher_test.exs", "test/test_iex2_test.exs", "test/test_iex_test.exs"]

TestIexTest [test/test_iex_test.exs]
  * test works (0.00ms) [L#4]

TestIex2Test [test/test_iex2_test.exs]
  * test works2 (0.00ms) [L#8]
  * test works (0.00ms) [L#4]

MatcherTest [test/matcher_test.exs]
  * test match matches single file with given line (0.2ms) [L#16]
  * test match matches multiple files (0.1ms) [L#9]

  1) test match matches multiple files (MatcherTest)
     test/matcher_test.exs:9
     ** (Mneme.InternalError) Mneme encountered an internal error. This is likely a bug in Mneme.

     Please consider reporting this error at https://github.com/zachallaun/mneme/issues. Thanks!

     ** (FunctionClauseError) no function clause matching in Sourceror.Zipper.node/1
         (sourceror 0.12.2) lib/sourceror/zipper.ex:89: Sourceror.Zipper.node(nil)
         (mneme 0.3.1) lib/mneme/patcher.ex:136: Mneme.Patcher.prepare_assertion/2
         (mneme 0.3.1) lib/mneme/patcher.ex:85: Mneme.Patcher.patch!/4
         (mneme 0.3.1) lib/mneme/server.ex:183: Mneme.Server.do_patch_assertion/2
         (mneme 0.3.1) lib/mneme/server.ex:171: Mneme.Server.handle_continue/2
         (stdlib 4.2) gen_server.erl:1123: :gen_server.try_dispatch/4
         (stdlib 4.2) gen_server.erl:865: :gen_server.loop/7
         (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3


     code: auto_assert(Matcher.match("test", @files))
     stacktrace:
       (mneme 0.3.1) lib/mneme/assertion.ex:146: Mneme.Assertion.handle_assertion/4
       (mneme 0.3.1) lib/mneme/assertion.ex:99: Mneme.Assertion.run/3
       test/matcher_test.exs:11: (test)



[Mneme] 17 new

Finished in 0.00 seconds (0.00s async, 0.00s sync)
5 tests, 1 failure

But honestly, it's probably not a good idea to test a stateful test runner with itself. So yeah, if this is not reproducible otherwise on "normal" code, I assume this is just too much statefulness to handle.

Feel free to close it, if I encounter similar behaviour somewhere else, I'll re-activate this issue.

Sorry to bother you!

Have a great weekend!

Answer 6 · 2023-04-15T17:49:52.000Z

I think you forgot to use Mneme.start(restart: true) 🙂

I was able to reproduce the weird behavior, but adding restart: true seems to clear it up.

There should be a better warning for this, however. Right now I'm making the mistake of ignoring the return value of Supervisor.start_link when I start the Mneme server, which is presumably returning {:error, {:already_started, ...}}. I'm going to match on it and report an error if the server fails to start, which would have made this situation a lot more clear.

Answer 7 · 2023-04-15T17:53:00.000Z

;) Oh, that was it! Thanks for taking a look! Uff... Weird, that it seem to be kinda working with 0.3.0. Not sure why.

But yes, clearer errors would have made the situation ... clearer :)
Will update to the latest now.
Cheers!

Answer 8 · 2023-04-15T17:55:08.000Z

Yep, it does work with restart! Uff, stupid me :)

Answer 9 · 2023-04-15T17:56:57.000Z

No worries at all! I really appreciate you finding all these edge cases and being willing to work through them with me. Smoothing over these rough edges makes everything a lot easier on the next person to give it a try 😄

Going to close this for now, but next release will include better errors when Mneme.start/1 is called multiple times.