clj-commons/etaoin

Possible bug in `fill-active-el`

dgr opened this issue · 7 comments

dgr commented

Version
This was a private development branch, but the most recent common parent is 602e0ec.

Platform
bb tools-versions reports:

Mac OS X
  version: 14.6.1
  arch: aarch64
Java 17.0.12 - /usr/bin/java
  openjdk version "17.0.12" 2024-07-16 LTS
  OpenJDK Runtime Environment Zulu17.52+17-CA (build 17.0.12+7-LTS)
  OpenJDK 64-Bit Server VM Zulu17.52+17-CA (build 17.0.12+7-LTS, mixed mode, sharing)
Babashka 1.3.191 - /opt/homebrew/bin/bb
  babashka v1.3.191
Image Magick 7.1.1 - /opt/homebrew/bin/magick
  Version: ImageMagick 7.1.1-36 Q16-HDRI aarch64 22352 https://imagemagick.org
Chrome 128.0.6613.85 - /Applications/Google Chrome.app
  128.0.6613.85
Chrome Webdriver 128.0.6613.84 - /opt/homebrew/bin/chromedriver
  ChromeDriver 128.0.6613.84 (606aa55c7d687518d34b55accc5a71ea0bd28727-refs/branch-heads/6613@{#1335})
Firefox 127.0.2 - /Applications/Firefox.app
  127.0.2
Firefox Webdriver 0.34.0 - /opt/homebrew/bin/geckodriver
  geckodriver 0.34.0
Edge 128.0.2739.42 - /Applications/Microsoft Edge.app
  128.0.2739.42
Edge Webdriver 127.0.2651.74 - /Users/dave/bin/msedgedriver
  Microsoft Edge WebDriver 127.0.2651.74 (dbf5b0aa014c4e70e3d5e2d73248e21264f82957)
Safari 17.6 - /Applications/Safari.app
  17.6
Safari Webdriver 17.6 - /System/Cryptexes/App/usr/bin/safaridriver
  Included with Safari 17.6 (19618.3.11.11.5)

Warnings
- Version mismatch: Chrome 128.0.6613.85 != Chrome Webdriver 128.0.6613.84
- Version mismatch: Edge 128.0.2739.42 != Edge Webdriver 127.0.2651.74

Symptom

Failure of a test. Browser was Edge. Rerunning the text resulted in success. You can see that the failure is the difference between "MyLogin" (expected) for the "login" field, versus "mMyLogin" (actual). This suggests that the field is not getting cleared properly. This might be a timing thing with Edge, or possibly a bug in the way that fill-human-el does the underlying filling of the text field.

=== test-input [jvm][edge]
.... fill multiple inputs
.... fill human multiple inputs
.... fill multiple vars
.... fill active
.... fill active human

FAIL in (test-input) (api_test.clj:239)
fill active human
expected: (str/ends-with? (e/get-url *driver*) "?login=MyLogin2&password=MyPassword2&message=Some+text+2")
  actual: (not (str/ends-with? "http://localhost:57916/test.html?login=mMyLogin2&password=MyPassword2&message=Some+text+2" "?login=MyLogin2&password=MyPassword2&message=Some+text+2"))

Reproduction
Difficult to reproduce. Monitor and see if it happens again, either with Edge (might suggest an issue with Edge timing that would require some browser-specific delays in the test suite for Edge, similar to what is done for Safari today) or another browser (would suggest a more subtle bug in fill-human-el, possibly with the semi-random "mistakes").

Actual behavior
Does not fill in the field correctly and that it caught by the test suite.

Expected behavior
Should fill in the field correctly.

Diagnosis
Nope. Could not reproduce on subsequent runs.

Action
Will monitor for now and report if it happens again.

dgr commented

Just happened again with Edge. Different error. First time happened at the beginning of the string. This time an extra "g" was inserted in the middle.

=== test-input [jvm][edge]
.... fill multiple inputs
.... fill human multiple inputs
.... fill multiple vars
.... fill active
.... fill active human

FAIL in (test-input) (api_test.clj:250)
fill active human
expected: (str/ends-with? (e/get-url *driver*) "?login=MyLogin2&password=MyPassword2&message=Some+text+2")
  actual: (not (str/ends-with? "http://localhost:49382/test.html?login=MyLoggin2&password=MyPassword2&message=Some+text+2" "?login=MyLogin2&password=MyPassword2&message=Some+text+2"))
lread commented

Happened again in a CI run on macOS JDK21:

FAIL in (test-input) (api_test.clj:291)
fill active human
expected: (str/ends-with? (e/get-url *driver*) "?login=MyLogin2&***")
  actual: (not (str/ends-with? "http://localhost:49266/test.html?login=lMyLogin2&***" "?login=MyLogin2&***"))
lread commented

I just noticed this sporadic failure on CI (notice the 2 gs in Loggin):

FAIL in (test-input) (api_test.clj:286)
fill active human
expected: (str/ends-with? (e/get-url *driver*) "?login=MyLogin2&***")
  actual: (not (str/ends-with? "http://localhost:49287/test.html?login=MyLoggin2&***" "?login=MyLogin2&***"))

@dgr asked me to reopen this issue.

dgr commented

More data from a failed CI run a few days ago. OS/browser was Windows Edge.

=== test-input [jvm][edge]
.... fill multiple inputs
.... fill-multi bad inputs
.... fill human multiple inputs
.... fill-human-multi bad inputs
.... fill multiple vars
.... fill active
.... fill active human

FAIL in (test-input) (api_test.clj:291)
fill active human
expected: (str/ends-with? (e/get-url *driver*) "?login=MyLogin2&***")
  actual: (not (str/ends-with? "http://localhost:56959/test.html?login=MylLogin2&***" "?login=MyLogin2&***"))
dgr commented

I just noticed this sporadic failure on CI (notice the 2 gs in Loggin):

FAIL in (test-input) (api_test.clj:286)
fill active human
expected: (str/ends-with? (e/get-url *driver*) "?login=MyLogin2&***")
  actual: (not (str/ends-with? "http://localhost:49287/test.html?login=MyLoggin2&***" "?login=MyLogin2&***"))

@dgr asked me to reopen this issue.

@lread , I'm not sure where you got the output for this one. Did you happen to notice the OS and browser?

dgr commented

Another trace from MacOS Edge:

=== test-input [jvm][edge]
.... fill multiple inputs
.... fill human multiple inputs
.... fill multiple vars
.... fill active
.... fill active human

FAIL in (test-input) (api_test.clj:212)
fill active human
expected: (str/ends-with? (e/get-url *driver*) "?login=MyLogin2&***")
  actual: (not (str/ends-with? "http://localhost:49266/test.html?login=MyLoggin2&***" "?login=MyLogin2&***"))
dgr commented

This seems to be Edge specific. At least, everything above that has a browser noted lists Edge. This may point to a bug with Edge that we need to workaround. Seems to be OS independent, as it has been seen on at least Windows and MacOS.