End of Math block sometimes cut off or accelerated in speech
Closed this issue · 11 comments
I was reproducing this fairly consistently earlier today and now it is a 50-50. In the same NVDA session (no restarting NVDA, no file reopening, no disabling/re-enabling MathCAT add-on), I can experience this bug sometimes and not other times (both seem roughly equally likely). What this looks like for a simple Math block that reads "lambda sub 2" is as follows:
Normal: lambda sub 2, slow and steady and respecting the relative speech rate and everything is spoken normally
Buggy: lambda 2, with the sub entirely inaudible and the "oo" in the 2 very sped up or cut off
This is making it harder to process small Math blocks (that contain just a sigma or X or such) as well as the endings of Math blocks (esp. if they contain an important number). There is no difference in the NVDA log for both these behaviours. I am using Microsoft Edge 117.0.2045.47, NVDA 2023.3beta3, and MathCAT version 0.3.6.
I arrow up to the Math block and it may read it in a normal or buggy manner, but identically log as follows:
IO - speech.speech.speak (22:50:31.219) - MainThread (9220):
Speaking ['clickable', RateCommand(multiplier=0.56), ' ', 'lambda sub 2', ' ', RateCommand()]
What speech engine are you using? espeak? SAPI? OneCore?
ESpeak-NG, Voice: English (Great Britain), Variant: Steph3.
That is what I am using too, so that is not the issue.
Confirming that this issue persists post a restart.
I am trying to test MathCAT with all other NVDA add-ons disabled to ensure that is not the source of this issue. Will update this ticket once that testing is done.
While I only spent a few minutes with SAPI and OneCore, this issue may be specific to ESpeak-NG. For Math which ESpeak-NG sped up the end of, OneCore and SAPI enunciated fully and at a steady rate. It is possible though that if I spent enough time with SAPI or OneCore, I might be able to reproduce this issue with them too. For now, let's say this issue occurs only when ESpeak-NG is in use.
@bhavyashah: Can you put the MathML that is (sometimes) causes problems into the issue? That will hopefully help me figure out what is going on.
Example:
Use ESpeak-NG set to Rate 35 with Rate boost enabled and MathCAT's Relative speech rate at 25. I was able to reproduce this on the above example near-consistently with ESpeak NG at 35 with rate boost enabled, but that ceased when I brought down the speech rate to 0 with rate boost enabled or switched to OneCore. Oddly specific configuration.
This issue likely has something to do with speech rates and ESpeak-NG. I have been reproducing this pretty consistently with short Math blocks the last few days where the end of a Math block is distinctly accelerated or garbled. When I reduce speech rate though, at some point the issue gets fixed. Earlier today, moving from 35 to 30 fixed it. Right now, I had to go from 40 to 15 to fix the issue. Quite strange.
There is another issue where for long Math blocks, NVDA decelerates dramatically midway through the Math and then speaks increasingly slowly. Testing to determine whether that too is exclusive to ESpeak-NG, a test case, and a separate ticket are necessary for that. I do wonder though if that issue is related to the one reported in this ticket.
This looks like it is going to be hard to debug...
I haven't been able to reproduce the slowing, but I did get the "4" cut off at the end problem (eventually). I'm not exactly using the same relative speech rate because the current version of the dialog (rhs=100% vs middle being 100% in older dialog) doesn't get me to a relative value of 25 (the closest is 22).
If you come up with any other ideas that make it reproducible, let me know.
The following error showed up in the log message, but it stopped showing up. It didn't reappear if I reloaded the page:
ERROR - scriptHandler.executeScript (14:42:06.080) - MainThread (46224):
error executing script: <bound method CursorManager.script_moveByLine_forward of <NVDAObjects.IAccessible.chromium.ChromeVBuf object at 0x089CDE70>> with gesture 'down arrow'
Traceback (most recent call last):
File "scriptHandler.pyc", line 295, in executeScript
File "cursorManager.pyc", line 274, in script_moveByLine_forward
File "cursorManager.pyc", line 171, in _caretMovementScriptHelper
File "speech\speech.pyc", line 1231, in speakTextInfo
File "speech\types.pyc", line 42, in __iter__
File "speech\speech.pyc", line 1402, in getTextInfoSpeech
File "speech\speech.pyc", line 1203, in _extendSpeechSequence_addMathForTextInfo
File "virtualBuffers\__init__.pyc", line 406, in getMathMl
AttributeError: 'NoneType' object has no attribute 'mathMl'
I did note one other strangeness. The "4" was cut off only on the down arrow, not on the up arrow. However, after reloading the page, it was cut off only on the up arrow, not the down arrow.
In the above message, none of the MathCAT files are mentioned. But it does mention a virtual buffer. So I suspect for speech getting cut off, it is a problem with math and the virtual buffer.
(mainly for my benefit)
The error in the log happens on the last line of
def getMathMl(self, field):
docHandle = int(field["controlIdentifier_docHandle"])
nodeId = int(field["controlIdentifier_ID"])
obj = self.obj.getNVDAObjectFromIdentifier(docHandle, nodeId)
return obj.mathMl
Hence, getNVDAObjectFromIdentifier is return None.
I believe the code for that is the version in MSHTML.py:
def getNVDAObjectFromIdentifier(self, docHandle, ID):
HTMLNode=NVDAObjects.IAccessible.MSHTML.locateHTMLElementByID(self.rootNVDAObject.HTMLNode.document,'ms__id%d'%ID)
if not HTMLNode:
return self.rootNVDAObject
return NVDAObjects.IAccessible.MSHTML.MSHTML(HTMLNode=HTMLNode)
Let me offer below a bunch of test cases.
Math:
Garbled: paren
Math:
Garbled: zero
Math:
Garbled: lambda
Math:
Garbled: n
Math:
Garbled: power
On further testing (with the latest Beta of version 0.3.7), I suspect the issue lies with the relative speech rate option. When I set it to 100, no matter how fast or slow ESpeak-NG is set to, there is no trailing cutoff. However, the moment I alter MathCAT's relative speech rate, I begin hearing something. The optimal values to reproduce the cutoff clearly is ESpeak-NG at 35% with rate boost enabled and MathCAT's relative speech rate at 60%. At my end, I don't get any NVDA errors and the issue occurs equally frequently for up/down arrow presses.
For a previous bug related to espeak slowing down (#51), I had to "monkey patch" the espeak "speak" function. I just double-checked, and the code I patched hasn't changed. My change simply got and reset the rate. I tried removing the patch and the problem was still there.
Here is what is getting sent to the espeak engine:
text=<prosody rate="39%"><say-as interpret-as="characters"><voice xml:lang="en-us">k</say-as> is equal to 2 comma 3 comma 4 </prosody><mark name="657" /></voice>
As you might be able to tell, that's not properly nested XML. However, epeak doesn't seem to mind. I pared this down to
text=<prosody rate="39%"><say-as interpret-as="characters">k</say-as> is equal to 2 comma 3 comma 4 </prosody>
And the problem persisted.
Since speech is cut off at the end, I tried add a <break ...> right before the end prosody tag. Eventually I cut the time all the way down to 1ms, and no cut off speech. It is a bit worrisome doing this because all epeak code is potentially affected, not just MathCAT speech -- that's the nature of monkey patching. However, most speech I saw didn't use <prosody>, so maybe it doesn't affect too much.
There is a new NVDA addon build here. Please test and let me know if the problem is fixed. Also, do you notice the pause?
Please reopen if this doesn't fix the problem.
I quickly tried the new build on the examples in my comment. Thanks for the fix - the speech cutoff seems to be gone. I will need to do further testing to comment on the pause. Also, the slowing down issue seems to persist for me so I will comment on #51.