When using game script, text box in main window doesn't change to script line
ShadowVolt opened this issue · 11 comments
So, I don't know if this is intended behavior or not, but when using the game script feature, it doesn't swap to the line from the script in the main window. I can see the correct lines in the log when i look in the drop-down menu, but selecting them there doesn't update it. I'm using version 0.4.9.9 on windows via chrome.
(As a sidenote, an option to automatically select the most probable line from the script without having to use the menu would be really nice, since the text i'm using includes every string in the game. I saw that part about confidence in #17, could basically just optionally always go with the most likely choice?)
The attached image is an example of what I'm talking about.
You are correct.
- Main window and log window out of sync is a bug.
- when latest log is modified in the log window -> main window text is updated
- when main window text is modified -> latest log in log window is updated.
(Neither works currently.)
- Auto-choosing most likely confidence is not yet implemented. Two reasons:
- not sure what default confidence value to use (70% up is pretty good for some games, but there are exceptions, especially when the matching text is too short)
- sometimes the correct one has lower confidence value.
I left this feature unfinished to include a texthooker, but I will come back to improve on this feature by allowing optional replacement with both custom confidence value and minimum matching text length.
Another thing I planned was exact match search by the highlighted words. That also has not been implemented.
Ah, alright. Good to know that it is indeed a bug.
Also, wanted to say thanks for working on this. It is exactly the thing I needed, no other software that I could find seems to perfectly cover this niche. As someone who doesn't know how to program, I was almost ready to give up on this idea until I stumbled across this. (edit: accidentally closed this, you may re-open if needed.)
Data Flow for text extraction and logging.
To change:
- allow optional logging in index.js : : updateOutput()
To add:
- game2text.py : : update_main_window_text()
- game2text.py : : update_log_window_text()
- index.js : : changeOutputText()
Added sync between main window and latest log in log window:
d47ffff
Can confirm that the window sync now works as expected as of 0.4.9.9b, but it still doesn't update after selecting the game script, unless something is manually typed afterwards. (Unsure if this is a new bug or just still work-in-progress, but I figured I should mention it.)
Can confirm that the window sync now works as expected as of 0.4.9.9b, but it still doesn't update after selecting the game script, unless something is manually typed afterwards. (Unsure if this is a new bug or just still work-in-progress, but I figured I should mention it.)
Added auto-replacement for matches that pass threshold. Default to 85, can be changed in config.ini
Synced main window to match replacements by both manual and auto matching.
Include original recognized text as an option in case auto-matching is incorrect.
Implemented game script cache and regional scan. In ideal situations, matching is instantaneous.
More on ideal situations:
- next line is within boundary (default boundary set to +/- 100 lines).
- the match bewteen the line and the ocr result reaches the confidence threshold, which is set to 85 by default
regional_scan.mp4
If neither requirement is satisfied, a regular match is performed. There have been cases where the correct match has a confidence score lower than 85 and an incorrect line a few thousand lines apart had a higher confidence score. There is no solution for this as far as I can tell apart from lowering the confidence threshold.
game script cache
regional scan.
With the potential speed of automatic matching, it seems like a UI/color change of the result is needed to tell the user the text has been changed.
The flashing main window text can also be an accessibility issue.
Any color changes would need to be intuitive in dark mode and light mode. For example a teal background for auto-matched text in light mode would not be suitable for dark mode.
I don't have a good design proposition yet.
One idea I had for that would be if you have a game script selected (this would not be default behavior if it's set to none), you could prevent the main window from updating until the match has either succeeded or failed, then update it from the log and add a UI indicator on the top bar (separate from the text, below the stream window) that lights up if it successfully matched a line. This would prevent the flashing text while still indicating it worked; and would possibly even speed it up a little, since it only pulls the text once? (not sure how the OCR backend works, this might be more difficult to implement than it sounds)
However, I don't think that idea about changing the text background color would go well, if you mean that literally. That would result in 95% of the lines being a weird color, instead of the normal browser behavior.
(And as a sidenote: wanted to mention that the addition of a limit to the log window fixed that slowdown I was having perfectly!)
you could prevent the main window from updating until the match has either succeeded or failed, then update it from the log and add a UI indicator on the top bar (separate from the text, below the stream window) that lights up if it successfully matched a line.
It sounds good if we always have ideal situations (quick and accurate matching), but when matching is slow the user will have to wait for the match before they can see the OCR result, not to mention if the match is incorrect they will be led to believe that the OCR result was incorrect.
but when matching is slow the user will have to wait for the match before they can see the OCR result,
For the first of those two cases, it would still be showing as the OCR result when first it comes in through the log, giving plenty of time to see it while it's working on the slow lines. (though i'm sure there are still some edge cases where the match is super fast despite the OCR taking forever, which leads me to my second point.)
not to mention if the match is incorrect they will be led to believe that the OCR result was incorrect.
For the second, possibly could add a script-match dropdown for the current line to the main window as well (possibly through the new indicator I mentioned, which would double as a button?), and highlight the original OCR as the first match (unlike the log window, this time with a background color to distinguish it), along with high-confidence lines just like in the log window. When I use the script matching feature, I'm almost always using the log in place of the main display since I can't pick the script lines anywhere aside from the log; so this could help that part as well.
In my experience using this feature, in most cases when it gets a false positive match, it is VERY different than the original line, making it hard to miss when it does happen. The only potential pitfall I can think of would be if you are using a partial gamescript (eg. less than half the lines in the file) it would make things more likely to false-positive, but this already happens as it stands.