Poor performance of LanguageTool command line checker

Question

Poor performance of LanguageTool command line checker

Closed this issue 3 years ago · 9 comments

I find that my CPU is unusually busy when using flycheck-languagetool.

I notice that languagetool-commandline.jar takes a long time to start up, even if there is no work for it to do. We pay this cost every time the check runs.

$ time java -jar languagetool-commandline.jar /dev/null
No language specified, using English (no spell checking active, specify a language variant like 'en-GB' if available)
Working on /dev/null...
Time: 76ms for 0 sentences (0.0 sentences/sec)

real    0m2.838s
user    0m11.031s
sys     0m0.255s

If we were to use languagetool-server.jar instead, subsequent checks would be much faster:

$ java -cp languagetool-server.jar org.languagetool.server.HTTPServer

$ time curl --data "language=en-GB&text=" http://localhost:8081/v2/check
{"software":{"name":"LanguageTool","version":"5.3","buildDate":"2021-03-29 11:16:13 +0000","apiVersion":1,"premium":false,"premiumHint":"You might be missing errors only the Premium version can find. Contact us at support<at>languagetoolplus.com.","status":""},"warnings":{"incompleteResults":false},"language":{"name":"English (GB)","code":"en-GB","detectedLanguage":{"name":"English (US)","code":"en-US","confidence":0.0}},"matches":[]}
real    0m2.365s
user    0m0.005s
sys     0m0.007s
$ time curl --data "language=en-GB&text=" http://localhost:8081/v2/check
{"software":{"name":"LanguageTool","version":"5.3","buildDate":"2021-03-29 11:16:13 +0000","apiVersion":1,"premium":false,"premiumHint":"You might be missing errors only the Premium version can find. Contact us at support<at>languagetoolplus.com.","status":""},"warnings":{"incompleteResults":false},"language":{"name":"English (GB)","code":"en-GB","detectedLanguage":{"name":"English (US)","code":"en-US","confidence":0.0}},"matches":[]}
real    0m0.064s
user    0m0.006s
sys     0m0.006s

Answer 1 · 2021-06-21T15:09:18.000Z

Work in progress at master...mavit:server

Answer 2 · 2021-06-21T15:54:19.000Z

Cool, good to see someone working on the server one. I don't have much experience with languagetool-server.jar yet. I think it would be great if we are able to keep both options running? So the user can choose to use commandline or server? WDYT? 😕

Answer 3 · 2021-06-22T11:48:50.000Z

I guess the question we should ask ourselves is, can we think of any reason why a user would prefer commandline? If not, providing it as an option only serves to set a trap for users.

Answer 4 · 2021-06-24T03:15:00.000Z

Sorry for the late reply! Since I don't have much experience with languagetool, I would agree to switch over to the server version if the result does not change (or better).

Anyway, I think the server version is still work in progress? I am getting this error. 😕

error in process filter: Wrong type argument: symbolp, 'languagetool

And again, thanks for spending time to improve this package! 😄 👍

Answer 5 · 2021-06-24T10:37:51.000Z

Thanks for trying the branch. It works for me, even with emacs -Q, so I wonder what’s different about our setups.

I have:

emacs 27.2
flycheck-20200527.1601
dash-2.18.1
pkg-info-0.6
epl-0.9

Answer 6 · 2021-06-24T13:07:21.000Z

Here is the full stacktrace

Debugger entered--Lisp error: (wrong-type-argument symbolp 'languagetool)
  symbol-name('languagetool)
  flycheck-error-list-make-last-column("The pronoun ‘We’ must be used with a non-third-per..." 'languagetool)
  flycheck-error-list-make-entry(#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10))
  mapcar(flycheck-error-list-make-entry (#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 3 :column 4 :message "The pronoun ‘He’ is usually used with a third-pers..." :level warning :id nil :group nil :-end-line nil :-end-column 6) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 1 :column 5 :message "The pronoun ‘She’ is usually used with a third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 7)))
  #f(compiled-function #'sequence #<bytecode 0x14768d5>)(flycheck-error-list-make-entry (#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 3 :column 4 :message "The pronoun ‘He’ is usually used with a third-pers..." :level warning :id nil :group nil :-end-line nil :-end-column 6) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 1 :column 5 :message "The pronoun ‘She’ is usually used with a third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 7)))
  apply(#f(compiled-function #'sequence #<bytecode 0x14768d5>) flycheck-error-list-make-entry (#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 3 :column 4 :message "The pronoun ‘He’ is usually used with a third-pers..." :level warning :id nil :group nil :-end-line nil :-end-column 6) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 1 :column 5 :message "The pronoun ‘She’ is usually used with a third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 7)) nil)
  seq-map(flycheck-error-list-make-entry (#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 3 :column 4 :message "The pronoun ‘He’ is usually used with a third-pers..." :level warning :id nil :group nil :-end-line nil :-end-column 6) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 1 :column 5 :message "The pronoun ‘She’ is usually used with a third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 7)))
  flycheck-error-list-entries()
  #f(compiled-function (&optional remember-pos update) "Populate the current Tabulated List mode buffer.\nThis sorts the `tabulated-list-entries' list if sorting is\nspecified by `tabulated-list-sort-key'.  It then erases the\nbuffer and inserts the entries with `tabulated-list-printer'.\n\nOptional argument REMEMBER-POS, if non-nil, means to move point\nto the entry with the same ID element as the current line and\nrecenter window line accordingly.\n\nNon-nil UPDATE argument means to use an alternative printing\nmethod which is faster if most entries haven't changed since the\nlast print.  The only difference in outcome is that tags will not\nbe removed from entries that haven't changed (see\n`tabulated-list-put-tag').  Don't use this immediately after\nchanging `tabulated-list-sort-key'." #<bytecode 0x1074dad>)(t)
  apply(#f(compiled-function (&optional remember-pos update) "Populate the current Tabulated List mode buffer.\nThis sorts the `tabulated-list-entries' list if sorting is\nspecified by `tabulated-list-sort-key'.  It then erases the\nbuffer and inserts the entries with `tabulated-list-printer'.\n\nOptional argument REMEMBER-POS, if non-nil, means to move point\nto the entry with the same ID element as the current line and\nrecenter window line accordingly.\n\nNon-nil UPDATE argument means to use an alternative printing\nmethod which is faster if most entries haven't changed since the\nlast print.  The only difference in outcome is that tags will not\nbe removed from entries that haven't changed (see\n`tabulated-list-put-tag').  Don't use this immediately after\nchanging `tabulated-list-sort-key'." #<bytecode 0x1074dad>) t)
  tabulated-list-print(t)
  tabulated-list-revert(nil nil)
  revert-buffer()
  flycheck-error-list-refresh()
  flycheck-finish-current-syntax-check((#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 3 :column 4 :message "The pronoun ‘He’ is usually used with a third-pers..." :level warning :id nil :group nil :-end-line nil :-end-column 6) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 1 :column 5 :message "The pronoun ‘She’ is usually used with a third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 7)) "c:/")
  flycheck-report-buffer-checker-status(#s(flycheck-syntax-check :buffer #<buffer cool.md> :checker languagetool :context #<killed buffer> :working-directory "c:/") finished (#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 3 :column 4 :message "The pronoun ‘He’ is usually used with a third-pers..." :level warning :id nil :group nil :-end-line nil :-end-column 6) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 1 :column 5 :message "The pronoun ‘She’ is usually used with a third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 7)))
  apply(flycheck-report-buffer-checker-status #s(flycheck-syntax-check :buffer #<buffer cool.md> :checker languagetool :context #<killed buffer> :working-directory "c:/") (finished (#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 3 :column 4 :message "The pronoun ‘He’ is usually used with a third-pers..." :level warning :id nil :group nil :-end-line nil :-end-column 6) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 1 :column 5 :message "The pronoun ‘She’ is usually used with a third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 7))))
  #f(compiled-function (&rest args) #<bytecode 0x3ef1045>)(finished (#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 3 :column 4 :message "The pronoun ‘He’ is usually used with a third-pers..." :level warning :id nil :group nil :-end-line nil :-end-column 6) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 1 :column 5 :message "The pronoun ‘She’ is usually used with a third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 7)))
  funcall(#f(compiled-function (&rest args) #<bytecode 0x3ef1045>) finished (#s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 5 :column 6 :message "The pronoun ‘We’ must be used with a non-third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 10) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 3 :column 4 :message "The pronoun ‘He’ is usually used with a third-pers..." :level warning :id nil :group nil :-end-line nil :-end-column 6) #s(flycheck-error :buffer #<buffer cool.md> :checker 'languagetool :filename "c:/cool.md" :line 1 :column 5 :message "The pronoun ‘She’ is usually used with a third-per..." :level warning :id nil :group nil :-end-line nil :-end-column 7)))
  (save-current-buffer (set-buffer source-buffer) (funcall callback 'finished (flycheck-increment-error-columns (mapcar #'(lambda (x) (apply #'flycheck-error-new-at (append x ...))) (condition-case err (flycheck-languagetool--check-all output) (error (funcall callback 'errored (error-message-string err)) (signal (car err) (cdr err))))))))
  (let ((output (car (flycheck-parse-json (buffer-substring (point) (point-max)))))) (kill-buffer) (save-current-buffer (set-buffer source-buffer) (funcall callback 'finished (flycheck-increment-error-columns (mapcar #'(lambda (x) (apply ... ...)) (condition-case err (flycheck-languagetool--check-all output) (error (funcall callback ... ...) (signal ... ...))))))))
  flycheck-languagetool--read-result(nil #<buffer cool.md> #f(compiled-function (&rest args) #<bytecode 0x3ef1045>))
  apply(flycheck-languagetool--read-result (nil #<buffer cool.md> #f(compiled-function (&rest args) #<bytecode 0x3ef1045>)))
  url-http-activate-callback()
  url-http-content-length-after-change-function(105 2442 2337)
  url-http-wait-for-headers-change-function(1 2447 2446)
  url-http-generic-filter(#<process localhost> "HTTP/1.1 200 OK\15\nDate: Thu, 24 Jun 2021 13:01:04 G...")

I think it's cause by line

https://github.com/mavit/flycheck-languagetool/blob/fcf8be2ab8b0ca86dfe357c0a3389b7dad4fad0d/flycheck-languagetool.el#L155

Remove the quote works for me

...
(apply #'flycheck-error-new-at `(,@x :checker languagetool)))
...

Answer 7 · 2021-06-24T13:12:20.000Z

Feel free to open a PR if you think this is ready to merge! :)

I think you have made decent amount of work on this package. Would you like to join the org and take some control of this package? In addition, you can add yourself as an author if you like to (top of the file). 👍

Answer 8 · 2021-06-27T12:11:52.000Z

I think you have made decent amount of work on this package. Would you like to join the org and take some control of this package?

Sure, why not.

Answer 9 · 2021-06-27T13:09:08.000Z

Cool, just invited you to the org! Welcome aboard!