JSON readtable error
Opened this issue · 8 comments
ob-ipython is great and works well, it is breaking other functionalities. It may be too vague, but weirdly enough, after loading ob-ipython, I cannot export other org files to pdf because it gives that "JSON readtable error", even though the .org doesn't have any code block.
I have no clue where to start looking at to find why ob-ipython is breaking other functionalities. The debug messages are not helping very much (considering the short time I have to invest in fixing it myself at the moment). Any help is much appreciated.
The problem only happens when I load emacs and add ipython to org-babel-load-langugage
with
(custom-set-variables '(org-babel-load-languages '((ipython . t))))
One situation the "JSON readtable error" happens is when I include another org file with:
#+INCLUDE: <another-org-file>.org
Use emacs-jupyter: https://github.com/dzop/emacs-jupyter
It requires emacs 26. I will try that later. Thanks for the suggestion.
Anyone?
I found the solution. The problem is the function
shell-command-to-string
of this function:
(defun ob-ipython--get-kernels ()
"Return a list of available jupyter kernels and their corresponding languages.
The elements of the list have the form (\"kernel\" \"language\")."
(and ob-ipython-command
(let ((kernelspecs (cdar (json-read-from-string
(shell-command-to-string
(s-concat ob-ipython-command " kernelspec list --json"))))))
(-map (lambda (spec)
(cons (symbol-name (car spec))
(->> (cdr spec)
(assoc 'spec)
cdr
(assoc 'language)
cdr)))
kernelspecs))))
In my .emacs.d
I had
(setq shell-command-switch "-ic")
The option -i
resulted in an error message from shell interactive command:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
The function ob-ipython--get-kernels
above is concatenating that error message to the json string, resulting in the error "JSON readtable error".
Solution 1: use just
(setq shell-command-switch "-c")
Solution 2 (better): change shell-command-to-string
in that function to make sure the output does not include any error or warning message from the shell. I will probably do that later, but maybe someone here knows a quick fix in that direction.
FWIW the shell-command-switch variable setting of "-ic" does not cause this problem for me. I am not sure why. I am using a Mac. A quick solution would be to let-bind shell-command-switch inside the function, e.g.
(defun ob-ipython--get-kernels ()
"Return a list of available jupyter kernels and their corresponding languages.
The elements of the list have the form (\"kernel\" \"language\")."
(and ob-ipython-command
(let* ((shell-command-switch "-c")
(kernelspecs (cdar (json-read-from-string
(shell-command-to-string
(s-concat ob-ipython-command " kernelspec list --json"))))))
(-map (lambda (spec)
(cons (symbol-name (car spec))
(->> (cdr spec)
(assoc 'spec)
cdr
(assoc 'language)
cdr)))
kernelspecs))))
This also happens if the command from jupyter
gives any error. Mine was complaining about the presence of both ~/.config/ipython
and ~/.ipython
. After "~/.config/ipython" was deleted, ob-ipython--get-kernels
could run successfully.
When this is broken, it also breaks other aspects of org, like captures, for some reason.
Solution 2 (better): change
shell-command-to-string
in that function to make sure the output does not include any error or warning message from the shell. I will probably do that later, but maybe someone here knows a quick fix in that direction.
If I understand your idea correctly, using the following instead of
(shell-command-to-string some-command)
might solve the problem:
(with-output-to-string
(with-current-buffer
standard-output
(process-file shell-file-name nil '(t nil) nil shell-command-switch some-command)))
See: https://emacs.stackexchange.com/questions/21422/how-to-discard-stderr-when-running-a-shell-function
Background: using Python 3.11 creates a similar problem. The command
python -Xfrozen_modules=off -m jupyter kernelspec list --json
outputs the following to stderr:
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
And this throws ob-ipython off.