Function sent to REPL with ,d is added to wrong package

Question

Function sent to REPL with ,d is added to wrong package

MaxGyver83 opened this issue a year ago · 15 comments

I have this minimal Common Lisp file:

(defpackage :my-package (:use :cl))
(in-package :my-package)
(defun my-function () (write-line "hello"))
(my-function)

When I connect to the swank server with ,c and then evaluate one line after another with ,d, the new function my-function is added to CL-USER instead of MY-PACKAGE. When I copy the defun line and paste it to the REPL, it's added to MY-PACKAGE.

Any idea what could be wrong?

SBCL 2.3.7  Port: 4005  Pid: 430116
; SWANK 2.26

CL-USER> (defpackage :my-package (:use :cl))
#<PACKAGE "MY-PACKAGE">

CL-USER> (in-package :my-package)
#<PACKAGE "MY-PACKAGE">

MY-PACKAGE> (defun my-function () (write-line "hello"))
MY-FUNCTION

MY-PACKAGE> (describe #'MY-FUNCTION)
The function MY-PACKAGE::MY-FUNCTION is undefined.
   [Condition of type UNDEFINED-FUNCTION]
; Evaluation aborted on NIL
MY-PACKAGE> ; Quit to level 1
MY-PACKAGE> ; Evaluation aborted on #<UNDEFINED-FUNCTION MY-FUNCTION {10023DF2B3}>

MY-PACKAGE> (describe #'CL-USER::MY-FUNCTION)
#<FUNCTION COMMON-LISP-USER::MY-FUNCTION>
  [compiled function]


Lambda-list: ()
Derived type: (FUNCTION NIL
               (VALUES (SIMPLE-ARRAY CHARACTER (5)) &OPTIONAL))
Source form:
  (LAMBDA () (BLOCK COMMON-LISP-USER::MY-FUNCTION (WRITE-LINE "hello")))
; No value

copy-pasting here:

MY-PACKAGE> (defun my-function () (write-line "hello"))
MY-FUNCTION
MY-PACKAGE> (describe #'MY-FUNCTION)
#<FUNCTION MY-FUNCTION>
  [compiled function]


Lambda-list: ()
Derived type: (FUNCTION NIL
               (VALUES (SIMPLE-ARRAY CHARACTER (5)) &OPTIONAL))
Source form:
  (LAMBDA () (BLOCK MY-FUNCTION (WRITE-LINE "hello")))
; No value

More information

:echo g:slimv_lisp
sbcl
:echo g:slimv_impl
sbcl

$ pgrep -fa sbcl
131524 sbcl --load /home/max/.vim/pack/plugins/start/slimv/slime/start-swank.lisp
$ sbcl --version
SBCL 2.3.7

Answer 1 · 2023-08-27T11:45:39.000Z

When I comment out this part in my ~/.sbclrc, it works again:

(if (member "--no-linedit" sb-ext:*posix-argv* :test 'equal)
    (setf sb-ext:*posix-argv* 
          (remove "--no-linedit" sb-ext:*posix-argv* :test 'equal))
    (when (interactive-stream-p *terminal-io*)
      (require :sb-aclrepl)
      (require :linedit)
      (funcall (intern "INSTALL-REPL" :linedit)
               :wrap-current t
               :eof-quits t
               :history "~/.sbcl_history")))

When I keep these lines in and set let g:slimv_lisp = 'sbcl --noinform --no-linedit' in my .vimrc, swank doesn't start at all (after typing ,c):

SWANK server is not running. Press ENTER to continue.

Same error when I set g:slimv_swank_cmd like this:

let g:slimv_swank_cmd = '!tmux new-window -d -n REPL-SBCL "sbcl --noinform --no-linedit --load ~/.vim/pack/plugins/start/slimv/slime/start-swank.lisp"'

On the Linedit homepage, they recommend to add this line to .emacs:

(setq inferior-lisp-program "sbcl --noinform --no-linedit")

This is why I tried adding --noinform --no-linedit to g:slimv_lisp and g:slimv_swank_cmd.

Answer 2 · 2023-08-27T11:49:06.000Z

This in my .vimrc works:

let g:slimv_swank_cmd = '!tmux new-window -d -n REPL-SBCL "sbcl --noinform --no-userinit --load ~/.vim/pack/plugins/start/slimv/slime/start-swank.lisp"'
let g:slimv_lisp = 'sbcl --noinform --no-userinit'

Update: let g:slimv_lisp = 'sbcl --noinform --no-userinit' isn't necessary.

Answer 3 · 2023-08-30T20:38:10.000Z

Thank you for the feedback and sorry for the delay. I tried to reproduce the problem but on my system my-function was added to my-package even if evaluated line-by-line.

Please enable the swank debug log info, so that we can see what is the communication between slimv and the swank server. The package is part of the communication, e.g. this is the function definition evaluation on my system:

(:emacs-rex (swank-repl:listener-eval "(defun my-function () (write-line \"hello\"))
") ":my-package" :repl-thread 6)

You can enable debug log info by adding this line to your .vimrc:

let g:swank_log = 1

After that a swank.log file will be created in your working directory. Please repeat the test and paste the part of the log where you evaluate the buffer line-by-line (or just simply send the whole swank.log file to me).

Thank you for the assistance.

Answer 4 · 2023-08-30T21:17:48.000Z

Thanks for your support!

It looks like I can't reproduce this issue. I have tried different values for g:slimv_lisp and g:slimv_swank_cmd and now it works for every combination I have tried.

I suspect that this issue was caused by a broken (quicklisp?) cache, maybe after installing roswell next to Arch's sbcl!? I was wondering that sbcl showed a long list of quicklisp command after every single start. Then I deleted the cache (~/.cache/common-lisp/sbcl-2.3.7-linux-x64/) and the quicklisp issue was gone. Maybe that's the reason I don't need this configuration anymore:

let g:slimv_swank_cmd = '!tmux new-window -d -n REPL-SBCL "sbcl --noinform --no-userinit --load ~/.vim/pack/plugins/start/slimv/slime/start-swank.lisp"'

Answer 5 · 2023-09-10T14:29:54.000Z

Now the problem is back. This time, I'm trying to connect to StumpWM's swank server. (Also see Hacking StumpWM with vim · stumpwm/stumpwm · Discussion #1139).

I have tried to debug this. The problem seems to be that initially after opening ~/.config/stumpwm/config, iskeyword doesn't contain a hyphen:

echo &iskeyword
@,48-57,_,192-255,&

When I send a form with ,d, the function SlimvFindPackage() sets s:swank_package and s:swank_package_form wrongly:

s:swank_package = '(in-package'
s:swank_package_form = '(in (in-package)'

The reason is that the second silent normal! w moves the cursor in (in-package :stumpwm) from i to -.

Only after inserting a blank and pressing Escape, iskeyword is extended:

echo &iskeyword
@,48-57,_,192-255,&,+,-,*,/,%,<,=,>,:,$,?,!,@-@,94,~,#,|,&,.,{,},[,]

Now a w moves the cursor from i to :.

Any idea how to fix?

I think replacing w by W would help but the cword expansion above would still be wrong. Temporarily setting the needed value for iskeyword might fix this.

Answer 6 · 2023-09-10T15:41:12.000Z

call s:SetKeyword() after line ftplugin/slimv.vim +1330 seems to fix this issue for me.

Answer 7 · 2023-09-10T15:50:59.000Z

Before:

[---Sent---] 1341.152077888
(:emacs-rex (swank-repl:create-repl nil) "(in-package" t 3)

[-Received-] 1341.157278291
(:return (:ok ("SWANK-FUZZY" "SWANK-FANCY-INSPECTOR" "SWANK-PACKAGE-FU" "SWANK-ASDF" "SWANK-ARGLISTS" "ASDF" "asdf" "UIOP" "uiop" "SWANK-PRESENTATIONS" "SWANK-REPL" "SWANK-C-P-C" "SWANK-UTIL" "SB-CLTL2" "SB-POSIX" "SB-INTROSPECT" "SB-BSD-SOCKETS")) 2)
[Actionlist] 1341.157453737
2: finished :swank-require 
3: pending  :create-repl 
params: ['"SWANK-FUZZY"', '"SWANK-FANCY-INSPECTOR"', '"SWANK-PACKAGE-FU"', '"SWANK-ASDF"', '"SWANK-ARGLISTS"', '"ASDF"', '"asdf"', '"UIOP"', '"uiop"', '"SWANK-PRESENTATIONS"', '"SWANK-REPL"', '"SWANK-C-P-C"', '"SWANK-UTIL"', '"SB-CLTL2"', '"SB-POSIX"', '"SB-INTROSPECT"', '"SB-BSD-SOCKETS"']
[-Received-] 1341.162787254
(:return (:ok ("COMMON-LISP-USER" "CL-USER")) 3)

After:

[---Sent---] 336.379557668
(:emacs-rex (swank-repl:create-repl nil) ":stumpwm" t 3)

[-Received-] 336.380848173
(:return (:ok ("COMMON-LISP-USER" "CL-USER")) 3)
[Actionlist] 336.380969588
3: finished :create-repl 
params: ['"COMMON-LISP-USER"', '"CL-USER"']

Somehow the sent form is accepted and applied in the stumpwm package but on the other hand, swank returns "COMMON-LISP-USER"/"CL-USER" and shows "CL-USER" in the REPL prompt:

  1 SBCL 2.3.7  Port: 4005  Pid: 775
  2 ; SWANK 2.28
  3 CL-USER> (set-prefix-key (kbd "s-space"))
  4 NIL
  5 CL-USER>

Answer 8 · 2023-09-11T15:47:02.000Z

Could you please check where is option iskeyword set in your system? Just open a .lisp file then enter this command:

:verbose set iskeyword?

For .lisp files it should contain some special characters that do not seem to be included on your system. I think the default vim lisp ftplugin contains these. This is the output on my system:

  iskeyword=@,48-57,_,192-255,+,-,*,/,%,<,=,>,:,$,?,!,@-@,94
        Last set from /usr/share/vim/vim82/ftplugin/lisp.vim line 20

Answer 9 · 2023-09-11T18:51:14.000Z

vim version 9.0.1882.

Just vim or vim -u NORC:

:verbose set iskeyword?
  iskeyword=@,48-57,_,192-255,&
        Last set from /usr/share/vim/vimfiles/after/syntax/lisp.vim line 5

When I open a lisp file with vim -u DEFAULTS or vim --noplugin:

:verbose set iskeyword?
  iskeyword=@,48-57,_,192-255,+,-,*,/,%,<,=,>,:,$,?,!,@-@,94,&
        Last set from /usr/share/vim/vimfiles/after/syntax/lisp.vim line 5

/usr/share/vim/vimfiles/after/syntax/lisp.vim line 5 just adds an ampersand:

set iskeyword+=&

The only syntax/lisp.vim that I can find in the vim repository looks different:

https://github.com/vim/vim/blob/master/runtime/syntax/lisp.vim

The difference between the two iskeyword outputs above seems to be in this line:

setl iskeyword+=+,-,*,/,%,<,=,>,:,$,?,!,@-@,94

https://github.com/vim/vim/blob/596ad66d1ddb742ef349e98eb06b8e4052f68f51/runtime/ftplugin/lisp.vim#L20

(Same in my local /usr/share/vim/vim90/ftplugin/lisp.vim.)

Answer 10 · 2023-09-11T19:21:18.000Z

When I comment out set iskeyword+=& in /usr/share/vim/vimfiles/after/syntax/lisp.vim and execute :verbose set iskeyword? again, the output is:

  iskeyword=@,48-57,_,192-255
        Last set from ~/.vimrc line 1

And line 1 in my ~/.vimrc is set nocompatible.

When I comment out this line, I get:

  iskeyword=@,48-57,_,192-255
        Last set from /usr/share/vim/vimfiles/archlinux.vim line 13

When I comment out this line in /usr/share/vim/vimfiles/archlinux.vim, I get only:

  iskeyword=@,48-57,_,192-255

Answer 11 · 2023-09-11T19:54:12.000Z

Finally I got it:

Only when I use vim -u DEFAULTS or vim --noplugin, line 20 of /usr/share/vim/vim90/ftplugin/lisp.vim extends iskeyword. Otherwise the file is skipped because my ~/.vim/ftplugin/lisp.vim (in which I set slimv options, for example) replaces the system's ftplugin.

The correct way is to move my ~/.vim/ftplugin/lisp.vim to ~/.vim/after/ftplugin/lisp.vim.

My fault! 🙃 Sorry for the trouble and thanks for your support. Using verbose set iskeyword? led me to my misconfiguration!

Answer 12 · 2023-09-11T20:26:16.000Z

One more question:

Should the REPL prompt show the current package?

When the REPL is created like this:

[---Sent---] 6439.276644843
(:emacs-rex (swank-repl:create-repl nil) "stumpwm" t 3)

I get a CL-USER> prompt. Even when I execute in-package, the prompt still shows CL-USER>:

SBCL 2.3.7  Port: 4005  Pid: 815
; SWANK 2.28
CL-USER> (in-package :stumpwm)
#<PACKAGE "STUMPWM">
CL-USER> *package*
#<PACKAGE "STUMPWM">
CL-USER>

Answer 13 · 2023-09-12T06:23:12.000Z

Every swank server message contains a package where the specific message is to be executed. If for an in-package message the current package differs from the one sent in the message, then the swank server responds with a :new-package, slimv catches it and changes the prompt. I assume that your swank server did not respond with a :new-package. I'm not sure why, but you can check it in the swank.log file.

When you eval an :(in-package :xxx) from a .lisp file by pressing ,d then the swank server again may not respond with :new-package, because there slimv checks the package definitions in the file and sets the current package for the swank server for every evaluation message. Therefore the swank server thinks that you are already in that package and does not respond with a :new-package.

This is a sample when I eval in-package via ,d:

(:emacs-rex (swank-repl:listener-eval "(in-package :my-package)") ":my-package" :repl-thread 5)

This is a sample when I enter the same in the REPL buffer, note that here `COMMON-LISP-USER' was passed as current package:

(:emacs-rex (swank-repl:listener-eval "(in-package :my-package)") "COMMON-LISP-USER" :repl-thread 7)
...
(:new-package "MY-PACKAGE" "MY-PACKAGE")

Answer 14 · 2023-09-12T19:47:58.000Z

Thanks for the explanation! I can confirm that swank never answers with :new-package. It always looks like in your first example: (in-package :stumpwm) is always executed in the stumpwm package, so the two packages always match.

When I set let g:slimv_package = 0, I can update the prompt by sending an in-package expression.

What is the intended behavior with let g:slimv_package = 1? Would it make sense to treat in-package expressions as a special case and not to reset the package after the expression has been evaluated? Or maybe even never to reset the package!?

Answer 15 · 2023-09-14T16:18:52.000Z

g:slimv_package is just for backward compatilibity with very old versions of slimv that did not handle packages at all. There is not much use of it right now, I think. Anyway, if it is set to 0, then slimv does not handle packages at all.

in-package: the package variable is reset because you can have multiple buffers open, some of have no packages at all. If we just simply keep the current package, then the other buffer with no package would be evaluated with an incorrect package. Or even in the same buffer, the first few lines may not contain any package information. So resetting the package after each evaluation seemed a reasonable approach.