takagi/cl-cuda

to be accepted in Quicklisp distribution

takagi opened this issue · 10 comments

takagi commented

Currently, cl-cuda is not available in Quicklisp distribution because of its testing policy (see #514 in quicklisp-projects).

It may be accepted if it just finished to be compiled without condition on an environment where CUDA SDK is not installed even though it does not work.

Approach:

  1. try to compile grovel files which include cuda.h before evaluate the defsystem form in cl-cuda.asd
  2. a condition would be signaled since CUDA SDK is not installed
  3. handle the condition and push a flag to *features* which mentions CUDA SDK is not installed
  4. in the defsystem form, avoid cffi-grovel:grovel-file form to be evaluated by looking *features*

Quetions:

  • may be warned that some symbols are not found if avoid cffi-grovel:grovel-file?
melisgl commented

Pushing to *FEATURES* may not interact well with saved images or executables whose runtime environment can vary.

takagi commented

Thanks. I will take it into account.

takagi commented

Following shows how to determine CUDA SDK is already installed or not.

Goal
Determining that CUDA SDK is already installed or not.

Approach
Processing a CFFI-Grovel specification file without defsystem forms and handling a condition signaled.

Detail
Put a CFFI-Grovel specification file foo.lisp containing following contents in cl-cuda system's directory.

(include "cuda.h")

Evaluate the following form to process the CFFI-Grovel specification file.

(asdf:perform
  (make-instance 'cffi-grovel::process-op)
  (make-instance 'cffi-grovel::grovel-file :pathname "foo"
                                           :parent (asdf:find-system "cl-cuda")))

As the result, if CUDA SDK is not installed, the following condition is signaled.

External process exited with code 1.
Command was: "cc" "-m64" "-I" "/opt/local/include/" "-I/Users/mtakagi/Lisp/quicklisp/dists/quicklisp/so..." "-o" "/Users/mtakagi/.cache/common-lisp/sbcl-1.0.55-maco..." "/Users/mtakagi/.cache/common-lisp/sbcl-1.0.55-maco..."
Output was:
/Users/mtakagi/.cache/common-lisp/sbcl-1.0.55-macosx-x64/Users/mtakagi/Lisp/cl-cuda/foo.c:6:10: fatal error: 'cud.h' file not found
#include <cuda.h>
         ^
1 error generated.

   [Condition of type SIMPLE-ERROR]

It can be determined that CUDA SDK is installed or not, although some false negatives remained that the compiler can not find cuda.h file because of some reasons such as appropriate environment variables are not specified.

Problems

  • CFFI-GROVEL package's private symbols are used
  • where to put or generate a CFFI-Grovel specification file to test?
  • (ASDF:FIND-SYSTEM "cl-cuda") can not be evaluated before cl-cuda's DEFSYSTEM form
guicho271828 commented

how about subclassing cffi-grovel::grovel-file (e.g. cuda-header) and make it have the :around method
that handles the error?
well, I'm not concerned with cl-cuda, but responding just on curiosity.

takagi commented

What you mean is something like this?

(defclass cuda-header (cffi-grovel:grovel-file) ())

(defmethod asdf:perform :around ((op cffi-grovel::process-op) (c cuda-header))
  (handler-case (call-next-method op c)
    (error (lambda (c) (handle-the-error))))

It would be nice.

takagi commented

Following shows consideration on guicho271828's suggestion.

Goal
To finish cl-cuda to be loaded without any conditions on environments where CUDA SDK is not installed even though it does not work.

Approach
Subclassing cffi-grovel:grovel-file with an :around method that handles the condition causing if CUDA SDK is not installed.

(defclass cuda-grovel-file (cffi-grovel:grovel-file) ())

(defmethod asdf:perform :around ((op cffi-grovel::process-op) (c cuda-grovel-file))
  (handler-case (call-next-method op c)
    (error (e))))

Then, use the following ASDF components in cl-cuda's defsystem form.

(cuda-grovel-file "type-grovel")
(cuda-grovel-file "enum-grovel")

Result
Cl-cuda is finished to be loaded without causing any conditions.

  • (added Dec. 26, 2014) can handle the error on cffi-grovel::process-op
  • (added Dec. 26, 2014) asdf:compile-op and asdf:load-op also cause file-not-found error after cffi-grovel::process-op
  • (added Dec. 26, 2014) undefined symbols are not resolved because of groveling skip

Problems

  • (added Dec. 26, 2014) skip or abort asdf:compile-op and asdf:load-op following cffi-grovel::process-op
  • (added Dec. 26, 2014) resolve undefined symbols when skip groveling: cu-result, cu-module, cu-function, ...
  • is this enough for Quicklisp acceptance?
  • cffi-grovel package's private symbol cffi-grovel::process-op is used
  • cl-cuda-test test will not pass, of course
  • how to behave when loaded cl-cuda is to be used on environments where CUDA SDK is not installed?
melisgl commented

Very cool. I hope it's enough for quicklisp.

takagi commented

I rewrite the result and problems on my previous comment.

takagi commented

For resolving undefined symbols, I will define them in working around for the case cuda.h not found. Additionally, I will modify to use grovel only for architecture depending types: CUdeviceptr and size_t.

takagi commented

Another context, I asked Quicklisp again because I found this IRC log and guess he thinks a physical board is required for just building cl-cuda.

quicklisp/quicklisp-projects#514