See if Seer can support the cuda-gdb debugger.

[epasveer]

The cuda project has their own version of gdb that has enhancements for some cuda related things.

Here's the cuda-gdb project.

https://github.com/NVIDIA/cuda-gdb

And this task I submitted to help me compile cuda-gdb.

NVIDIA/cuda-gdb#14

Documentation on using cuda-gdb.

https://docs.nvidia.com/cuda/cuda-gdb/

[epasveer]

Here's my build commands. Note, it points to the cuda-12.3 installation directory that you should have built earlier. It has the missing header file. Later cuda releases should change "12.3" to the new version.

$ ./configure --program-prefix=cuda- \
    --prefix=/usr/local/cuda \
    --enable-cuda \
    --enable-targets="x86_64-unknown-linux-gnu,arm-elf-linux-gnu,m68k-unknown-linux-gnu" \
    CFLAGS="-I/usr/local/cuda-12.3/extras/Debugger/include/ -I/usr/local/cuda-12.3/include" \
    CXXFLAGS="-I/usr/local/cuda-12.3/extras/Debugger/include/ -I/usr/local/cuda-12.3/include" \
    LDFLAGS='-lpthread'

$ make all-gdb
$ sudo make install

[epasveer]

These are better instructions. It creates and installs the cuda-gdbserver as well.

$ ./configure --program-prefix=cuda- \
    --prefix=/usr/local/cuda \
    --enable-cuda \
    --enable-targets="x86_64-unknown-linux-gnu,arm-elf-linux-gnu,m68k-unknown-linux-gnu" \
    CFLAGS="-I/usr/local/cuda-12.4/extras/Debugger/include/ -I/usr/local/cuda-12.4/include" \
    CXXFLAGS="-I/usr/local/cuda-12.4/extras/Debugger/include/ -I/usr/local/cuda-12.4/include" \
    LDFLAGS='-lpthread'

$ make all-gdb all-gdbserver
$ sudo make install-gdb install-gdbserver

[epasveer]

Looks like these MI commands are available.

erniep@gracie:/nas/erniep/Development/cuda-gdb/cuda-gdb-cuda-toolkit-12.4-release/gdb/mi> grep -i cuda *.c
mi-cmd-break.c:/* NVIDIA CUDA Debugger CUDA-GDB
mi-cmd-break.c:   Modified from the original GDB file referenced above by the CUDA-GDB
mi-cmd-break.c:   team at NVIDIA <cudatools@nvidia.com>. */
mi-cmd-break.c:#ifdef NVIDIA_CUDA_GDB
mi-cmd-break.c:#ifdef NVIDIA_CUDA_GDB
mi-cmd-break.c:  /* CUDA - pending conditional breakpoints*/
mi-cmds.c:/* NVIDIA CUDA Debugger CUDA-GDB
mi-cmds.c:   Modified from the original GDB file referenced above by the CUDA-GDB
mi-cmds.c:   team at NVIDIA <cudatools@nvidia.com>. */
mi-cmds.c:#ifdef NVIDIA_CUDA_GDB
mi-cmds.c:  add_mi_cmd_mi ("cuda-info-devices", mi_cmd_cuda_info_devices);
mi-cmds.c:  add_mi_cmd_mi ("cuda-info-sms", mi_cmd_cuda_info_sms);
mi-cmds.c:  add_mi_cmd_mi ("cuda-info-warps", mi_cmd_cuda_info_warps);
mi-cmds.c:  add_mi_cmd_mi ("cuda-info-lanes", mi_cmd_cuda_info_lanes);
mi-cmds.c:  add_mi_cmd_mi ("cuda-info-kernels", mi_cmd_cuda_info_kernels);
mi-cmds.c:  add_mi_cmd_mi ("cuda-info-blocks", mi_cmd_cuda_info_blocks);
mi-cmds.c:  add_mi_cmd_mi ("cuda-info-threads", mi_cmd_cuda_info_threads);
mi-cmds.c:  add_mi_cmd_mi ("cuda-info-launch-trace", mi_cmd_cuda_info_launch_trace);
mi-cmds.c:  add_mi_cmd_mi ("cuda-info-contexts",  mi_cmd_cuda_info_contexts);
mi-cmds.c:  add_mi_cmd_mi ("cuda-focus-query", mi_cmd_cuda_focus_query);
mi-cmds.c:  add_mi_cmd_mi ("cuda-focus-switch", mi_cmd_cuda_focus_switch);
mi-interp.c:/* NVIDIA CUDA Debugger CUDA-GDB
mi-interp.c:   Modified from the original GDB file referenced above by the CUDA-GDB
mi-interp.c:   team at NVIDIA <cudatools@nvidia.com>. */
mi-interp.c:#ifdef NVIDIA_CUDA_GDB
mi-interp.c:#include "cuda/cuda-api.h"
mi-interp.c:#ifdef NVIDIA_CUDA_GDB
mi-interp.c:  /* CUDA - Attach/Detach support
mi-interp.c:     If cuda-gdb is attaching or detaching, the inferior might
mi-interp.c:  if (cuda_debugapi::attach_or_detach_in_progress ())
mi-main.c:/* NVIDIA CUDA Debugger CUDA-GDB
mi-main.c:   Modified from the original GDB file referenced above by the CUDA-GDB
mi-main.c:   team at NVIDIA <cudatools@nvidia.com>. */
mi-main.c:#ifdef NVIDIA_CUDA_GDB
mi-main.c:#include "cuda/cuda-tdep.h"
mi-main.c:#ifdef NVIDIA_CUDA_GDB
mi-main.c:/* CUDA - memory segments*/
mi-main.c:  *segment = TYPE_CUDA_ALL(type);
mi-main.c:#ifdef NVIDIA_CUDA_GDB
mi-main.c:#ifdef NVIDIA_CUDA_GDB
mi-main.c:  if (TYPE_CUDA_ALL(&dummy_type))
mi-main.c:      rc = cuda_read_memory_partial (addr, mbuf.data (), total_bytes, &dummy_type);
erniep@gracie:/nas/erniep/Development/cuda-gdb/cuda-gdb-cuda-toolkit-12.4-release/gdb/mi> 

[epasveer]

Completed the rough framework. For now, it's another panel, underneath the Threads panel.

I realize I need to get a machine that actually has a NVidia card. So I will pause this task for now until I get one (cheap).

[epasveer]

Changed my mind, the Cuda window is a Visualizer, for now.

[epasveer]

Small success!

Can get the list of Nvidia devices.

Need this set before debugging.

export CUDA_DEBUGGER_SOFTWARE_PREEMPTION=1
cuda-gdb ./a.out
(cuda-gdb) r