google/ghost-userspace

sched_setscheduler(tid) failed: Bad file descriptor when using pushtosched

NGUETOUM opened this issue · 3 comments

Hello,
I have meet an issue when i tried to spawn threads of multithreads app in ghost kernel. Just recently with my previous computer i was able to spawn threads on ghost kernel normally by using pushtosched. Today i now use a new computer, i have compiled and install ghost kernel on Ubuntu 20.04 but after the installation i had met 02 issues.
The first issue was about ghost-userspace compilation where during the compilation I had met an error with the following lines in third_party/bpf/common.bpf.h:

static long (*bpf_ghost_wake_agent)(__u32 cpu) = (void *) 3000;
static long (*bpf_ghost_run_gtid)(__s64 gtid, __u32 task_barrier, __s32 run_flags) = (void *) 3001;
static long (*bpf_ghost_resched_cpu)(__u32 cpu, __u64 cpu_seqnum) = (void *) 3002;

image

To skip this issue I have commented those three lines and the compilation completed successfully.

The second issue was about /sys/fs/ghost/ folder which was not present in my computer, but i had followed #18 (modify the init/kconfig file in ghost-kernel) and that solution solved my problem. I am now able to see /sys/fs/ghost folder.

My problem now is that when I use pushtosched app to spawn thread on ghost kernel. It fail with the following error: sched_setscheduler(tid) failed: Bad file descriptor where tid is the process id of client app.

image

Please can you help me to solve this issue.
thanks.

brho commented

for setsched, there are a few things that can go wrong. do you have more than one enclave in the system? pushtosched assumes it is /sys/fs/ghost/enclave_1.

can you try manually moving the task into the enclave? to do so:

echo $PID > /sys/fs/ghost/enclave_1/tasks

that's basically what pushtosched does internally. and if you have another enclave, then push to e.g. enclave_2/tasks instead.

if that works, my next guess is to make sure pushtosched was rebuilt too.

Hello @brho,
I just have one enclave in /sys/fs/ghost. When I run the command echo $PID > /sys/fs/ghost/enclave_1/tasks it works and the threads are spawn on ghost kernel. I have try to run pushtosched again but i got the same error as above.

It is only Fifo_per_cpu which work, I have try to run that command with Sol and Fifo_centralized but nothing is done when i run the command. I have not try with Shinjuku and EDF schedulers since my client app don't implement the shared memory.