The int GhostShmem::OpenGhostShmemFd(const char* suffix, pid_t pid) function still return value lower than 0, How to solve it ?
NGUETOUM opened this issue · 16 comments
Hi,
Firstly I want to thanks @jackhumphries for his help. On my previous post I was trying to spawn threads on ghost kernel by modifying multithread application with the code get on experiments folder since when i running shinjuku scheduler and use external app (pushtosched.c script) to spawn threads on ghost kernel I still get Segfault at address 0x10 error. To solve this, @jackhumphries tell me to set all my threads on a PrioTable firstly and then set them as SCHED_ITEM_RUNNABLE. The first approach that I have used is to modify the multithread app by adding a code take on experiments folder. Now I want to spawn threads on ghost kernel by using any multithread application. The aims there is to able to write some pushtosched app which will be able to spawn threads of any multithreads app in ghost kernel without modify the code of that multithread app. To achieve this, I have firstly create a little project with bazel and I have try to externalize (Taking the following files on ghost-userspace and add them in my project) prio_table.cc, prio_table.h, shmem.cc, shmem.h and their dependencies such as base.cc, base.h, ghost_uapi.h, logging.h and Abseil library on my own project. In a main file (The file where I am suppose to spawn threads on ghost kernel) I have try to get pid of all threads of that multithread app given by the following command when I start the program: ls /proc/pid/task | ./launcher 18. the code is look like this:
while (fscanf(stdin, "%d\n", &pid) != EOF) {
threads_pid[numThreads] = pid;
numThreads++;
}
After this, I create a new PrioTable object by using the following code:
ghost::PrioTable table(numThreads, numThreads, ghost::PrioTable::StreamCapacity::kStreamCapacity83);
After I try to attach all threads on a PrioTable by using this code:
for(int j = 0; j <= numThreads; j++){
if(j == 0){
if(table.Attach(threads_pid[j])){
printf("\nThe threads have been attached on PrioTable\n");
}else{
printf("\n The threads cannot be attach on PrioTable \n");
}
}
}
Since the first pid on threads_pid array is the pid of a process during the execution Attach function cannot attach threads on a PrioTable. I have try to track a code and I have see that it is int GhostShmem::OpenGhostShmemFd(const char suffix, pid_t pid)* function in shmem.cc which return value lower than zero and I don't know why.
For my understand (Excuse me if I tell something wrong) I think that the value of p and the value of needle at this line
if (p.rfind(needle, 0) == 0) {
will never be the same since I have try to make some printf on that part of code and this is the way that I obtain:
Please can you help me to solve this issue? thanks.
Hello,
It looks like you are constructing the PrioTable in the launcher process rather than the multithreaded application. If you look at the PrioTable constructor, you'll see that it in turn constructs GhostShmem. GhostShmem associates itself with the PID of whatever process constructed it, so in your case, GhostShmem associates itself with the PID of the launcher process, not the multithreaded app. Thus, the OpenGhostShmemFd() function returns -1 because it is looking for a shared memory region associated with the PID of the multithreaded app, but no such region exists.
You either need to construct the PrioTable first in the multithreaded app, or you need to change the PrioTable and GhostShmem constructors to allow you to pass the PID of the multithreaded app so that they are associated with that PID instead of the PID of the launcher app.
Hello @jackhumphries, I am sorry for my silence and thanks for your answer.
But I have try to implement your solution which consist to modify the constructors of PrioTable and GhostShmem by passing PID of my multithread app since my objective is to implement some app which spawn threads without modify multithread app. But OpenGhostShmemFd(suffix, pid) still return me -1 value. I have try to look the value of memfd_ and that vrariable get value 4 in memfd_ = memfd_create(name.c_str(), MFD_FLAGS);, on void GhostShmem::CreateShmem(int64_t client_version, const char* suffix, size_t size) function which mean that the the shmem have been created successfully. I don't know why OpenGhostShmemFd(suffix, pid) return -1 value now.
At the end of function GhostShmem::CreateShmem(int64_t client_version, const char* suffix, size_t size, pid_t pid), after the shmem is created I have try to print the content of /dev/pts/ folder. The code look like this:
printf("\n/***********************************/\n");
std::string path = "/dev/pts/";
for (auto& f : fs::directory_iterator(path)) {
std::string str = fs::path(f);
//printf("\n p: %s \n", p.c_str());
printf("\n File in /dev/pts : %s \n", str.c_str());
printf("\n shared memory name which is suppose to be create: %s \n", name.c_str());
if (str.rfind(name, 0) == 0) {
printf("\n 1 \n");
}
}
printf("\n/***********************************/\n");
But this is the output that I obtain:
I don't understand why I am not see the shmem in that location Sir.
hello @neelnatu, thanks for your answer.
I have try to find the shared memory file in /proc/self/fd/fd where self is the pid of multithread app (I have just apply the solution that @jackhumphries tell me above). But OpenGhostShmemFd still return -1 since it not found any match of shmem in /proc/self/fd/fd. I don't know what I am suppose to do since on CreateShmem the shmem is created normally, memfd_ = 4 and mmap return me an address in shmem_ variable. Please help me to solve this issue.
Hello @jackhumphries, @neelnatu
Please excuse me, I want to better understand something in a shinjuku scheduler. For my understand when I have browsed the code I have seen this (Please may be I will tell something wrong excuse me):
At the first time during the execution of shinjuku scheduler, on a taskNew function when a new thread is spawned on a kernel, this function call HandleNewGtid(ShinjukuTask* task, pid_t tgid) function, which create a shared memory object and try to bind the spawned thread pid to that shared memory object (if (!orch->Init(tgid))). Me I think that the segFault error get during the execution come from this action since if I have understand the solution that @jackhumphries had proposed me, the Init(tgid) function of Orchestrator class call the Attach(pid) function of PrioTable class which is also to is case call Attach(pid) function of Shmem class which try to connect on shared memory file of client app. But the problem is that the client app don't have a shared memory. This is why shinjuku scheduler get SegFault error.
My problem now is it is possible to create a shared memory of client app using another exernal app such as launcher ? In the case where we don't want to modify the code of client app.
Hello @jackhumphries, @neelnatu
I have try to optimize launcher app in such a way that it can be able to create multiple shared memory with different suffixes (for example /memfd:ghost-shmem-priotable-1, /memfd:ghost-shmem-priotable-2 where -1,-2 represent suffixes ) for multiple client apps, I have also modify the code of shinjuku scheduler by force it to read the PID of launcher (There shinjuku scheduler read on a predefine file that i have create in /tmp and read the pid of launcher with a suffix then use that pid with that suffix to connect on a shared memory) in order to ensure that it connect to a shared memory created by launcher corresponding to a particular client app. After this, launcher app use pid of client app to get the pid of all threads belonging to that app in order to spawn them on ghost kernel. in launcher app I have in my first test before to spawn threads of client app on ghost kernel attached those threads on a priotable by using this line of code ghost_.table_.Attach(thread, suffix); (There I have try to write trhreads information on shared memory before to spawn them) and in my second test I have not used that line of code. For both tests after spawned threads on ghost kernel I have try to make update of sched_item in a priotable by using the following code:
for(int k = 0; k < 6; k++){
ghost::sched_item si;
ghost_.GetSchedItem(k, si);
const uint32_t seq = si.seqcount.write_begin();
si.sid = k;
si.wcid = kWorkClassIdentifier;
ghost::Gtid gtid(gtid_app);
si.gpid = gtid.id();
printf("\n tid : %d \n", gtid.id());
si.deadline = absl::ToUnixNanos(ghost::MonotonicNow() + absl::Milliseconds(50));
si.flags |= SCHED_ITEM_RUNNABLE;
si.seqcount.write_end(seq);
ghost_.SetSchedItem(k, si);
ghost_.table_.MarkUpdatedIndex(k, /* num_retries = */ 3);
}
At the end I obtain the same result for both tests. the threads of client app are spawned on ghost kernel but they did not scheduled by ghost and I don't know why. this is the result that I obtain:
On launcher app it show me that the threads have been spawned on ghost kernel:
Please can you help me to better understand what happen in order to solve this issue ? thanks for your comprehension.
Hello, For my understand I think that, the threads are not scheduled by ghost kernel when I run shinjuku scheduler because those threads are not a type GhostThread. Since the constructor of GhostThread class is define as GhostThread(KernelScheduler ksched, std::function<void()> work); Please it is possible to change that constructor to obtain something like GhostThread(KernelScheduler ksched, pid_t thread); where thread represent the pid of one of threads of client app ?
Hello, I have try to check why the shinjuku scheduler is not able to schedule threads that I have spawned on ghost kernel. I have observe that in callback function void ShinjukuScheduler::SchedParamsCallback(ShinjukuOrchestrator& orch, const ShinjukuSchedParams sp, Gtid oldgtid), at this line ShinjukuTask task = allocator()->GetTask(gtid);, the allocator still return nullptr. It is because in launcher app when i write the threads parameters on a priotable before to spawn them on a kernel, using the following lines of code:
ghost::sched_item si;
ghost_.GetSchedItem(num_items, si);
si.sid = num_items; //num_items represent the number of a thread for example 0 will be thread number 0
si.wcid = kWorkClassIdentifier; // kWorkClassIdentifier = 0
ghost::Gtid gtid((uint64_t)thread); // thread is a value of thread got in /proc/parent_pid/task/
si.gpid = gtid.id();
si.flags |= SCHED_ITEM_RUNNABLE;
ghost_.SetSchedItem(num_items, si);
ghost_.table_.MarkUpdatedIndex(num_items, /* num_retries = */ 3);
At this line si.gpid = gtid.id(); the value return by id() function is still the same value that I read in /proc/parent_pid/task/.
But when I print the value of each thread id in DumpAllTasks function by using gtid.id() I get some value like this 12910465533351670 for each thread in shinjuku scheduler.
I think that the allocator is not able to return task because the gtid that it get and which come from priotable contain thread id value get in /proc/parent_pid/task/.
My question is since in launcher it is not possible for me to have the source code of multithread app, how can I got the unique identifier of each thread ?
It is not possible for me to use some syscall like pid_t x = syscall(__NR_gettid), or gettid(), or pthread_self() because they will return me the thread identifier of launcher.
Hello,
finally I have found solution to my problem. In order to solve that issue in shinjuku scheduler, at this line ShinjukuTask task = allocator()->GetTask(gtid); I have write my own function which help me to retrieve task based on sid parameter got in Priotable since it is not possible for me to get OS thread identifier in a userspace. Launcher is now able to spawn any threads coming from any multithreads app on any scheduler in ghost framework. Thanks for your help and you can close now this issue.
Hello @jackhumphries Please if it is possible can you add this line of code ghost::set_verbose(1); in main function in fifo_agent.cc on centralized fifo scheduler in order to print some information during scheduling process ?
Hi,
I will close this issue -- glad you got everything working. As for set_verbose(), you can specify the verbose level via a command line argument:
./fifo_centralized_agent --verbose 1 ...
Jack
Ok thanks.