byteduck/duckOS

[TODO] Graceful kernel stack overflow handling

Closed this issue · 1 comments

The Problem

As it stands, kernel stacks have absolutely no protection whatsoever and will just trample whatever happens to be adjacent in memory when they overflow. Obviously, this is a problem.

The fix

The obvious fix is to add sentinel pages on either side of each kernel stack so that a fault occurs when the stack overflows (assuming there's no malice involved). However, as we handle page faults currently, this means that we triple-fault; if the stack pointer is in an unusable area of virtual memory, then we properly proceed with the ISR in the first place.

So, we need to change how we handle page faults. We'll probably need to use a task gate in the descriptor table for the page fault ISR, so that we can have a known safe stack to use for it. And I'll have to learn how hardware tasks work. :)

This is fixed! We now set up a separate hardware task with a small emergency stack in the GDT and use a task gate for the double fault ISR so that we can handle it more gracefully instead of triple-faulting. We also grab the stack / base pointers from the main TSS to determine if we probably hit a stack overflow & to give a backtrace.