Trepan-Debuggers/python3-trepan

Issue with breakpoints in multithreading

Opened this issue · 1 comments

Hi,
It seems trepan3k is not able to stop in breakpoints put inside a function launched as a thread target.
For example, for the script:

import threading
from queue import Queue

fibo_dict = {}
input_list = [3,5,7,9]
shared_queue = Queue()
queue_condition = threading.Condition()

def fibonacci_task(condition):
      with condition: # line 10

        while shared_queue.empty():
            print("[{}] - waiting for elements in queue..".format(threading.current_thread().name))
            condition.wait()

        else:
            value = shared_queue.get()
            a, b = 0, 1
            for item in range(value):
                a, b = b, a + b
                fibo_dict[value] = a

        shared_queue.task_done()
        print("[{}] fibonacci of key [{}] with result [{}]".
              format(threading.current_thread().name, value, fibo_dict[value]))

def queue_task(condition):
    print('Starting queue_task...') # line 29
    with condition:
        for item in input_list:
            shared_queue.put(item)

        print("Notifying fibonacci task threads that the queue is ready to consume...")
        condition.notifyAll()

def main():
    threads = []
    for i in range(3):
        thread = threading.Thread(target=fibonacci_task, args=(queue_condition,))
        thread.daemon = False
        threads.append(thread)

    [thread.start() for thread in threads]

    prod = threading.Thread(name='queue_task_thread', target=queue_task, args=(queue_condition,))
    prod.daemon = False
    prod.start()

    [thread.join() for thread in threads]

    print("[{}] - Result {}".format(threading.current_thread().name, fibo_dict))

if __name__=='__main__':
    main()

If we put breakpoints inside fibonacci_task or queue_task functions:
Trepan3k Execution flow:

$ trepan3k parallel_fibonacci.py
(/tmp/python-debug/trepan3k_issue/parallel_fibonacci.py:1): <module>
-> 1 import threading
(trepan3k) break 10
Breakpoint 1 set at line 10 of file /tmp/python-debug/trepan3k_issue/parallel_fibonacci.py
(trepan3k) break 29
Breakpoint 2 set at line 29 of file /tmp/python-debug/trepan3k_issue/parallel_fibonacci.py
(trepan3k) continue
[Thread-1] - waiting for elements in queue..
[Thread-2] - waiting for elements in queue..
[Thread-3] - waiting for elements in queue..
Starting queue_task...
Notifying fibonacci task threads that the queue is ready to consume...
[Thread-3] fibonacci of key [3] with result [2]
[Thread-2] fibonacci of key [5] with result [5]
[Thread-1] fibonacci of key [7] with result [13]
[MainThread] - Result {3: 2, 5: 5, 7: 13}
The program finished - quit or restart
(/tmp/python-debug/trepan3k_issue/parallel_fibonacci.py:56 @131): <module>
-> 56     main()   

The debug session ends without stopping in the breakpoints.

If we launch the same debug sessoin with pydb:
Pydb Execution flow:

$ pydb --threading parallel_fibonacci.py 
--Call level -1 
Current thread is MainThread
(/tmp/python-debug/trepan3k_issue/parallel_fibonacci.py:1):  <module>
1 import threading
(Pydb) break 10
Breakpoint 1 set in file /tmp/python-debug/trepan3k_issue/parallel_fibonacci.py, line 10.
(Pydb) break 29
Breakpoint 2 set in file /tmp/python-debug/trepan3k_issue/parallel_fibonacci.py, line 29.
(Pydb) continue
Current thread is Thread-1
(/tmp/python-debug/trepan3k_issue/parallel_fibonacci.py:10):  fibonacci_task
10     with condition:
(Pydb) continue
Current thread is Thread-2
(/tmp/python-debug/trepan3k_issue/parallel_fibonacci.py:10):  fibonacci_task
10     with condition:
(Pydb) continue
[Thread-1] - waiting for elements in queue..
Current thread is Thread-3
(/tmp/python-debug/trepan3k_issue/parallel_fibonacci.py:10):  fibonacci_task
10     with condition:
(Pydb) continue
Current thread is queue_task_thread
(/tmp/python-debug/trepan3k_issue/parallel_fibonacci.py:29):  queue_task
29     print('Starting queue_task...')
(Pydb) [Thread-2] - waiting for elements in queue..

Hope this is useful
Regards

rocky commented

Ok. Thanks. I'll look at when I get a chance.

Feel free to take code from pydb and put it the other or redo trepan3k. Or make trepan3k work more like pdb in setting breakpoints. I suspect the filtering I did a long long time ago to make the semantics of stopping more precise are failing here.

At at this point, I'm happy to rip out all of that code and live with what I recall were the somewhat flaky semantics of how pdb does it. (pdb's methods for step, next, continue were imprecise at least when I last looked at it, but since they have less overhead, they are also faster for things like "next" and "continue".

Python 3.7 adds a breakpoint() command, so I may also revisit that sometime after that comes out.