Don't print stack frames from certain libraries
Closed this issue · 5 comments
Is it possible to not print stack frames from third party library? Ideally with a confiuration ignore=['pandas', 'numpy'] or a function ignore(['pandas', 'numpy'])?
Unfortunately not, no. I'm not sure how much I want that either, to be honest - I'm not a fan of hiding information from error messages. They should be helpful, not terse. Can you show me an example of an undesirable stacktrace?
Below is a code snippet that perfectly illustrates the issue:
from apscheduler.schedulers.blocking import BlockingScheduler
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore
from apscheduler.executors.pool import ProcessPoolExecutor
from datetime import datetime, timedelta, timezone
from loguru import logger
from pytz import utc
def create_scheduler():
jobstores = {
'default': SQLAlchemyJobStore(url='sqlite:///jobs.sqlite')
}
executors = {
'default': ProcessPoolExecutor(1),
}
job_defaults = {
'coalesce': False,
'max_instances': 4
}
scheduler = BlockingScheduler(jobstores=jobstores, executors=executors, job_defaults=job_defaults, timezone=utc)
return scheduler
def add_adhoc_job(scheduler, task_func, task_name, task_args=None):
scheduler.add_job(
id=task_name,
trigger='date',
func=task_func,
args=task_args,
next_run_time=datetime.now(timezone.utc) + timedelta(minutes=1),
name=f"scheduled_{task_name}",
)
logger.debug(f"Added adhoc task {task_name}")
def print_greeting(greeting, name):
print(f"{greeting} {name}")
try:
raise Exception("NOOOOOOOOOOOOOO")
except:
logger.exception('')
scheduler = create_scheduler()
add_adhoc_job(scheduler, print_greeting, "print_task1", task_args=("Hello", "World",))
scheduler.start()Here's the stacktrace from the better-exceptions lib:
2021-10-18 20:10:47.239 | ERROR | __main__:print_greeting:120 -
Traceback (most recent call last):
File "python-test/apscheduler-test.py", line 127, in <module>
scheduler.start()
│ └ <function BlockingScheduler.start at 0x7f1078a7e7b8>
└ <apscheduler.schedulers.blocking.BlockingScheduler object at 0x7f1075f86cf8>
File "/usr/local/lib/python3.6/dist-packages/apscheduler/schedulers/blocking.py", line 19, in start
self._main_loop()
│ └ <function BlockingScheduler._main_loop at 0x7f1077883d90>
└ <apscheduler.schedulers.blocking.BlockingScheduler object at 0x7f1075f86cf8>
File "/usr/local/lib/python3.6/dist-packages/apscheduler/schedulers/blocking.py", line 30, in _main_loop
wait_seconds = self._process_jobs()
│ └ <function BaseScheduler._process_jobs at 0x7f1077883b70>
└ <apscheduler.schedulers.blocking.BlockingScheduler object at 0x7f1075f86cf8>
File "/usr/local/lib/python3.6/dist-packages/apscheduler/schedulers/base.py", line 974, in _process_jobs
executor.submit_job(job, run_times)
│ │ │ └ [datetime.datetime(2021, 10, 19, 0, 10, 47, 222096, tzinfo=datetime.timezone.utc)]
│ │ └ <Job (id=print_task1 name=scheduled_print_task1)>
│ └ <function BaseExecutor.submit_job at 0x7f107898f488>
└ <apscheduler.executors.pool.ProcessPoolExecutor object at 0x7f107652fd68>
File "/usr/local/lib/python3.6/dist-packages/apscheduler/executors/base.py", line 71, in submit_job
self._do_submit_job(job, run_times)
│ │ │ └ [datetime.datetime(2021, 10, 19, 0, 10, 47, 222096, tzinfo=datetime.timezone.utc)]
│ │ └ <Job (id=print_task1 name=scheduled_print_task1)>
│ └ <function BasePoolExecutor._do_submit_job at 0x7f10788e0d08>
└ <apscheduler.executors.pool.ProcessPoolExecutor object at 0x7f107652fd68>
File "/usr/local/lib/python3.6/dist-packages/apscheduler/executors/pool.py", line 22, in _do_submit_job
f = self._pool.submit(run_job, job, job._jobstore_alias, run_times, self._logger.name)
│ │ │ │ │ │ │ │ │ │ └ 'apscheduler.executors.default'
│ │ │ │ │ │ │ │ │ └ <Logger apscheduler.executors.default (WARNING)>
│ │ │ │ │ │ │ │ └ <apscheduler.executors.pool.ProcessPoolExecutor object at 0x7f107652fd68>
│ │ │ │ │ │ │ └ [datetime.datetime(2021, 10, 19, 0, 10, 47, 222096, tzinfo=datetime.timezone.utc)]
│ │ │ │ │ │ └ <member '_jobstore_alias' of 'Job' objects>
│ │ │ │ │ └ <Job (id=print_task1 name=scheduled_print_task1)>
│ │ │ │ └ <Job (id=print_task1 name=scheduled_print_task1)>
│ │ │ └ <function run_job at 0x7f10789aad08>
│ │ └ <function ProcessPoolExecutor.submit at 0x7f10788e0840>
│ └ <concurrent.futures.process.ProcessPoolExecutor object at 0x7f107652fe10>
└ <apscheduler.executors.pool.ProcessPoolExecutor object at 0x7f107652fd68>
File "/usr/lib/python3.6/concurrent/futures/process.py", line 466, in submit
self._start_queue_management_thread()
│ └ <function ProcessPoolExecutor._start_queue_management_thread at 0x7f10788e0730>
└ <concurrent.futures.process.ProcessPoolExecutor object at 0x7f107652fe10>
File "/usr/lib/python3.6/concurrent/futures/process.py", line 427, in _start_queue_management_thread
self._adjust_process_count()
│ └ <function ProcessPoolExecutor._adjust_process_count at 0x7f10788e07b8>
└ <concurrent.futures.process.ProcessPoolExecutor object at 0x7f107652fe10>
File "/usr/lib/python3.6/concurrent/futures/process.py", line 446, in _adjust_process_count
p.start()
│ └ <function BaseProcess.start at 0x7f10789699d8>
└ <Process(Process-1, started)>
File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
│ │ │ │ └ <Process(Process-1, started)>
│ │ │ └ <staticmethod object at 0x7f1078983ac8>
│ │ └ <Process(Process-1, started)>
│ └ None
└ <Process(Process-1, started)>
File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
│ │ └ <Process(Process-1, started)>
│ └ <function DefaultContext.get_context at 0x7f10788d0510>
└ <multiprocessing.context.DefaultContext object at 0x7f10789518d0>
File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
│ └ <Process(Process-1, started)>
└ <class 'multiprocessing.popen_fork.Popen'>
File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
│ │ └ <Process(Process-1, started)>
│ └ <function Popen._launch at 0x7f1075f04a60>
└ <multiprocessing.popen_fork.Popen object at 0x7f1075f65b00>
File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch
code = process_obj._bootstrap()
│ └ <function BaseProcess._bootstrap at 0x7f10789611e0>
└ <Process(Process-1, started)>
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
│ └ <function BaseProcess.run at 0x7f1078969950>
└ <Process(Process-1, started)>
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
│ │ │ │ │ └ {}
│ │ │ │ └ <Process(Process-1, started)>
│ │ │ └ (<multiprocessing.queues.Queue object at 0x7f107652feb8>, <multiprocessing.queues.SimpleQueue object at 0x7f1075f86a90>)
│ │ └ <Process(Process-1, started)>
│ └ <function _process_worker at 0x7f10788e0378>
└ <Process(Process-1, started)>
File "/usr/lib/python3.6/concurrent/futures/process.py", line 175, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
│ │ │ │ │ └ {}
│ │ │ │ └ <concurrent.futures.process._CallItem object at 0x7f1075f86a58>
│ │ │ └ (<Job (id=print_task1 name=scheduled_print_task1)>, 'default', [datetime.datetime(2021, 10, 19, 0, 10, 47, 222096, tzinfo=dat...
│ │ └ <concurrent.futures.process._CallItem object at 0x7f1075f86a58>
│ └ <function run_job at 0x7f10789aad08>
└ <concurrent.futures.process._CallItem object at 0x7f1075f86a58>
File "/usr/local/lib/python3.6/dist-packages/apscheduler/executors/base.py", line 125, in run_job
retval = job.func(*job.args, **job.kwargs)
│ │ │ │ │ └ <member 'kwargs' of 'Job' objects>
│ │ │ │ └ <Job (id=print_task1 name=scheduled_print_task1)>
│ │ │ └ <member 'args' of 'Job' objects>
│ │ └ <Job (id=print_task1 name=scheduled_print_task1)>
│ └ <member 'func' of 'Job' objects>
└ <Job (id=print_task1 name=scheduled_print_task1)>
> File "python-test/apscheduler-test.py", line 118, in print_greeting
raise Exception("NOOOOOOOOOOOOOO")
Exception: NOOOOOOOOOOOOOO
As you can see most of the stacktrace from apscheduler and multiprocessing libs don't give any valuable info. Rather they create so much noice that it's hard to find the frames of my own code. That's why if there's an option to hide stackframes from blacklisted libs, it'd be a great help in debugging.
Yep, agreed.
Perhaps it would be useful to ignore paths that exist outside of the root of the module entirely, with the ability to enable them in scenarios where it's necessary via an environment variable.
@Delgan (hi! long time 👋🏻 hope you're doing well) what do you think about that idea?
Hey @Qix-! Sorry for this answer a month later, I had taken a little break. :)
Regarding the issue raised here, it looks to me that it has little to do with better_exceptions and is rather a problem on loguru side. The traceback is displayed entirely which leads to printing undesired frames from third libraries. Using solely better_exceptions it would be displayed as follow:
Traceback (most recent call last):
File "python-test/apscheduler-test.py", line 118, in print_greeting
raise Exception("NOOOOOOOOOOOOOO")
Exception: NOOOOOOOOOOOOOO
However, if there exist cases for which better_exceptions formatting is considered too verbose, we could indeed imagine a solution to make it more readable. Adding an option to hide external paths seems like a good idea. Another possibility that I think we discussed a few years ago is to highlight the most interesting ones as done by tbvaccine:
Two others libraries that could be inspiring are PrettyErrors and rich.
Anyway, thanks for asking my opinion, I hope you're doing well too!
@Delgan I've created a new issue since it's specific to loguru Delgan/loguru#622. I am closing this one.