Another metrics related crash
pipermerriam opened this issue · 0 comments
pipermerriam commented
What is wrong?
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: Traceback (most recent call last):
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/asyncio_run_in_process/_child_trio.py", line 63, in _run_on_trio
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: result = trio.run(_do_async_fn, async_fn, args, to_parent)
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trio/_core/_run.py", line 1896, in run
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: raise runner.main_task_outcome.error
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/asyncio_run_in_process/_child_trio.py", line 55, in _do_async_fn
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: result = await async_fn(*args)
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trinity/extensibility/trio.py", line 80, in _do_run
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: nursery.cancel_scope.cancel()
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trio/_core/_run.py", line 741, in __aexit__
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: raise combined_error_from_nursery
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trinity/extensibility/trio.py", line 39, in run_process
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: await self.do_run(event_bus)
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trinity/components/builtin/metrics/component.py", line 159, in do_run
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: await run_background_trio_services([
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trinity/_utils/services.py", line 35, in run_background_trio_services
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: await wait_first_trio([
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/.pyenv/versions/3.8.5/lib/python3.8/contextlib.py", line 679, in __aexit__
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: raise exc_details[1]
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/async_generator/_util.py", line 53, in __aexit__
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: await self._agen.athrow(type, value, traceback)
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/async_service/trio.py", line 409, in background_trio_service
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: yield manager
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/.pyenv/versions/3.8.5/lib/python3.8/contextlib.py", line 662, in __aexit__
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: cb_suppress = await cb(*exc_details)
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/async_generator/_util.py", line 53, in __aexit__
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: await self._agen.athrow(type, value, traceback)
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/async_service/trio.py", line 411, in background_trio_service
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: await manager.stop()
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trio/_core/_run.py", line 741, in __aexit__
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: raise combined_error_from_nursery
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/async_service/trio.py", line 205, in run
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: raise trio.MultiError(
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/async_service/base.py", line 324, in _run_and_manage_task
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: await task.run()
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/async_service/trio.py", line 76, in run
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: await self._async_fn(*self._async_fn_args)
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/async_service/base.py", line 75, in run
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: await service_fn(self.manager, *self._args, **self._kwargs)
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trinity/components/builtin/metrics/system_metrics_collector.py", line 147, in collect_process_metrics
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: process_stats=read_process_stats(),
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trinity/components/builtin/metrics/system_metrics_collector.py", line 112, in read_process_stats
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: main_trinity_process = get_main_trinity_process()
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trinity/components/builtin/metrics/system_metrics_collector.py", line 104, in get_main_trinity_process
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: python_processes = get_all_python_processes()
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/eth_utils/functional.py", line 45, in inner
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: return callback(fn(*args, **kwargs))
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/trinity/components/builtin/metrics/system_metrics_collector.py", line 94, in get_all_python_processes
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: commands = process.cmdline()
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/psutil/__init__.py", line 694, in cmdline
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: return self._proc.cmdline()
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: File "/root/env/lib/python3.8/site-packages/psutil/_pslinux.py", line 1522, in wrapper
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: raise NoSuchProcess(self.pid, self._name)
Oct 01 05:38:52 localhost launch-trinity.sh[10210]: psutil.NoSuchProcess: psutil.NoSuchProcess process no longer exists (pid=11698)
How can it be fixed
🤷 probably error handling.