Sentry initialisation uses outdated raven packages resulting in constant warnings and inability to set environments
karlokr-p opened this issue · 4 comments
Checklist
- I have included information about relevant versions
- I have verified that the issue persists when using the
master
branch of Faust.
Steps to reproduce
Initialise sentry normally using the built-in function setup_sentry()
Expected behavior
There should be no warnings about deprecated stuff
The sentry environment is properly set (ie, differing between production/staging environments)
Actual behavior
Many warnings are seen in the logs
The sentry environment is not properly set and all errors are sent to the same environment in sentry, so I'm unable to differentiate between which environment the error occurred on.
Full traceback
TypeError: As of 3.10, the *loop* parameter was removed from Queue() since it is no longer necessary
File "raven/base.py", line 732, in send_remote
transport = self.remote.get_transport()
File "raven/conf/remote.py", line 71, in get_transport
self._transport = self._transport_cls(**self.options)
File "raven_aiohttp.py", line 195, in __init__
self._queue = asyncio.Queue(maxsize=qsize, loop=self._loop)
File "asyncio/queues.py", line 34, in __init__
super().__init__(loop=loop)
File "asyncio/mixins.py", line 17, in __init__
raise TypeError(
Known cause
The raven and raven-aiohttp packages have not been updated since 2019. These packages do not support python 3.10, hence the constant warnings about the loop
parameter. According sentry, raven has been deprecated https://docs.sentry.io/clients/python/
My solution
Initialise the current version of sentry-sdk manually:
sentry_sdk.init(
dsn=settings.SENTRY_DSN,
integrations=[_sdk_aiohttp.AioHttpIntegration()],
traces_sample_rate=settings.SENTRY_TRACES_SAMPLE_RATE,
environment=settings.SENTRY_ENVIRONMENT,
release=settings.RELEASE_VERSION,
)
Versions
- Python version: 3.10.9
- Faust version: All versions since 3.10 support was added
- Operating system: Docker (built from debian-slim-bullseye image)
- Kafka version: confluentinc/cp-kafka 7.2.2
- RocksDB version (if applicable): N/A
Unfortunately I'm not familiar with sentry. This was implemented well before my time as a maintainer. Could you please open a PR with your solution or provide a more broad example?
The setup_sentry()
function makes use of a custom sentry handler, FaustSentryHandler
which includes some additional logic:
# 1) We override SentryHandler to write internal log messages
# mto sys.__stderr__ instead of sys.stderr, as the latter
# is redirected to a logger, causing noisy internal logs
# to be sent to Sentry.
#
# 2) We augment can_record to skip logging for CancelledError
# exceptions that have a ``.is_expected`` attribute set to True.
# That way once a CancelledError showing up in Sentry is
# investigated and resolved, we can silence that particular
# error in the future.
Which I am not sure how to implement without digging into the sentry-sdk documentation. My solution does not include this custom handler so I'm not sure my solution is good enough to include as a PR, unfortunately. I can attempt to re-write this custom handler, but it may take some time
Can confirm, it is better to remove the built-in (outdated) sentry integration and point users to the stock solution. I spent some time trying to make the built-in solution to work, only to discover it does not work for any Python version newer than 3.9.
The stock solution is:
from sentry_sdk.integrations.aiohttp import AioHttpIntegration
from sentry_sdk.integrations.logging import EventHandler
faust_app = faust.App(...)
sentry_sdk.init(
environment=settings.environment_name,
dsn=settings.sentry_dsn,
integrations=[AioHttpIntegration()],
traces_sample_rate=1.0,
)
sentry_log_handler = EventHandler(level=WARNING) # send errors and warnings to sentry
faust_app.conf.loghandlers.append(sentry_log_handler)
Can confirm, it is better to remove the built-in (outdated) sentry integration and point users to the stock solution. I spent some time trying to make the built-in solution to work, only to discover it does not work for any Python version newer than 3.9. The stock solution is:
from sentry_sdk.integrations.aiohttp import AioHttpIntegration from sentry_sdk.integrations.logging import EventHandler faust_app = faust.App(...) sentry_sdk.init( environment=settings.environment_name, dsn=settings.sentry_dsn, integrations=[AioHttpIntegration()], traces_sample_rate=1.0, ) sentry_log_handler = EventHandler(level=WARNING) # send errors and warnings to sentry faust_app.conf.loghandlers.append(sentry_log_handler)
Thanks for the example, I'm probably going to deprecate the existing sentry integrations and replace it with their suggested solution.