vectordotdev/timber-python

Use of multiprocessing prevents use on AWS Lambda

Opened this issue · 0 comments

Because this library uses multiprocessing for parallelization, it's not available for use on AWS Lambda. While this can be mitigated by streaming logs from AWS, it still prevents the use of Timber with layered services that run on AWS Lambda (e.g., Zeit Now).

Is there a known workaround for AWS Lambda? If not, is it feasible to use threading as a fallback if multiprocessing is not available?

[Errno 38] Function not implemented: OSError
Traceback (most recent call last):
  File "/var/task/now_python_wsgi/handler.py", line 130, in now_handler
    wsgi_module = import_module(wsgi_module_name)
  File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/var/task/index.py", line 12, in <module>
    api_key=os.environ.get('TIMBER_API_KEY')
  File "/var/task/timber/handler.py", line 35, in __init__
    self.pipe = multiprocessing.JoinableQueue(maxsize=buffer_capacity)
  File "/var/lang/lib/python3.6/multiprocessing/context.py", line 107, in JoinableQueue
    return JoinableQueue(maxsize, ctx=self.get_context())
  File "/var/lang/lib/python3.6/multiprocessing/queues.py", line 272, in __init__
    Queue.__init__(self, maxsize, ctx=ctx)
  File "/var/lang/lib/python3.6/multiprocessing/queues.py", line 42, in __init__
    self._rlock = ctx.Lock()
  File "/var/lang/lib/python3.6/multiprocessing/context.py", line 67, in Lock
    return Lock(ctx=self.get_context())
  File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 59, in __init__
    unlink_now)
OSError: [Errno 38] Function not implemented