EPypes (for event-driven piplines) is a Python library for developing data processing algorithms in a form of computational graphs and their integration with distributed systems based on publish-subscribe communication. The initial use case of EPypes is computer vision alogorithms development, although it is suitable for any algorithm that can be expressed as a directed acyclic graph.
EPypes facilitates flexibility of algorithm prototyping, as well as provides a structured approach to managing algorithm logic and exposing the developed pipelines as a part of on-line publish-subscribe systems. Currently, ZeroMQ middleware is supported, with data serialization based on Protocol Buffers.
The most important modules include:
compgraph
,graph
-- primitives for construction and execution of computational graphspipeline
,node
-- primitives for extendind computational graphs with additional functionality, specifically the reactive behaviorzeromq
-- adapters to ZeroMQ middlewarereactivevision
-- functionality for creation of reactive computer vision components
The core dependencies for the EPypes codebase include pyzmq
, protobuf
, and networkx>=2.0
. They are listed in the requirements.txt
file, and can be installed in one of the following ways:
# using pip
$ pip install -r requirements.txt
# using conda
$ while read requirement; do conda install --yes $requirement -c conda-forge; done < requirements.txt
The example below demostrates construction and execution of a computational graph. The demonstrated algorithm accepts a BGR image, converts it to grayscale, blurs the grayscale, and feeds the blurred image to the Canny edge detector.
import cv2
from epypes.compgraph import CompGraph
from epypes.compgraph import CompGraphRunner
def grayscale(im):
return cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
def gaussian_blur(im, kernel_size):
return cv2.GaussianBlur(im, (kernel_size, kernel_size), 0)
if __name__ == '__main__':
func_dict = {
'grayscale': grayscale,
'canny': cv2.Canny,
'blur': gaussian_blur
}
func_io = {
'grayscale': ('image', 'image_gray'),
'blur': (('image_gray', 'blur_kernel'), 'image_blurred'),
'canny': (('image_blurred', 'canny_lo', 'canny_hi'), 'edges'),
}
hparams = {
'blur_kernel': 11,
'canny_lo': 70,
'canny_hi': 200
}
cg = CompGraph(func_dict, func_io)
runner = CompGraphRunner(cg, hparams)
im = cv2.imread('my_image.jpg', cv2.IMREAD_COLOR)
runner.run(image=im)
For more complex computational graphs examples, refer to the following Jupyter notebooks: