Mediapipe-lite

Builds the mediapipe graph module using cmake, conan, and allows conan packaging for various graphing tasks.

Change Log

2023/08/30 supports building on Mac, Linux, Windows, Android

Quick Start

Build the Docker image

cd docker 
docker build -t x86_u20_cpp_gcc9 -f Dockerfile.x86_u20_gcc9 --no-cache .

Start docker

docker run -v ${PWD}:/mediapipe-lite --name x86_u20_med -it x86_u20_cpp_gcc9:latest -c "cd /mediapipe-lite && bash docker/init_zsh.sh && zsh"

Compile the test

# conan install dependencies, run only on first build
conan install . --build=missing -pr:h=docker/x86_gcc_profile
cd build
cmake ..
make -j 4
# Declare the path to the dynamic library, unset to run source deactivate_conanrun.sh
source conanrun.sh
# run unit test
make test
# run object detection example 
# The model in the config file is at https://drive.google.com/file/d/1U9cm5qfOxnGwyB6ypJjYvB6OeOjLZqpC/view?usp=drive_link
# Download and place in mediapipe/models folder
# The test image package can be downloaded from https://drive.google.com/file/d/1IjP8aT_iQ8fV_FCuUJk8TH3_e1X2Y_3Q/view?usp=drive_link
bin/object_detection --calculator_graph_config_file=../examples/desktop/object_detection/object_detection_desktop_live.pbtxt --input_video_path=$IMAGE_DIR --output_video_path=$OUTPUT_DIR
# run object detection example with all verbose logs
GLOG_v=5 bin/object_detection --calculator_graph_config_file=../examples/desktop/object_detection/object_detection_desktop_live.pbtxt --input_video_path=$IMAGE_DIR --output_video_path=$OUTPUT_DIR

Project Architecture
- libgraph depends only on protobuf, abseil and glog, and is 1.7M in size for x86 environments.
- libframework contains libgraph and other auxiliary projects.

conan package

conan create . --build=missing -s build_type=Release  -pr:h=docker/x86_gcc_profile  -o 'export_package=True'

Examples of using Graph

See the tutorial folder for example code.

// main.cpp
#include "mediapipe/framework/calculator_graph.h"
#include "mediapipe/framework/port/logging.h"
#include "mediapipe/framework/port/parse_text_proto.h"
#include "mediapipe/framework/port/status.h"

namespace mediapipe {

absl::Status PrintHelloWorld() {
// Configures a simple graph, which concatenates 2 PassThroughCalculators.
CalculatorGraphConfig config =
    ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        input_stream: "in"
        output_stream: "out"
        node {
        calculator: "PassThroughCalculator"
        input_stream: "in"
        output_stream: "out1"
        }
        node {
        calculator: "PassThroughCalculator"
        input_stream: "out1"
        output_stream: "out"
        }
    )pb");
LOG(INFO) << config.DebugString();
CalculatorGraph graph;
MP_RETURN_IF_ERROR(graph.Initialize(config)) << "init graph failed";
ASSIGN_OR_RETURN(OutputStreamPoller poller,
                graph.AddOutputStreamPoller("out"));
MP_RETURN_IF_ERROR(graph.StartRun({}));
// Give 10 input packets that contains the same string "Hello World!".
for (int i = 0; i < 10; ++i) {
    MP_RETURN_IF_ERROR(graph.AddPacketToInputStream(
        "in", MakePacket<std::string>("Hello World!").At(Timestamp(i))));
}
// Close the input stream "in".
MP_RETURN_IF_ERROR(graph.CloseInputStream("in"));
mediapipe::Packet packet;
// Get the output packets string.
while (poller.Next(&packet)) {
    LOG(INFO) << packet.Get<std::string>();
}
return graph.WaitUntilDone();
}
}  // namespace mediapipe

int main(int argc, char** argv) {
google::InitGoogleLogging(argv[0]);
FLAGS_stderrthreshold=google::INFO;
FLAGS_colorlogtostderr=true;
mediapipe::PrintHelloWorld().ok();
return 0;
}

CMakeList.txt

add_executable(hello_world hello_world.cc
${PROJECT_SOURCE_DIR}/mediapipe/calculators/core/pass_through_calculator.cc)
target_link_libraries(new_node PUBLIC 
                          -Wl,--whole-archive 
                          stream_handler)
target_link_libraries(hello_world PUBLIC 
                          -Wl,--no-whole-archive 
                        graph)

Note: Since stream_handler, calculator in mediapipe are called dynamically by registering them with a registrar, there are two ways to make sure the registered classes are available in the final executable.

compile the handler, the used calculator into static libraries and force linking of all symbols by passing a parameter to ld, as in the above cmake operation for the stream_handler library
compile the source code of the handler and calculator implementations directly with the target files, as in the test_package example.

Running output

I20230721 17:08:09.523499 140015 hello_world.cc:41] node {
  calculator: "PassThroughCalculator"
  input_stream: "in"
  output_stream: "out1"
}
node {
  calculator: "PassThroughCalculator"
  input_stream: "out1"
  output_stream: "out"
}
input_stream: "in"
output_stream: "out"
I20230721 17:08:09.525810 140015 hello_world.cc:57] Hello World!
I20230721 17:08:09.525838 140015 hello_world.cc:57] Hello World!
I20230721 17:08:09.525846 140015 hello_world.cc:57] Hello World!
I20230721 17:08:09.525852 140015 hello_world.cc:57] Hello World!
I20230721 17:08:09.525859 140015 hello_world.cc:57] Hello World!
I20230721 17:08:09.525866 140015 hello_world.cc:57] Hello World!
I20230721 17:08:09.525872 140015 hello_world.cc:57] Hello World!
I20230721 17:08:09.525879 140015 hello_world.cc:57] Hello World!
I20230721 17:08:09.525885 140015 hello_world.cc:57] Hello World!
I20230721 17:08:09.525892 140015 hello_world.cc:57] Hello World!

TODO List

Clean up redundant code step by step
Add doc, CI, CD, CT, code formatting checking and other related processes.
Improve tutorials and code examples
Add pipieline test benchmark.
Improve python interface, allow adding python node.

lzx1413/mediapipe-lite

Mediapipe-lite

Change Log

Quick Start

TODO List