BMF(Babit Multimedia Framework, BabitMF) is a universal multimedia processing framework launched by ByteDance that provides a concise and easy-to-use cross-language interface, flexible scheduling and scalability. It dynamically expands, manages and reuses the atomic capabilities of video processing in a modular way, and builds high-performance multimedia processing links in a graph/pipeline manner or implements engineering integration by directly invoking individual processing capabilities
Our collaborative contributor includes NVIDIA, and we have our own official website, welcome to browse and put forward your valuable opinions: https://babitmf.github.io/
BMF helps multimedia users easily and efficiently implement projects in production environments. the cases used BMF cover video transcoding, video frame extraction, video enhancement, video analysis, video interpolation, video editing, video conferencing, VR, and etc. Currently, hundreds of millions of videos are processed using BMF daily. During the implementation of these business scenarios, BMF's functional diversity, ease of use, compatibility, stability and performance have been fully polished.
In this section, we will directly showcase the capabilities of the BMF framework around five dimensions: Transcode, Edit, Meeting/Broadcaster, CPU+GPU acceleration, and AI. For all the demos provided below, corresponding implementations and documentation are available on Google Colab, allowing you to experience them intuitively.
This demo describes step-by-step how to use BMF to develop a transcoding program, including video transcoding, audio transcoding, and image transcoding. In it, you can familiarize yourself with how to use BMF and how to use FFmpeg-compatible options to achieve the capabilities you need.
If you want to have a quick experiment, you can try it on
The Edit Demo will show you how to implement a high-complexity audio and video editing pipeline through the BMF framework. We have implemented two Python modules, video_concat and video_overlay, and combined various atomic capabilities to construct a complex BMF Graph.
If you want to have a quick experiment, you can try it on
This demo uses BMF framework to construct a simple broadcast service. The service provides an API that enables dynamic video source pulling, video layout control, audio mixing, and ultimately streaming the output to an RTMP server. This demo showcases the modularity of BMF, multi-language development, and the ability of dynamically adjusting the pipeline.
Below is a screen recording demonstrating the operation of broadcaster:
The video frame extraction acceleration demo shows:
-
BMF flexible capability of:
- Multi-language programming，we can see multi-language module work together in the demo
- Ability extend easily, there are new C++, Python modules added simply
- FFmpeg ability fully compatible
-
Hardware acceleration quickly enablement and CPU/GPU pipeline support
- Heterogeneous pipeline is supported in BMF, such as process between CPU and GPU
- Usefull hardware color space convertion in BMF
If you want to have a quick experiment, you can try it on
The GPU transcoding and filter module demo shows:
- Common video/image filters in BMF accelerated by GPU
- How to write GPU modules in BMF
The demo builds a transcoding pipeline which fully runs on GPU:
decode->scale->flip->rotate->crop->blur->encode
If you want to have a quick experiment, you can try it on
This demo shows the how to integrate the state of art AI algorithms into the BMF video processing pipeline. The famous open source colorization algorithm DeOldify is wrapped as an BMF pyhton module in less than 100 lines of codes. The final effect is illustrated below, with the original video on the left side and the colored video on the right.
If you wan't to have a quick experiment, you can try it on
This demo implements the super-resolution inference process of Real-ESRGAN as a BMF module, showcasing a BMF pipeline that combines decoding, super-resolution inference and encoding.
If you wan't to have a quick experiment, you can try it on
This demo shows how to invoke our aesthetic assessment model using bmf. Our deep learning model Aesmode has achieved a binary classification accuracy of 83.8% on AVA dataset, reaching the level of academic SOTA, and can be directly used to evaluate the aesthetic degree of videos by means of frame extraction processing.
If you wan't to have a quick experiment, you can try it on
-
- Install
- Create a Graph
- one of transcode example with 3 languages
- Use Module Directly
-
sync mode with 3 languages. You can try it on:
Python:
C++:
-
- Create a Module
- customize module with python, C++ and Go. You can try it on
- customize module with python, C++ and Go. You can try it on
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
We welcome contributions. Please follow these guidelines.
We use GitHub issues to track and resolve bugs. If you have any questions, please feel free to join the discussion and work with us to find a solution.