Real Time Bidding (RTB) - Demand Side Platform framework
open-source library utilizing modern C++11/14 features and latest Boost.
What makes us different from other open-source RTB projects we have seen?
Our stack is fairly small and easy to integrate with any cmake project. As a model project please see https://github.com/vanilla-rtb/rapid-bidder
vanilla-rtb stack is completely decoupled by C++ templates and has minimum dependency on outside vendors.
Multi-bidder-model-with-communicator-for-Win-notifications
best performance compared to other stacks - 50K QPS
runs on cloud with docker - see instructions
Recommended build environment: Linux or macOS, CMake >= 3.7.2, GCC >= 5.1, Boost >= 1.60
Structure :
-
/ -- the root directory
- benchmarks/ -- optionaly built benchmarks for IPC caches, json parsers and low overhead IPC audit logger
- CRUD/ -- C++11 high performance HTTP-restful handlers based on boost.ASIO and CRUD API
- docker/ -- vanilla docker files and instructions on how to build and run
- jsonv/ -- DSL mapper of json encoded objects to C++ structures
- parsers/ -- fast zero copy, zero memory allocation parsers
- rapidjson/ -- very fast json tokenizer including SAX / DOM API but requires more memory
- rtb/ -- RTB framework
- core/ -- generic structures shared in the project ( RTB specific )
- common/ -- generic RTB agnostic structures
- datacache/ -- IPC cache generic classes for fast targeting and other lookups
- exchange/ -- exchange handlers implementation
- DSL/ -- DSL formats for jsonv , boost::any and rapidjson
- examples/ -- root to our sandbox with examples
- bidder/ -- collection of application specific classes to support targeting
- loader/ -- collection of application specific classes to support campaign loading
- campaign/ -- add/modify/delete campaign API + UI ( work in progress )
- datacache/ -- IPC cache implementation based on rtb/datacache model
- UI/ -- HTML and javascript for campaign budget management
-
[CMakeLists.txt] - cmake file
The stack of vanilla-rtb includes other C++11 projects and is referencing them via gh-subree. To update to the latest version of json-voorhees or CRUD we use the following commands :
- git subtree pull --prefix jsonv git@github.com:tgockel/json-voorhees.git master --squash
- git subtree pull --prefix CRUD git@github.com:venediktov/CRUD.git master --squash
(installing dependencies before building vanilla stack)
$ mkdir Release
$ cd Release
$ cmake -DCMAKE_BUILD_TYPE=Release .. -G "Unix Makefiles"
$ make -j4 install
####### Creating Debug build #######
$ cd ..
$ mkdir Debug
$ cd Debug
$ cmake -DCMAKE_BUILD_TYPE=Debug .. -G "Unix Makefiles"
$ make -j4 install
same steps as above for linux , only difference is depending on your environment either Visual Studio or NMake project can be used
######### for NMake ####################
cd Release
cmake -DCMAKE_BUILD_TYPE=Release .. -G "NMake Makefiles"
cd ../Debug
cmake -DCMAKE_BUILD_TYPE=Debug .. -G "NMake Makefiles"
######### for Visual Studio ############
cd Release
cmake -DCMAKE_BUILD_TYPE=Release .. -G "Visual Studio 14 2015"
cd ../Debug
cmake -DCMAKE_BUILD_TYPE=Debug .. -G "Visual Studio 14 2015"
For the reliable results it is suggested to have the build directory out of source tree.
The process involves creating a build directory, generating an Xcode
project in that directory with CMake
,
opening the project file generated in the build directory with Xcode
, and lastly, adjusting project
settings as requried and kicking off the build.
To generate an Xcode
project invoke cmake from an empty build directory with command line similar to cmake -G Xcode -DCMAKE_BUILD_TYPE=Release
.
$ xcode-select --install
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ brew doctor
$ brew install cmake
$ brew install boost
$ mkdir Release
$ cd Release
$ cmake -DCMAKE_BUILD_TYPE=Release .. -G "Unix Makefiles"
$ make -j4 install
$ brew doctor
$ brew install cmake
$ brew install boost
$ brew install --with-clang llvm
$ mkdir Release
$ cd Release
$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=/usr/local/opt/llvm/bin/clang -DCMAKE_CXX_COMPILER=/usr/local/opt/llvm/bin/clang++ -DCMAKE_RANLIB=/usr/local/opt/llvm/bin/llvm-ranlib -DCMAKE_AR=/usr/local/opt/llvm/bin/llvm-ar .. -G "Unix Makefiles"
When building on Linux and Mac OS X with Make it's possible to automatically adjust the concurreny of the build using nproc
command line tool that returns number of CPUs available to the Make execution context:
$ make -j $(nproc) ...
It's also possible to specifying the target Load Average with -l
flag to prevent machine overloading:
$ make -l $(nproc) ...
And lastly, on Linux, it's possible to run the build with the BATCH scheduling mode (throughput oriented) as:
$ chrt --batch 0 make ...
All above considered the ultimate Make invocation combo on Linux would be something like:
$ chrt --batch 0 make -j$(nproc) -l$(nproc)
- HTTP-Bidder
- vanilla-rtb/Release/examples/bin$ ./http_bidder_test --config etc/config.cfg
- test with curl and apache benchmark
- vanilla-rtb/Release/examples/bin$ ./curl.sh --bidder
- vanilla-rtb/Release/examples/bin$ ./ab.sh -n30000 -c10 --bidder
- Start Exchange Handler with HTTP handler
- vanilla-rtb/Release/examples/bin$ ./exchange_handler_test --config etc/config.cfg
- vanilla-rtb/Release/examples/bin$ ./ab.sh -n30000 -c10 --auction
- vanilla-rtb/Release/examples/bin$ ./ab.sh -n30000 -c10 --auction-any
- vanilla-rtb/Release/examples/bin$ ./ab.sh -n30000 -c10 --auction-rapid
- Start Exchange Handler distributing to multi-bidders via communicator
- vanilla-rtb/Release/examples/bin$ ./exchange_handler_test --config etc/config.cfg
- Mock-bidders starting multiple in one swoop, currently configured as 5 bidders in config
- vanilla-rtb/Release/examples/bin$ ./mock_bidder_test --config etc/config.cfg
- vanilla-rtb/Release/examples/bin$ ./ab.sh -n30000 -c10 --mock-bidders
- multi-bidders starting multiple in one swoop (currently configured as 3 bidders in config ) and starting exchange handler
- vanilla-rtb/Release/examples/bin$ ./multi_bidder --config etc/config.cfg
- vanilla-rtb/Release/examples/bin$ ./multi_exchange_handler --config etc/config.cfg
Testing cache loader of ad campaigns and ad budgets ( needs to be extended to get cache update events from outside )
- Cache loader
- vanilla-rtb/Release/examples/bin$ ./cache_loader_test --config etc/config.cfg
- Mock exchange - emulating bid requests
- To run mock exchange you need any python, and python "requests" library installed.
- for simple exchange please run
- vanilla-rtb/Release/examples/bin/mock_exchange$ python mock-x.py
- for various geo and size rotation please run
- vanilla-rtb/Release/examples/bin/mock_exchange$ python mock-x.py --geo "Russia:Moscow USA:NY USA:Washington USA:Chicago" --size '100:300 240:400 420:280'
- for more info please run
- vanilla-rtb/Release/examples/bin/mock_exchange$ python mock-x.py --help
-
Campaign manager - Budget
- vanilla-rtb/Release/examples/bin$ ./campaign_manager_test --config etc/config.cfg
-
Notification service and Slave-Banker
- vanilla-rtb/Release/examples/bin$ ./notification_service_test --config etc/config.cfg
- vanilla-rtb/Release/examples/bin$ ./slavebanker_service_test --config etc/config.cfg
-
To add/delete/modify campign budgets fire up UI by connecting to manager via browser http:://localhost:11081/campaign/index.html