AREG SDK

Project Status

Introduction

AREG (Automated Real-time Event Grid) is a powerful and lightweight cross-platform communication engine designed to facilitate seamless data transmission in IoT fog- and mist-network. AREG creates a grid of services using interface-centric approach, allowing multiple connected devices and software nodes to operate like thin distributed servers and clients. It automates real-time data transmission, utilizing both Client-Server and Publish-Subscribe models to ensure efficient and reliable communication between connected software nodes.

Motivation
Interface-centricity
More than embedded
Composition
Software build
Integration
Roadmap
Use cases and benefits
Examples
License
Call to action

Motivation

Traditionally, devices act as connected clients to stream data to the cloud or fog servers for further processing.

As data is generated and collected at the edge of the network (mist network), it is logical to redefine the role of connected Things and enable network-accessible services (Public Services) on them, thereby extending the Cloud to the extreme edge. This approach serves as a strong foundation for implementing robust solutions such as:

Increase data privacy, which is an important factor for sensitive data.
Decrease data streaming, which is a fundamental condition to optimize network communication.
Autonomous, intelligent and self-aware devices with services directly in the environment of data origin.

[ ↑ to top ↑ ]

Interface-centricity

ORPC, or Object Remote Procedure Call, is a remote procedure call concept that targets an interface on an object. It enables the construction of a service mesh (or service grid), where applications offer reusable services, and programmable client objects request method execution of programmable server objects without needing to know their location in the network.

ORPC concept is interface-centric, similar to object-oriented programming, and allows multiple instances of the same object to be instantiated and handled transparently. There are no protocol restrictions, although the bi-directional communication is required to send messages to all connected nodes. The programmable server objects are called service providers, and the programmable clients are called service consumers. This approach enables AREG to combine the features of action-centric (Client-Server / Request-Reply) and data-centric (Publish-Subscribe / PubSub) models.

[ ↑ to top ↑ ]

More than embedded

AREG is designed to provide a homogeneous solution for multithreading, multiprocessing, and internet communications through categorized services (Local, Public, and Internet). These services are software components with predefined interfaces, which methods are invoked remotely.

💡 Currently AREG engine supports Local (multithreading) and Public (multiprocessing) service categories.

AREG forms a fault-tolerant system that automatically discovers and automates communications between services, allowing developers to focus on application logic development. The system ensures:

The crash of one application does not affect the entire system.
The clients automatically receive service availability notifications.
The requests, responses, and notifications are invoked in their own thread context.

[ ↑ to top ↑ ]

Composition

The major modules of AREG SDK are:

Multicast router (mcrouter) is a console application or OS-managed service that routes message.
AREG engine is a library to link with every application.
Code generator is a tool that creates service provider / consumer objects from interface documents.

💡 The multiple examples demonstrate the features and fault tolerant behavior of AREG communication engine.

[ ↑ to top ↑ ]

Software build

💡 Check the Wiki page of AREG SDK. We change the content and add more details.

The source codes of AREG framework and examples support following platform, CPU and compilers:

Platforms	Linux (list of POSIX API), Cygwin, Windows.
CPU	x86, x86_64, arm and aarch64.
Compilers	GCC, Clang, MSVC and Cygwin gcc.

The tools to compile sources:

Tool	Solution	Platforms	API	Quick actions to compile
`cmake`	`CMakeLists.txt`	Linux, Cygwin, Windows	POSIX, Win32	- Build with `cmake`. - Build in VSCode; - Build in MSVS.
`make`	`Makefile`	Linux, Cygwin	POSIX	- Build with `make`.
`msbuild`	`areg-sdk.sln`	Windows	Win32	- Build with `msbuild`. - Open and build in MSVS.

To customize the build, the tools accept parameters. For information on available options, refer to the Wiki pages. By default, after build the binaries are in <areg-sdk>/product/build/<family>-<compiler>/<platform>-<bitness>-<cpu>-<build type>/bin folder (for example, areg-sdk/product/build/gnu-gcc/linux-64-x86_64-release/bin).

💡 The other POSIX-compliant OS and compilers are not tested yet.

Clone sources

To get AREG SDK source codes and dependent modules, open Terminal in your projects folder and clone:

# This clones the source codes of AREG SDK and dependent module
$ git clone --recurse-submodules https://github.com/aregtech/areg-sdk.git
$ cd areg-sdk

If you already have AREG SDK sources and only need to load or update submodules to latest versions, take the steps:

$ cd areg-sdk

# Step 1: If you cloned AREG SDK and need to load submodules
$ git submodule update --init

# Step 2: If you need to pull all latest source of submodules
$ git submodule update --remote

After cloning sources, use cmake, make, or msbuild tools to build applications. You should as well have C++17 GNU, Clang, Cygwin, or MSVC compiler installed on your machine.

Build with `cmake`

Firstly, clone the sources properly. To build with cmake, using default (g++) compiler and options, open Terminal in areg-sdk folder and take the steps:

# Step 1: Initialize cache and build configuration in folder './build' folder.
#         Default options: g++ compiler, release build, enabled examples and unit tests
$ cmake -B ./build

# Step 2: Compile sources.
$ cmake --build ./build -j 8

Build with `make`

Firstly, clone the sources properly. To build with make, using default (g++) compiler and options, open Terminal in areg-sdk folder and take the steps:

# Compile sources with default options: g++ compiler, release build, enabled examples and unit tests
$ make -j 8

Build with `msbuild`

Firstly, clone the sources properly. To build with MSBuild, using MSVC default options, open Terminal in areg-sdk folder and take the steps:

# Compile sources with default options: msvc compiler, debug build, enabled examples and unit tests
$ MSBuild .

Build with IDE

Firstly, clone the sources properly. Here we consider builds with Microsoft Visual Studio and Visual Studio Code.

Open areg-sdk.sln solution file in Microsoft Visual Studio and build the solution with MSVC.
Open areg-sdk folder in Visual Studio Code => select CMakeLists.txt in Explorer => mouse right click => click on Configure All Projects in menu => wait for completion => mouse right click on CMakeLists.txt => click on Build All Projects to build the sources with default (g++) compiler and default options.

[ ↑ to top ↑ ]

Integration

💡 Check the Wiki page of AREG SDK. We change the content and add more details.

Integrate for development

Firstly, clone the sources properly. Use following parameters to change compilation default settings:

Parameter	Description, possible values
AREG_OUTPUT_BIN	Change the default output folder of compiled shared libraries and executables.
AREG_OUTPUT_LIB	Change the default output folder of compiled static libraries.
AREG_COMPILER_FAMILY	Set compilers by specifying gnu, llvm, cygwin or msvc.
AREG_COMPILER	Set compiler by specifying g++, gcc, clang++, clang or cl.
AREG_BINARY	Set AREG library type by specifying shared or static.
AREG_BUILD_TYPE	Set the build type by specifying Debug or Release.
AREG_ENABLE_EXT	Set the flag to indicate the build of a special library with extended features.

More details of parameters for each tool are described in the appropriate user configuration files:

For builds with cmake, in the conf/cmake/user.cmake file.
For builds with make, in the conf/make/user.mk file.
For builds with MSBuild, in the conf/msvc/user.props file.

An example to configure and compile codes with cmake:

# Step 1: configure, make appropriate settings
$ cmake -B ./build -DAREG_COMPILER=clang++ -DAREG_BINARY=static -DAREG_BUILD_TYPE=Debug -DAREG_ENABLE_EXT:BOOL=ON AREG_OUTPUT_BIN="~/product/areg-sdk/"

# Step 2: build applications
$ cmake --build -j 8

In this scenario, the source codes are set up to build using clang++. The AREG engine is created as a static library with enabled extensions (may require additional dependencies). All applications are built in Debug mode, and the resulting binaries are stored in the ~/product/areg-sdk/ directory.

The additional development guidance and step-by-step example to create a simple service-enabled application are described in Hello Service! document, which demonstrate the various scenarios of creating service provider and service consumer within the same thread, within the same process and within the same network. The other useful documents are in DEVELOP.md.

Configure multicast router

Configure router.init file to set the IP-address and the port of multicast router (mcrouter):

connection.address.tcpip    = 127.0.0.1	# the address of mcrouter host
connection.port.tcpip       = 8181      # the connection port of mcrouter

The multicast router is required only for multiprocessing applications and can be ignored in case of multithreading. It forms a network and requires GPOS.

Configure logging

Configure log.init to set scopes, priorities and file name for logging:

log.file        = %home%/logs/%appname%_%time%.log # create logs in 'log' subfolder of user home 
scope.mcrouter.*= NOTSET ;                         # disable logs for mcrouter.

scope.my_app.*                   = DEBUG | SCOPE ; # enable all logs of my_app
scope.my_app.ignore_this_scope   = NOTSET ;        # disable logs of certain scopes in my_app
scope.my_app.ignore_this_group_* = NOTSET ;        # disable logs of certain scope group in my_app

💡 By default, the router.init and log.init files are located in the config subfolder of binaries.
💡 To enable all logs of all applications, use scope.* = DEBUG | SCOPE ; .
💡 Currently logging is possible only in the file.

[ ↑ to top ↑ ]

Roadmap

The aim of the AREG SDK is a lightweight, self-sufficient development and testing system to help developers to create complex Desktop, Embedded, and IoT edge applications in a shorter amount of time (30-50% faster).

Planned features of the framework:
- Multi-channel and multi-protocol communication.
- Logging service to collect logs in the network.
- Internet (web) category services.
Planned tools:
- Service interface designer.
- Interactive log viewer.
- Service testing and Data simulation tool.

[ ↑ to top ↑ ]

Use cases and benefits

AREG SDK can be used in a very large scope of multithreading and multiprocessing application development.

Distributed solution

Click to show / hide distributed solution.

The AREG SDK is a distributed services solution that enables components to interact seamlessly across nodes on the network, appearing as if they are located within a single process. To define relationships and distribute services across processes, developers create runtime loadable models.

The following is a demonstration of 2 models defined in 2 different processes, where one model defines service provider and the second is at the same time is a service consumer (client) and a service provider. For simplicity, let's name them service.cpp and mixed.cpp.

A: model of service provider component in service.cpp:

// service.cpp, define service provider
BEGIN_MODEL("MyModel")

  BEGIN_REGISTER_THREAD( "Thread1" )
    BEGIN_REGISTER_COMPONENT( "SystemShutdown", SystemShutdownService )
      REGISTER_IMPLEMENT_SERVICE( NESystemShutdown::ServiceName, NESystemShutdown::InterfaceVersion )
    END_REGISTER_COMPONENT( "SystemShutdown" )
  END_REGISTER_THREAD( "Thread1" )

END_MODEL("MyModel")

B: model of service consumer and service provider (mixed) component in mixed.cpp:

// mixed.cpp, define service provider and service consumer.
BEGIN_MODEL("MyModel")

  BEGIN_REGISTER_THREAD( "Thread1" )
    BEGIN_REGISTER_COMPONENT( "RemoteRegistry", RemoteRegistryService )
      REGISTER_IMPLEMENT_SERVICE( NERemoteRegistry::ServiceName, NERemoteRegistry::InterfaceVersion )
      REGISTER_DEPENDENCY("SystemShutdown")
    END_REGISTER_COMPONENT( "RemoteRegistry" )
  END_REGISTER_THREAD( "Thread1" )

END_MODEL("MyModel")

After declaration of services, each project can load them in the main() method and load them as it is shown in the next example.

C: common code for service.cpp and mixed.cpp files:

int main()
{
    Application::initApplication( );    // initialize defaults
    Application::loadModel("MyModel");  // load model, start services
    Application::waitAppQuit( );        // wait for quit signal
    Application::unloadModel("MyModel");// unload model, stop services
    Application::releaseApplication();  // release resources

    return 0;
}

In these codes:

service.cpp contains model to register service "SystemShutdown" that implements interface NESystemShutdown::ServiceName.
mixed.cpp contains model to register service "RemoteRegistry" that implements interface NERemoteRegistry::ServiceName and consumes (requires) service "SystemShutdown".
int main() is identical in both files. It initializes resources, loads model and waits for the completion.

The developers should create SystemShutdownService and RemoteRegistryService objects that implement the business logic of request methods of provided and response methods of used service interfaces. When these processes start, the services are automatically discovered via mcrouter. With this technique, the projects easily develop multiprocessing applications where provided services can be distributed and accessed remotely within the network formed by mcrouter.

An example of developing a service provider and consumer in one and multiple processes is in Hello Service! guide. As well there are multiple examples of multiprocessing and multithreading applications.

Driverless devices

Click to show / hide driverless devices.

Normally, the devices are supplied with the drivers to install in the system.

Proposed are driverless service-enabled devices that do not require installation of drivers in the system.

The services, described in the interface prototype documents, are faster and easier to develop than drivers, as they do not require special techniques and can be developed like user mode applications. Service development reduces risks and the code generator simplifies client object generation from a service interface document.

Real-time solutions

Click to show / hide real-time solutions.

AREG engine automatically generates and delivers messages to the target and invokes the exact methods of the target objects in real-time with ultra-low networking latency. This makes it ideal for developing time-sensitive applications for industries such as automotive, drone fleets, medtech, real-time manufacturing, and monitoring.

Digital twin

Click to show / hide digital twin.

AREG framework's event-driven and service-oriented architecture, coupled with real-time communication, offers a robust solution for digital twin applications. This framework allows for visualization, monitoring, and control of external devices, while also enabling immediate reaction to changes in the environment or device state. AREG's approach eliminates the need for additional communication layers, making it an ideal solution for emergency, security, and safety applications.

Simulation and tests

Click to show / hide simulation and tests.

Simulating a software environment is a practical solution for testing and checking rapidly changing software, especially when hardware provisioning is not feasible. Simulations are portable, accessible, and help to optimize solutions while avoiding unnecessary risks. Projects using simulations are better prepared for remote work and easier to outsource.

The distributed and service-oriented solution of AREG engine eases application testing in a simulated environment. For example, the Simulation application may provide identical Data layer services for seamless testing of the rest of the application.

The same technique of simulating data can be used to create API-driven test automations.

[ ↑ to top ↑ ]

Examples

There are various examples to demonstrate features of the AREG SDK. The examples are listed in the examples/README.md document.

[ ↑ to top ↑ ]

License

AREG SDK is under free open source Apache License Version 2.0. However, AREG SDK can be commercially licensed, which includes the commercial support, full rights to create and distribute software without open source license obligations. For commercial license, support or additional information, please visit Aregtech website or contact info[at]aregtech[dot]com.

[ ↑ to top ↑ ]

Call to action

Join our project and provide assistance by:

Checking out the list of open issues where we need help.
If you need new features, please open a new issue or start a discussion.
hen creating a pull request, kindly consider the time it takes for reviewing and testing, and maintain proper coding style.
If you require invoiced commercial support or training, or if you wish to support AREG SDK commercially, kindly contact us at info[at]aregtech[dot]com.

If AREG SDK has helped your project or inspired you with new ideas, please don't forget to this repository. This gesture of appreciation encourages contributors and helps us grow our community.