/srf

Streaming Reactive Framework (SRF)

Primary LanguageC++Apache License 2.0Apache-2.0

!!! Work In Progress !!!

While were close to our public release, we still have a few things to finalize before the repository is ready for public use. Check back in a few days for more details.

Streaming Reactive Framework (SRF)

The Streaming Reactive Framework (SRF) library (pronounced "surf") is designed to build high throughput, low latency pipelines.

Major features include:

  • Built in C++ for performance, with Python bindings for ease of use
  • Code-first pipeline design
  • Distributed computation with message transfers over RMDA using UCX
  • Dynamic reconfiguration to scale up and out at runtime​. Requires no changes to pipeline configuration
  • Unopinionated data model. Messages of any type can be used in the pipeline
  • Built from the ground up with asynchronous computation for mitigation of I/O and GPU blocking
  • Automatically handles backpressure and reschedules computation as needed
  • Focused on rapid pipeline development, with options for maximizing performance

Quickstart

To get started with SRF, see the SRF Quickstart Repository located here.

Contributing

SRF is licensed under the Apache v2.0 license. All new source files including CMake and other build scripts should contain the Apache v2.0 license header. Any edits to existing source code should update the date range of the copyright to the current year. The format for the license header is:

/*
 * SPDX-FileCopyrightText: Copyright (c) <year>, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 * SPDX-License-Identifier: Apache-2.0
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

Thirdparty code

Thirdparty code included in the source tree (that is not pulled in as an external dependency) must be compatible with the Apache v2.0 license and should retain the original license along with a url to the source. If this code is modified, it should contain both the Apache v2.0 license followed by the original license of the code and the url to the original code.

Ex:

/**
 * SPDX-FileCopyrightText: Copyright (c) 2018-2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 * SPDX-License-Identifier: Apache-2.0
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

//
// Original Source: https://github.com/org/other_project
//
// Original License:
// ...