Kinesis Video Stream hangs on getMedia request

Question

Kinesis Video Stream hangs on getMedia request

intergavg opened this issue 6 months ago · 5 comments

Describe the bug

We are using cpp SDK to read video from Kinesis Video Streams, GetEndpoint is returning us a response without any issues, GetMedia endpoint however hangs forever, no response is received.

Expected Behavior

GetMedia should return a proper response, so we can read payload continuously

Current Behavior

GetMedia request is getting stuck

Reproduction Steps

Here is a sample code we try to consume the Kinesis Video Streams API:

  `Aws::KinesisVideo::Model::GetDataEndpointOutcome get_end_point(const std::string& stream_name, const 
    Aws::Client::ClientConfiguration& client_config, std::string& client_name)
    {
        Aws::KinesisVideo::KinesisVideoClient client(aws_creds_, client_config);
        auto outcome = client.GetDataEndpoint(Aws::KinesisVideo::Model::GetDataEndpointRequest()
                                                           .WithStreamName(stream_name)
                                                           .WithAPIName(Aws::KinesisVideo::Model::APIName::GET_MEDIA));
        client_name = client.GetServiceClientName();
        if (!outcome.IsSuccess()) {
            std::cout << "Failed to get data endpoint: " << outcome.GetError().GetMessage() << std::endl;
        }
        client.OverrideEndpoint(outcome.GetResultWithOwnership().GetDataEndpoint());
        return outcome;
    }

    void read(const std::string& stream_name, const std::string& client_name)
    {
        Aws::Client::ClientConfiguration client_config;
        client_config.region = Aws::Region::US_EAST_2;
        std::string client_name_api;
        auto outcome = get_end_point(stream_name, client_config, client_name_api);
        client_config.endpointOverride = outcome.GetResult().GetDataEndpoint();
        Aws::KinesisVideoMedia::KinesisVideoMediaClient video_media_client(aws_creds_, client_config);
        video_media_client.EnableRequestProcessing();

        auto output = video_media_client.GetMedia(Aws::KinesisVideoMedia::Model::GetMediaRequest()
                                                          .WithStreamName(stream_name)
                                                          .WithStartSelector(Aws::KinesisVideoMedia::Model::StartSelector()
                                                                                     .WithStartSelectorType(Aws::KinesisVideoMedia::Model::StartSelectorType::NOW)));
        if (!output.IsSuccess()) {
            std::cout << "Failed to retrieve data from Kinesis Video Stream: " << output.GetError().GetMessage() << std::endl;
        }

    }`

Code is being stuck on GetMedia call forever.

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.272

Compiler and Version used

gcc version 11.4.0 / g++ version 11.4.0

Operating System and version

Ubuntu 22.04 Desktop

Answer 1 · 2024-02-28T15:28:32.000Z

hey @intergavg I tried reproducing and I couldn't, i modified your test code a little bit

#include <aws/core/Aws.h>
#include <aws/kinesisvideo/KinesisVideoClient.h>
#include <aws/kinesisvideo/model/GetDataEndpointRequest.h>
#include <aws/kinesis-video-media/KinesisVideoMediaClient.h>
#include <aws/kinesis-video-media/model/GetMediaRequest.h>

using namespace Aws;
using namespace Aws::Utils;
using namespace Aws::KinesisVideo;
using namespace Aws::KinesisVideo::Model;
using namespace Aws::KinesisVideoMedia;
using namespace Aws::KinesisVideoMedia::Model;

const char* STREAM_NAME = "your-stream-name";

auto main() -> int {
  SDKOptions options;
  options.loggingOptions.logLevel = Logging::LogLevel::Trace;
  InitAPI(options); {
    const auto client = Aws::MakeUnique<KinesisVideoClient>("test");
    const auto outcome = client->GetDataEndpoint(GetDataEndpointRequest()
      .WithStreamName(STREAM_NAME)
      .WithAPIName(APIName::GET_MEDIA));
    assert(outcome.IsSuccess());
    const auto enpoint = outcome.GetResult().GetDataEndpoint();
    std::cout << "Found endpoint: " << enpoint << "\n";

    KinesisVideoMediaClientConfiguration configuration;
    configuration.endpointOverride = enpoint;
    const auto mediaClient = Aws::MakeUnique<KinesisVideoMediaClient>("test", configuration);
    const auto mediaOutcome = mediaClient->GetMedia(GetMediaRequest()
      .WithStreamName(STREAM_NAME)
      .WithStartSelector(StartSelector()
      .WithStartSelectorType(StartSelectorType::NOW)));
    assert(mediaOutcome.IsSuccess());
  }
  ShutdownAPI(options);
  return 0;
}

So looks like we might have a "works on my machine" issue here. one thing i want to rule out first and foremost is could you try setting you logs to trace i.e.

  SDKOptions options;
  options.loggingOptions.logLevel = Logging::LogLevel::Trace;
  InitAPI(options);

then in your workdir you should see a logfile aws_sdk_{DATE}.log

in that file near the end you should see a curl debug statement

[DEBUG] 2024-02-28 15:16:02.468 CURL [0x1d6f05c40] (HeaderOut) POST /getMedia HTTP/1.1
Host: REDACTED-HOST-PREFIX.kinesisvideo.us-east-1.amazonaws.com
Accept: */*
amz-sdk-invocation-id: DC534AB6-FB00-4F77-A07D-B389E81D3A1C
amz-sdk-request: attempt=1
authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20240228/us-east-1/kinesisvideo/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=REDACTED
content-length: 84
content-type: application/json
user-agent: REDACTED
x-amz-api-version: 2017-09-30
x-amz-content-sha256: REDACTED
x-amz-date: 20240228T151602Z
x-amz-security-token: REDACTED

what im curious specifically about is if the Host: REDACTED-HOST-PREFIX.kinesisvideo.us-east-1.amazonaws.com header is reflective of what your override was set to or if it is something unexpected? gunna try creating a docker image so we can be on same page of "on my machine" for further debugging.

also give my test code a shot and see if it works for you. i know your code is likely a snippet of a larger piece of code but InitAPI and ShutdownAPI are requried for sdk usage. Just trying to rule out everything so let me know if the test code changes the outcome.

Answer 2 · 2024-02-28T17:23:29.000Z

Alright created a docker file and example that cannot reproduce

Docker File:

FROM public.ecr.aws/amazonlinux/amazonlinux:2023

#Install g++
RUN yum groupinstall "Development Tools" -y

#Install required dependencies
RUN yum install -y curl-devel openssl-devel ninja-build cmake3 graphviz

# Clone repo
RUN git clone --depth 1 --recurse-submodules https://github.com/aws/aws-sdk-cpp && \
    cd aws-sdk-cpp && \
    mkdir build && \
    cd build && \
    cmake -DAUTORUN_UNIT_TESTS=OFF -DBUILD_ONLY="kinesisvideo;kinesis-video-media" .. && \
    cmake --build . && \
    cmake --install .

# Copy Code Over 
RUN mkdir sdk-example
COPY CMakeLists.txt sdk-example/CMakeLists.txt
COPY main.cpp sdk-example/main.cpp
RUN cd sdk-example && \
    mkdir build && \
    cd build && \
    cmake .. && \
    cmake --build . \

CMakeLists.txt

cmake_minimum_required(VERSION 3.22)
project(sdk_usage_workspace)
set(CMAKE_CXX_STANDARD 20)
find_package(ZLIB)
find_package(AWSSDK REQUIRED COMPONENTS kinesisvideo kinesis-video-media)
add_executable(${PROJECT_NAME} "main.cpp")
target_link_libraries(${PROJECT_NAME} ${AWSSDK_LINK_LIBRARIES})

main.cpp

#include <aws/core/Aws.h>
#include <aws/kinesisvideo/KinesisVideoClient.h>
#include <aws/kinesisvideo/model/GetDataEndpointRequest.h>
#include <aws/kinesis-video-media/KinesisVideoMediaClient.h>
#include <aws/kinesis-video-media/model/GetMediaRequest.h>

using namespace Aws;
using namespace Aws::Utils;
using namespace Aws::KinesisVideo;
using namespace Aws::KinesisVideo::Model;
using namespace Aws::KinesisVideoMedia;
using namespace Aws::KinesisVideoMedia::Model;

const char* STREAM_NAME = "your stream name";

auto main() -> int {
  SDKOptions options;
  options.loggingOptions.logLevel = Logging::LogLevel::Trace;
  InitAPI(options); {
    KinesisVideoClientConfiguration videoConfig;
    videoConfig.connectTimeoutMs = 5000;
    const auto client = Aws::MakeUnique<KinesisVideoClient>("test", videoConfig);
    const auto outcome = client->GetDataEndpoint(GetDataEndpointRequest()
      .WithStreamName(STREAM_NAME)
      .WithAPIName(APIName::GET_MEDIA));
    if (outcome.IsSuccess()) {
      std::cout << "Successfully called Get Endpoint\n";
    } else {
      std::cout << "Failed call to  Get Endpoint: " << outcome.GetError().GetMessage() << "\n";
    }
    assert(outcome.IsSuccess());
    const auto endpoint = outcome.GetResult().GetDataEndpoint();
    std::cout << "Found endpoint: " << endpoint << "\n";

    KinesisVideoMediaClientConfiguration configuration;
    configuration.endpointOverride = endpoint;
    configuration.connectTimeoutMs = 5000;
    const auto mediaClient = Aws::MakeUnique<KinesisVideoMediaClient>("test", configuration);
    const auto mediaOutcome = mediaClient->GetMedia(GetMediaRequest()
      .WithStreamName(STREAM_NAME)
      .WithStartSelector(StartSelector().WithStartSelectorType(StartSelectorType::NOW)));
    if (mediaOutcome.IsSuccess()) {
      std::cout << "Successfully called Get Media\n";
    } else {
      std::cout << "Failed call to  Get Media: " << mediaOutcome.GetError().GetMessage() << "\n";
    }
    assert(mediaOutcome.IsSuccess());
  }
  ShutdownAPI(options);
  return 0;
}

when built like

docker build -t kinesis-test-image .

and run like

docker run \
    -e AWS_ACCESS_KEY_ID=YOUR_SECRET_KEY \
    -e AWS_SECRET_ACCESS_KEY=YOUR_SECRECT_ACCESS \
    -e AWS_SESSION_TOKEN=YOUR_SESSION_TOKEN_IF_APPLICABLE \
    --name kinesis-test-image kinesis-test-image /sdk-example/build/sdk_usage_workspace

I saw the output

Successfully called Get Endpoint
Found endpoint: https://REACTED_ENDPOINT_PREFIX.kinesisvideo.us-east-1.amazonaws.com
Successfully called Get Media

For further debugging I will need you update this example to fail in the way you expect it too. I will also note that while developing the example i saw curl timeouts in the example in which i had to update with config.connectTimeoutMs = 5000; which might also be your problem but would need to see your logs to be sure.

let me know if you see any errors in your logs, or if you can update the example to replicate your failure.

Answer 3 · 2024-02-29T09:01:17.000Z

Thank you for making this sample, I tried this on my demo Ubuntu 20.04 PC, Docker version 25.0.3, build 4debf41

I got the following in the output:

sdk_usage_workspace: /sdk-example/main.cpp:31: int main(): Assertion outcome.IsSuccess()' failed.`

Answer 4 · 2024-02-29T16:36:01.000Z

main.cpp:31

line 31 is the the endpoint call, which previously worked for you, there should be std::out right before the assertion about why the call failed. What does the std::out say why it failed? what do the logs say?

Answer 5 · 2024-03-11T00:16:21.000Z

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.