Rekognition `GetContentModeration` response differs from documentation
Closed this issue · 7 comments
ifellinaholeonce commented
Describe the bug
I have recently started using the Rekognition API for video content moderation. I noticed that the responses I have been getting do not match the documentation in a couple of ways - the labels themselves use different words than the docs, and a number of fields on the object are nil.
I was able to find older documentation using Wayback Machine here that seems to better match the current behaviour. The labels properly align and the nil fields are not documented.
Is it possible the gem updated the struct for the responses but is not hitting the right version of the API or something? I haven't noticed anywhere that specifying an API version was necessary. Am I possibly simply missing a configuration setting?
Expected Behavior
struct Aws::Rekognition::Types::ContentModerationDetection should properly match documented behaviour found here and here.
Current Behavior
struct Aws::Rekognition::Types::ContentModerationDetection is missing some fields and uses undocumented labels. Notice in the below snippet in the moderation_labels array that each element is missing values for taxonomy_level, start_timestamp_milis, end_timestamp_milis and duration_milis. Also notice that some of the names of labels are "Tobacco", for example, which do not match any of the name in the docs. I would expect it to be either "Drugs & Tobacco" or "Drugs & Tobacco Paraphernalia & Use" in this case.
#<struct Aws::Rekognition::Types::GetContentModerationResponse
job_status="SUCCEEDED",
status_message=nil,
video_metadata=#<struct Aws::Rekognition::Types::VideoMetadata codec="h264", duration_millis=13760, format="QuickTime / MOV", frame_rate=25.0, frame_height=1920, frame_width=1080, color_range="LIMITED">,
moderation_labels=
[#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.2705247402191162, name="Smoking", parent_name="Tobacco", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.2705247402191162, name="Tobacco", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=1000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.0228523015975952, name="Smoking", parent_name="Tobacco", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=1000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.0228523015975952, name="Tobacco", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=1520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.377088189125061, name="Explicit Nudity", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=1520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.1258432865142822, name="Graphic Female Nudity", parent_name="Explicit Nudity", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=1520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.3413856029510498, name="Sexual Activity", parent_name="Explicit Nudity", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=1520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=2.4593212604522705, name="Smoking", parent_name="Tobacco", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=1520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=2.4593212604522705, name="Tobacco", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=2000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=3.6541855335235596, name="Explicit Nudity", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=2000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=2.030139207839966, name="Graphic Female Nudity", parent_name="Explicit Nudity", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=2000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=3.6497228145599365, name="Sexual Activity", parent_name="Explicit Nudity", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=2000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=5.710151672363281, name="Smoking", parent_name="Tobacco", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=2000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=5.710151672363281, name="Tobacco", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=2520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.7081072330474854, name="Explicit Nudity", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=2520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.7058758735656738, name="Sexual Activity", parent_name="Explicit Nudity", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=2520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.3264174461364746, name="Smoking", parent_name="Tobacco", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=2520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.3264174461364746, name="Tobacco", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=3000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.4072818756103516, name="Explicit Nudity", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=3000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=2.879955291748047, name="Graphic Violence Or Gore", parent_name="Violence", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=3000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.4072818756103516, name="Sexual Activity", parent_name="Explicit Nudity", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=3000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.1348352432250977, name="Smoking", parent_name="Tobacco", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=3000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.1348352432250977, name="Tobacco", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=3000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=2.879955291748047, name="Violence", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=3520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=4.893579006195068, name="Graphic Violence Or Gore", parent_name="Violence", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=3520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=4.893579006195068, name="Violence", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=4000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=2.2859199047088623, name="Graphic Violence Or Gore", parent_name="Violence", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=4000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=2.2859199047088623, name="Violence", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=4520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=2.868089437484741, name="Graphic Violence Or Gore", parent_name="Violence", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=4520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=2.868089437484741, name="Violence", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=5000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.615448236465454, name="Graphic Violence Or Gore", parent_name="Violence", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=5000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.615448236465454, name="Violence", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=5520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=3.0176239013671875, name="Graphic Violence Or Gore", parent_name="Violence", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=5520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=3.0176239013671875, name="Violence", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=6000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=3.4282007217407227, name="Graphic Violence Or Gore", parent_name="Violence", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=6000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=3.4282007217407227, name="Violence", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=6520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.4649137258529663, name="Smoking", parent_name="Tobacco", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=6520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.4649137258529663, name="Tobacco", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=10520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.5620267391204834, name="Graphic Violence Or Gore", parent_name="Violence", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=10520, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.5620267391204834, name="Violence", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=11000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.3054022789001465, name="Graphic Violence Or Gore", parent_name="Violence", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>,
#<struct Aws::Rekognition::Types::ContentModerationDetection timestamp=11000, moderation_label=#<struct Aws::Rekognition::Types::ModerationLabel confidence=1.3054022789001465, name="Violence", parent_name="", taxonomy_level=nil>, start_timestamp_millis=nil, end_timestamp_millis=nil, duration_millis=nil, content_types=nil>],
next_token=nil,
moderation_model_version="6.1",
job_id="6399...e",
video=#<struct Aws::Rekognition::Types::Video s3_object=#<struct Aws::Rekognition::Types::S3Object bucket="ifellinaholeonce-storage", name="7230860-hd_1080_1920_25fps.mp4", version=nil>>,
job_tag=nil,
get_request_metadata=#<struct Aws::Rekognition::Types::GetContentModerationRequestMetadata sort_by="TIMESTAMP", aggregate_by="TIMESTAMPS">>Reproduction Steps
Upload a video to S3, start content moderation and then get content moderation..
client = Aws::Rekognition::Client.new(
credentials: Aws::Credentials.new(
aws_access_key_id,
aws_access_key
),
region: aws_region,
)
content_key = "your-s3-item-content-key"
resp = client.start_content_moderation(
video: {
s3_object: {
bucket:,
name: content_key
}
},
min_confidence: 1 # set this really low so that we can easily get some moderation flags returned
)
job_id = resp.job_id
# wait a minute for processing to complete
result = client.get_content_moderation(job_id:)Possible Solution
No response
Additional Information/Context
No response
Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version
aws-sdk-rekognition
Environment details (Version of Ruby, OS environment)
ruby 3.2.3p157
mullermp commented
Thanks for opening up an issue. What version of aws-sdk-rekognition are you using? The latest version is 1.94.0.
mullermp commented
I don't use rekognition, but it looks like to use the V7 label categories (the Drugs & Tobacco label), you need to opt in to some private version?
https://docs.aws.amazon.com/rekognition/latest/dg/moderation.html#moderation-labels-update-sdk
ifellinaholeonce commented
Sorry I should have specified the gem version. It is indeed the newest
aws-sdk-rekognition (1.94.0)
aws-sdk-core (~> 3, >= 3.191.0)
aws-sigv4 (~> 1.1)
That is interesting @mullermp about the private version. The response does state moderation_model_version="6.1", so maybe this is the missing piece.
ifellinaholeonce commented
@mullermp I think this is the information I was missing (I swear this wasn't in the docs yesterday 😂). Looks like I just happened to show up during a transition time. I don't think there is anything more to do here so I am going to close this. Thanks for the help.
github-actions commented
This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
mullermp commented
No worries. I'm glad that was the answer. @RanVaknin from the support team informed me that on May 24, that new version will become default.
mullermp commented
Let me know about those nil members. I'm fairly confident (99.99% sure) that we are parsing the service responses correctly - we have very extensive tests on this per protocol. It's likely that the version of the model from the service simply just does not return any values for those.