awslabs/amazon-kinesis-client

how to read from dynamodb + kinesis stream

Opened this issue · 0 comments

I have data like this:

val raw_json = """{"awsRegion":"us-west-2","eventID":"ba5735ba-db0f-44eb-b0a0-7915c6c99e1a","eventName":"MODIFY","userIdentity":null,"recordFormat":"application/json","tableName":"qa.customer_data.userevent_customer_rollup","dynamodb":{"ApproximateCreationDateTime":1721101251338321,"Keys":{"_feature_store_internal__primary_keys":{"S":"[\"444a3fa3-b3c5-44ee-ac99-f861203a66ab\"]"}},"NewImage":{"models_visited_1d":{"M":{"Chevrolet|Trailblazer":{"N":"2"}}},"last_action_time":{"N":"1721101243000000"},"makes_visited_1d":{"M":{"Chevrolet":{"N":"2"}}},"window.end":{"N":"1721174400000000"},"customer_id":{"S":"444a3fa3-b3c5-44ee-ac99-f861203a66ab"},"newused_models_visited_30d":{"M":{"new|chevrolet|trailblazer":{"N":"2"}}},"unique_pages_30d":{"SS":["dealer_site","dealer_srp","dealer_vdp","eva_appraisal_entry","new_model_car_inventory_vin_detail","new_model_car_inventory_vin_detail_shopping"]},"total_actions_30d":{"N":"233"},"models_visited_30d":{"M":{"Chevrolet|Trailblazer":{"N":"2"},"|":{"N":"15"}}},"_feature_store_internal__primary_keys":{"S":"[\"444a3fa3-b3c5-44ee-ac99-f861203a66ab\"]"},"makes_visited_30d":{"M":{"Chevrolet":{"N":"2"}}},"vehicle_viewed_events_30d":{"L":[{"S":"{\"viewedAt\":1720805122000,\"vin\":\"JF2GTAAC2PG255353\",\"make\":\"\",\"model\":\"\",\"year\":null,\"bodyType\":\"\"}"},{"S":"{\"viewedAt\":1720805325000,\"vin\":\"JF2GTAAC2PG255353\",\"make\":\"\",\"model\":\"\",\"year\":null,\"bodyType\":\"\"}"},{"S":"{\"viewedAt\":1720806219000,\"vin\":\"1C4PJMMB2PD104071\",\"make\":\"\",\"model\":\"\",\"year\":null,\"bodyType\":\"\"}"},{"S":"{\"viewedAt\":1720806894000,\"vin\":\"3N1CP5DV6NL475449\",\"make\":\"\",\"model\":\"\",\"year\":null,\"bodyType\":\"\"}"},{"S":"{\"viewedAt\":1721101013000,\"vin\":\"KL79MSSL7RB191624\",\"make\":\"Chevrolet\",\"model\":\"Trailblazer\",\"year\":\"2024\",\"bodyType\":\"suv\"}"},{"S":"{\"viewedAt\":1721101069000,\"vin\":\"KL79MSSL7RB191624\",\"make\":\"Chevrolet\",\"model\":\"Trailblazer\",\"year\":\"2024\",\"bodyType\":\"suv\"}"}]},"newused_bodytype_visited_30d":{"M":{"new|suv":{"N":"2"}}},"first_action_time":{"N":"1720729962433000"},"window.start":{"N":"1720742400000000"},"unique_sessions_30d":{"SS":["524666941738586619","597604370061016119"]},"page_enters_30d":{"N":"17"}},"OldImage":{"models_visited_1d":{"M":{"Chevrolet|Trailblazer":{"N":"2"}}},"last_action_time":{"N":"1721101232000000"},"makes_visited_1d":{"M":{"Chevrolet":{"N":"2"}}},"window.end":{"N":"1721174400000000"},"customer_id":{"S":"444a3fa3-b3c5-44ee-ac99-f861203a66ab"},"newused_models_visited_30d":{"M":{"new|chevrolet|trailblazer":{"N":"2"}}},"unique_pages_30d":{"SS":["dealer_site","dealer_srp","dealer_vdp","eva_appraisal_entry","new_model_car_inventory_vin_detail","new_model_car_inventory_vin_detail_shopping"]},"total_actions_30d":{"N":"232"},"models_visited_30d":{"M":{"Chevrolet|Trailblazer":{"N":"2"},"|":{"N":"15"}}},"_feature_store_internal__primary_keys":{"S":"[\"444a3fa3-b3c5-44ee-ac99-f861203a66ab\"]"},"makes_visited_30d":{"M":{"Chevrolet":{"N":"2"}}},"vehicle_viewed_events_30d":{"L":[{"S":"{\"viewedAt\":1720805122000,\"vin\":\"JF2GTAAC2PG255353\",\"make\":\"\",\"model\":\"\",\"year\":null,\"bodyType\":\"\"}"},{"S":"{\"viewedAt\":1720805325000,\"vin\":\"JF2GTAAC2PG255353\",\"make\":\"\",\"model\":\"\",\"year\":null,\"bodyType\":\"\"}"},{"S":"{\"viewedAt\":1720806219000,\"vin\":\"1C4PJMMB2PD104071\",\"make\":\"\",\"model\":\"\",\"year\":null,\"bodyType\":\"\"}"},{"S":"{\"viewedAt\":1720806894000,\"vin\":\"3N1CP5DV6NL475449\",\"make\":\"\",\"model\":\"\",\"year\":null,\"bodyType\":\"\"}"},{"S":"{\"viewedAt\":1721101013000,\"vin\":\"KL79MSSL7RB191624\",\"make\":\"Chevrolet\",\"model\":\"Trailblazer\",\"year\":\"2024\",\"bodyType\":\"suv\"}"},{"S":"{\"viewedAt\":1721101069000,\"vin\":\"KL79MSSL7RB191624\",\"make\":\"Chevrolet\",\"model\":\"Trailblazer\",\"year\":\"2024\",\"bodyType\":\"suv\"}"}]},"newused_bodytype_visited_30d":{"M":{"new|suv":{"N":"2"}}},"first_action_time":{"N":"1720729962433000"},"window.start":{"N":"1720742400000000"},"unique_sessions_30d":{"SS":["524666941738586619","597604370061016119"]},"page_enters_30d":{"N":"17"}},"SizeBytes":2817,"ApproximateCreationDateTimePrecision":"MICROSECOND"},"eventSource":"aws:dynamodb"}"""

Can it be read with amazon-kinesis-client? I've tried various approaches and have not been successful. Perhaps it is just regular kinesis or dynamodb objects?