How to specify query parameters for getImageFrame

[terry-wilson]

I am interested in trying out the tile-level-marker-proxy sample, however, it is not clear to me how to add the necessary query parameters (startLevel, endLevel) to the getImageFrame method. I am using the official AWS SDK V3 (it takes care of the authentication for me) but don't really see any obvious way to add these parameters as the standard method does not define any in the SDK (GetImageFrameCommand) ... any suggestions on how to do this?

[yyao84]

Hi @terry-wilson,

The idea behind retrieving discrete lower-resolutioned images from HealthImaging using HTJ2K is:

• initiate the get frame request, creating the read stream for the frame

• begin reading the stream, and decode the header

  • only the first 128 bytes are required to get this data
  • header information includes width/height, pixel format, components (rgb), etc
  • one piece of header information is the number of discrete decompositions (discrete resolutions)
    • this data includes both the decomposition level, and the resolution at that level

• continue reading the stream, taking note of the SOT (start of tile) marker. this marks the decomposition level boundary

• when arriving at the interested level (endLevel), stop reading the stream and return the data

You can read from levels 0 - n to return a sub or full frame. If you've read a sub-frame and want to continue rendering the full resolutioned image, you can read levels n - m, then append it to 0 - n. This results in an object of 0 - m size, and you can decode that for a higher resolutioned image. This way you don't have to keep reading from the start of the frame to render the image at higher resolutions. As a side note, if you don't need discrete image resolutions, you can simply decode and display the frame as you're reading it from a stream.

The TLM sample project sets up a fastify environment and accepts a startLevel, endLevel parameter,

• creates a GetImageFrame stream

• uses the getResolutionRange function (written by @chafey , using the htj2k-js decoder) to read and parse the image frame stream

• this returns one of three objects: 0 - endLevel, startLevel - endLevel, or the full image frame (where endLevel is the the last decomposition level)

• the proxy passes this object back to the

Please let us know if you have any further questions. Happy to set up a call too, if you'd like.

[terry-wilson]

Hi @yyao84, thanks for the detailed explanation, that all makes sense ... however, I still have my original question of HOW to pass the startLevel & endLevel query parameters on the getImageFrame API using the official AWS SDK V3 for javascript?

I also have an additional question on how the authentication works between the proxy and the corresponding HealthImaging data store? When I looked at the code in the proxy, I can see the call in tlm-proxy.js where the GetImageFrameCommand is sent, but I don't see any sort of authentication set up in the imagingClient ... do you somehow forward any authentication token from the incoming request?

At this point, I am just doing some testing with anonymized data, so I would be fine using no authentication at all, but I thought the back-end AWS data store required it.

[yyao84]

I'll answer this in reverse since I think the authentication question helps to answer the query parameter question -

Authentication to AWS APIs uses AWS Signature v4, which the SDK takes care of for us. There's a number of ways to configure credentials for the SDK.

In the case of the TLM proxy, since it's running on an ECS cluster, the SDK picks up credentials from the ECS task role, which is an IAM role for specifically for this task. This uses the SDK container provider behind the scenes, and transparent to us. Processes running in this container can medical-imaging:GetImageFrame but nothing else.

The TLM proxy itself is exposed publically using an Application Load Balancer (ALB), which can be accessed using an HTTPS endpoint. This requires an Amazon Certificate Manager certificate, which is free if you already have a domain to use. The ALB has a public DNS name that you'll use to create a CNAME record, e.g. tlm.proxy.your.domain <> tlm-alb-123-abc.us-east-1.elb.amazonaws.com. This is where you'd authenticate with the JWT from Cognito or leave it open for testing, but keep in mind this request is signed and passed in as an authenticated to request to HealthImaging in the container.

Since the TLM proxy is not an AWS API, we're not able to use the official AWS SDK with it. You would call the endpoint using the same URL and body as the HealthImaging GetImageFrame request, but with the authentication that the TLM Proxy expects and the query parameters appended. This is implemented in the Sample Viewer's tlmLoader, where it loads the frames one level at a time:

• initially it requests from the proxy tlm level 0 (startLevel = 0, endLevel = 0) using a simple fetch

  • this small object has the header and the smallest discrete resolution for the frame
  • url example: https://tlm.proxy.your.domain/datastore/<datastoreId>/imageSet/<imageSetId>/getImageFrame?startLevel=0&endLevel=0
  • post body: {"imageFrameId":"<imageFrameId>"}

• it then requests subsequent TLM levels one at a time, and appends them to an array

  • https://tlm.proxy.your.domain/datastore/<datastoreId>/imageSet/<imageSetId>/getImageFrame?startLevel=1&endLevel=1
  • https://tlm.proxy.your.domain/datastore/<datastoreId>/imageSet/<imageSetId>/getImageFrame?startLevel=2&endLevel=2
  • https://tlm.proxy.your.domain/datastore/<datastoreId>/imageSet/<imageSetId>/getImageFrame?startLevel=3&endLevel=3

• after all the requests are complete, it joins the array buffers into a partial or full frame, and decodes and displays it

This is just an example implementation where we request one TLM level at a time and put them together client-side. But you can do other things like startLevel=0, endLevel=3, etc.

Hope this helps!

[terry-wilson]

Thx, that helps to explain things and gives me the info I needed.

I do have a domain and created a certificate with AWS Certificate Mgr and configured the corresponding ARN in config.ts ... I also set AUTH_MODE to 'null'. I ran "cdk deploy" without issue, however, "cdk deploy" seems to mostly finish before stalling and eventually timing out after 3 hours and then rolling back (see screenshot). I have tried running the deploy command twice and both times had the exact same problem ... any suggestions for how to determine what the issue with the deployment is?

[yyao84]

I've seen this occur when the ECS service doesn't enter into a consistent state for whatever reason (container doesn't start, health check fails, etc.). You can typically see the reason from ECS logging:

• go into the ECS console (verify this is the right region),
• select the TileLevelMakerProxyStack-... cluster

To check the service logs:

• select the TileLevelMarkerProxyStack-ecsfargateservice... service
• select the Logs tab.

To check the individual task logs:

• select the tasks tab
• select one of the tasks
• select the Logs tab

In either case, there should be logs on why the container is failing to start.

I'll attempt to reproduce this on my side as well.

[terry-wilson]

I ran deploy again and I see this in the logs for the service ... AHI_REGION is set to 'us-east-1' in config.ts so not sure what exactly is the issue ... the only changes I made to config.ts was to add the ARN for the cert and set AUTH_MODE to 'null':

[terry-wilson]

@yyao84 I was looking at the code that was failing in tlm-proxy.js and it didn't seem correct to me so I changed it as follows (commented out code is the original):

//if (process.env.AHI_ENDPOINT) imagingClientConfig.endpoint = AHI_ENDPOINT;
if (process.env.AHI_ENDPOINT) imagingClientConfig.endpoint = process.env.AHI_ENDPOINT;
if (process.env.AHI_REGION) {
    // imagingClientConfig.endpoint = AHI_REGION;
    imagingClientConfig.region = process.env.AHI_REGION;
} else {
    imagingClientConfig.region = 'us-east-1';
}

Re-running the deployment with these changes works this time. I haven't tried actually using the system yet, but please let me know if these code changes are not appropriate for whatever reason.

[yyao84]

Thanks @terry-wilson for finding that bug in the code. Feel free to submit a PR, otherwise I'll update it later today