possible to switch register serving input/output feature map?

[NinaHimmelsbach]

Hello, is it possible to first solve the input feature map in FMEM0 and solve the output feature map in FMEM1 in the first level, but for the next level use FMEM1 as input and solve the output feature map in FMEM0, and switch so on?

Thanks in advance :)

[JiacongSun]

Hi NinaHimmelsbach,

Sorry I have difficulty understanding your question. Could you provide more context: what you are trying to accomplish and elaborate more on what "level" and "FMEM" you are referring to?

Best regards,
Jiacong

[NinaHimmelsbach]

Hi Jiacong,

Yes of course. I would like to have three feature map memories. FMEM0, FMEM1 and FMEM2. The three feature map memories store the input
and output feature maps and are switched dynamically to
exploit the fact that the output feature map of the previous
layer is the input feature map of the next layer. For instance,
the initial feature map is loaded into FMEM0 and the output
feature map of the first layer is stored into FMEM1. For the
next layer, FMEM1 will provide the input feature map and
the generated output feature map is stored back into FMEM0. Cause I would like to implement a ResNet workload where an additional
feature map for the parallel path needs to be managed, FMEM2 should store this feature map of the parallel path.

Thanks a lot for your help!

Best regards,
Nina Himmelsbach

[JiacongSun]

Hi Nina,

Thank you for your detailed explanation.

Currently, the memory hierarchy and memory serving operands are defined within the hardware template for entire network. This means that the same memory topology will be consistently applied across all layers, and the intended dataflow changes in your case won't happen automatically. However, there are 2 possible approaches you could consider:

1. Define FMEM0, FMEM1, FMEM2 as a memory shared by activation and output.
Treat FMEM0, FMEM1, and FMEM2 as a memory cluster and define a shared memory with the same size in the input file. This setup will automatically facilitate the operation you mentioned within the memory. Note: This solution is applicable only when the memory bandwidth of FMEM0, FMEM1, and FMEM2 is the same.

2. Define FMEM0, FMEM1 and FMEM2 as separate memories and iteratively update the memory serving operands in your intended way for different layers. There are 2 possible ways to achieve this:

• If you are familiar with how the accelerator object propagates within ZigZag framework, you could change the serving operands of FMEM0, FMEM1 in the accelerator object after the WorkloadStage. Use the layer id within WorkloadStage as a handle.

• If you are unfamiliar with ZigZag framework, create another script to call ZigZag run for only a single layer. For each layer, update the hardware input file --> skip all other layers except the targeted layer in WorkloadStage --> collect output data. By traversing all layers, you will obtain the total cost for the entire network.

Best regards,
Jiacong

[NinaHimmelsbach]

Thanks, I'll try it :D

[JiacongSun]

No problem! Let me know if you meet any problem.

Best regards,
Jiacong