Modeling of Inter-layer Data Transfer

[chensy7]

Hi, thanks a lot for open-sourcing the great work. I've been playing around with the demos and one thing I noticed is that for almost all layers, energy is dominated by DRAM accesses of I and O operands, even for Edge TPU example which has 2MB of on-chip SRAM for I&O. When we are modeling a network (e.g., Resnet18), are all the layers treated as independent workload (so the I&O has to go to DRAM) or is inter-layer data transfer considered (i.e., output from layer 0 stays on-chip as input for layer 1)? It would be great if you can point me to any code/text explanation of this. Thank you!

[asyms]

Hi,

Thanks for your interest.

You are correct that ZigZag models networks using a "single-layer-at-a-time" approach, where data always travels from/to the highest memory level. We are currently working on a Stage that can automatically infer the highest required memory level. @JiacongSun can give more updates on this.

In the meantime, you could play around with Stream, defining a single core. Stream models the entire network and considers data stationarity at the highest memory level defined within a Core, if possible.
Keep in mind that for some platforms data will always travel between offchip. As far as I know this is the case for the Edge TPU platform.

[JiacongSun]

Hi chensy7,

Thanks for your interest!

As mentioned by asyms, currently the input/output data of each layer is treated as an independent workload, and it travels from/to the top memory level.

We are working on developing stages that can automatically infer the required memory level based on the current layer size and remove unnecessary ones. This feature will allow data to remain in the local memory for use by the next layer. The new feature will be released in an upcoming version soon.

If you have any other features or suggestions that you believe would enhance the framework, please don't hesitate to let us know!

[chensy7]

Hi guys, thank you for the prompt reply. It's great to hear that there's a new version coming, looking forward to it! I'll check out Stream in the mean time. We are trying to use Zigzag for a new project so you will probably see me again with questions as I continue experimenting different things with the framework.