[EdgeTPU Backend] How to distinguish regular tflite model from edgeTPU-compiled tflite model?
Hyeon-Uk opened this issue · 7 comments
Here is our project introduction
Project Introduction : #1621
Background : outputs after compiling tflite model with EdgeTPU
File before compiled with EdgeTPU and file after compiled with EdgeTPU are the same type, .tflite
.
The image below illustrates the procedure of converting a .tflite
model into an Edge TPU Model using the Edge TPU compiler. The red box located at the bottom represents the .tflite
file intended for compilation, while the blue box symbolizes the resulting output of the compilation process.
It means that baseModel and Product are all tflite file. However, tflite file compiled with EdgeTPU doesn’t need to convert to circle file because edgetpu-custom-op cannot be inferenced in ONE-vscode. So this special tflite file is not a Base Model, it is a Product.
As a result, ONE explorer has to show the tflite file in Product type instead of Base Model type after detecting tflite file and checks whether the tflite is a regular file or is from EdgeTPU Compiler.
The way to distinguish Base Model from Product in ONE-vscode
- Find the Base Model with filename extension such as
.pb
,.tflite.
- Read the Base Model’s
.cfg
file and finds the results that corresponding with Output Path. - Display the results.
Of course, it assumes that the user performs all works in ONE-vscode interface.
Problem
As I mentioned, because the file type of the EdgeTPU Compiler’s result is .tflite
, ONE-vscode reads this result two times, while finding Base Model and finding Product with Output Path.
We considered about the way to distinguish those two types(regular tflite and tflite from EdgeTPU Compiler) of tflite files.
Process we discussed
- After generating Base Models, set Products of each Models corresponding to Output Path in cfg files of each Models.
- At this point, EdgeTPU Compiler’s outputs saved seperately
- Exclude those files from Base Model and generate the rest files as the Base Model.
However, we believe that this straightforward method might incur a high time complexity
. So we keep thinking about the better method.
If you have a better idea about it, please give us an innovative idea.
Process we discussed
- After generating Base Models, set Products of each Models corresponding to Output Path in cfg files of each Models.
- At this point, EdgeTPU Compiler’s outputs saved seperately
- Exclude those files from Base Model and generate the rest files as the Base Model.
Could you explain more about this?
I agree that excluding edgetpu-compiled tflite files from BaseModel
(data structure) is necessary in our view.
I suppose that we could define edgetpu-compiled files to have a certain postfix as *_edgetpu.tflite
, like edgetpu-compiler do as default.
- Before creating the TreeView component, we first utilize the getCfgList() function from OneStorage to fetch a list of
cfg
files. - Following this, we collect and save the output path generated by the EdgeTPU compiler into a variable.
- Then, within the _buildChildren() code, we implement a conditional check to determine if the file is a
.tflite
file representing thebaseModel
. - In this check, we verify whether the model exists within the previously collected variables.
- If we find the model in the collection, we've made the decision not to categorize it as a 'baseModel' because it originates from the EdgeTPU Compiler.
The method described above is what we have considered. We opted for this approach because we believed that if we differentiated them solely by postfix, it might limit our ability to customize the output path, and we would always need to include the postfix.
Initially, we will use postfix to differentiate them, and if we identify an additional method for distinguishing them, we will incorporate it as well.
We found a method
When view .tflite
file compiled with edge tpu compiler using netron, we notice that description : Exported from Subgraph
. We observed that this text appears in .tflite
file compiled with edge tpu compiler, regardless of the original description.
Therefore, we concluded that if .tflite
file has description : Exported from Subgraph
, it is a edgetpu-compiled tflite file.
We attempted to open tflite file using flatbuffers
library. We compiled schema.fbs
file on Tensorflow's github using flatbuffers-compiler
. And we success to read tflite file's description with schema_generated.ts
that is result of flatbuffers-compiler.
Regarding this
- We used Tensorflow's schema.fbs file that licensed under the Apache License V2. Is it ok?
- When we open 6~7mb tflie file for the first time, it spend 20ms. After second times, it spend 3~4ms. Would this have significant impact on performace?
- Then, within the _buildChildren() code, we implement a conditional check to determine if the file is a .tflite file representing the baseModel.
How do you 'check to determine'? 🤔
Do you mean that you see whether the file is written as 'Product' in the cfgs?
(I am still curious about this approach even though you have suggested another flatbuffer way below. We may opt this one in case of latency problem arises)
The method described above is what we have considered. We opted for this approach because we believed that if we differentiated them solely by postfix, it might limit our ability to customize the output path, and we would always need to include the postfix.
I totally agree with the point.
When view
.tflite
file compiled with edge tpu compiler using netron, we notice thatdescription : Exported from Subgraph
. We observed that this text appears in.tflite
file compiled with edge tpu compiler, regardless of the original description.
That's cool :-D
- We used Tensorflow's schema.fbs file that licensed under the Apache License V2. Is it ok?
Yes, Apache License is okay. :-D
It's the license we can use. (Apache 2.0)
Also the MIT License is fine to use.
- When we open 6~7mb tflie file for the first time, it spend 20ms. After second times, it spend 3~4ms. Would this have significant impact on performace?
I think it matters... Imagine we have 500+ tflite files and latency gets 20ms * 500 = 10sec to load the view.
But it still sounds like a good thing that you discovered this one.
I think it's why we need to try modulization, to ensure the basic functionality of ONE-vscode while we try some optional functions for our backend.
To make long short; we concluded that using _edgetpu.tflite prefix for edgetpu compiled models.