Extending `RModelWrapper` to allow dynamic switching between models
Closed this issue · 1 comments
Part of Robustar v0.3 Main Features (#175)
All model-related logic is implemented in back-end/objects/RModelWrapper
. It declares the model architecture (init_model
function) and tries to load in any model weight if provided (load_net
function). Currently the model is only initialized when launching Robustar, and there is no way of switching to another model later. To support dynamically switching to another model through API calls, we need to expose some functions for the API layer to use. Specifically, we need to
-
Add more fields to
RModelWrapper
class to keep track of current model information (name, description, created_time, last_trained, architecture, ...). -
Model definition logic should be moved out from
__init__
function into a new function that the API layer calls. For exampledef switch_model(self, model_id: str): """ Query the database to pull out a model information and load accordingly """ if not is_valid(model_id) return error model_info = get_model(model_id) if model_info: self.aaa = model_info.aaa self.bbb = model_info.bbb # more logic here ... if model_info.weight_path: try: self.load_net(weight_path) except: # ...
-
Create a file (if not exist)
back-end/utils/model_utils.py
and implement any required database queries / other related utility functions. Database queries should utilize generic database query functions defined inback-end/utils/db_ops.py
. You may take a look atback-end/objects/RImageFolder.py
to see how to use these generic database query functions.
As the first step, I think we can try implementing the following (which I think will also be a great onboarding task). Feel free to DM me or post questions in group chat on slack if anything is not clear to you.
- Create an model API definition file
model.py
underback-end/apis/
and a utils fileback-end/utils/model_utils.py
. - Define a blueprint and an endpoint that looks like following to set the current model. You may refer to other API definitions how this is done.
@route(POST, "/model/current/<id>")
def SetCurrModel(id: string):
""" return 200 on success """
- Expose the API endpoint in
back-end/apis/__init__.py
. - In
RModelWrapper
, create a datastructure (a dictionary that maps id to a model for now) that stores multiple PyTorch models. You may hardcode models and ids, initialize these models and add them to the datastructure in__init__
function. - Implement the
switch_model
function (see the description of the issue). You don't have to follow the pseudocode, because you may have designed things differently. - Fill in missing details in
back-end/apis/model.py
andback-end/utils/model_utils.py
. The end goal is that when the front end send a request to set current model with an id, you should query the datastructure to pull the model out and set it as the current model. When other parts of Robustar code is trying to access the model, it should get the current one. - Implement a new test case in
back-end/tests/test_model.py
. The test case should switch between models and expect different predictions. Specifically:- Ask Robustar server to make prediction on an image and store the prediction result as
result 1
- Using the API you just defined, ask Robustar to switch to another model
- Ask Robustar server to make a prediction again, save result in
result 2
- Switch back to the first model
- Make prediction again, save result in
result 3
. result 1
should be equal toresult 3
, andresult 1
should be different fromresult 2
You may refer totest_predict_success
function inback-end/tests/test_predict.py
to see how test cases are implemented.
- Ask Robustar server to make prediction on an image and store the prediction result as
- Under
back-end
, with dev environment fully setup, runpython -m pytest
. If you pass all the test cases, then you are done with the first step. Congrats!